US20130254482A1 - Storage system - Google Patents
Storage system Download PDFInfo
- Publication number
- US20130254482A1 US20130254482A1 US13/897,672 US201313897672A US2013254482A1 US 20130254482 A1 US20130254482 A1 US 20130254482A1 US 201313897672 A US201313897672 A US 201313897672A US 2013254482 A1 US2013254482 A1 US 2013254482A1
- Authority
- US
- United States
- Prior art keywords
- real
- pdev
- virtual
- page
- storage devices
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0665—Virtualisation aspects at area level, e.g. provisioning of virtual or logical volumes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
- G06F2211/10—Indexing scheme relating to G06F11/10
- G06F2211/1002—Indexing scheme relating to G06F11/1076
- G06F2211/103—Hybrid, i.e. RAID systems with parity comprising a mix of RAID types
Definitions
- the present invention relates to storage control to which Dynamic Provisioning (also called Thin Provisioning) is applied.
- Dynamic Provisioning also called Thin Provisioning
- RAID Redundant Array of Independent (or Inexpensive) Disks
- thin provisioning is being introduced to enhance the utilization efficiency of storage system capacity.
- a RAID group is configured from a plurality of storage devices, and a logical volume is configured on the RAID group.
- the logical volume is provided to a host, and a storage area of a storage device having a capacity that is equivalent to the entire storage capacity of the logical volume is allocated to this logical volume from the time the logical volume is created.
- a storage area of the storage device corresponding to the initial virtual storage area is not allocated to the virtual storage area inside this virtual volume, but rather, the storage area of the storage device that is allocated to the logical volume is allocated only to the virtual storage area when there has been a write request with respect to the virtual storage area. For this reason, only a storage device of the capacity required for the time being is provided when the storage system is introduced, and a storage device can easily be added as the need arises.
- Patent Literature 1 proposes a method for rearranging data on a plurality of storage devices in logical volume units in a storage system that uses thin provisioning.
- Patent Literature 2 proposes a method for adding storage devices in storage device units by changing the data redundancy configuration after adding a storage device.
- a RAID group is configured from a plurality of storage devices prior to these storage devices being allocated to the virtual volume, and data management is carried out in logical volume units inside the RAID group. For example, data redundancy is maintained by storing the data that is inside the logical volume so as to extend across a plurality of storage devices of this RAID group. For this reason, storage device addition must be carried out by adding a plurality of storage devices during a single addition to achieve a group of storage devices in the numbers required to configure a RAID group.
- the premise is that a RAID group is configured from a plurality of storage devices prior to these storage devices being allocated to the virtual volume the same as in the above-mentioned Patent Literature 1, and that a logical volume in this RAID group is provided to the virtual volume. In so doing, the RAID group must be reconfigured after adding a single storage device, giving rise to the need to reorganize the data redundancy configuration.
- An object of the present invention is to provide a method for enhancing the flexibility and lowering the cost of storage device addition in a storage system that makes use of thin provisioning.
- An object of the present invention is to suppress the influence, which is exerted by the addition of storage devices, on business or other operations in a storage system to which the thin provisioning is employed.
- Another object of the present invention is to provide a storage system that does not require a spare disk, i.e., a specific storage device provided in preparation for a storage device failure.
- a storage area of a storage device is provided directly to a virtual volume instead of providing a logical volume inside the RAID group to the virtual volume. That is, the storage system, upon receiving a write request with respect to a virtual storage area, first of all, specifies the data redundancy configuration (a number of data partitions and a number of created parities) and the RAID level set to the virtual volume comprising this virtual storage area. The storage system selects a number of storage devices in accordance with the specified RAID level and redundancy configuration for this virtual storage area. The storage system selects, from among the selected storage devices, a storage area that is not allocated to a virtual storage area, and allocates this storage area to this virtual storage area. The storage system partitions the data and writes this data together with the parity to this allocated storage area.
- FIG. 1 shows the configuration of a storage system related to an embodiment of the present invention.
- FIG. 2 shows an overview of dynamic capacity expansion related to this embodiment.
- FIG. 3 shows an example of a first configuration of a DP pool.
- FIG. 4 shows an example of a second configuration of the DP pool.
- FIG. 5 shows a computer program and information stored in a memory 123 .
- FIG. 6 shows the configuration of a DP common specification table 503 .
- FIG. 7 shows the configuration of a pool management table 505 .
- FIG. 8 shows the configuration of a virtual LU management table 507 .
- FIG. 9 shows the configuration of a PDEV management table 509 .
- FIG. 10 shows the configuration of a real page management table 511 .
- FIG. 11 shows the configuration of a mapping management table 513 .
- FIG. 12 shows the configuration of a real access status table 515 .
- FIG. 13 shows the configuration of a virtual access status table 517 .
- FIG. 14 is a flowchart showing the flow of processing for creating a DP pool.
- FIG. 15 is a flowchart showing the flow of processing for creating a virtual LU 201 .
- FIG. 16 shows an overview of the flow of a write to a virtual page.
- FIG. 17 is a flowchart showing the flow of processing for writing to a virtual page.
- FIG. 18 is a flowchart showing the flow of processing for reading from a virtual page.
- FIG. 19 shows how data stored in accordance with a plurality of RAID types is mixed together in a single DP pool.
- FIG. 20 shows an example of the arrangement of data units (generic name for data elements and parity) in a case where a PDEV allocated to a single virtual page is selected without relying on the PDEV number.
- FIG. 21 is a flowchart for processing carried out when adding a PDEV.
- FIG. 22 shows a flowchart of a rearrange process.
- FIG. 23 shows an example of rearranging carried out in a case where a PDEV has been added.
- FIG. 24 is a flowchart of processing carried out when removing a PDEV.
- FIG. 25 shows an example of rearranging based on access frequency.
- FIG. 26 shows an overview of a restore process in the case of a failure in a PDEV.
- FIG. 27 is a flowchart of a priority-restore type restore process.
- FIG. 28 is a flowchart of a priority-replace type restore process.
- FIG. 1 is shows the configuration of a storage system related to an embodiment of the present invention.
- One or more (for example, two) host apparatuses (hereinafter, host) 101 are coupled to the storage system 103 .
- Communication between the host 101 and the storage system 103 is carried out using a protocol that conforms to either a Fibre Channel (FC) or an iSCSI (SCSI is the abbreviation for Small Computer System Interface).
- FC Fibre Channel
- iSCSI Small Computer System Interface
- the storage system 103 manages a virtual LU (LU is the abbreviation for a Logical Unit).
- the virtual LU is a virtual LU that conforms to Dynamic Provisioning (also called Thin Provisioning) technology.
- the virtual LU comprises a plurality of virtual pages (virtual storage areas).
- One or more addresses for example, a LBA (Logical Block Address) belong to each virtual page.
- the host 101 sends an I/O command for carrying out I/O (Input/Output) with respect to the virtual LU.
- the I/O command comprises I/O destination information.
- the I/O destination information includes the LUN (Logical Unit Number) of a virtual LU and the virtual page address of the virtual LU.
- the host 101 is generally a computer, but another storage system may be used instead of a computer.
- the storage system 103 comprises a plurality of physical storage devices (PDEV) 117 , and a storage control apparatus (hereinafter referred to as “DKC”) 111 that is coupled to the plurality of PDEV 117 .
- the DKC 111 is multiplexed (for example, duplexed).
- the plurality of PDEV 117 may be configured from the same type of PDEV, or may be a mixture of a plurality of types of PDEV.
- types of PDEV for example, there are a HDD (Hard Disk Drive) and a SSD (Solid State Drive).
- the DKC 111 receives and processes an I/O command (either a write command or a read command) from the host 101 .
- the DKC upon receiving a write command, for example, allocates a plurality of real pages (substantial storage areas) to each of one or more write-destination virtual pages specified in the received write command. For the sake of convenience, the allocation of a real page to a virtual page will be called “dynamic capacity expansion”.
- a plurality of PDEV 117 are coupled to the DKC 111 via one or more backend coupling devices 113 .
- the backend coupling devices 113 are multiplexed (for example, duplexed).
- a backend coupling device 113 for example, a FC-AL (Fibre Channel-Arbitrated Loop) or a SAS (Serial Attached SCSI) expander may be used.
- FC-AL Fibre Channel-Arbitrated Loop
- SAS Serial Attached SCSI expander
- a management apparatus 105 is coupled to at the least one DKC 111 .
- the management apparatus 105 for example, is coupled to the at the least one DKC 111 via a LAN (Local Area Network).
- the management apparatus 105 for example, may be a computer and may carry out various types of settings and references with respect to a LU.
- Each DKC 111 comprises a FE-IF (front-end interface) 121 , a BE-IF (backend interface) 127 , and a controller.
- the controller for example, comprises a transfer circuit 129 , a memory 123 , and an MP (a microprocessor) 125 .
- the controller may comprise a hardware circuit for creating parity instead of the MP 125 .
- the FE-IF 121 , the memory 123 , the MP 125 , and the BE-IF 127 are coupled to the transfer circuit 129 .
- the transfer circuit 129 relays data and a command exchanged among these elements 121 , 123 , 125 , and 127 .
- the FE-IF 121 is an interface device that is coupled to the host 101 .
- the FE-IF 121 comprises a plurality of communication ports, and a host computer 101 is coupled to each communication port.
- the FE-IF receives an I/O command from the host 101 and transfers the received I/O command to the MP 125 .
- the BE-IF 127 is an interface device that is coupled to the PDEV 117 .
- the BE-IF 127 comprises a plurality of communication ports, and a PDEV 117 is coupled to each communication port via the backend coupling device 113 .
- the memory 123 is configured from one or more memories.
- the memory 123 stores information (for example, tables) and a computer program.
- the memory 123 comprises a cache area. Data that is exchanged between the host 101 and the PDEV 117 is temporarily stored in the cache area.
- the MP 125 processes an I/O command from the host 101 . Further, in response to a request from the management apparatus 105 , the MP 125 also sets information (for example, LU-related information) in the memory 123 and provides the information (for example, LU-related information) stored in the memory 123 to the management apparatus 105 .
- information for example, LU-related information
- FIG. 2 shows an overview of dynamic capacity expansion related to this embodiment.
- a virtual LU 201 and a DP (Dynamic Provisioning) pool 203 are managed by the DKC 111 .
- the virtual LU 201 is associated with one DP pool 203 , and a real page is allocated from the DP pool 203 to this virtual LU 201 .
- the virtual LU 201 is configured from a plurality of virtual pages, and is recognized by the host 101 .
- a RAID level is defined for the virtual LU 201 .
- the combination, the RAID level and capacity of the virtual LU 201 can be set from the management apparatus 105 (refer to FIG. 1 ).
- the RAID level here is obtained by classifying the RAID technology in accordance with the function thereof, and is expressed as RAID 5 and RAID 6. For example, in RAID 5, data for which there is a write request is partitioned into n parts (n being a natural number), one parity data is created from this data, and this partitioned data and parity are distributively written to (n+1) storage devices.
- nD+mP or D
- the DP pool 203 is configured from either a portion or all of the PDEV 117 of the plurality of PDEV 117 of the storage system 103 .
- the DP pool 203 is configured from a large number of real pages.
- the DP pool 203 may lack a RAID definition, and a real page may be provided from individual PDEV rather than from a RAID group configured from a plurality of PDEV.
- a real page allocated to a virtual LU is formed on the basis of a predefined RAID group storage space.
- the pool is configured from either one or a plurality of RAID groups.
- Each RAID group is configured from two or more PDEV on the basis of the RAID level.
- a real page is configured on the basis of one RAID group, and one real page extends over two or more PDEV that configure a RAID group. For this reason, in a write to a real page that is allocated to a virtual page, the write-targeted data is written on the basis of the RAID level and combination of the RAID group that provides this real page.
- the individual PDEV 117 that configure the DP pool 203 are not RAID group-configuring members.
- a real page is provided from an individual PDEV.
- the RAID level and combination are set for the virtual LU 201 .
- a number of real pages determined on the basis of the RAID level and combination of this virtual LU 201 is allocated from the DP pool 203 to a virtual page of the virtual LU 201 , and write-targeted data with respect to this virtual page is written to the allocated plurality of real pages.
- two or more data elements (or write-targeted data and a replicate of this data) from among a plurality of data elements that configure the write-targeted data are not stored in a single PDEV 117 .
- two or more real pages from a single PDEV 117 will not be allocated to the same virtual page. That is, real pages from respectively different PDEV 117 are allocated to a single virtual page. Therefore, it is preferable that a plurality of real pages be allocated to a virtual page so that the utilization rate of a plurality of PDEV belonging to a DP pool (the ratio of the total capacity of the allocated real pages with respect to the capacity of the PDEV) becomes uniform.
- the plurality of real pages may be allocated to a virtual page so that not the utilization rate of the plurality of PDEV but free capacity of a plurality of PDEV becomes uniform.
- the virtual LU 201 and the DP pool 203 There may be either one or a plurality of the virtual LU 201 and the DP pool 203 .
- FIG. 2 it is supposed that there are two virtual LU # 1 and # 2 as the virtual LU 201 , there is one DP pool # 1 as the DP pool 203 , and the virtual LU # 1 and # 2 are associated with the DP pool # 1 .
- the RAID level and combination of the virtual LU # 1 is RAID 5 (2D+1P), and that the RAID level and combination of the virtual LU # 2 is RAID 6 (4D+2P).
- the DP pool # 1 is configured from six PDEV # 01 through # 06 .
- RAID level and combination of virtual LU # 1 According to RAID 5 (2D+1P), data that is targeted for a write to virtual page V 11 of the virtual LU # 1 is divided into two data elements by the DKC 111 , and, in addition, one parity is created. For this reason, three real pages R 11 , R 22 , and R 31 are allocated to the virtual page V 11 from the three PDEV # 01 through # 03 . The corresponding relationship between the virtual page V 11 and the real pages R 11 , R 22 , and R 31 is managed by the DKC 111 . Then, two data elements are respectively written to two of the three real pages R 11 , R 22 , and R 31 , and a parity is written to the remaining one real page.
- RAID level and combination of virtual LU # 2 According to RAID 6 (4D+2P), data that is targeted for a write to virtual page V 21 of the virtual LU # 2 is divided into four data elements, and, in addition, two parities are created. For this reason, six real pages R 18 , R 23 , R 33 , R 41 , R 51 , and R 61 are allocated to the virtual page V 21 from the six PDEV # 01 through # 06 . The corresponding relationship between the virtual page V 21 and the real pages R 18 , R 23 , R 33 , R 41 , R 51 , and R 61 is managed by the DKC 111 . Then, four data elements are respectively written to four of the six real pages R 18 , R 23 , R 33 , R 41 , R 51 , and R 61 , and two parities are written to the remaining two real page.
- One DP pool 203 may be configured by one or more physical storage devices (PDEV) having the same attributes, and may also be configured by PDEV having different attributes. Attributes, for example, may include the I/O performance or reliability (for example, the MTBF (Mean Time Between Failures)) of the PDEV.
- the I/O performance of the PDEV for example, may include the type of the PDEV itself (for example, HDD, SSD, and so forth), the type of interface (for example, an FC (Fibre Channel), a SATA (Serial ATA), an SAS (Serial Attached SCSI), and the rotational speed of the disk.
- a single DP pool 203 may be configured from PDEV (in the example of FIG. 3 , HDD and SSD) having different types of attributes.
- PDEV in the example of FIG. 3 , HDD and SSD
- performance drops to the level of the PDEV with the lowest I/O performance when writing or reading data to or from the respective PDEV, making it preferable that each virtual page be configured from PDEV having the same attribute.
- respective virtual LU are assigned to one pool. For this reason, the mixing together of PDEV with different attributes inside the same pool makes it possible to allocate real pages inside PDEV with different attributes to each virtual page of the same virtual LU.
- a DP pool 203 of this type of configuration makes it possible to efficiently use disk capacity.
- one or more prescribed types of attributes of a plurality of attributes may be configured in the same PDEV of one DP pool 203 .
- a single DP pool 203 is configured from PDEV for which the PDEV performance, the PDEV interface and the type of the PDEV itself are all the same.
- a DP pool 203 of this type configuration it is possible to anticipate a desired I/O performance with respect to accessing the virtual LU 201 .
- this virtual LU may be associated with a DP pool (for example, DP pool #D) configured from PDEV having improved I/O performance than the PDEV that configure the DP pool #E.
- a DP pool for example, DP pool #D
- the data elements and parity are migrated from the pre-change DP pool to the post-change DP pool.
- Information related to the DP pool 203 and information denoting the corresponding relationship between a virtual page and a plurality of real pages is managed by the DKC 111 .
- This information for example, is stored in the memory 123 (for example, the cache area) of the DKC 111 .
- FIG. 5 shows the computer program and information stored in the memory 123 .
- the memory 123 stores a control processing program 501 .
- the control processing program 501 is executed by the MP 125 .
- processing that is carried out by the program 501 is actually performed by the MP 125 .
- the memory 123 stores a DP common specification table 503 , a pool management table 505 , a virtual LU management table 507 , a PDEV management table 509 , a real page management table 511 , a mapping management table 513 , a real access status table 515 , and a virtual access status table 517 .
- the real page management table 511 exists in each DP pool 203 .
- the information in at least one of these tables may be managed in a structure other than a table.
- FIG. 6 shows the configuration of the DP common specification table 503 .
- the DP common specification table 503 comprises common information related to the specification of the DP pool. Specifically, for example, the table 503 comprises an item 200 and a value 201 for each item related to the DP pool specification.
- the item 200 is the name of an item, and the value 201 is the value for this item.
- the maximum number of creatable pools (the maximum number of DP pools 203 capable of being created in the storage system 103 ) is 50, and the real page capacity is 32 KB (kilobytes).
- the real page capacity is the same in all of the DP pools.
- the capacity of a virtual page that configures the virtual LU 201 is determined on the basis of the real page capacity, RAID level and combination.
- the combination is nD+mP, it is possible to store in the virtual page data of n-times the capacity of the real page, and (n+m) times the capacity is allocated.
- the capacity of a virtual page of a RAID 6 (4D+2P) virtual LU 201 is twice the capacity of a virtual page of a RAID 5 (2D+1P) virtual LU 201 .
- the number of real pages allocated to a virtual page of a RAID 6 (4D+2P) virtual LU 201 is twice the number of real pages that is allocated to a virtual page of a RAID 5 (2D+1P) virtual LU 201 .
- six real pages are allocated to the virtual page of the RAID 6 (4D+2P) virtual LU 201
- three real pages are allocated to the virtual page of the RAID 5 (2D+1P) virtual LU 201 .
- FIG. 7 shows the configuration of the pool management table 505 .
- the pool management table 505 comprises information related to the DP pool 203 .
- the table 505 comprises a pool number 210 , a PDEV number list 211 , a real allocation capacity 212 , a utilization capacity 213 , and a first PDEV number 214 for each DP pool 203 .
- the pool number 210 is the number of the DP pool 203 .
- the PDEV number list 211 is a list of numbers of the PDEV that configure the DP pool 203 .
- the real allocation capacity 212 is the total capacity of the PDEV that configure the DP pool 203 . Therefore, in a case where a new PDEV is added to the DP pool 203 , the real allocation capacity 212 will increase in proportion to the capacity of the added PDEV. By contrast, in a case where a PDEV is removed from the DP pool 203 , the real allocation capacity 212 will decrease in proportion to the capacity of the removed PDEV.
- the utilization capacity 213 is the total capacity of a real page that has been allocated from the DP pool 203 .
- the first PDEV number 214 is the number of the PDEV that is the next allocation source of a real page.
- the two or more PDEV that provide two or more real pages that are allocated to a virtual page are in PDEV number order.
- the RAID level and combination of the virtual LU is RAID 5 (2D+1P)
- an initial write is generated to an unallocated virtual page (a virtual page to which a real page is not allocated) in this virtual LU
- the first PDEV number 214 is “1”, and therefore, respective real pages are allocated from PDEV # 01 , # 02 , and # 03 to the write-destination virtual page. Thereafter, the first PDEV number 214 is updated to “4”.
- the first PDEV number 214 is “4”
- respective real pages are allocated from PDEV # 04 , # 01 , and # 02 to the write-destination virtual page.
- the first PDEV number 214 is updated to “3”.
- the two or more PDEV that are the provision sources of the two or more real pages allocated to the virtual page are determined based on the order of the PDEV numbers.
- the numbers of the plurality of PDEV configuring the DP pool may be consecutive or nonconsecutive.
- the two or more PDEV that are the provision sources of the two or more real pages allocated to the virtual page may be arranged in descending order from the largest number rather than in ascending order from the smallest number. Since the two or more provision-source PDEV are determined in PDEV number order, the PDEV inside the DP pool are used equably.
- FIG. 8 shows the configuration of the virtual LU management table 507 .
- the virtual LU management table 507 comprises information related to the virtual LU 201 , and can be used to set and check the virtual LU. Specifically, for example, the table 507 comprises a LUN 220 , an assigned pool number 221 , a RAID level 222 , a combination 223 and a capacity 224 for each virtual LU 201 .
- the LUN 220 is the LUN of the virtual LU 201 .
- the assigned pool number 221 is the number of the DP pool (the DP pool with which the virtual LU 201 is associated) 203 to which the virtual LU 201 is assigned.
- the RAID level 222 is the RAID level of the virtual LU 201 .
- the combination 223 is the combination of the virtual LU 201 .
- the capacity 224 is the capacity of the virtual LU 201 .
- FIG. 9 shows the configuration of the PDEV management table 509 used to create, for example, a DP pool.
- the PDEV management table 509 comprises information related to the PDEV 117 .
- the table 509 comprises a PDEV number 230 , an assigned pool number 231 , an IF type 232 , a PDEV performance 233 , a capacity 234 , a free capacity 235 , and a device type 236 for each PDEV 117 .
- the PDEV number 230 is the number of a PDEV 117 .
- the assigned pool number 231 is the number of the DP pool (the DP pool in which the PDEV 117 is one of the components) to which the PDEV 117 is assigned. “null” shows that the PDEV 117 is not allocated to any pool.
- the IF type 232 is the type (for example, SAS, SATA, and so forth) of the interface of the PDEV 117 .
- the PDEV performance 233 is expressed in accordance with the device type 236 of the PDEV 117 .
- the PDEV performance 233 is shown by the rotational speed of the disk (for example, 7,200 rpm).
- the capacity 234 is the capacity of the PDEV 117 , that is, the total capacity of the real pages that that PDEV 117 provides.
- the free capacity 235 is the total capacity of the real page that is not allocated to the virtual page from among the real pages that the PDEV 117 provides, but can be allocated to the virtual page.
- the device type 236 is the type of the PDEV 117 itself.
- the device type 236 for example, includes HDD and SSD.
- FIG. 10 shows the configuration of the real page management table 511 showing the relationship between the virtual page and the real page.
- the real page management table 511 is used when allocating a real page to a virtual page.
- the real page management table 511 exists for each DP pool.
- the real page management table 511 comprises information related to a real page inside the DP pool corresponding to this table 511 .
- the table 511 comprises a PDEV number 240 , a real page number 241 , a LUN 242 , a virtual page number 243 , and an access counter 244 for each real page.
- the PDEV number 240 is the number of the PDEV 117 that provides the real page.
- the real page number 241 is the number of the real page.
- the LUN 242 is the LUN of the virtual LU 201 comprising the virtual page that is the allocation destination of the real page.
- the virtual page number 243 is the number of the allocation-destination virtual page of the real page.
- the access counter 244 is the number of times that an I/O (access) has been carried out to the real page. Furthermore, for example, in a case where there is a real access status table 515 as alternative information as in this embodiment, the access counter 244 may be eliminated.
- FIG. 11 shows the configuration of the mapping management table 513 used, for example, when judging whether the real page is allocated to the virtual page that has received a write command.
- the mapping management table 513 shows the corresponding relationship between the virtual page and the real page. Specifically, for example, the table 513 comprises a LUN 250 , a virtual page number 251 , a real page identification information 252 , and an access counter 253 for each virtual page.
- the LUN 250 is the LUN of the virtual LU 201 of the virtual page.
- the virtual page number 251 is the number of the virtual page.
- the real page identification information 252 is information for identifying the real pages that are allocated to the virtual page, and, for example, is a combination of the real page numbers and the numbers of the PDEV 117 of these real pages.
- the access counter 253 is the number of times that an I/O (access) has been carried out with respect to the real page. Furthermore, for example, in a case where there is a virtual access status table 517 as alternative information as in this embodiment, the access counter 253 may be eliminated.
- FIG. 12 shows the configuration of the real access status table 515 referenced, for example, when determining a storage device for rearrange and the like.
- the real access status table 515 comprises information, which is common to one or more DP pools 203 , and which shows the access state for each real page.
- the table 515 comprises a cell for each real page, and the cell comprises information showing the access frequency with respect to the real page (hereinafter, real access frequency information).
- the real access frequency information for example, includes information showing the usage of the real page corresponding to this information, and shows the write frequency and the read frequency separately.
- the write frequency is the number of writes per unit period, specifically, for example, the number of writes per minute, hour, day and week.
- the read frequency is the number of reads per unit period, specifically, for example, the number of reads per minute, hour, day and week.
- a record corresponding to this added PDEV 117 is added to the table 515 . Further, in a case where one PDEV 177 is removed from any DP pool 203 , the record corresponding to this removed PDEV 117 is removed from the table 515 .
- FIG. 13 shows the configuration of the virtual access status table 517 referenced, for example, when determining the virtual page for rearrange and the like.
- the virtual access status table 517 comprises information denoting the access status of each virtual page.
- the table 517 comprises a cell for each virtual page, and the cell comprises information showing the access frequency with respect to the virtual page (hereinafter, virtual access frequency information).
- the virtual access frequency information for example, includes information showing the usage of the virtual page corresponding to this information, and shows both the write frequency and the read frequency.
- the write frequency is the number of writes per unit period, specifically, for example, the number of writes per minute, hour, day and week.
- the read frequency is the number of reads per unit period, specifically, for example, the number of reads per minute, hour, day and week.
- At the least one of the virtual access frequency information and the above-mentioned real access frequency information may show the access frequency without regard for a write or a read, may show only the write frequency, or may show only the read frequency.
- At the least one of the virtual access frequency information and the above-mentioned real access frequency information may be updated each time an access occurs with respect to a virtual area or a real page, or may be updated on a predetermined schedule (for example, regularly).
- the control processing program 501 may manage a processed command log in the memory 123 .
- the command log may comprise, for example, a type, a date/time (for example, date/time received, process completion date/time, or a date/time of an I/O command time stamp), the number of the access-destination virtual page, and the identification information of the access-destination real page (for example, a combination of the PDEV number and the real page number) for each processed I/O command.
- control processing program 501 may add to the command log the type, date/time, number of the access-destination virtual page, and identification information for the access-destination real page related to this I/O command.
- the control processing program 501 may update the virtual access frequency information and the real access frequency information based on this command log.
- FIG. 14 is a flowchart showing the flow of processing for creating a DP pool.
- the control processing program 501 receives a create pool instruction from the management apparatus 105 .
- the parameter of the create pool instruction for example, is at the least (p 1 ) of the following (p 1 ) through (p 3 ).
- the (p 1 ) is a parameter that must be input by the user
- the (p 2 ) and the (p 3 ) are parameters that are input at the discretion of the user.
- the user-requested pool capacity differs from the real allocation capacity 212 . This is because the real allocation capacity 212 is the total capacity of the PDEV that belong to a DP pool.
- the control processing program 501 determines whether or not the number of DP pools 203 specified from the pool management table 505 is smaller that the value 201 of the maximum number of creatable pools. In a case where the result of this determination is affirmative, S 12 is performed. In a case where the result of this determination is negative, the control processing program 501 sends information denoting an error to the management apparatus 105 , and the management apparatus 105 displays an error (S 16 ).
- the control processing program 501 selects PDEV 117 so that the total capacity is greater than the user-requested pool capacity. Specifically, for example, the control processing program 501 carries out the following processing.
- the control processing program 501 references the PDEV management table 509 , and specifies a PDEV (a PDEV that is not assigned to any DP pool 203 ) for which the assigned pool number 231 is “null”.
- the specified PDEV is a DP pool component candidate.
- the assigned pool number 231 is “null”, and, in addition, the PDEV that conforms to this PDEV information is specified as the DP pool component candidate.
- the PDEV specified in this (S 12 - 1 ) will be called the “candidate PDEV”.
- the control processing program 501 selects from a plurality of candidate PDEV the PDEV having a total capacity greater than the user-requested pool capacity.
- the total capacity of the selected PDEV (the real allocation capacity of the DP pool) is either the same as or larger than the user-requested pool capacity.
- the RAID level and combination may also be taken into consideration in PDEV selection. For example, in the case where a user-requested pool capacity is 1 TB (one terabyte), the control processing program 501 may select PDEV from a plurality of PDEV so that the total capacity of the selected PDEV is 1.3 times the user-requested pool capacity (1.3 TB).
- a PDEV may be selected on the basis of the parity ratio of the RAID level and combination having the highest parity ratio. Further, in a case where the RAID level and combination of the virtual LU associated with the DP pool has been decided, the PDEV may be selected on the basis of the parity ratio of this RAID level and combination. Furthermore, the “parity ratio” is the percentage of the total size of the created parity with respect to the size of the I/O-targeted data for the virtual page.
- the control processing program 501 adds information related to the created DP pool to the pool management table 505 .
- the following information is added as information related to the created DP pool.
- the control processing program 501 updates the assigned pool number 231 of the PDEV selected in S 12 from “null” to the number of the created DP pool (the number in S 13 ) in the PDEV management table 509 .
- the control processing program 501 partitions the storage space of the respective PDEV (the respective PDEV selected in S 12 ) configuring the created DP pool with respect to each real page capacity, and manages the respected real pages obtained in accordance with this. Specifically, the control processing program 501 creates a real page management table 511 corresponding to the created DP pool, and stores this table 511 in the memory 123 .
- the respective records of the created real page management table 511 comprise the following information.
- FIG. 15 is a flowchart showing the flow of processing for creating a virtual LU 201 .
- the control processing program 501 receives a virtual LU create instruction from the management apparatus 105 .
- the virtual LU create instruction for example, has as a parameter at the least (p 15 ) of the following (p 11 ) through (p 15 ).
- the (p 15 ) is a parameter that must be inputted by the user, and the (p 11 ) through the (p 14 ) are parameters that are inputted at the discretion of the user.
- a parameter from among the parameters (p 11 ) through (p 14 ) that is not inputted by the user is a value (a default value) determined in accordance with a prescribed rule.
- the default value for example, is the smallest number of a group of numbers capable of being assigned as the LUN of the virtual LU.
- the control processing program 501 adds to the virtual LU management table 507 information related to the virtual LU to be created for each virtual LU in the number of virtual LU denoted in the above-mentioned (p 12 ). The following information is added as information related to the virtual LU to be created.
- the control processing program 501 adds to the mapping management table 513 information related to the respective virtual pages of the virtual LU to be created for each virtual LU in the number of virtual LU denoted in the above-mentioned (p 12 ).
- the information added for the respective the virtual pages is as follows.
- FIG. 16 shows an overview of the flow of a write to a virtual page.
- the flow of a write to a virtual page for which a real page is not allocated, and the flow of a write to a virtual page for which a real page has been allocated will be explained below.
- ⁇ Case 1 Write to a Virtual Page for which a Real Page is not Allocated>
- the control processing program 501 receives from the host # 1 a write command specifying the virtual page V 11 of the virtual LU # 11 , references the mapping management table 513 , and specifies the fact that a real page is not allocated to the write-destination virtual page V 11 .
- the control processing program 501 references the virtual LU management table 507 , and specifies that the RAID level of the virtual LU # 11 that comprises the virtual page V 11 is RAID 5 (2D+1P).
- the control processing program 501 references the real page management table 511 corresponding to the DP pool # 11 to which the virtual LU # 11 is assigned, and selects the three real pages having different PDEV numbers 240 (the number of real pages based on the RAID level (RAID 5 (2D+1P)) specified in (w 12 )) from the real pages for which the LUN 242 and the virtual page number 243 are null.
- the real page R 11 inside PDEV # 11 , the real page R 21 inside PDEV # 12 , and the real page R 31 inside PDEV # 13 are selected here.
- the control processing program 501 allocates the real pages R 11 , R 21 , and R 31 selected in (w 13 ) to the virtual page V 11 and updates the mapping management table 513 .
- the control processing program 501 writes two data elements obtained by partitioning the write-targeted data for the virtual page V 11 to two of the three real pages R 11 , R 21 , and R 31 , and writes the parity based on these two data elements to the one remaining real page of these three real pages.
- ⁇ Case 2 Write to a Virtual Page for which a Real Page has Been Allocated>
- the control processing program 501 receives from the host # 2 a write command specifying the virtual page V 22 of the virtual LU # 12 , references the mapping management table 513 , and specifies the real page that has been allocated to the write-destination virtual page V 22 . Since the RAID level of the virtual LU # 12 that comprises the virtual page V 22 is RAID 6 (3D+2P), five real pages are allocated to the virtual page V 22 , and these five real pages are specified in (w 21 ).
- the control processing program 501 writes three data elements obtained by partitioning the write-targeted data for the virtual page V 22 among three of the five real pages R 32 , R 41 , R 51 , R 61 , and R 71 specified in (w 21 ), and writes two parities based on these three data elements to the two remaining real pages of these five real pages.
- FIG. 17 is a flowchart showing the flow of processing for a write to a virtual page.
- the control processing program 501 receives a write command comprising write-destination information and write-targeted data.
- the write-destination information includes a LUN and a LBA.
- the write-targeted data is written to the memory 123 (the cache area).
- the control processing program 501 references the mapping management table 513 , and determines whether or not a real page has been allocated to the virtual page (write-destination virtual page) specified from the write-destination information.
- S 37 is carried out without performing S 32 through S 36 . That is, the control processing program 501 writes the write-targeted data (for example, the two or more data elements and one or more parities obtained by partitioning this data) to the real pages (for example, the real pages R 32 , R 41 , R 51 , R 61 , and R 71 of FIG. 16 ) that have been allocated to the write-destination virtual page (for example, the virtual page V 22 of FIG. 16 ) based on the RAID level of the virtual LU (for example, the virtual LU # 12 of FIG. 16 ) comprising the write-destination virtual page. Then, the control processing program 501 performs the following write status update process:
- the control processing program 501 specifies the RAID level 222 and combination 223 (for example, RAID 5 (2D+1P) corresponding to the write-destination virtual LU (for example, the virtual LU # 11 of FIG. 16 ).
- the control processing program 501 specifies the assigned pool number 221 corresponding to the write-destination virtual LU and the first PDEV number 214 corresponding to the pool number 210 of the same value.
- the control processing program 501 selects the same number of PDEV numbers as the number K selected in S 32 from the assigned pool number 221 corresponding to the write-destination virtual LU and the PDEV number list 211 corresponding to the pool number 210 of the same value. At this time, the control processing program 501 , first selects the same PDEV number (for example, # 11 ) as the first PDEV number 214 selected in S 33 , makes this PDEV number the first PDEV number, and selects the PDEV numbers in order from the smallest PDEV number until the number of selected PDEV numbers reaches K.
- the same PDEV number for example, # 11
- the selected PDEV number reaches the largest number of the PDEV number list 211 corresponding to the DP pool (for example, DP pool # 11 of FIG. 16 ) to which the write-destination virtual LU is assigned prior to the number of selected PDEV numbers having reached the K, the smallest number of this PDEV number list 211 is selected.
- the PDEV numbers (PDEV) may be selected as follows instead of selecting the PDEV number (PDEV) in accordance with the PDEV number order.
- the control processing program 501 selects one free real page from the respective PDEV specified from the PDEV numbers selected in S 34 . Specifically, for example, the control processing program 501 selects any real page number 241 that corresponds to a LUN 242 and virtual page number 243 that are null for each selected PDEV number based on the real page management table 511 corresponding to the DP pool to which the write-destination virtual LU belongs. According to the example of FIG. 16 , the real page number R 11 is selected for the PDEV number # 11 , the real page number R 21 is selected for the PDEV number # 12 , and the real page number R 31 is selected for the PDEV number # 13 .
- the control processing program 501 updates the real page identification information 252 corresponding to the write-destination virtual page from null to a combination of the real page numbers 241 in S 35 and the PDEV numbers 240 corresponding to these numbers 241 (for example, # 11 -R 11 , # 12 -R 21 , # 13 -R 31 ). That is, the control processing program 501 allocates the two or more free real pages selected in S 35 to the write-destination virtual page (for example, the virtual page V 11 in FIG. 16 ).
- the control processing program 501 based on the RAID level and combination of the write-destination virtual LU, writes the write-targeted data to the two or more real pages allocated in S 36 . According to the example of FIG. 16 , the two data elements and one parity are written to the three real pages R 11 , R 21 , and R 31 . Then, the control processing program 501 performs the write status update process as described hereinabove.
- the value of the access counter 253 corresponding to the write-destination virtual page, the virtual access frequency information corresponding to the write-destination virtual page, the value of the access counter 244 corresponding to the respective real pages allocated to the write-destination virtual page, and the real access frequency information corresponding to the respective real pages allocated to the write-destination virtual page are updated.
- FIG. 18 is a flowchart showing the flow of processing of a read with respect to a virtual page.
- the control processing program 501 receives a read command comprising read-source information.
- the read-source information for example, comprises a LUN and a LBA.
- the control processing program 501 references the mapping management table 513 , and determines whether or not a real page is allocated to the virtual page specified from the read-source information (the read-source virtual page). In a case where the result of the determination in S 41 is negative, S 42 is carried out. In a case where the result of the determination in S 41 is affirmative, S 43 is carried out.
- the control processing program 501 sends an error to the host that is the source of the read command.
- the control processing program 501 reads two or more data elements (and one or more parities) from the two or more real pages allocated to the read-source virtual page, and sends to the read command-source host the data configured by the two or more data elements that have been read (that is, the read-targeted data) (the read-targeted data may be data that includes a data element that was recovered using parity). Then, the control processing program 501 performs the following read status update process:
- a real page that configures a DP pool 203 is not a substantial storage area provided from a RAID group, but rather is a storage area that is provided from one PDEV. Then, it is possible to associate a plurality of virtual LU with different RAID level definitions in a single DP pool 203 . For this reason, as shown in the example of FIG.
- data respectively stored at different RAID levels such as a set of data (data elements a, b and c) and parity P stored, for example, at RAID 5 (3D+1P) is mixed together with a set of data (data elements e, f, g, h and i) and parities P 1 and P 2 stored at RAID 6 (5D+2P).
- RAID levels such as a set of data (data elements a, b and c) and parity P stored, for example, at RAID 5 (3D+1P) is mixed together with a set of data (data elements e, f, g, h and i) and parities P 1 and P 2 stored at RAID 6 (5D+2P).
- the control processing program 501 carries out, for each virtual page, the following processing—(k 1 ) to (k 3 )—for each virtual LU to which a real page is allocated:
- (k 1 ) Reads v data units from v real pages allocated to the virtual page U, and constructs the data based on the v data units (a “data unit” as used in the explanation of this embodiment is a generic term for either a data element or a parity that is stored in the real page); (k 2 ) respectively selects from w PDEV of the DP pool to which the virtual LU is associated w real pages on the basis of the second RAID level and combination, and allocates the selected w real pages to the virtual page instead of the v real pages of (k 1 ); and (k 3 ) writes the w data units based on the data constructed in (k 1 ) to the allocated w real pages.
- the w real pages may be selected so that the utilization rates of the respective PDEV in the DP pool associated with the virtual LU for which the RAID level (or combination) changed become uniform.
- the utilization rates of the respective PDEV that belong to the DP pool with which this virtual LU is associated can be expected to be made uniform as a result of changing the RAID level (or combination) that is defined for the virtual LU.
- the w real pages may be selected so that not the utilization rates of the respective PDEV but the free capacity of the respective PDEV become uniform.
- the source PDEV for the real page(s) allocated to the virtual page is selected in PDEV number order, but, for example, as shown in FIG. 20 , the PDEV may be selected at random.
- the data elements a, b, and c and the parity P in conformance with the RAID 5 (3D+1P), and the data elements e, f, g, h, and i and the parities P 1 and P 2 in conformance with the RAID 6 (5D+2P) are stored in the real pages of the respective PDEV that were selected without relying on the PDEV number.
- the PDEV utilization rate (or free capacity) may be taken into consideration. This is to make the utilization rates (or free capacity) of the plurality of PDEV in the DP pool equable.
- control processing program 501 may allocate a plurality of real pages to one virtual page from a plurality of PDEV that are coupled to the same backend coupling device 113 .
- the access performance to these plurality of real pages can be expected to be made uniform.
- a plurality of real pages may be allocated to the virtual page from a plurality of PDEV coupled to a plurality of backend coupling devices 113 .
- a rearrange process by which a data unit is migrated from a real page inside a certain HDD to a real page inside a different PDEV, is performed.
- this rearrange process is carried out when a PDEV is added or removed.
- the addition of a PDEV is carried out to increase the free real pages.
- the addition of a PDEV to the DP pool 203 may be carried out in PDEV units.
- the control processing program 501 it is possible for the control processing program 501 to detect the fact that the total capacity of the free real pages is insufficient by referencing the pool management table 505 .
- the control processing program 501 detects that the total capacity of the free real pages is insufficient, the control processing program 501 sends the management apparatus 105 a warning and the number of the DP pool in which the total capacity of the free real pages is insufficient.
- the management apparatus 105 displays the warning and DP pool number. The user sees this warning and DP pool number, adds a PDEV, and makes the added PDEV a member of the DP pool identified from the DP pool number displayed together with the warning.
- the addition of a PDEV is not limited to when the total capacity of the free real pages is insufficient, and may also be carried out at other times (for example, in accordance with a prescribed plan).
- FIG. 21 is a flowchart of the processing carried out when adding a PDEV.
- the control processing program 501 adds information (a record) related to the added PDEV (hereinafter, the added PDEV) to the PDEV management table 509 .
- the added PDEV information related to the PDFV
- the IF type 232 , the PDEV performance 233 , the capacity 234 , and the free capacity 235 portions of the information related to the PDEV may be inputted by the user via the management apparatus 105 , or may be acquired from the added PDEV.
- the PDEV number 230 may be inputted by the user via the management apparatus 105 , or may be determined by the control processing program 501 .
- the control processing program 501 adds the assigned pool number 231 , that is, the number of the DP pool in which the added PDEV has become a member to the information related to the added PDEV in the PDEV management table 509 .
- This assigned pool number 231 may be a number inputted by the user via the management apparatus 105 , or may be a number that was determined by the control processing program 501 (for example, the number of the DP pool in which the shortage of free real page total capacity was detected).
- the control processing program 501 adds the number of the added PDEV to the PDEV number list 211 (hereinafter, referred to as the “target list 211 ” in the explanation of FIG. 21 ) corresponding to the same pool number 210 (refer to FIG. 7 ) as the assigned pool number 231 added in S 1801 , and, in addition, adds the capacity of the added PDEV to the real allocation capacity 212 corresponding to this pool number 210 . Further, the control processing program 501 adds the information (records) corresponding to the respective real pages of the added PDEV to the real page management table 511 that corresponds to the assigned pool number 231 added in S 1801 .
- the control processing program 501 From among the capacity 234 and the free capacity 235 managed in the PDEV management table 509 , the control processing program 501 references the capacity 234 and the free capacity 235 corresponding to the PDEV that belongs to the target list 211 , and computes the utilization rate (the ratio of the free capacity 235 to the capacity 234 ) of the respective PDEV in the target list 211 . Then, the control processing program 501 specifies the PDEV for which the utilization rate exceeds a predetermined threshold.
- the control processing program 501 executes the rearrange process until the utilization rate of the PDEV specified in S 1803 becomes less than the predetermined threshold.
- the “rearrange process” is the process for migrating a data unit from a real page inside a certain PDEV to a real page inside a different PDEV in a single DP pool.
- the control processing program 501 executes the rearrange process until the utilization rates of all the PDEV in the DP pool comprising the added PDEV become equable.
- the PDEV which is the target of the rearrange process may be decided based on not the utilization rates of the PDEV but free capacity of the PDEV.
- FIG. 22 shows a flowchart of the rearrange process. Furthermore, in the following explanation, data created based on a plurality of data units (data for which an I/O is performed with respect to a virtual area) will be called “I/O data”.
- the control processing program 501 selects the data unit migration-source real page and the data unit migration-destination real page.
- this rearrange process is the rearrange process in S 1804 of FIG. 21
- the migration-source real page is selected from the PDEV specified in S 1803 of FIG. 21 (the PDEV for which the utilization rate exceeds the predetermined threshold)
- the migration-destination real page is selected from the PDEV for which the utilization rate does not exceed the predetermined threshold (for example, the PDEV with the lowest utilization rate) even when the data unit is stored in at least one free real page.
- this rearrange process is the rearrange process in S 1805 of FIG. 21
- the migration-source real page is selected from the PDEV with the highest utilization rate
- the migration-destination real page is selected from the PDEV with the lowest utilization rate.
- the control processing program 501 determines whether or not the RAID is supported. “Whether or not the RAID is supported” refers to whether or not it is possible to create I/O data on the basis of RAID technology. Specifically, it refers to whether or not a different data unit, which will become the basis for I/O data created based on a data unit stored in the migration-source real page, exists in a different real page inside the PDEV comprising the migration-destination real page. More specifically, for example, the control processing program 501 determines whether or not the number of the PDEV comprising the migration-destination real page is included in the real page identification information 252 corresponding to the virtual page that is the allocation destination of the migration-source real page, and determines whether the RAID is supported. In a case where the result of this determination is affirmative, S 1903 is carried out, and in a case where the result of this determination is negative, S 1904 is carried out.
- the control processing program 501 migrates the data unit from the migration-source real page to the migration-destination real page. Then, the control processing program 501 updates the identification information of the migration-source real page (the combination of the PDEV number and the real page number), which is included in the real page identification information 252 corresponding to the virtual page that was the allocation destination of the migration-source real page, to the identification information of the migration-destination real page. That is, the control processing program 501 allocates the migration-destination real page to the virtual page in place of the migration-source real page.
- the control processing program 501 selects the migration-destination real page from another PDEV.
- the other PDEV may be a PDEV of which the usage ratio is in a level next lower than the PDEV having the migration-destination selected in S 1901 , regardless of whether the rearrange process in FIG. 21 is in S 1804 or S 1805 . Further, in this rearrange process, the PDEV may be selected based on not the utilization rates of the PDEV but free capacity of the PDEV.
- a data unit is migrated as shown in FIG. 23 .
- PDEV # 01 and # 05 store four data units
- the remaining PDEV # 02 through # 04 and # 06 through # 09 store three data units.
- the added PDEV # 10 is assigned to the DP pool here, one data unit each is migrated from the PDEV # 01 and # 05 to the added PDEV # 10 .
- the utilization rate of the PDEV # 01 through # 10 becomes as uniform as possible.
- a PDEV may be selected based on not the utilization rates of the PDEV but free capacity of the PDEV.
- control processing program 501 always selects the real page, which is to be allocated to a write-destination virtual page, from the added PDEV in the processing of each write command. For example, in a case where PDEV # 51 through # 53 are in DP pool # 20 and PDEV # 54 is added to this DP pool # 20 , the control processing program 501 , in the processing of a write command, always does so such that the two or more real pages to be allocated to the write-destination virtual page comprise real pages inside the PDEV # 54 .
- the selection of a real page from the added PDEV in the processing of a write command is carried out until the utilization rate of the added PDEV is about the same as the utilization rates of the other PDEV inside the DP pool that includes the added PDEV (for example, either the same as the average of the utilization rates of the PDEV in the DP pool, or a difference with this average of less than a predetermined value). Thereafter, the plurality of PDEV in the DP pool are selected equably.
- the migration-destination real page is determined based on the access frequency of the migration-source real page (the access frequency specified from the real access status table 515 ). For example, in a case where the I/O performance of the PDEV comprising the migration-source real page is high compared to the access frequency of the migration-source real page, the migration-destination real page may be selected from the PDEV (for example, a HDD) with a lower I/O performance than the PDEV (for example, a SSD) that comprises the migration-source real page.
- the PDEV for example, a HDD
- the PDEV for example, a SSD
- the migration-destination real page may be selected from the PDEV (for example, a SSD) with a higher I/O performance than the PDEV (for example, a HDD) that comprises the migration-source real page.
- FIG. 24 is a flowchart of the processing carried out when a PDEV is removed.
- the control processing program 501 receives the input of the number of the PDEV targeted for removal from the user via the management apparatus 105 .
- the control processing program 501 references the real page management table 511 , and specifies one or more real pages, which are respectively allocated to one or more virtual pages from among the plurality of real pages of the PDEV targeted for removal, from the real page for which LUN 242 and the virtual page No. 243 are not “null”.
- the control processing program 501 supports the RAID, and, in addition, migrates data units so that the utilization rates are equable. Specifically, for example, the control processing program 501 selects one real page (the migration-destination real page) from the PDEV with the lowest utilization rate of the PDEV other than the PDEV targeted for removal for each real page (migration-source real page) that was specified in S 2702 , and migrates a data unit from the migration-source real page to the selected migration-destination real page. It is preferable that the I/O performance of the PDEV comprising the migration-source real page be the same as the I/O performance of the PDEV comprising the migration-destination real page.
- the control processing program 501 allocates to the virtual page, to which the migration-source real page is allocated, the migration-destination real page in place of the migration-source real page for each real page (migration-source real page) specified in S 2702 . Specifically, for example, the control processing program 501 updates the identification information of the migration-source real page to the migration-destination real page identification information in the real page identification information 252 corresponding to the virtual page to which the migration-source real page is allocated for each real page (migration-source real page) specified in S 2702 .
- the control processing program 501 deletes the information related to the PDEV targeted for removal from the table stored in the memory 123 . Specifically, for example, the control processing program 501 performs the following processing:
- a rearrange process is carried out based on the real access status table 515 and/or the virtual access status table 517 , which are updated in accordance with the processing of an I/O command.
- the following three examples (EX 1 ) through (EX 3 ) are conceivable with respect to the rearrange process.
- the control processing program 501 references the real access status table 515 and/or the virtual access status table 517 , and determines whether or not there is a PDEV (hereinafter, high-load PDEV) for which the access frequency or operating rate exceeds a first threshold.
- the PDEV access frequency for example, is the average access frequency of the real pages of the PDEV.
- the control processing program 501 migrates a data unit inside the high-load PDEV to a free real page inside the PDEV with the lowest operating rate.
- the real page storing the migration-targeted data unit is either the real page that is allocated to the virtual page with the highest access frequency or the real page with the highest access frequency.
- the data unit “J” inside the high-load PDEV # 05 is migrated to the PDEV # 10 with the lowest operating rate.
- the operating rate of the high-load PDEV # 05 is lowered from 40% to 20%, while the operating rate of the PDEV # 10 is raised from 10% to 20%.
- the migration of the data unit be carried out such that the operating rates of a plurality of PDEV become uniform.
- the control processing program 501 references the real access status table 515 and/or the virtual access status table 517 , and determines whether or not there is a PDEV (hereinafter, low-load PDEV) for which the access frequency or operating rate falls below a second threshold (the second threshold ⁇ the first threshold).
- a low-load PDEV the control processing program 501 migrates a data unit inside the PDEV with the highest operating rate to a free real page inside the low-load PDEV.
- the real page storing the migration-targeted data unit is either the real page that is allocated to the virtual page with the highest access frequency or the real page with the highest access frequency.
- the control processing program 501 references the real access status table 515 and/or the virtual access status table 517 , and computes the read ratio and the write ratio for either each real page or each virtual page.
- the read ratio and the write ratio are the ratios of reads and writes to overall accesses. Specifically, for example, the read ratio and the write ratio may be computed on the basis of the read frequency and the write frequency, and the total of the read frequency and the write frequency.
- the control processing program 501 migrates a data unit inside the real page comprising either the computed read ratio or the computed write ratio to a real page comprising an attribute (for example, I/O performance) that is suitable for either the computed read ratio or the computed write ratio.
- (X 1 ) Either the computed read ratio or the computed write ratio does not conform to the attribute (for example, the I/O performance) of the PDEV providing the real page that comprises either this read ratio or this write ratio.
- (X 2 ) Either the computed read ratio or the computed write ratio does not conform to the attribute (for example, the I/O performance) of the PDEV providing the real page that is allocated to the virtual page that comprises either this read ratio or this write ratio.
- (f 1 ) Data units are respectively migrated from the plurality of real pages inside the first DP pool, which are allocated to the virtual page, to the plurality of real pages inside the second DP pool; and (f 2 ) the plurality of migration-destination real pages are allocated to the virtual page in place of the plurality of migration-source real pages. Furthermore, the migration-destination real pages in (f 1 ) are selected from the PDEV that comprises the attribute (for example, I/O performance) that conforms to the access frequency of the virtual page (the access frequency specified from the virtual access status table 517 ).
- the migration-destination real page may be selected from the PDEV with high I/O performance, and alternatively, in a case where the access frequency of the virtual page is lower than the predetermined threshold, the migration-destination real page may be selected from the PDEV with low I/O performance.
- the real page of the DP pool is provided from the PDEV rather than from the RAID group.
- a data unit restore need only be carried out for the real page allocated to the virtual page from among the plurality of real pages of the PDEV in which the failure occurred (hereinafter, the failed PDEV).
- the expression “restore” means the restoration of data stored in the real page in the failed PDEV from the data elements or parity written in the real pages other than the real page in the failed PDEV from among real pages assigned with the same real page. Specifically, for example, in a case where only data element “b” is stored in the failed PDEV as shown in FIG.
- the control processing program 501 restores the data element “b” on the basis of the other data elements “a” and “c” and the parity “P”, and writes the restored data element “b” to a PDEV in which a failure has not occurred.
- the PDEV that becomes the write destination is selected such that the utilization rates (or free capacity) in the DP pool are equable.
- a priority-restore type restore process As processes related to a data unit restore inside a failed PDEV, there may be a priority-restore type restore process and a priority-replace type restore process.
- the priority-restore type restore process is as follows. That is, the control processing program 501 restores data units inside one or more real pages allocated to one or more virtual pages within the failed PDEV, and distributes the restored one or more data units to one or more other PDEV. After the failed PDEV has been replaced with another PDEV, the control processing program 501 migrates the data units from the plurality of other PDEV to the post-replace PDEV by carrying out a rearrange process such that the utilization rates of the plurality of PDEV are uniform.
- the priority-replace type restore process for example, is as follows. That is, the failed PDEV is replaced with another PDEV.
- the control processing program 501 restores data units inside one or more real pages allocated to one or more virtual pages within the failed PDEV, and writes the restored one or more data units to the post-replace PDEV.
- FIG. 27 is a flowchart of the priority-restore type restore process.
- the control processing program 501 references the mapping management table 513 and specifies an allocated real page within the failed PDEV.
- the control processing program 501 carries out (S 2502 ) through (S 2504 ) below for each allocated real page that is specified.
- the control processing program 501 reads a data unit from one or more other real pages allocated to the allocation-destination virtual page of the allocated real pages.
- the control processing program 501 restores the data unit to be stored in the real page inside the failed PDEV based on the one or more data units read in S 2502 .
- the control processing program 501 supports the RAID, and, in addition, selects a free real page as the write-destination for the data unit restored in S 2503 from another PDEV inside the DP pool such that the utilization rates of the PDEV inside this DP pool of the failed PDEV are uniform, and writes the restored data unit to the selected real page.
- the control processing program 501 allocates this write-destination real page to the virtual page in place of the real page inside the failed PDEV.
- the failed HDD is replaced with another HDD.
- the control processing program 501 migrates the data unit from the plurality of other PDEV inside the DP pool to the post-replace PDEV such that the utilization rates of the PDEV inside the DP pool are uniform.
- the control processing program 501 allocates the migration-destination real page inside the post-replace PDEV to the virtual page in place of the data unit migration-source real page.
- a migration-destination PDEV may be selected so that not the utilization rates of the PDEV but free capacity of the PDEV become uniform.
- control processing program 501 receives a read command from the host 101 , and, in addition, detects the fact that a real page inside the failed PDEV is included in the plurality of real pages that are allocated to this read-source virtual page, the control program 510 may carry out S 2503 and S 2504 on the basis of the one or more data units that were read in response to the read command. After S 2503 and S 2504 have been carried out for all the allocated real pages in the failed PDEV, S 2505 and S 2506 may be carried out.
- FIG. 28 is a flowchart of the priority-replace type restore process.
- the failed PDEV is replaced with another PDEV.
- the control processing program 501 references the real page management table 511 , and specifies an allocated real page within the failed PDEV.
- the control processing program 501 carries out (S 2603 ) through (S 2605 ) below for each allocated real page that is specified.
- the control processing program 501 reads a data unit from one or more other real pages allocated to the virtual page that is the allocation destination of the allocated real page.
- the control processing program 501 restores the data unit to be stored in the real page inside of the failed PDEV on the basis of the one or more data units read in S 2603 .
- the control processing program 501 writes the restored data unit to the post-replace PDEV.
- the decision to perform a priority-restore or a priority-replace may be made in accordance with the access status with respect to the storage system 103 , or may be preset by the user.
- each element may be identified by another type of identification information either instead of or in addition to a number and a name.
- the creation of a parity may be carried out by a hardware circuit instead of the control processing program 501 .
- at the least one of the access counter 244 for each real page and the access counter 253 for each virtual page may be at the least one of a write counter and a read counter instead of the access counter.
- the real access status table 515 may be eliminated. In this case, it is possible to regard the access frequency of the virtual page as the access frequency of the real page corresponding to the virtual page.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
- This is a continuation of U.S. Ser. No. 12/670,498, filed Jan. 25, 2010 which is a 371 National Stage of PCT/JP2010/000166, filed on Jan. 14, 2010. The entire disclosures of all these applications are hereby incorporated by reference.
- The present invention relates to storage control to which Dynamic Provisioning (also called Thin Provisioning) is applied.
- Storage systems configured from RAID (Redundant Array of Independent (or Inexpensive) Disks) are being used widely today to safely store corporate data. In addition, thin provisioning is being introduced to enhance the utilization efficiency of storage system capacity.
- In a RAID system, a RAID group is configured from a plurality of storage devices, and a logical volume is configured on the RAID group. In a storage system that does not use thin provisioning, the logical volume is provided to a host, and a storage area of a storage device having a capacity that is equivalent to the entire storage capacity of the logical volume is allocated to this logical volume from the time the logical volume is created. By contrast, in thin provisioning, in which a virtual volume is provided to the host, a storage area of the storage device corresponding to the initial virtual storage area is not allocated to the virtual storage area inside this virtual volume, but rather, the storage area of the storage device that is allocated to the logical volume is allocated only to the virtual storage area when there has been a write request with respect to the virtual storage area. For this reason, only a storage device of the capacity required for the time being is provided when the storage system is introduced, and a storage device can easily be added as the need arises.
- As a thin provisioning technique,
Patent Literature 1 proposes a method for rearranging data on a plurality of storage devices in logical volume units in a storage system that uses thin provisioning. - Further,
Patent Literature 2 proposes a method for adding storage devices in storage device units by changing the data redundancy configuration after adding a storage device. -
- Japanese Patent Application Laid-open No. 2008-234158
-
- Japanese Patent Application Laid-open No. 2009-230352
- In the above-mentioned
Patent Literature 1, a RAID group is configured from a plurality of storage devices prior to these storage devices being allocated to the virtual volume, and data management is carried out in logical volume units inside the RAID group. For example, data redundancy is maintained by storing the data that is inside the logical volume so as to extend across a plurality of storage devices of this RAID group. For this reason, storage device addition must be carried out by adding a plurality of storage devices during a single addition to achieve a group of storage devices in the numbers required to configure a RAID group. - In the above-mentioned
Patent Literature 2 method, the premise is that a RAID group is configured from a plurality of storage devices prior to these storage devices being allocated to the virtual volume the same as in the above-mentionedPatent Literature 1, and that a logical volume in this RAID group is provided to the virtual volume. In so doing, the RAID group must be reconfigured after adding a single storage device, giving rise to the need to reorganize the data redundancy configuration. - An object of the present invention is to provide a method for enhancing the flexibility and lowering the cost of storage device addition in a storage system that makes use of thin provisioning.
- An object of the present invention is to suppress the influence, which is exerted by the addition of storage devices, on business or other operations in a storage system to which the thin provisioning is employed.
- In addition, another object of the present invention is to provide a storage system that does not require a spare disk, i.e., a specific storage device provided in preparation for a storage device failure.
- In a storage system having a thin provisioning function, with a RAID group being configured from a plurality of storage devices, a storage area of a storage device is provided directly to a virtual volume instead of providing a logical volume inside the RAID group to the virtual volume. That is, the storage system, upon receiving a write request with respect to a virtual storage area, first of all, specifies the data redundancy configuration (a number of data partitions and a number of created parities) and the RAID level set to the virtual volume comprising this virtual storage area. The storage system selects a number of storage devices in accordance with the specified RAID level and redundancy configuration for this virtual storage area. The storage system selects, from among the selected storage devices, a storage area that is not allocated to a virtual storage area, and allocates this storage area to this virtual storage area. The storage system partitions the data and writes this data together with the parity to this allocated storage area.
- According to an embodiment of the present invention, it is possible to expand storage capacity while holding the cost of storage device addition to the minimum.
-
FIG. 1 shows the configuration of a storage system related to an embodiment of the present invention. -
FIG. 2 shows an overview of dynamic capacity expansion related to this embodiment. -
FIG. 3 shows an example of a first configuration of a DP pool. -
FIG. 4 shows an example of a second configuration of the DP pool. -
FIG. 5 shows a computer program and information stored in amemory 123. -
FIG. 6 shows the configuration of a DP common specification table 503. -
FIG. 7 shows the configuration of a pool management table 505. -
FIG. 8 shows the configuration of a virtual LU management table 507. -
FIG. 9 shows the configuration of a PDEV management table 509. -
FIG. 10 shows the configuration of a real page management table 511. -
FIG. 11 shows the configuration of a mapping management table 513. -
FIG. 12 shows the configuration of a real access status table 515. -
FIG. 13 shows the configuration of a virtual access status table 517. -
FIG. 14 is a flowchart showing the flow of processing for creating a DP pool. -
FIG. 15 is a flowchart showing the flow of processing for creating avirtual LU 201. -
FIG. 16 shows an overview of the flow of a write to a virtual page. -
FIG. 17 is a flowchart showing the flow of processing for writing to a virtual page. -
FIG. 18 is a flowchart showing the flow of processing for reading from a virtual page. -
FIG. 19 shows how data stored in accordance with a plurality of RAID types is mixed together in a single DP pool. -
FIG. 20 shows an example of the arrangement of data units (generic name for data elements and parity) in a case where a PDEV allocated to a single virtual page is selected without relying on the PDEV number. -
FIG. 21 is a flowchart for processing carried out when adding a PDEV. -
FIG. 22 shows a flowchart of a rearrange process. -
FIG. 23 shows an example of rearranging carried out in a case where a PDEV has been added. -
FIG. 24 is a flowchart of processing carried out when removing a PDEV. -
FIG. 25 shows an example of rearranging based on access frequency. -
FIG. 26 shows an overview of a restore process in the case of a failure in a PDEV. -
FIG. 27 is a flowchart of a priority-restore type restore process. -
FIG. 28 is a flowchart of a priority-replace type restore process. -
FIG. 1 is shows the configuration of a storage system related to an embodiment of the present invention. - One or more (for example, two) host apparatuses (hereinafter, host) 101 are coupled to the
storage system 103. Communication between thehost 101 and thestorage system 103, for example, is carried out using a protocol that conforms to either a Fibre Channel (FC) or an iSCSI (SCSI is the abbreviation for Small Computer System Interface). - The
storage system 103 manages a virtual LU (LU is the abbreviation for a Logical Unit). The virtual LU is a virtual LU that conforms to Dynamic Provisioning (also called Thin Provisioning) technology. The virtual LU comprises a plurality of virtual pages (virtual storage areas). One or more addresses (for example, a LBA (Logical Block Address) belong to each virtual page. - The
host 101 sends an I/O command for carrying out I/O (Input/Output) with respect to the virtual LU. The I/O command comprises I/O destination information. The I/O destination information includes the LUN (Logical Unit Number) of a virtual LU and the virtual page address of the virtual LU. Thehost 101 is generally a computer, but another storage system may be used instead of a computer. - The
storage system 103 comprises a plurality of physical storage devices (PDEV) 117, and a storage control apparatus (hereinafter referred to as “DKC”) 111 that is coupled to the plurality ofPDEV 117. TheDKC 111 is multiplexed (for example, duplexed). The plurality ofPDEV 117 may be configured from the same type of PDEV, or may be a mixture of a plurality of types of PDEV. As types of PDEV, for example, there are a HDD (Hard Disk Drive) and a SSD (Solid State Drive). - The
DKC 111 receives and processes an I/O command (either a write command or a read command) from thehost 101. The DKC, upon receiving a write command, for example, allocates a plurality of real pages (substantial storage areas) to each of one or more write-destination virtual pages specified in the received write command. For the sake of convenience, the allocation of a real page to a virtual page will be called “dynamic capacity expansion”. - A plurality of
PDEV 117 are coupled to theDKC 111 via one or morebackend coupling devices 113. Thebackend coupling devices 113 are multiplexed (for example, duplexed). As abackend coupling device 113, for example, a FC-AL (Fibre Channel-Arbitrated Loop) or a SAS (Serial Attached SCSI) expander may be used. Onebackend coupling device 113 is coupled to two or more PDEV 117 of a plurality ofPDEV 117. - A
management apparatus 105 is coupled to at the least oneDKC 111. Themanagement apparatus 105, for example, is coupled to the at the least oneDKC 111 via a LAN (Local Area Network). Themanagement apparatus 105, for example, may be a computer and may carry out various types of settings and references with respect to a LU. - Each
DKC 111 comprises a FE-IF (front-end interface) 121, a BE-IF (backend interface) 127, and a controller. The controller, for example, comprises atransfer circuit 129, amemory 123, and an MP (a microprocessor) 125. In addition to these elements, for example, the controller may comprise a hardware circuit for creating parity instead of theMP 125. - The FE-
IF 121, thememory 123, theMP 125, and theBE-IF 127 are coupled to thetransfer circuit 129. Thetransfer circuit 129 relays data and a command exchanged among theseelements - The FE-
IF 121 is an interface device that is coupled to thehost 101. The FE-IF 121 comprises a plurality of communication ports, and ahost computer 101 is coupled to each communication port. The FE-IF receives an I/O command from thehost 101 and transfers the received I/O command to theMP 125. - The
BE-IF 127 is an interface device that is coupled to thePDEV 117. TheBE-IF 127 comprises a plurality of communication ports, and aPDEV 117 is coupled to each communication port via thebackend coupling device 113. - The
memory 123 is configured from one or more memories. Thememory 123 stores information (for example, tables) and a computer program. Thememory 123 comprises a cache area. Data that is exchanged between thehost 101 and thePDEV 117 is temporarily stored in the cache area. - The
MP 125 processes an I/O command from thehost 101. Further, in response to a request from themanagement apparatus 105, theMP 125 also sets information (for example, LU-related information) in thememory 123 and provides the information (for example, LU-related information) stored in thememory 123 to themanagement apparatus 105. -
FIG. 2 shows an overview of dynamic capacity expansion related to this embodiment. - A
virtual LU 201 and a DP (Dynamic Provisioning)pool 203 are managed by theDKC 111. Thevirtual LU 201 is associated with oneDP pool 203, and a real page is allocated from theDP pool 203 to thisvirtual LU 201. - The
virtual LU 201 is configured from a plurality of virtual pages, and is recognized by thehost 101. A RAID level is defined for thevirtual LU 201. The combination, the RAID level and capacity of thevirtual LU 201, for example, can be set from the management apparatus 105 (refer toFIG. 1 ). The RAID level here is obtained by classifying the RAID technology in accordance with the function thereof, and is expressed asRAID 5 and RAID 6. For example, inRAID 5, data for which there is a write request is partitioned into n parts (n being a natural number), one parity data is created from this data, and this partitioned data and parity are distributively written to (n+1) storage devices. Further, in RAID 6, two types of parity data are created with respect to the partitioned n pieces of data, and this partitioned data and parity are distributively written to (n+2) storage devices. Further, a combination is a combination of the partition number n of the data elements arbitrarily determined for configuring the RAID, and the number m (m being an integer of greater than 0) of parity determined by the RAID level, and is expressed as nD+mP (or D). That is, the data written to a virtual page is partitioned into n data elements, m parity is created, and these partitioned data elements and corresponding parity are distributively written to (n+m) storage devices. - The
DP pool 203 is configured from either a portion or all of thePDEV 117 of the plurality ofPDEV 117 of thestorage system 103. TheDP pool 203 is configured from a large number of real pages. - As one characteristic feature of this embodiment, the
DP pool 203 may lack a RAID definition, and a real page may be provided from individual PDEV rather than from a RAID group configured from a plurality of PDEV. - Generally speaking, according to the Dynamic Provisioning (Thin Provisioning) technology, a real page allocated to a virtual LU is formed on the basis of a predefined RAID group storage space. Specifically, for example, the pool is configured from either one or a plurality of RAID groups. Each RAID group is configured from two or more PDEV on the basis of the RAID level. A real page is configured on the basis of one RAID group, and one real page extends over two or more PDEV that configure a RAID group. For this reason, in a write to a real page that is allocated to a virtual page, the write-targeted data is written on the basis of the RAID level and combination of the RAID group that provides this real page.
- By contrast, according to this embodiment, there is no RAID group definition, and the
individual PDEV 117 that configure theDP pool 203 are not RAID group-configuring members. A real page is provided from an individual PDEV. Further, the RAID level and combination are set for thevirtual LU 201. A number of real pages determined on the basis of the RAID level and combination of thisvirtual LU 201 is allocated from theDP pool 203 to a virtual page of thevirtual LU 201, and write-targeted data with respect to this virtual page is written to the allocated plurality of real pages. - According to RAID technology, two or more data elements (or write-targeted data and a replicate of this data) from among a plurality of data elements that configure the write-targeted data are not stored in a
single PDEV 117. For this reason, in this embodiment, two or more real pages from asingle PDEV 117 will not be allocated to the same virtual page. That is, real pages from respectivelydifferent PDEV 117 are allocated to a single virtual page. Therefore, it is preferable that a plurality of real pages be allocated to a virtual page so that the utilization rate of a plurality of PDEV belonging to a DP pool (the ratio of the total capacity of the allocated real pages with respect to the capacity of the PDEV) becomes uniform. For example, this is because a plurality of real pages are not able to be allocated from a single PDEV to a single virtual page even when there are lots of free real pages in this one PDEV, thereby making it likely that there will be an excess of free real pages as a result. Further, in this case, the plurality of real pages may be allocated to a virtual page so that not the utilization rate of the plurality of PDEV but free capacity of a plurality of PDEV becomes uniform. - There may be either one or a plurality of the
virtual LU 201 and theDP pool 203. According to the example ofFIG. 2 , it is supposed that there are twovirtual LU # 1 and #2 as thevirtual LU 201, there is oneDP pool # 1 as theDP pool 203, and thevirtual LU # 1 and #2 are associated with theDP pool # 1. It is also supposed that the RAID level and combination of thevirtual LU # 1 is RAID 5 (2D+1P), and that the RAID level and combination of thevirtual LU # 2 is RAID 6 (4D+2P). Further, it is supposed that theDP pool # 1 is configured from sixPDEV # 01 through #06. - RAID level and combination of virtual LU #1: According to RAID 5 (2D+1P), data that is targeted for a write to virtual page V11 of the
virtual LU # 1 is divided into two data elements by theDKC 111, and, in addition, one parity is created. For this reason, three real pages R11, R22, and R31 are allocated to the virtual page V11 from the threePDEV # 01 through #03. The corresponding relationship between the virtual page V11 and the real pages R11, R22, and R31 is managed by theDKC 111. Then, two data elements are respectively written to two of the three real pages R11, R22, and R31, and a parity is written to the remaining one real page. - RAID level and combination of virtual LU #2: According to RAID 6 (4D+2P), data that is targeted for a write to virtual page V21 of the
virtual LU # 2 is divided into four data elements, and, in addition, two parities are created. For this reason, six real pages R18, R23, R33, R41, R51, and R61 are allocated to the virtual page V21 from the sixPDEV # 01 through #06. The corresponding relationship between the virtual page V21 and the real pages R18, R23, R33, R41, R51, and R61 is managed by theDKC 111. Then, four data elements are respectively written to four of the six real pages R18, R23, R33, R41, R51, and R61, and two parities are written to the remaining two real page. - One
DP pool 203 may be configured by one or more physical storage devices (PDEV) having the same attributes, and may also be configured by PDEV having different attributes. Attributes, for example, may include the I/O performance or reliability (for example, the MTBF (Mean Time Between Failures)) of the PDEV. The I/O performance of the PDEV, for example, may include the type of the PDEV itself (for example, HDD, SSD, and so forth), the type of interface (for example, an FC (Fibre Channel), a SATA (Serial ATA), an SAS (Serial Attached SCSI), and the rotational speed of the disk. - As shown in
FIG. 3 , asingle DP pool 203 may be configured from PDEV (in the example ofFIG. 3 , HDD and SSD) having different types of attributes. In a case where a virtual page is configured from a plurality of real pages having different attributes, performance drops to the level of the PDEV with the lowest I/O performance when writing or reading data to or from the respective PDEV, making it preferable that each virtual page be configured from PDEV having the same attribute. However, respective virtual LU are assigned to one pool. For this reason, the mixing together of PDEV with different attributes inside the same pool makes it possible to allocate real pages inside PDEV with different attributes to each virtual page of the same virtual LU. Further, even in a case where real pages from PDEV having the same attribute are allocated to the same virtual page, it is possible to allocate real pages from PDEV having different attributes when the capacity of a PDEV of a certain attribute becomes insufficient. After adding a PDEV with this capacity storage attribute, it is possible to allocate to the added PDEV data elements allocated to PDEV with different attributes. ADP pool 203 of this type of configuration makes it possible to efficiently use disk capacity. - Further, as shown in
FIG. 4 , one or more prescribed types of attributes of a plurality of attributes may be configured in the same PDEV of oneDP pool 203. According to the example ofFIG. 4 , asingle DP pool 203 is configured from PDEV for which the PDEV performance, the PDEV interface and the type of the PDEV itself are all the same. According to aDP pool 203 of this type configuration, it is possible to anticipate a desired I/O performance with respect to accessing thevirtual LU 201. For example, in a case where it is preferable to improve the I/O performance of a virtual LU that is associated with the DP Pool #E, this virtual LU may be associated with a DP pool (for example, DP pool #D) configured from PDEV having improved I/O performance than the PDEV that configure the DP pool #E. Furthermore, in a case where the DP pool with which the virtual LU is associated is changed to a different DP pool, the data elements and parity are migrated from the pre-change DP pool to the post-change DP pool. - Information related to the
DP pool 203 and information denoting the corresponding relationship between a virtual page and a plurality of real pages is managed by theDKC 111. This information, for example, is stored in the memory 123 (for example, the cache area) of theDKC 111. -
FIG. 5 shows the computer program and information stored in thememory 123. - The
memory 123 stores acontrol processing program 501. Thecontrol processing program 501 is executed by theMP 125. In the following explanation, processing that is carried out by theprogram 501 is actually performed by theMP 125. - The
memory 123 stores a DP common specification table 503, a pool management table 505, a virtual LU management table 507, a PDEV management table 509, a real page management table 511, a mapping management table 513, a real access status table 515, and a virtual access status table 517. The real page management table 511 exists in eachDP pool 203. The information in at least one of these tables may be managed in a structure other than a table. -
FIG. 6 shows the configuration of the DP common specification table 503. - The DP common specification table 503 comprises common information related to the specification of the DP pool. Specifically, for example, the table 503 comprises an
item 200 and avalue 201 for each item related to the DP pool specification. Theitem 200 is the name of an item, and thevalue 201 is the value for this item. According to the example ofFIG. 6 , the maximum number of creatable pools (the maximum number ofDP pools 203 capable of being created in the storage system 103) is 50, and the real page capacity is 32 KB (kilobytes). - Furthermore, in this embodiment, the real page capacity is the same in all of the DP pools. For this reason, the capacity of a virtual page that configures the
virtual LU 201 is determined on the basis of the real page capacity, RAID level and combination. When the combination is nD+mP, it is possible to store in the virtual page data of n-times the capacity of the real page, and (n+m) times the capacity is allocated. For example, the capacity of a virtual page of a RAID 6 (4D+2P)virtual LU 201 is twice the capacity of a virtual page of a RAID 5 (2D+1P)virtual LU 201. As explained hereinabove, this is because the real page capacity is the same, and the number of real pages allocated to a virtual page of a RAID 6 (4D+2P)virtual LU 201 is twice the number of real pages that is allocated to a virtual page of a RAID 5 (2D+1P)virtual LU 201. Specifically, six real pages are allocated to the virtual page of the RAID 6 (4D+2P)virtual LU 201, and three real pages are allocated to the virtual page of the RAID 5 (2D+1P)virtual LU 201. -
FIG. 7 shows the configuration of the pool management table 505. - The pool management table 505 comprises information related to the
DP pool 203. Specifically, for example, the table 505 comprises apool number 210, aPDEV number list 211, areal allocation capacity 212, autilization capacity 213, and afirst PDEV number 214 for eachDP pool 203. - The
pool number 210 is the number of theDP pool 203. - The
PDEV number list 211 is a list of numbers of the PDEV that configure theDP pool 203. - The
real allocation capacity 212 is the total capacity of the PDEV that configure theDP pool 203. Therefore, in a case where a new PDEV is added to theDP pool 203, thereal allocation capacity 212 will increase in proportion to the capacity of the added PDEV. By contrast, in a case where a PDEV is removed from theDP pool 203, thereal allocation capacity 212 will decrease in proportion to the capacity of the removed PDEV. - The
utilization capacity 213 is the total capacity of a real page that has been allocated from theDP pool 203. - The
first PDEV number 214 is the number of the PDEV that is the next allocation source of a real page. For example, in this embodiment, the two or more PDEV that provide two or more real pages that are allocated to a virtual page are in PDEV number order. Specifically, for example, in a case where the DP pool is configured from the fourPDEV # 01 through #04, the RAID level and combination of the virtual LU is RAID 5 (2D+1P), and, in addition, an initial write is generated to an unallocated virtual page (a virtual page to which a real page is not allocated) in this virtual LU, thefirst PDEV number 214 is “1”, and therefore, respective real pages are allocated fromPDEV # 01, #02, and #03 to the write-destination virtual page. Thereafter, thefirst PDEV number 214 is updated to “4”. Then, in a case where a second write is generated to an unallocated virtual page in this virtual LU, since thefirst PDEV number 214 is “4”, respective real pages are allocated fromPDEV # 04, #01, and #02 to the write-destination virtual page. After this, thefirst PDEV number 214 is updated to “3”. In this embodiment, the two or more PDEV that are the provision sources of the two or more real pages allocated to the virtual page are determined based on the order of the PDEV numbers. Furthermore, the numbers of the plurality of PDEV configuring the DP pool may be consecutive or nonconsecutive. Further, the two or more PDEV that are the provision sources of the two or more real pages allocated to the virtual page may be arranged in descending order from the largest number rather than in ascending order from the smallest number. Since the two or more provision-source PDEV are determined in PDEV number order, the PDEV inside the DP pool are used equably. -
FIG. 8 shows the configuration of the virtual LU management table 507. - The virtual LU management table 507 comprises information related to the
virtual LU 201, and can be used to set and check the virtual LU. Specifically, for example, the table 507 comprises a LUN 220, an assigned pool number 221, aRAID level 222, acombination 223 and acapacity 224 for eachvirtual LU 201. - The LUN 220 is the LUN of the
virtual LU 201. - The assigned pool number 221 is the number of the DP pool (the DP pool with which the
virtual LU 201 is associated) 203 to which thevirtual LU 201 is assigned. - The
RAID level 222 is the RAID level of thevirtual LU 201. - The
combination 223 is the combination of thevirtual LU 201. - The
capacity 224 is the capacity of thevirtual LU 201. -
FIG. 9 shows the configuration of the PDEV management table 509 used to create, for example, a DP pool. - The PDEV management table 509 comprises information related to the
PDEV 117. Specifically, for example, the table 509 comprises aPDEV number 230, an assignedpool number 231, an IFtype 232, aPDEV performance 233, acapacity 234, afree capacity 235, and adevice type 236 for eachPDEV 117. - The
PDEV number 230 is the number of aPDEV 117. - The assigned
pool number 231 is the number of the DP pool (the DP pool in which thePDEV 117 is one of the components) to which thePDEV 117 is assigned. “null” shows that thePDEV 117 is not allocated to any pool. - The
IF type 232 is the type (for example, SAS, SATA, and so forth) of the interface of thePDEV 117. - The
PDEV performance 233 is expressed in accordance with thedevice type 236 of thePDEV 117. For example, in a case where thedevice type 236 of thePDEV 117 is HDD, thePDEV performance 233 is shown by the rotational speed of the disk (for example, 7,200 rpm). - The
capacity 234 is the capacity of thePDEV 117, that is, the total capacity of the real pages that thatPDEV 117 provides. - The
free capacity 235 is the total capacity of the real page that is not allocated to the virtual page from among the real pages that thePDEV 117 provides, but can be allocated to the virtual page. - The
device type 236 is the type of thePDEV 117 itself. Thedevice type 236, for example, includes HDD and SSD. -
FIG. 10 shows the configuration of the real page management table 511 showing the relationship between the virtual page and the real page. The real page management table 511 is used when allocating a real page to a virtual page. - The real page management table 511 exists for each DP pool. The real page management table 511 comprises information related to a real page inside the DP pool corresponding to this table 511. Specifically, for example, the table 511 comprises a
PDEV number 240, areal page number 241, aLUN 242, avirtual page number 243, and anaccess counter 244 for each real page. - The
PDEV number 240 is the number of thePDEV 117 that provides the real page. - The
real page number 241 is the number of the real page. - The
LUN 242 is the LUN of thevirtual LU 201 comprising the virtual page that is the allocation destination of the real page. - The
virtual page number 243 is the number of the allocation-destination virtual page of the real page. - The
access counter 244 is the number of times that an I/O (access) has been carried out to the real page. Furthermore, for example, in a case where there is a real access status table 515 as alternative information as in this embodiment, theaccess counter 244 may be eliminated. -
FIG. 11 shows the configuration of the mapping management table 513 used, for example, when judging whether the real page is allocated to the virtual page that has received a write command. - The mapping management table 513 shows the corresponding relationship between the virtual page and the real page. Specifically, for example, the table 513 comprises a
LUN 250, avirtual page number 251, a realpage identification information 252, and anaccess counter 253 for each virtual page. - The
LUN 250 is the LUN of thevirtual LU 201 of the virtual page. - The
virtual page number 251 is the number of the virtual page. - The real
page identification information 252 is information for identifying the real pages that are allocated to the virtual page, and, for example, is a combination of the real page numbers and the numbers of thePDEV 117 of these real pages. - The
access counter 253 is the number of times that an I/O (access) has been carried out with respect to the real page. Furthermore, for example, in a case where there is a virtual access status table 517 as alternative information as in this embodiment, theaccess counter 253 may be eliminated. -
FIG. 12 shows the configuration of the real access status table 515 referenced, for example, when determining a storage device for rearrange and the like. - The real access status table 515 comprises information, which is common to one or
more DP pools 203, and which shows the access state for each real page. Specifically, for example, the table 515 comprises a cell for each real page, and the cell comprises information showing the access frequency with respect to the real page (hereinafter, real access frequency information). The real access frequency information, for example, includes information showing the usage of the real page corresponding to this information, and shows the write frequency and the read frequency separately. The write frequency is the number of writes per unit period, specifically, for example, the number of writes per minute, hour, day and week. The read frequency is the number of reads per unit period, specifically, for example, the number of reads per minute, hour, day and week. - In a case where one PDEV 177 is added to any
DP pool 203, a record corresponding to this addedPDEV 117 is added to the table 515. Further, in a case where one PDEV 177 is removed from anyDP pool 203, the record corresponding to this removedPDEV 117 is removed from the table 515. -
FIG. 13 shows the configuration of the virtual access status table 517 referenced, for example, when determining the virtual page for rearrange and the like. - The virtual access status table 517 comprises information denoting the access status of each virtual page. Specifically, for example, the table 517 comprises a cell for each virtual page, and the cell comprises information showing the access frequency with respect to the virtual page (hereinafter, virtual access frequency information). The virtual access frequency information, for example, includes information showing the usage of the virtual page corresponding to this information, and shows both the write frequency and the read frequency. The write frequency is the number of writes per unit period, specifically, for example, the number of writes per minute, hour, day and week. The read frequency is the number of reads per unit period, specifically, for example, the number of reads per minute, hour, day and week.
- In a case where one
virtual LU 201 is associated with anyDP pool 203, a record corresponding to this associatedvirtual LU 201 is added to the table 517. Further, in a case where onevirtual LU 201 is disassociated from anyDP pool 203, the record corresponding to this disassociatedvirtual LU 201 is removed from the table 517. - Furthermore, instead of showing the write frequency and the read frequency separately, at the least one of the virtual access frequency information and the above-mentioned real access frequency information may show the access frequency without regard for a write or a read, may show only the write frequency, or may show only the read frequency.
- Further, at the least one of the virtual access frequency information and the above-mentioned real access frequency information may be updated each time an access occurs with respect to a virtual area or a real page, or may be updated on a predetermined schedule (for example, regularly). For example, the
control processing program 501 may manage a processed command log in thememory 123. The command log may comprise, for example, a type, a date/time (for example, date/time received, process completion date/time, or a date/time of an I/O command time stamp), the number of the access-destination virtual page, and the identification information of the access-destination real page (for example, a combination of the PDEV number and the real page number) for each processed I/O command. In a case where an I/O command has been processed, thecontrol processing program 501 may add to the command log the type, date/time, number of the access-destination virtual page, and identification information for the access-destination real page related to this I/O command. Thecontrol processing program 501 may update the virtual access frequency information and the real access frequency information based on this command log. -
FIG. 14 is a flowchart showing the flow of processing for creating a DP pool. - The
control processing program 501 receives a create pool instruction from themanagement apparatus 105. The parameter of the create pool instruction, for example, is at the least (p1) of the following (p1) through (p3). The (p1) is a parameter that must be input by the user, the (p2) and the (p3) are parameters that are input at the discretion of the user. - (p1) Capacity requested by the user (for example, the administrator) (user-requested pool capacity). The user-requested pool capacity differs from the
real allocation capacity 212. This is because thereal allocation capacity 212 is the total capacity of the PDEV that belong to a DP pool.
(p2)Information denoting PDEV 117 attribute (PDEV information). For example, the device type, IF type and PDEV performance.
(p3) The number assigned to a pool (pool number). - The
control processing program 501 determines whether or not the number ofDP pools 203 specified from the pool management table 505 is smaller that thevalue 201 of the maximum number of creatable pools. In a case where the result of this determination is affirmative, S12 is performed. In a case where the result of this determination is negative, thecontrol processing program 501 sends information denoting an error to themanagement apparatus 105, and themanagement apparatus 105 displays an error (S16). - The
control processing program 501 selectsPDEV 117 so that the total capacity is greater than the user-requested pool capacity. Specifically, for example, thecontrol processing program 501 carries out the following processing. - (S12-1): The
control processing program 501 references the PDEV management table 509, and specifies a PDEV (a PDEV that is not assigned to any DP pool 203) for which the assignedpool number 231 is “null”. The specified PDEV is a DP pool component candidate. Furthermore, in a case where the PDEV information is the parameter of the create pool instruction, the assignedpool number 231 is “null”, and, in addition, the PDEV that conforms to this PDEV information is specified as the DP pool component candidate. Hereinafter, the PDEV specified in this (S12-1) will be called the “candidate PDEV”.
(S12-2): Thecontrol processing program 501 selects from a plurality of candidate PDEV the PDEV having a total capacity greater than the user-requested pool capacity. As a result of this, the total capacity of the selected PDEV (the real allocation capacity of the DP pool) is either the same as or larger than the user-requested pool capacity. Furthermore, the RAID level and combination may also be taken into consideration in PDEV selection. For example, in the case where a user-requested pool capacity is 1 TB (one terabyte), thecontrol processing program 501 may select PDEV from a plurality of PDEV so that the total capacity of the selected PDEV is 1.3 times the user-requested pool capacity (1.3 TB). In a case where virtual LU having different RAID levels and combination are associated with the created DP pool, a PDEV may be selected on the basis of the parity ratio of the RAID level and combination having the highest parity ratio. Further, in a case where the RAID level and combination of the virtual LU associated with the DP pool has been decided, the PDEV may be selected on the basis of the parity ratio of this RAID level and combination. Furthermore, the “parity ratio” is the percentage of the total size of the created parity with respect to the size of the I/O-targeted data for the virtual page. - The
control processing program 501 adds information related to the created DP pool to the pool management table 505. The following information is added as information related to the created DP pool. -
- Pool number 210: A number determined in accordance with a prescribed rule (for example, the number after the largest number of the
pool number 210 registered in the pool management table 505). Or, the pool number denoted by the above-mentioned parameter (p3) of S10. - PDEV number list 211: The PDEV numbers selected in S12.
- Real allocation capacity 212: The total capacity of the PDEV selected in S12.
- Utilization capacity 213: Zero. This is because a real page is not allocated from the DP pool.
- First PDEV number 214: A number determined in accordance with a prescribed rule from among the PDEV numbers selected in S12. For example, the smallest number of the PDEV numbers selected in S12.
- Pool number 210: A number determined in accordance with a prescribed rule (for example, the number after the largest number of the
- The
control processing program 501 updates the assignedpool number 231 of the PDEV selected in S12 from “null” to the number of the created DP pool (the number in S13) in the PDEV management table 509. - The
control processing program 501 partitions the storage space of the respective PDEV (the respective PDEV selected in S12) configuring the created DP pool with respect to each real page capacity, and manages the respected real pages obtained in accordance with this. Specifically, thecontrol processing program 501 creates a real page management table 511 corresponding to the created DP pool, and stores this table 511 in thememory 123. The respective records of the created real page management table 511 comprise the following information. -
- PDEV number 240: Any PDEV number selected in S12.
- Real page number 241: Any serial number assigned to the real pages provided by the PDEV selected in S12.
- LUN 242: null. This is because none of the real pages has been allocated to any of the virtual pages.
- Virtual page number 243: null. This is because none of the real pages has been allocated to any of the virtual pages.
- Access counter 244: Zero. This is because an I/O is not generated for any real page.
-
FIG. 15 is a flowchart showing the flow of processing for creating avirtual LU 201. - The
control processing program 501 receives a virtual LU create instruction from themanagement apparatus 105. The virtual LU create instruction, for example, has as a parameter at the least (p15) of the following (p11) through (p15). The (p15) is a parameter that must be inputted by the user, and the (p11) through the (p14) are parameters that are inputted at the discretion of the user. A parameter from among the parameters (p11) through (p14) that is not inputted by the user is a value (a default value) determined in accordance with a prescribed rule. - (p11) LUN assigned to the virtual LU to be created. The default value, for example, is the smallest number of a group of numbers capable of being assigned as the LUN of the virtual LU.
(p12) The number of virtual LU to be created. The default value is “1”.
(p13) The DP pool number of the assigned destination of the virtual LU to be created (the pool number). The default value is the smallest number of the created DP pool numbers.
(p14) The RAID level (simultaneous setting of a combination). The default number, for example, is RAID 5 (2D+1P).
(p15) The capacity of the virtual LU. - The
control processing program 501 adds to the virtual LU management table 507 information related to the virtual LU to be created for each virtual LU in the number of virtual LU denoted in the above-mentioned (p12). The following information is added as information related to the virtual LU to be created. -
- LUN 220: LUN denoted by the above-mentioned parameter (p11).
- Assigned pool number 221: Pool number denoted by the above-mentioned parameter (p13). The number of virtual LU denoted by the above-mentioned (p12) is associated with the DP pool having the pool number denoted by the above-mentioned parameter (p13).
- RAID level 222: The RAID level denoted by the above-mentioned parameter (p14).
- Combination 223: The combination denoted by the above-mentioned parameter (p14).
- Capacity 224: The capacity denoted by the above-mentioned parameter (p15).
- The
control processing program 501 adds to the mapping management table 513 information related to the respective virtual pages of the virtual LU to be created for each virtual LU in the number of virtual LU denoted in the above-mentioned (p12). For example, the information added for the respective the virtual pages is as follows. -
- LUN 250: LUN of the virtual LU comprising the virtual page.
- Virtual page number 251: The number of the virtual page.
- Real page identification information 252: null. This is because a real page is not allocated to any of the virtual pages.
- Access counter 253: Zero. This is because an I/O is not generated for any virtual page.
-
FIG. 16 shows an overview of the flow of a write to a virtual page. The flow of a write to a virtual page for which a real page is not allocated, and the flow of a write to a virtual page for which a real page has been allocated will be explained below. - <Case 1: Write to a Virtual Page for which a Real Page is not Allocated>
- The following processing is carried out.
- (w11) The
control processing program 501 receives from the host #1 a write command specifying the virtual page V11 of thevirtual LU # 11, references the mapping management table 513, and specifies the fact that a real page is not allocated to the write-destination virtual page V11.
(w12) Thecontrol processing program 501 references the virtual LU management table 507, and specifies that the RAID level of thevirtual LU # 11 that comprises the virtual page V11 is RAID 5 (2D+1P).
(w13) Thecontrol processing program 501 references the real page management table 511 corresponding to theDP pool # 11 to which thevirtual LU # 11 is assigned, and selects the three real pages having different PDEV numbers 240 (the number of real pages based on the RAID level (RAID 5 (2D+1P)) specified in (w12)) from the real pages for which theLUN 242 and thevirtual page number 243 are null. The real page R11 insidePDEV # 11, the real page R21 insidePDEV # 12, and the real page R31 insidePDEV # 13 are selected here.
(w14) Thecontrol processing program 501 allocates the real pages R11, R21, and R31 selected in (w13) to the virtual page V11 and updates the mapping management table 513.
(w15) Thecontrol processing program 501 writes two data elements obtained by partitioning the write-targeted data for the virtual page V11 to two of the three real pages R11, R21, and R31, and writes the parity based on these two data elements to the one remaining real page of these three real pages.
<Case 2: Write to a Virtual Page for which a Real Page has Been Allocated> - The following processing is carried out.
- (w21) The
control processing program 501 receives from the host #2 a write command specifying the virtual page V22 of thevirtual LU # 12, references the mapping management table 513, and specifies the real page that has been allocated to the write-destination virtual page V22. Since the RAID level of thevirtual LU # 12 that comprises the virtual page V22 is RAID 6 (3D+2P), five real pages are allocated to the virtual page V22, and these five real pages are specified in (w21). Specifically, the real page R32 insidePDEV # 13, the real page R41 insidePDEV # 14, the real page R51 insidePDEV # 15, the real page R61 insidePDEV # 16, and the real page R71 insidePDEV # 17 are selected.
(w22) Thecontrol processing program 501 writes three data elements obtained by partitioning the write-targeted data for the virtual page V22 among three of the five real pages R32, R41, R51, R61, and R71 specified in (w21), and writes two parities based on these three data elements to the two remaining real pages of these five real pages. -
FIG. 17 is a flowchart showing the flow of processing for a write to a virtual page. - The
control processing program 501 receives a write command comprising write-destination information and write-targeted data. The write-destination information includes a LUN and a LBA. The write-targeted data is written to the memory 123 (the cache area). - The
control processing program 501 references the mapping management table 513, and determines whether or not a real page has been allocated to the virtual page (write-destination virtual page) specified from the write-destination information. - <A Case where the Result of the Determination of (S31) is Affirmative (a Case where the
PDEV Identification Information 252 For the Write-Destination Virtual Page is not Null)> - This corresponds to the above-mentioned
Case 2. Specifically, S37 is carried out without performing S32 through S36. That is, thecontrol processing program 501 writes the write-targeted data (for example, the two or more data elements and one or more parities obtained by partitioning this data) to the real pages (for example, the real pages R32, R41, R51, R61, and R71 ofFIG. 16 ) that have been allocated to the write-destination virtual page (for example, the virtual page V22 ofFIG. 16 ) based on the RAID level of the virtual LU (for example, thevirtual LU # 12 ofFIG. 16 ) comprising the write-destination virtual page. Then, thecontrol processing program 501 performs the following write status update process: -
- Increments by 1 the value of the
access counter 253 corresponding to the write-destination virtual page; - Updates the virtual access frequency information (the information inside the virtual access status table 517) corresponding to the write-destination virtual page (specifically, performs an update that increases the write frequency (one minute, one hour, one day, or one week));
- Increments by 1 the value of the
access counter 244 corresponding to the respective real pages allocated to the write-destination virtual page; and - Updates the real access frequency information (the information inside the real access status table 515) corresponding to the respective real pages allocated to the write-destination virtual page (specifically, performs an update that increases the write frequency (one minute, one hour, one day, or one week)). Furthermore, the real pages allocated to the write-destination virtual page are able to be specified from the mapping management table 513. The virtual LU comprising the write-destination virtual page (hereinafter, the write-destination virtual LU) is able to be identified from the LUN of the write-destination information. The RAID level of the write-destination virtual LU is able to be specified from the virtual LU management table 507.
<A Case where the Result of the Determination of (S31) is Negative (a Case where thePDEV Identification Information 252 For the Write-Destination Virtual Page is Null)>
- Increments by 1 the value of the
- This corresponds to the above-mentioned
Case 1. For this reason, S32 through S36 are carried out, and thereafter, S37 is performed. - The
control processing program 501 specifies theRAID level 222 and combination 223 (for example, RAID 5 (2D+1P) corresponding to the write-destination virtual LU (for example, thevirtual LU # 11 ofFIG. 16 ). Thecontrol processing program 501 determines the number K of selected PDEV based on the specifiedRAID level 222 andcombination 223. For example, in a case where the combination denoting the RAID level is “nD+mP (or D)” (where n is a natural number and m is an integer of 0 or more), the number K of selected PDEV is (n+m). Specifically, for example, in a case where the combination is “2D+1P”, K=2+1=3. - The
control processing program 501 specifies the assigned pool number 221 corresponding to the write-destination virtual LU and thefirst PDEV number 214 corresponding to thepool number 210 of the same value. - The
control processing program 501 selects the same number of PDEV numbers as the number K selected in S32 from the assigned pool number 221 corresponding to the write-destination virtual LU and thePDEV number list 211 corresponding to thepool number 210 of the same value. At this time, thecontrol processing program 501, first selects the same PDEV number (for example, #11) as thefirst PDEV number 214 selected in S33, makes this PDEV number the first PDEV number, and selects the PDEV numbers in order from the smallest PDEV number until the number of selected PDEV numbers reaches K. In a case where the selected PDEV number reaches the largest number of thePDEV number list 211 corresponding to the DP pool (for example,DP pool # 11 ofFIG. 16 ) to which the write-destination virtual LU is assigned prior to the number of selected PDEV numbers having reached the K, the smallest number of thisPDEV number list 211 is selected. - Furthermore, in this (S34), the PDEV numbers (PDEV) may be selected as follows instead of selecting the PDEV number (PDEV) in accordance with the PDEV number order.
-
- Preferentially select the PDEV with the most
free capacity 235. In accordance with this, the PDEV in the DP pool can be expected to be used equably. - Select PDEV for which either one or both of the
IF type 232 and thePDEV performance 233 are the same. In accordance with this, I/O performance with respect to the real pages allocated to one virtual page can be expected to be the same.
- Preferentially select the PDEV with the most
- The
control processing program 501 selects one free real page from the respective PDEV specified from the PDEV numbers selected in S34. Specifically, for example, thecontrol processing program 501 selects anyreal page number 241 that corresponds to aLUN 242 andvirtual page number 243 that are null for each selected PDEV number based on the real page management table 511 corresponding to the DP pool to which the write-destination virtual LU belongs. According to the example ofFIG. 16 , the real page number R11 is selected for thePDEV number # 11, the real page number R21 is selected for thePDEV number # 12, and the real page number R31 is selected for thePDEV number # 13. - The
control processing program 501 updates the realpage identification information 252 corresponding to the write-destination virtual page from null to a combination of thereal page numbers 241 in S35 and thePDEV numbers 240 corresponding to these numbers 241 (for example, #11-R11, #12-R21, #13-R31). That is, thecontrol processing program 501 allocates the two or more free real pages selected in S35 to the write-destination virtual page (for example, the virtual page V11 inFIG. 16 ). - The
control processing program 501, based on the RAID level and combination of the write-destination virtual LU, writes the write-targeted data to the two or more real pages allocated in S36. According to the example ofFIG. 16 , the two data elements and one parity are written to the three real pages R11, R21, and R31. Then, thecontrol processing program 501 performs the write status update process as described hereinabove. In accordance with this, the value of theaccess counter 253 corresponding to the write-destination virtual page, the virtual access frequency information corresponding to the write-destination virtual page, the value of theaccess counter 244 corresponding to the respective real pages allocated to the write-destination virtual page, and the real access frequency information corresponding to the respective real pages allocated to the write-destination virtual page are updated. -
FIG. 18 is a flowchart showing the flow of processing of a read with respect to a virtual page. - The
control processing program 501 receives a read command comprising read-source information. The read-source information, for example, comprises a LUN and a LBA. - The
control processing program 501 references the mapping management table 513, and determines whether or not a real page is allocated to the virtual page specified from the read-source information (the read-source virtual page). In a case where the result of the determination in S41 is negative, S42 is carried out. In a case where the result of the determination in S41 is affirmative, S43 is carried out. - The
control processing program 501 sends an error to the host that is the source of the read command. - The
control processing program 501 reads two or more data elements (and one or more parities) from the two or more real pages allocated to the read-source virtual page, and sends to the read command-source host the data configured by the two or more data elements that have been read (that is, the read-targeted data) (the read-targeted data may be data that includes a data element that was recovered using parity). Then, thecontrol processing program 501 performs the following read status update process: -
- Increments by 1 the value of the
access counter 253 corresponding to the read-source virtual page; - Updates the virtual access frequency information (the information inside the virtual access status table 517) corresponding to the read-source virtual page (specifically, performs an update that increases the read frequency (one minute, one hour, one day, or one week));
- Increments by 1 the value of the
access counter 244 corresponding to the respective real pages allocated to the read-source virtual page; and - Updates the real access frequency information (the information inside the real access status table 515) corresponding to the respective real pages allocated to the read-source virtual page (specifically, performs an update that increases the read frequency (one minute, one hour, one day, or one week)).
- Increments by 1 the value of the
- As explained hereinabove, in this embodiment, a real page that configures a
DP pool 203 is not a substantial storage area provided from a RAID group, but rather is a storage area that is provided from one PDEV. Then, it is possible to associate a plurality of virtual LU with different RAID level definitions in asingle DP pool 203. For this reason, as shown in the example ofFIG. 19 , in oneDP pool # 40, data respectively stored at different RAID levels, such as a set of data (data elements a, b and c) and parity P stored, for example, at RAID 5 (3D+1P) is mixed together with a set of data (data elements e, f, g, h and i) and parities P1 and P2 stored at RAID 6 (5D+2P). In accordance with this, in a case where a RAID level or a combination defined for a certain virtual LU is changed to a different RAID level or a combination, the reconstruction of the data may be carried out only for the real page(s) allocated to this virtual LU. For example, in a case where the RAID level (or combination) of the virtual LU changes from a first RAID level (or combination) to a second RAID level (or combination), thecontrol processing program 501 carries out, for each virtual page, the following processing—(k1) to (k3)—for each virtual LU to which a real page is allocated: - (k1) Reads v data units from v real pages allocated to the virtual page U, and constructs the data based on the v data units (a “data unit” as used in the explanation of this embodiment is a generic term for either a data element or a parity that is stored in the real page);
(k2) respectively selects from w PDEV of the DP pool to which the virtual LU is associated w real pages on the basis of the second RAID level and combination, and allocates the selected w real pages to the virtual page instead of the v real pages of (k1); and
(k3) writes the w data units based on the data constructed in (k1) to the allocated w real pages. Furthermore, in (k2), the w real pages may be selected so that the utilization rates of the respective PDEV in the DP pool associated with the virtual LU for which the RAID level (or combination) changed become uniform. In accordance with this, the utilization rates of the respective PDEV that belong to the DP pool with which this virtual LU is associated can be expected to be made uniform as a result of changing the RAID level (or combination) that is defined for the virtual LU. In this case, the w real pages may be selected so that not the utilization rates of the respective PDEV but the free capacity of the respective PDEV become uniform. - Further, in the above explanation, the source PDEV for the real page(s) allocated to the virtual page is selected in PDEV number order, but, for example, as shown in
FIG. 20 , the PDEV may be selected at random. According to the example ofFIG. 20 , the data elements a, b, and c and the parity P in conformance with the RAID 5 (3D+1P), and the data elements e, f, g, h, and i and the parities P1 and P2 in conformance with the RAID 6 (5D+2P) are stored in the real pages of the respective PDEV that were selected without relying on the PDEV number. Furthermore, instead of relying on the PDEV number, the PDEV utilization rate (or free capacity) may be taken into consideration. This is to make the utilization rates (or free capacity) of the plurality of PDEV in the DP pool equable. - Further, the
control processing program 501 may allocate a plurality of real pages to one virtual page from a plurality of PDEV that are coupled to the samebackend coupling device 113. In accordance with this, the access performance to these plurality of real pages can be expected to be made uniform. Furthermore, in a case where the utilization rates (or free capacity) of the plurality of PDEV that are coupled to the samebackend coupling device 113 are not equable, a plurality of real pages may be allocated to the virtual page from a plurality of PDEV coupled to a plurality ofbackend coupling devices 113. - Now then, in this embodiment, a rearrange process, by which a data unit is migrated from a real page inside a certain HDD to a real page inside a different PDEV, is performed.
- For example, this rearrange process is carried out when a PDEV is added or removed. Specifically, for example, in a case where the total capacity of the free real pages in the
DP pool 203 is insufficient (for example, a case in which the ratio of theutilization capacity 213 to thereal allocation capacity 212 of theDP pool 203 is not less than a predetermined threshold), the addition of a PDEV is carried out to increase the free real pages. The addition of a PDEV to theDP pool 203 may be carried out in PDEV units. Furthermore, it is possible for thecontrol processing program 501 to detect the fact that the total capacity of the free real pages is insufficient by referencing the pool management table 505. In a case where thecontrol processing program 501 detects that the total capacity of the free real pages is insufficient, thecontrol processing program 501 sends the management apparatus 105 a warning and the number of the DP pool in which the total capacity of the free real pages is insufficient. Themanagement apparatus 105 displays the warning and DP pool number. The user sees this warning and DP pool number, adds a PDEV, and makes the added PDEV a member of the DP pool identified from the DP pool number displayed together with the warning. Furthermore, the addition of a PDEV is not limited to when the total capacity of the free real pages is insufficient, and may also be carried out at other times (for example, in accordance with a prescribed plan). -
FIG. 21 is a flowchart of the processing carried out when adding a PDEV. - In a case where a PDEV has been added (for example, a case in which the
control processing program 501 has detected that a PDEV has been added), thecontrol processing program 501 adds information (a record) related to the added PDEV (hereinafter, the added PDEV) to the PDEV management table 509. Of the information related to the PDFV, theIF type 232, thePDEV performance 233, thecapacity 234, and thefree capacity 235 portions of the information related to the PDEV may be inputted by the user via themanagement apparatus 105, or may be acquired from the added PDEV. ThePDEV number 230 may be inputted by the user via themanagement apparatus 105, or may be determined by thecontrol processing program 501. Thecontrol processing program 501 adds the assignedpool number 231, that is, the number of the DP pool in which the added PDEV has become a member to the information related to the added PDEV in the PDEV management table 509. This assignedpool number 231 may be a number inputted by the user via themanagement apparatus 105, or may be a number that was determined by the control processing program 501 (for example, the number of the DP pool in which the shortage of free real page total capacity was detected). - The
control processing program 501 adds the number of the added PDEV to the PDEV number list 211 (hereinafter, referred to as the “target list 211” in the explanation ofFIG. 21 ) corresponding to the same pool number 210 (refer toFIG. 7 ) as the assignedpool number 231 added in S1801, and, in addition, adds the capacity of the added PDEV to thereal allocation capacity 212 corresponding to thispool number 210. Further, thecontrol processing program 501 adds the information (records) corresponding to the respective real pages of the added PDEV to the real page management table 511 that corresponds to the assignedpool number 231 added in S1801. - From among the
capacity 234 and thefree capacity 235 managed in the PDEV management table 509, thecontrol processing program 501 references thecapacity 234 and thefree capacity 235 corresponding to the PDEV that belongs to thetarget list 211, and computes the utilization rate (the ratio of thefree capacity 235 to the capacity 234) of the respective PDEV in thetarget list 211. Then, thecontrol processing program 501 specifies the PDEV for which the utilization rate exceeds a predetermined threshold. - The
control processing program 501 executes the rearrange process until the utilization rate of the PDEV specified in S1803 becomes less than the predetermined threshold. The “rearrange process” is the process for migrating a data unit from a real page inside a certain PDEV to a real page inside a different PDEV in a single DP pool. - The
control processing program 501 executes the rearrange process until the utilization rates of all the PDEV in the DP pool comprising the added PDEV become equable. In this process executed in a case where a PDEV is added, the PDEV which is the target of the rearrange process may be decided based on not the utilization rates of the PDEV but free capacity of the PDEV. -
FIG. 22 shows a flowchart of the rearrange process. Furthermore, in the following explanation, data created based on a plurality of data units (data for which an I/O is performed with respect to a virtual area) will be called “I/O data”. - The
control processing program 501 selects the data unit migration-source real page and the data unit migration-destination real page. In a case where this rearrange process is the rearrange process in S1804 ofFIG. 21 , the migration-source real page is selected from the PDEV specified in S1803 ofFIG. 21 (the PDEV for which the utilization rate exceeds the predetermined threshold), and the migration-destination real page is selected from the PDEV for which the utilization rate does not exceed the predetermined threshold (for example, the PDEV with the lowest utilization rate) even when the data unit is stored in at least one free real page. Alternatively, in a case where this rearrange process is the rearrange process in S1805 ofFIG. 21 , the migration-source real page is selected from the PDEV with the highest utilization rate, and the migration-destination real page is selected from the PDEV with the lowest utilization rate. - The
control processing program 501 determines whether or not the RAID is supported. “Whether or not the RAID is supported” refers to whether or not it is possible to create I/O data on the basis of RAID technology. Specifically, it refers to whether or not a different data unit, which will become the basis for I/O data created based on a data unit stored in the migration-source real page, exists in a different real page inside the PDEV comprising the migration-destination real page. More specifically, for example, thecontrol processing program 501 determines whether or not the number of the PDEV comprising the migration-destination real page is included in the realpage identification information 252 corresponding to the virtual page that is the allocation destination of the migration-source real page, and determines whether the RAID is supported. In a case where the result of this determination is affirmative, S1903 is carried out, and in a case where the result of this determination is negative, S1904 is carried out. - The
control processing program 501 migrates the data unit from the migration-source real page to the migration-destination real page. Then, thecontrol processing program 501 updates the identification information of the migration-source real page (the combination of the PDEV number and the real page number), which is included in the realpage identification information 252 corresponding to the virtual page that was the allocation destination of the migration-source real page, to the identification information of the migration-destination real page. That is, thecontrol processing program 501 allocates the migration-destination real page to the virtual page in place of the migration-source real page. - The
control processing program 501 selects the migration-destination real page from another PDEV. For example, the other PDEV may be a PDEV of which the usage ratio is in a level next lower than the PDEV having the migration-destination selected in S1901, regardless of whether the rearrange process inFIG. 21 is in S1804 or S1805. Further, in this rearrange process, the PDEV may be selected based on not the utilization rates of the PDEV but free capacity of the PDEV. - The preceding is an explanation of the processing carried out when a PDEV is added. According to S1805 of
FIG. 21 , for example, a data unit is migrated as shown inFIG. 23 . For example, it is supposed that in thePDEV # 01 through #09 that belong to a single DP pool,PDEV # 01 and #05 store four data units, and the remainingPDEV # 02 through #04 and #06 through #09 store three data units. In a case where the addedPDEV # 10 is assigned to the DP pool here, one data unit each is migrated from thePDEV # 01 and #05 to the addedPDEV # 10. In accordance with this, the utilization rate of thePDEV # 01 through #10 becomes as uniform as possible. - The process related to adding a PDEV is not limited to the explanation given above, and variations such as those described below are conceivable. In this process, a PDEV may be selected based on not the utilization rates of the PDEV but free capacity of the PDEV.
- S1803 through S1805 are not carried out. In this variation, the
control processing program 501 always selects the real page, which is to be allocated to a write-destination virtual page, from the added PDEV in the processing of each write command. For example, in a case where PDEV #51 through #53 are inDP pool # 20 and PDEV #54 is added to thisDP pool # 20, thecontrol processing program 501, in the processing of a write command, always does so such that the two or more real pages to be allocated to the write-destination virtual page comprise real pages inside the PDEV #54. Furthermore, at all times the selection of a real page from the added PDEV in the processing of a write command is carried out until the utilization rate of the added PDEV is about the same as the utilization rates of the other PDEV inside the DP pool that includes the added PDEV (for example, either the same as the average of the utilization rates of the PDEV in the DP pool, or a difference with this average of less than a predetermined value). Thereafter, the plurality of PDEV in the DP pool are selected equably. - The migration-destination real page is determined based on the access frequency of the migration-source real page (the access frequency specified from the real access status table 515). For example, in a case where the I/O performance of the PDEV comprising the migration-source real page is high compared to the access frequency of the migration-source real page, the migration-destination real page may be selected from the PDEV (for example, a HDD) with a lower I/O performance than the PDEV (for example, a SSD) that comprises the migration-source real page. By contrast, for example, in a case where the I/O performance of the PDEV comprising the migration-source real page is low compared to the access frequency of the migration-source real page, the migration-destination real page may be selected from the PDEV (for example, a SSD) with a higher I/O performance than the PDEV (for example, a HDD) that comprises the migration-source real page.
-
FIG. 24 is a flowchart of the processing carried out when a PDEV is removed. - The
control processing program 501 receives the input of the number of the PDEV targeted for removal from the user via themanagement apparatus 105. - The
control processing program 501 references the real page management table 511, and specifies one or more real pages, which are respectively allocated to one or more virtual pages from among the plurality of real pages of the PDEV targeted for removal, from the real page for whichLUN 242 and the virtual page No. 243 are not “null”. - The
control processing program 501 supports the RAID, and, in addition, migrates data units so that the utilization rates are equable. Specifically, for example, thecontrol processing program 501 selects one real page (the migration-destination real page) from the PDEV with the lowest utilization rate of the PDEV other than the PDEV targeted for removal for each real page (migration-source real page) that was specified in S2702, and migrates a data unit from the migration-source real page to the selected migration-destination real page. It is preferable that the I/O performance of the PDEV comprising the migration-source real page be the same as the I/O performance of the PDEV comprising the migration-destination real page. - The
control processing program 501 allocates to the virtual page, to which the migration-source real page is allocated, the migration-destination real page in place of the migration-source real page for each real page (migration-source real page) specified in S2702. Specifically, for example, thecontrol processing program 501 updates the identification information of the migration-source real page to the migration-destination real page identification information in the realpage identification information 252 corresponding to the virtual page to which the migration-source real page is allocated for each real page (migration-source real page) specified in S2702. - The
control processing program 501 deletes the information related to the PDEV targeted for removal from the table stored in thememory 123. Specifically, for example, thecontrol processing program 501 performs the following processing: -
- Deletes the record corresponding to the PDEV targeted for removal from the PDEV management table 509;
- Deletes the number of the PDEV targeted for removal from the
PDEV number list 211 corresponding to the DP pool of the remove-targeted PDEV in the pool management table 505, and, in addition, subtracts the capacity of the remove-targeted PDEV from thereal allocation capacity 212; and in a case where the remove-targeted PDEV number is thefirst PDEV number 214, the value for thefirst PDEV number 214 is updated to a value for the PDEV number in the level next larger than the remove-targeted PDEV number. - deletes the information (records) corresponding to the respective real pages of the remove-targeted PDEV from the real page management table 511 that corresponds to the DP pool of the remove-targeted PDEV. In this process executed in a case where a PDEV is removed, the PDEV may be selected based on not the utilization rates of the PDEV but free capacity of the PDEV.
- Now then, there are times when the access frequency of the specified real page becomes high and the operating rate (load) of the PDEV comprising this real page increases in accordance with the number and types of I/O commands and the I/O destination information received per unit of time by the
storage system 103 while thestorage system 103 is in operation. - Accordingly, in this embodiment, a rearrange process is carried out based on the real access status table 515 and/or the virtual access status table 517, which are updated in accordance with the processing of an I/O command. For example, the following three examples (EX1) through (EX3) are conceivable with respect to the rearrange process.
- (EX1) The
control processing program 501 references the real access status table 515 and/or the virtual access status table 517, and determines whether or not there is a PDEV (hereinafter, high-load PDEV) for which the access frequency or operating rate exceeds a first threshold. The PDEV access frequency, for example, is the average access frequency of the real pages of the PDEV. In a case where a high-load PDEV is specified, thecontrol processing program 501 migrates a data unit inside the high-load PDEV to a free real page inside the PDEV with the lowest operating rate. The real page storing the migration-targeted data unit is either the real page that is allocated to the virtual page with the highest access frequency or the real page with the highest access frequency. According to the example ofFIG. 25 , the data unit “J” inside the high-load PDEV # 05 is migrated to thePDEV # 10 with the lowest operating rate. As a result of this, the operating rate of the high-load PDEV # 05 is lowered from 40% to 20%, while the operating rate of thePDEV # 10 is raised from 10% to 20%. It is preferable that the migration of the data unit be carried out such that the operating rates of a plurality of PDEV become uniform.
(EX2) Thecontrol processing program 501 references the real access status table 515 and/or the virtual access status table 517, and determines whether or not there is a PDEV (hereinafter, low-load PDEV) for which the access frequency or operating rate falls below a second threshold (the second threshold<the first threshold). In a case where a low-load PDEV is specified, thecontrol processing program 501 migrates a data unit inside the PDEV with the highest operating rate to a free real page inside the low-load PDEV. The real page storing the migration-targeted data unit is either the real page that is allocated to the virtual page with the highest access frequency or the real page with the highest access frequency.
(EX3) Thecontrol processing program 501 references the real access status table 515 and/or the virtual access status table 517, and computes the read ratio and the write ratio for either each real page or each virtual page. The read ratio and the write ratio are the ratios of reads and writes to overall accesses. Specifically, for example, the read ratio and the write ratio may be computed on the basis of the read frequency and the write frequency, and the total of the read frequency and the write frequency. Thecontrol processing program 501, in a case where the condition of either (X1) or (X2) below is met, migrates a data unit inside the real page comprising either the computed read ratio or the computed write ratio to a real page comprising an attribute (for example, I/O performance) that is suitable for either the computed read ratio or the computed write ratio.
(X1) Either the computed read ratio or the computed write ratio does not conform to the attribute (for example, the I/O performance) of the PDEV providing the real page that comprises either this read ratio or this write ratio.
(X2) Either the computed read ratio or the computed write ratio does not conform to the attribute (for example, the I/O performance) of the PDEV providing the real page that is allocated to the virtual page that comprises either this read ratio or this write ratio. - There are times when the assigned destination of the virtual LU is switched from a first DP pool to a second DP pool. In accordance with this, the following (f1) and (f2) processes are carried out for each virtual page to which a real page inside the first DP pool is allocated:
- (f1) Data units are respectively migrated from the plurality of real pages inside the first DP pool, which are allocated to the virtual page, to the plurality of real pages inside the second DP pool; and
(f2) the plurality of migration-destination real pages are allocated to the virtual page in place of the plurality of migration-source real pages. Furthermore, the migration-destination real pages in (f1) are selected from the PDEV that comprises the attribute (for example, I/O performance) that conforms to the access frequency of the virtual page (the access frequency specified from the virtual access status table 517). For example, in a case where the access frequency of the virtual page is greater than a predetermined threshold, the migration-destination real page may be selected from the PDEV with high I/O performance, and alternatively, in a case where the access frequency of the virtual page is lower than the predetermined threshold, the migration-destination real page may be selected from the PDEV with low I/O performance. - As described hereinabove, in this embodiment, the real page of the DP pool is provided from the PDEV rather than from the RAID group. For this reason, in a case where a failure occurs in the PDEV, a data unit restore need only be carried out for the real page allocated to the virtual page from among the plurality of real pages of the PDEV in which the failure occurred (hereinafter, the failed PDEV). Here, the expression “restore” means the restoration of data stored in the real page in the failed PDEV from the data elements or parity written in the real pages other than the real page in the failed PDEV from among real pages assigned with the same real page. Specifically, for example, in a case where only data element “b” is stored in the failed PDEV as shown in
FIG. 26 , thecontrol processing program 501 restores the data element “b” on the basis of the other data elements “a” and “c” and the parity “P”, and writes the restored data element “b” to a PDEV in which a failure has not occurred. The PDEV that becomes the write destination is selected such that the utilization rates (or free capacity) in the DP pool are equable. - As processes related to a data unit restore inside a failed PDEV, there may be a priority-restore type restore process and a priority-replace type restore process.
- The priority-restore type restore process, for example, is as follows. That is, the
control processing program 501 restores data units inside one or more real pages allocated to one or more virtual pages within the failed PDEV, and distributes the restored one or more data units to one or more other PDEV. After the failed PDEV has been replaced with another PDEV, thecontrol processing program 501 migrates the data units from the plurality of other PDEV to the post-replace PDEV by carrying out a rearrange process such that the utilization rates of the plurality of PDEV are uniform. - The priority-replace type restore process, for example, is as follows. That is, the failed PDEV is replaced with another PDEV. The
control processing program 501 restores data units inside one or more real pages allocated to one or more virtual pages within the failed PDEV, and writes the restored one or more data units to the post-replace PDEV. - These respective restore processes will be explained in detail below.
-
FIG. 27 is a flowchart of the priority-restore type restore process. - The
control processing program 501 references the mapping management table 513 and specifies an allocated real page within the failed PDEV. Thecontrol processing program 501 carries out (S2502) through (S2504) below for each allocated real page that is specified. - The
control processing program 501 reads a data unit from one or more other real pages allocated to the allocation-destination virtual page of the allocated real pages. - The
control processing program 501 restores the data unit to be stored in the real page inside the failed PDEV based on the one or more data units read in S2502. - The
control processing program 501 supports the RAID, and, in addition, selects a free real page as the write-destination for the data unit restored in S2503 from another PDEV inside the DP pool such that the utilization rates of the PDEV inside this DP pool of the failed PDEV are uniform, and writes the restored data unit to the selected real page. Thecontrol processing program 501 allocates this write-destination real page to the virtual page in place of the real page inside the failed PDEV. - After carrying out S2502 through S2504 for all of the allocated real pages in the failed PDEV, the failed HDD is replaced with another HDD.
- The
control processing program 501 migrates the data unit from the plurality of other PDEV inside the DP pool to the post-replace PDEV such that the utilization rates of the PDEV inside the DP pool are uniform. Thecontrol processing program 501 allocates the migration-destination real page inside the post-replace PDEV to the virtual page in place of the data unit migration-source real page. In the priority-replace type restore process, a migration-destination PDEV may be selected so that not the utilization rates of the PDEV but free capacity of the PDEV become uniform. - The following is conceivable as a variation of this restore process. For example, in a case where the
control processing program 501 receives a read command from thehost 101, and, in addition, detects the fact that a real page inside the failed PDEV is included in the plurality of real pages that are allocated to this read-source virtual page, the control program 510 may carry out S2503 and S2504 on the basis of the one or more data units that were read in response to the read command. After S2503 and S2504 have been carried out for all the allocated real pages in the failed PDEV, S2505 and S2506 may be carried out. -
FIG. 28 is a flowchart of the priority-replace type restore process. - The failed PDEV is replaced with another PDEV.
- The
control processing program 501 references the real page management table 511, and specifies an allocated real page within the failed PDEV. Thecontrol processing program 501 carries out (S2603) through (S2605) below for each allocated real page that is specified. - The
control processing program 501 reads a data unit from one or more other real pages allocated to the virtual page that is the allocation destination of the allocated real page. - The
control processing program 501 restores the data unit to be stored in the real page inside of the failed PDEV on the basis of the one or more data units read in S2603. - The
control processing program 501 writes the restored data unit to the post-replace PDEV. - The decision to perform a priority-restore or a priority-replace may be made in accordance with the access status with respect to the
storage system 103, or may be preset by the user. - The preceding has been an explanation of one embodiment of the present invention, but, needless to say, the present invention is not limited to this embodiment, and a variety of changes are possible without departing from the scope thereof. For example, each element may be identified by another type of identification information either instead of or in addition to a number and a name. Further, for example, the creation of a parity may be carried out by a hardware circuit instead of the
control processing program 501. Further, for example, at the least one of theaccess counter 244 for each real page and theaccess counter 253 for each virtual page may be at the least one of a write counter and a read counter instead of the access counter. Further, for example, the real access status table 515 may be eliminated. In this case, it is possible to regard the access frequency of the virtual page as the access frequency of the real page corresponding to the virtual page. -
- 103 Storage system
Claims (14)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/897,672 US8650360B2 (en) | 2010-01-14 | 2013-05-20 | Storage system |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2010/000166 WO2011086598A1 (en) | 2010-01-14 | 2010-01-14 | Storage system |
US12/670,498 US8468302B2 (en) | 2010-01-14 | 2010-01-14 | Storage system |
US13/897,672 US8650360B2 (en) | 2010-01-14 | 2013-05-20 | Storage system |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2010/000166 Continuation WO2011086598A1 (en) | 2010-01-14 | 2010-01-14 | Storage system |
US12/670,498 Continuation US8468302B2 (en) | 2010-01-14 | 2010-01-14 | Storage system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20130254482A1 true US20130254482A1 (en) | 2013-09-26 |
US8650360B2 US8650360B2 (en) | 2014-02-11 |
Family
ID=42562589
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/670,498 Active 2030-11-15 US8468302B2 (en) | 2010-01-14 | 2010-01-14 | Storage system |
US13/897,672 Expired - Fee Related US8650360B2 (en) | 2010-01-14 | 2013-05-20 | Storage system |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/670,498 Active 2030-11-15 US8468302B2 (en) | 2010-01-14 | 2010-01-14 | Storage system |
Country Status (3)
Country | Link |
---|---|
US (2) | US8468302B2 (en) |
JP (1) | JP5444464B2 (en) |
WO (1) | WO2011086598A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190004968A1 (en) * | 2017-06-30 | 2019-01-03 | EMC IP Holding Company LLC | Cache management method, storage system and computer program product |
US10255176B1 (en) * | 2015-12-02 | 2019-04-09 | Pure Storage, Inc. | Input/output (‘I/O’) in a storage system that includes multiple types of storage devices |
Families Citing this family (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9134922B2 (en) | 2009-03-12 | 2015-09-15 | Vmware, Inc. | System and method for allocating datastores for virtual machines |
WO2011108027A1 (en) * | 2010-03-04 | 2011-09-09 | 株式会社日立製作所 | Computer system and control method therefor |
TWI417721B (en) * | 2010-11-26 | 2013-12-01 | Etron Technology Inc | Method of decaying hot data |
US8775773B2 (en) * | 2011-08-26 | 2014-07-08 | Vmware, Inc. | Object storage system |
US8595460B2 (en) | 2011-08-26 | 2013-11-26 | Vmware, Inc. | Configuring object storage system for input/output operations |
US8775774B2 (en) | 2011-08-26 | 2014-07-08 | Vmware, Inc. | Management system and methods for object storage system |
US8677085B2 (en) | 2011-08-29 | 2014-03-18 | Vmware, Inc. | Virtual machine snapshotting in object storage system |
US8769174B2 (en) | 2011-08-29 | 2014-07-01 | Vmware, Inc. | Method of balancing workloads in object storage system |
US8719620B2 (en) | 2011-08-29 | 2014-05-06 | Hitachi, Ltd. | Storage control apparatus and storage control apparatus control method |
US8856440B2 (en) * | 2011-09-12 | 2014-10-07 | Microsoft Corporation | Volatile memory representation of nonvolatile storage device set |
WO2013061368A1 (en) * | 2011-10-26 | 2013-05-02 | Hitachi, Ltd. | Storage apparatus and method of controlling storage apparatus |
US9229637B2 (en) * | 2011-11-18 | 2016-01-05 | Hitachi, Ltd. | Volume copy management method on thin provisioning pool of storage subsystem |
JP2013114624A (en) * | 2011-11-30 | 2013-06-10 | Hitachi Ltd | Storage system and control method for pool capacity reduction |
US20140136643A1 (en) * | 2012-11-13 | 2014-05-15 | Motorola Mobility Llc | Dynamic Buffer Management for a Multimedia Content Delivery System |
US8464095B1 (en) * | 2012-11-15 | 2013-06-11 | DSSD, Inc. | Method and system for multi-dimensional raid reconstruction and defect avoidance |
US20150286531A1 (en) * | 2012-12-20 | 2015-10-08 | Hewlett-Packard Development Company, L.P. | Raid storage processing |
DE112013006504B4 (en) * | 2013-01-25 | 2022-06-15 | Hitachi, Ltd. | Storage system and data management method |
JP5713412B2 (en) * | 2013-01-31 | 2015-05-07 | 株式会社日立製作所 | Management device, management system, and management method |
JP5907100B2 (en) * | 2013-03-21 | 2016-04-20 | 日本電気株式会社 | Disk control device, disk reduction method, storage system, and computer program |
US9378075B2 (en) * | 2013-05-15 | 2016-06-28 | Amazon Technologies, Inc. | Reducing interference through controlled data access |
CN104937561B (en) * | 2013-05-17 | 2018-01-02 | 株式会社日立制作所 | Storage device |
US11150807B2 (en) | 2015-02-23 | 2021-10-19 | Avago Technologies International Sales Pte. Limited | Dynamic storage system configuration |
JP6458565B2 (en) * | 2015-03-11 | 2019-01-30 | 日本電気株式会社 | Provisioning setting support apparatus, provisioning setting support method, program and system thereof |
US9811280B2 (en) * | 2015-12-23 | 2017-11-07 | Netapp, Inc. | Efficient method of combining parity groups for uniform load distribution and maximizing parallelization in parity de-clustered and sliced disk raid architecture |
KR102581479B1 (en) * | 2016-05-31 | 2023-09-21 | 삼성전자주식회사 | Storage System and Volume Management Method thereof |
US10459663B2 (en) * | 2017-01-23 | 2019-10-29 | International Business Machines Corporation | Thin provisioning of raid storage |
EP3695319B1 (en) * | 2017-09-27 | 2022-11-23 | Telefonaktiebolaget LM Ericsson (publ) | Method and reallocation component for managing reallocation of information from source to target memory sled |
CN110858124B (en) * | 2018-08-24 | 2021-06-01 | 华为技术有限公司 | Data migration method and device |
KR20200043814A (en) * | 2018-10-18 | 2020-04-28 | 에스케이하이닉스 주식회사 | Memory system and operating method thereof |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7548975B2 (en) * | 2002-01-09 | 2009-06-16 | Cisco Technology, Inc. | Methods and apparatus for implementing virtualization of storage within a storage area network through a virtual enclosure |
JP4236590B2 (en) * | 2002-03-13 | 2009-03-11 | 富士通株式会社 | RAID device control device |
EP1668486A2 (en) | 2003-08-14 | 2006-06-14 | Compellent Technologies | Virtual disk drive system and method |
JP2008146574A (en) * | 2006-12-13 | 2008-06-26 | Hitachi Ltd | Storage controller and storage control method |
JP5379956B2 (en) | 2007-03-19 | 2013-12-25 | 株式会社日立製作所 | Storage device and storage area arrangement method |
US7958303B2 (en) * | 2007-04-27 | 2011-06-07 | Gary Stephen Shuster | Flexible data storage system |
JP5075699B2 (en) | 2008-03-21 | 2012-11-21 | 株式会社日立製作所 | Storage capacity expansion method and storage system using the method |
US8099623B1 (en) | 2008-10-08 | 2012-01-17 | Netapp, Inc. | Efficient distributed hot sparing scheme in a parity declustered RAID organization |
US7904749B2 (en) * | 2008-10-24 | 2011-03-08 | Hitachi, Ltd. | Fast data recovery from HDD failure |
JP5214502B2 (en) * | 2009-03-12 | 2013-06-19 | 株式会社日立製作所 | Computer and method for managing storage device |
-
2010
- 2010-01-14 US US12/670,498 patent/US8468302B2/en active Active
- 2010-01-14 JP JP2012517600A patent/JP5444464B2/en not_active Expired - Fee Related
- 2010-01-14 WO PCT/JP2010/000166 patent/WO2011086598A1/en active Application Filing
-
2013
- 2013-05-20 US US13/897,672 patent/US8650360B2/en not_active Expired - Fee Related
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10255176B1 (en) * | 2015-12-02 | 2019-04-09 | Pure Storage, Inc. | Input/output (‘I/O’) in a storage system that includes multiple types of storage devices |
US10970202B1 (en) | 2015-12-02 | 2021-04-06 | Pure Storage, Inc. | Managing input/output (‘I/O’) requests in a storage system that includes multiple types of storage devices |
US20190004968A1 (en) * | 2017-06-30 | 2019-01-03 | EMC IP Holding Company LLC | Cache management method, storage system and computer program product |
US11093410B2 (en) * | 2017-06-30 | 2021-08-17 | EMC IP Holding Company LLC | Cache management method, storage system and computer program product |
Also Published As
Publication number | Publication date |
---|---|
WO2011086598A1 (en) | 2011-07-21 |
US8468302B2 (en) | 2013-06-18 |
JP2013506886A (en) | 2013-02-28 |
US8650360B2 (en) | 2014-02-11 |
US20120011315A1 (en) | 2012-01-12 |
JP5444464B2 (en) | 2014-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8650360B2 (en) | Storage system | |
US9037829B2 (en) | Storage system providing virtual volumes | |
US8984221B2 (en) | Method for assigning storage area and computer system using the same | |
US8527702B2 (en) | Storage system providing virtual volumes | |
US8533419B2 (en) | Method for controlling data write to virtual logical volume conforming to thin provisioning, and storage apparatus | |
JP5112003B2 (en) | Storage device and data storage method using the same | |
US9258364B2 (en) | Virtualization engine and method, system, and computer program product for managing the storage of data | |
US8225039B2 (en) | Storage controller and virtual volume control method | |
US10353602B2 (en) | Selection of fabric-attached storage drives on which to provision drive volumes for realizing logical volume on client computing device within storage area network | |
US20090292870A1 (en) | Storage apparatus and control method thereof | |
US20190317682A1 (en) | Metrics driven expansion of capacity in solid state storage systems | |
US20110145528A1 (en) | Storage apparatus and its control method | |
WO2018138813A1 (en) | Computer system | |
JP2011227898A (en) | Method and apparatus to manage tier information | |
KR20100077156A (en) | Thin provisioning migration and scrubbing | |
US11899533B2 (en) | Stripe reassembling method in storage system and stripe server | |
US9298388B2 (en) | Computer system, data management apparatus, and data management method | |
US11474901B2 (en) | Reliable RAID system with embedded spare capacity and flexible growth | |
US11314608B1 (en) | Creating and distributing spare capacity of a disk array | |
US11327666B2 (en) | RAID member distribution for granular disk array growth | |
US10402095B1 (en) | Online data expansion in redundant data storage systems | |
US11847351B2 (en) | Automatic identification and ranking of migration candidate storage groups based on relative performance impact to current storage array components | |
US11544005B2 (en) | Storage system and processing method | |
US20210026566A1 (en) | Storage control system and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20220211 |