WO2017128802A1 - 一种实现存储空间管理的架构及方法 - Google Patents

一种实现存储空间管理的架构及方法 Download PDF

Info

Publication number
WO2017128802A1
WO2017128802A1 PCT/CN2016/105075 CN2016105075W WO2017128802A1 WO 2017128802 A1 WO2017128802 A1 WO 2017128802A1 CN 2016105075 W CN2016105075 W CN 2016105075W WO 2017128802 A1 WO2017128802 A1 WO 2017128802A1
Authority
WO
WIPO (PCT)
Prior art keywords
logical volume
lun
san
data
service
Prior art date
Application number
PCT/CN2016/105075
Other languages
English (en)
French (fr)
Inventor
秦长鹏
牛克强
杨长江
刘卯银
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017128802A1 publication Critical patent/WO2017128802A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present invention relates to network storage technologies, and more particularly to an architecture and method for implementing storage space management.
  • Network storage services mainly have two types of standard service interfaces: network file service interface and block service interface.
  • the network file service interface is mainly based on a network file system (NFS, Netwok File System), a network file sharing system (CIFS, Common Internet File System), a file transfer protocol (FTP, File Transfer Protocol), and the like, and the server can pass the file. Semantic access to data on the storage device.
  • the basic access object is a file.
  • the storage device provides a network attached storage (NAS) service.
  • the block service interface is mainly based on a small computer system interface (SCSI, Small Computer System Interface).
  • the protocol family, the server accesses the data on the storage device through the semantics of the block device.
  • the basic access object is a logical volume. At this time, the storage device provides a storage area network (SAN) service.
  • SAN storage area network
  • NAS services and SAN services were provided by different devices, each of which was called NAS storage device and SAN storage device.
  • Netapp's FAS900 provided both NAS and SAN storage services on the same storage device for the first time, called unified storage devices.
  • unified storage devices have significantly reduced user costs and are rapidly becoming widely used.
  • a unified storage device provides both NAS and SAN services. Different storage services share the physical storage space of the device, and the management presents the user as a single complete device.
  • the physical form of the unified storage device is mainly composed of controller integration and controller separation.
  • the physical form of the unified storage device with controller integration is shown in Figure 1.
  • the physical shape of the unified storage device separated by the controller is shown in Figure 2. Show.
  • the unified storage device software architecture of the controller is mainly composed of space management based on file system and space management based on capacity pool, as shown in FIG. 3 and FIG. 4 respectively.
  • the NAS interface software maps network file access requests to the local file system. Access to file objects, the byte-granular access service provided by the file object exactly matches the network file service; and the SAN interface software maps access requests for logical volumes to access to file objects in the local file system, logical volume objects and file objects One-to-one correspondence, the access volume of the logical volume object requires granularity, and the file object provides a finer access granularity than the sector, and can also meet the service requirements of the logical volume.
  • the NAS interface software processing is the same as that shown in Figure 3.
  • the NAS interface software provides NAS services on one side and file system on the other, that is, the external interface and internal logic of the NAS interface software are
  • the SAN interface software maps the access request of the logical volume to the access of the volume object on the capacity pool.
  • the logical volume object and the capacity pool volume object correspond one-to-one.
  • the access service granularity of the capacity pool is sector, and the logic The service requirements for the volume are exactly matched.
  • FIG. 5 illustrates two common ways.
  • the storage space available to users is divided into the same or different granularity. Pages, the information of these pages (such as page addresses) is stored in the metadata.
  • the leaf node of the tree is the data page available to the user, and the branch node of the tree is metadata; in the iNode page pointer mode shown in FIG. 6, the iNode and the page pointer are metadata. .
  • the metadata brings a certain space overhead, and the smaller the data page, the more the number of pages divided by the physical space of the same size, the larger the amount of metadata required.
  • a fine-grained page such as 4 KB is managed for the space, and a complete file system enables various business functions to be conveniently supported, such as a thin provisioning function (Thin Provision, Also known as auto-reduction (de-duplication), deduplication, etc., but the amount of metadata in this way is relatively large, can not be all in memory, although it can be specific to some kind of input and output (IO) flow Optimized, but in general, compared to traditional SAN devices, its performance is not high.
  • Thin Provision Also known as auto-reduction (de-duplication), deduplication, etc.
  • the NAS service data path processing is more complicated and redundant, and the performance is compared with the traditional NAS device. Not high; if the space is coarse-grained Pages such as 1MB management may put all metadata in memory and perform well. However, this coarse-grained page will make some advanced functions of SAN services difficult to implement, or the implementation will not work well, such as Thin Provision. With a granularity of more than 64k, it is difficult to achieve better results in general application scenarios.
  • the space management mode of the existing storage software architecture the granularity of the space management is fine, which is beneficial to the function development, but the performance will be slightly worse; and the spatial management granularity is coarse, the performance is better, but the function support is not good. That is to say, the existing space management method has problems in that performance and functions are difficult to balance, and cannot meet the different requirements for performance and function of various applications.
  • the embodiments of the present invention provide an architecture and a method for implementing storage space management, which can solve the problem that performance and functions are difficult to balance, thereby flexibly satisfying different requirements for performance and functions of various applications.
  • An embodiment of the present invention provides an architecture for implementing storage space management, including a controller.
  • the controller includes at least: a capacity pool configured to provide only coarse-grained management of data blocks; a file system that manages fine-grained data blocks; a NAS interface component configured to complete processing of the NAS interface protocol stack and provide NAS services; and a SAN interface component configured to perform processing of the SAN interface protocol stack and provide SAN services; among them,
  • the SAN interface component is also configured to: determine a logical volume whose performance requirement is lower than a preset threshold, or a logical volume that needs to enable advanced data management functions; and provide a SAN service by using a space management method including a file system and a capacity pool.
  • the SAN interface component is configured to: map the access request of the host to the logical volume by using a logical volume whose determined performance requirement is lower than a preset threshold, or a logical volume that needs to enable advanced data management function Access to the created file object, providing SAN services to the host;
  • the created file object is: the system automatically creates a virtual volume on the capacity pool, and creates a file system on the virtual volume, the virtual volume and the file system are reserved internally; from the reserved file system A file object is created in the system, where the size of the file object is equal to the size of the logical volume created by the user.
  • the SAN interface component is further configured to provide a SAN service by using a space management manner of the capacity pool for a logical volume whose performance requirement is higher than or equal to the preset threshold.
  • the NAS interface component is further configured to provide a NAS service by using a space management manner including the file system and the capacity pool.
  • the architecture further includes: a data migration module
  • the data migration module is configured to notify the SAN interface component to perform data migration to another data path of the SAN service when it is detected that the logical volume LUN-A on the specified data path of the SAN service meets the preset automatic migration condition.
  • the data migration module is configured to:
  • the logical volume identifying the LUN ID of the logical volume LUN-A' is the same as the logical volume identifier of the logical volume LUN-A, but does not provide a SAN service;
  • the SAN interface component is further configured to: receive the first notification, and then send the write to the logical volume LUN-A to the data migration module; receive the second notification, stop the service to the logical volume LUN-A, and start to
  • the logical volume LUN-A' provides a SAN service that maps the original service to the logical volume LUN-A to access to the logical volume LUN-A' object on the file system.
  • the logical volume LUN-A continues to provide the SAN service
  • the SAN interface component is further configured to: send data written by the host to the logical volume LUN-A to the data migration module;
  • the data migration module is further configured to: write data from the specified data path to a virtual volume corresponding to the logical volume LUN-A, and simultaneously write the file corresponding to the logical volume LUN-A' through the another data path. Object.
  • the embodiment of the present invention further provides a method for implementing storage space management, including: determining a logical volume whose performance requirement is lower than a preset threshold, or a logical volume that needs to enable an advanced data management function;
  • SAN services in a space management manner that includes file systems and capacity pools.
  • the providing the SAN service by using a space management manner including a file system and a capacity pool includes:
  • the system automatically creates a virtual volume on the capacity pool and creates a file system on the virtual volume, which is reserved internally;
  • the size of the file object is equal to the size of the logical volume created by the user
  • the SAN interface component maps host access requests for logical volumes to access to created file objects and SAN services to hosts.
  • the method further includes: providing a SAN service by using a space management manner of the capacity pool for a logical volume whose performance requirement is higher than or equal to the preset threshold.
  • the method further includes providing the NAS service by using a space management manner including the file system and the capacity pool.
  • the SAN interface component is notified to perform data migration to another data path of the SAN service.
  • the performing another data path of the data migration to the SAN service includes:
  • the data migration module periodically detects whether the logical volume LUN-A on the specified data path currently meets the preset automatic migration condition, starts data migration when the automatic migration condition is met, and creates logic on the another data path.
  • Volume LUN-A' the logical volume identifier LUN ID of the logical volume LUN-A' is the same as the logical volume identifier of the logical volume LUN-A, but does not provide a SAN service; the first notification is sent to the SAN interface component; Reading data of the virtual volume corresponding to the logical volume LUN-A from the specified data path; writing the read data from the another data path to the file object corresponding to the logical volume LUN-A' until the logical volume LUN The data on -A' is identical to the data on the logical volume LUN-A; the second notification is sent to the SAN interface component; the logical volume LUN-A is deleted, and the related resources are released;
  • the SAN interface component receives the first notification, and then sends a write to the logical volume LUN-A to the data migration module; receives the second notification, stops the service to the logical volume LUN-A, and starts to the logical volume LUN- A' provides a SAN service that maps the original service to logical volume LUN-A to access to the logical volume LUN-A' object on the file system.
  • the automatic migration condition includes: the IO access of the logical volume in a period of time is lower than a preset IO access threshold, or the creation time of the logical volume is greater than a preset time threshold.
  • the data migration module reads data of the virtual volume corresponding to the logical volume LUN-A from the specified data path, and writes the read data from the another data path to the logic
  • the method further includes:
  • the SAN interface component sends data written by the host to the logical volume LUN-A to the data migration module;
  • the data migration module writes data from the specified data path to a virtual volume corresponding to the logical volume LUN-A, and simultaneously writes a file object corresponding to the logical volume LUN-A' through the another data path.
  • the technical solution of the present application includes: determining a logical volume whose performance requirement is lower than a preset threshold, or a logical volume that needs to enable advanced data management function; adopting a space management manner including two levels of a file system and a capacity pool Provide SAN services.
  • the technical solution provided by the embodiment of the present invention provides a SAN service to the host through space management including a file system and a capacity pool, thereby solving the problem that performance and functions are difficult to balance, thereby flexibly satisfying various applications for performance and functions. Different needs on.
  • the embodiment of the present invention further includes: for a logical volume whose performance requirement is higher than or equal to a preset threshold, the SAN service is provided by using a single-level space management manner of the capacity pool. And, NAS services are provided in a space management manner including a file system and a capacity pool.
  • the embodiment of the present invention further includes: when the data migration module detects that the specified data path of the SAN service, such as the logical volume LUN-A on the data path 3, meets a preset automatic migration condition, notifying the SAN interface component to perform data migration.
  • the SAN interface component of the embodiment of the present invention implements an access request for different logical volumes in the system to be respectively mapped to a file object or an access request for a volume object on a capacity pool, and the mapping relationship is implemented without affecting access to the logical volume. In the case of a change in the life cycle of a logical volume.
  • FIG. 1 is a schematic diagram of a unified storage device in which a controller is integrated
  • FIG. 2 is a schematic diagram of a unified storage device separated by an existing controller
  • FIG. 3 is a schematic diagram of a unified unified storage device software architecture of a file system-based space management controller
  • FIG. 4 is a schematic diagram of a software architecture of a unified unified storage device based on a space pool-based space management controller
  • FIG. 5 is a schematic diagram of a B-tree mode in an existing space management manner
  • FIG. 6 is a schematic diagram of an iNode page pointer mode in an existing space management manner
  • FIG. 7 is a schematic diagram of an architecture for implementing storage space management according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of migration of a SAN storage service according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of a method for implementing storage space management according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of an architecture for implementing storage space management according to an embodiment of the present invention.
  • the controller includes at least a capacity pool, a file system, a NAS interface component, and a SAN interface component.
  • the NAS interface component is configured to complete the processing of the NAS interface protocol stack, such as NFS/CIFS/FTP, and the NAS service is provided by the NAS interface component;
  • the SAN interface component is configured to complete the processing of the SAN interface protocol stack, such as FC/iSCSI+SCSI, and the SAN service is provided through the SAN interface component.
  • the SAN interface protocol stack such as FC/iSCSI+SCSI
  • a file system set to manage data blocks at a finer granularity such as KB granularity. Based on this fine-grained management, compression, encryption, Thin Porvision, Deduplication and other advanced storage The storage function is easy to implement. Different file systems have their own methods of space management, such as EXT3 (which is the name of a Linux kernel's native file system).
  • EXT3 which is the name of a Linux kernel's native file system.
  • ZFS (a file system developed by Sun) uses a similar image to Figure 5. structure;
  • the capacity pool is set to only manage the data blocks at a coarser granularity such as MB granularity. Based on this coarse-grained management capacity pool, basic functions such as RAID (Redundant Arrays of Independent Disks) and automatic data classification are provided.
  • RAID Redundant Arrays of Independent Disks
  • automatic data classification A feasible capacity pool space management method is described in patent US6823442.
  • the SAN interface component is also configured to: determine a logical volume whose performance requirement is lower than a preset threshold, or a logical volume that needs to enable advanced data management functions; and provide a SAN service by using a space management method including a file system and a capacity pool.
  • a space management method including a file system and a capacity pool.
  • the host's access request for the logical volume is mapped to the access to the created file object, and the host is provided with the SAN service. ;among them,
  • the created file object is: the system automatically creates a virtual volume on the capacity pool, and creates a file system on the virtual volume, the virtual volume and the file system are reserved internally; and the file object is created from the reserved file system, wherein the file The size of the object is equal to the logical volume size created by the user.
  • FIG. 7 three feasible data paths are formed, wherein the NAS service is provided by the data path 1 as shown by the thick dotted line in FIG. 7, and the SAN service passes the data path 2 and the coarse double point indicated by the thick one-dot chain line.
  • the data path 3 shown by the line is provided.
  • the SAN service not only has a single path but has two different data paths.
  • a single quality of service is provided on a single data path, and only one strategy can be selected in the tradeoff between IO performance and space efficiency, which is difficult to meet the needs of many different applications.
  • different strategies are provided by introducing two different data paths, namely, data path 2 and data path 3, thereby better satisfying different applications. Different needs.
  • Data path 1 in Figure 7 provides the NAS service, on data path 1:
  • the disk space is added to the capacity pool.
  • the capacity pool divides the disk space into pages of granularity above MB and generates corresponding metadata.
  • the capacity pool completes capacity management functions such as RAID, online capacity expansion, and automatic data grading, and virtualizes disk space into capacity pool space.
  • the user specifies or the system automatically sets the initial capacity quota of the file system.
  • the system allocates the virtual volume corresponding to the space size from the capacity pool. For example, when the automatic classification function is enabled, the virtual volume attribute adopts the default attribute; A file system is created on the volume; the file system divides the capacity of the virtual volume into pages of 4 to 128 KB, and creates storage objects such as directories and files thereon.
  • the NAS interface component maps the host's network file access request to access to file system objects and NAS services to the host.
  • the SAN interface component may use a service provided by a file system, or may use a service provided by a capacity pool.
  • the user specifies the performance requirements of the logical volume, and the system selects the actual data path based on this option of the user. among them,
  • the SAN service may select the data path 3 as shown in FIG. 7.
  • the capacity pool space management manner is the same as described above, that is, the NAS data path.
  • the system allocates a virtual volume corresponding to the space size from the capacity pool, and its attributes satisfy user settings (such as automatic hierarchical attribute); the SAN interface component maps the host's access request for the logical volume to the capacity pool. Access to virtual volumes to provide SAN services to hosts.
  • the data path 3 has only a coarse-grained space management of the capacity pool, and the metadata can be implemented in the resident memory, and the IO performance is optimal.
  • the SAN service can select data path 2 as shown in Figure 7, on data path 2: the capacity pool space management mode and
  • the above NAS data path is the same, that is, the system automatically creates a virtual volume on the capacity pool, and creates a file system on the virtual volume, the virtual volume and the file system are reserved internally, are invisible to the user; are created from the reserved file system File pair
  • the size of the file object is equal to the size of the logical volume created by the user; the SAN interface component maps the host's access request for the logical volume to the access to the created file object, and provides the SAN service to the host.
  • the data path 2 has two levels of space management of the file system and the capacity pool.
  • the metadata of the file system cannot be all resident in the memory, and the performance is lower than the data path 3; however, due to the small granularity of the space management, Very good support for all types of data management functions.
  • the SAN interface component externally maps access requests of different logical volumes in the system to: access requests to file objects or volume objects on the capacity pool, but each volume only There can be a mapping method that, once formed, does not change throughout the lifecycle of the logical volume.
  • FIG. 8 is a schematic diagram of a migration of a SAN storage service according to an embodiment of the present invention.
  • the architecture for implementing space management in the embodiment of the present invention further includes: a data migration function added to complete data migration within the system in FIG. Software component.
  • the set automatic migration condition may include, but is not limited to, the IO access of the logical volume within a certain period is lower than a certain advance.
  • the set IO access threshold, or the creation time of the logical volume is greater than a preset time threshold.
  • the data migration module inside the system performs automatic migration of logical volume data from the capacity pool to the file system, including:
  • the data migration module is configured to perform data migration and notify the SAN interface component when it is detected that the logical volume LUN-A on the data path 3 satisfies the preset automatic migration condition.
  • Logical volume ID (LUN ID) of the logical volume LUN-A’ Same as the logical volume identifier of the logical volume LUN-A on the data path 3, but does not provide the SAN service; sends the first notification to the SAN interface component; reads the data of the virtual volume corresponding to the logical volume LUN-A from the data path 3; Write the read data from the data path 2 to the file object corresponding to the logical volume LUN-A' until the data on the logical volume LUN-A' and the data on the logical volume LUN-A are identical; send to the SAN interface component Second notification; delete LUN-A and release related resources;
  • the SAN interface component is also configured to receive notifications from the data migration module and perform data migration.
  • the specific settings are:
  • the subsequent write of the logical volume LUN-A is sent to the data migration module; receiving the second notification, stopping the service to the logical volume LUN-A, and starting to provide the SAN service to the logical volume LUN-A'
  • the original service for logical volume LUN-A is mapped to access to the logical volume LUN-A' object on the file system.
  • the data migration module reads the data of the virtual volume corresponding to the logical volume LUN-A from the data path 3, and writes the read data from the data path 2 to the file object corresponding to the logical volume LUN-A', if Logical volume LUN-A continues to provide SAN services, then,
  • the SAN interface component is further configured to: send data written by the host to the logical volume LUN-A to the data migration module;
  • the data migration module is further configured to: write data from the data path 3 to the virtual volume corresponding to the logical volume LUN-A, and also write the file object corresponding to the logical volume LUN-A' through the data path 2.
  • the user can also set policies to control the logical volume data for reverse migration.
  • the specific implementation is easy to implement on the basis of the embodiment shown in Embodiment 8 of the present invention, and details are not described herein again.
  • the SAN interface component of the embodiment of the present invention implements mapping access requests for different logical volumes in the system to access to a file object or a volume object on a capacity pool, and the mapping relationship may be Changes are made in the lifecycle of a logical volume without affecting access to the logical volume.
  • FIG. 9 is a flowchart of a method for implementing storage space management according to an embodiment of the present invention, as shown in FIG. include:
  • Step 900 Determine a logical volume whose performance requirement is lower than a preset threshold, or a logical volume that needs to enable advanced data management functions.
  • Step 901 Provide a SAN service by using a space management manner including a file system and a capacity pool. This step specifically includes:
  • the system automatically creates a virtual volume on the capacity pool and creates a file system on the virtual volume that is internal to the file system and is invisible to the user;
  • the size of the file object is equal to the size of the logical volume created by the user
  • the SAN interface component maps host access requests for logical volumes to access to created file objects and SAN services to hosts.
  • the method of the embodiment of the present invention provides a SAN service to a host by using a space management system including a file system and a capacity pool.
  • a space management system including a file system and a capacity pool.
  • the metadata of the file system cannot be all resident in the memory, the performance is compared with the single-level space management mode of the capacity pool.
  • the provision of SAN services is low.
  • due to the small granularity of space management it supports various data management functions, solves the problem that performance and functions are difficult to balance, and thus flexibly meets the performance and function of various applications. Different needs.
  • the method of the embodiment of the present invention further includes: providing a SAN service by using a single-level space management manner of the capacity pool for a logical volume whose performance requirement is higher than or equal to a preset threshold. Specifically include:
  • the system allocates a virtual volume corresponding to the size of the space from the capacity pool, and its attributes satisfy user settings (such as automatic hierarchical attributes); the SAN interface component maps the host's access request for the logical volume to the virtual pool on the capacity pool. Volume access, providing SAN services to hosts.
  • the method of the embodiment of the present invention further includes: providing the NAS service by using a space management manner including two levels of a file system and a capacity pool. Specifically include:
  • Disk space is added to the capacity pool, which divides the disk space into pages of granularity above MB. Generate corresponding metadata; the capacity pool completes capacity management functions such as RAID, online capacity expansion, and automatic data classification, and virtualizes disk space into capacity pool space;
  • the user specifies or the system automatically sets the initial capacity quota of the file system.
  • the system allocates the virtual volume corresponding to the space size from the capacity pool. For example, when the automatic classification function is enabled, the virtual volume attribute adopts the default attribute; A file system is created on the volume; the file system divides the capacity of the virtual volume into 4 to 128 KB pages, and creates storage objects such as directories and files thereon;
  • the NAS interface component maps the host's network file access request to access to file system objects and NAS services to the host.
  • the method of the embodiment of the present invention further includes:
  • the SAN interface component is notified to perform data migration to another data path of the SAN service.
  • it includes:
  • the data migration module periodically detects the specified data path of the SAN service, such as whether the logical volume LUN-A on the data path 3 currently meets the preset automatic migration condition.
  • the automatic migration condition is met, data migration is initiated; another data path for creating a SAN service is the logical volume LUN-A' on data path 2.
  • the logical volume identifier (LUN ID) of the logical volume LUN-A' and the specified data path of the SAN service are the same as the logical volume identifier of the logical volume LUN-A on the data path 3, but the SAN service is not provided; the SAN interface component is sent The first notification; reading the data of the virtual volume corresponding to the logical volume LUN-A from the designated data path of the SAN service, such as the data path 3; writing the read data from another data path of the SAN service, such as the data path 2, to the logic Volume file corresponding to volume LUN-A' until the data on logical volume LUN-A' is identical to the data on logical volume LUN-A; send a second notification to SAN interface component; delete logical volume LUN-A, release related Resources.
  • the SAN interface component receives the first notification, and the subsequent write to the logical volume LUN-A is sent.
  • the data migration module receives the second notification, stops the service to the logical volume LUN-A, starts to provide the SAN service to the logical volume LUN-A', and maps the original service of the logical volume LUN-A to the pair on the file system. Access to logical volume LUN-A' objects.
  • the automatic migration condition may include, but is not limited to, the IO access of the logical volume in a period of time is lower than a preset IO access threshold, or the creation time length of the logical volume is greater than a preset time threshold.
  • the SAN interface component of the embodiment of the present invention implements mapping access requests for different logical volumes in the system to access requests for file objects or volume objects on the capacity pool, and the mapping relationship is implemented without affecting the logic. In the case of volume access, changes occur during the life cycle of the logical volume.
  • the data migration module reads the data of the virtual volume corresponding to the logical volume LUN-A from the specified data path of the SAN service, such as the data path 3, and writes the read data from another data path of the SAN service, such as the data path 2, into the logic.
  • the specified data path of the SAN service such as the data path 3
  • the read data from another data path of the SAN service, such as the data path 2
  • the process of the file object corresponding to the volume LUN-A' if the logical volume LUN-A continues to provide the SAN service, then
  • the SAN interface component sends the data written by the host to the logical volume LUN-A to the data migration module;
  • the data migration module writes data from the specified data path of the SAN service, such as data path 3, to the virtual volume corresponding to the logical volume LUN-A, and also writes the logical volume LUN-A' through another data path of the SAN service, such as data path 2.
  • the corresponding file object
  • the access request for different logical volumes in the system is respectively mapped to the access request for the file object or the volume object on the capacity pool by using the SAN interface component, and the mapping relationship is implemented. Changes occur in the lifecycle of a logical volume without affecting access to the logical volume.
  • Ability to solve performance and functionality is difficult The problem of Gu, so as to flexibly meet the different needs of performance and function of various applications.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种实现存储空间管理的架构及方法，包括确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；采用包括文件***和容量池两个层次的空间管理方式提供SAN服务。上述技术方案通过包括有文件***和容量池两个层次的空间管理对主机提供SAN服务，解决了性能和功能难以兼顾的问题，从而灵活满足了各类应用对于性能和功能上的不同需求。

Description

一种实现存储空间管理的架构及方法 技术领域
本发明涉及网络存储技术，尤指一种实现存储空间管理的架构及方法。
背景技术
网络存储服务主要有两类标准服务接口：网络文件服务接口和块服务接口。其中，网络文件服务接口主要基于网络文件***(NFS，Netwok File System)，网络文件共享***(CIFS，Common Internet File System)，文件传输协议(FTP，File Transfer Protocol)等标准协议，服务器可以通过文件语义方式访问存储设备上的数据，基本访问对象是文件，此时称存储设备提供了网络附加存储(NAS，Network Attack Storage)服务；块服务接口主要基于小型计算机***接口(SCSI，Small Computer System Interface)协议族，服务器通过块设备的语义来访问存储设备上的数据，基本访问对象是逻辑卷，此时称存储设备提供了存储区域网络(SAN，Storage Area Network)服务。
二十世纪九十年代，NAS服务与SAN服务分别由不同的设备提供，各自分别称为NAS存储设备和SAN存储设备。2002年，Netapp公司的FAS900第一次在同一个存储设备上同时提供NAS和SAN两类存储服务，称为统一存储设备。在一些应用场景，统一存储设备显著降低了用户成本，从而迅速得到了广泛应用。
统一存储设备同时提供NAS和SAN两类服务，不同存储服务共享设备的物理存储空间，管理上对使用者呈现为单个完整设备。统一存储设备物理形态主要有控制器合一和控制器分离两种方式，其中，控制器合一的统一存储设备物理形态如图1所示，控制器分离的统一存储设备物理形态如图2所示。控制器合一的统一存储设备软件架构主要有基于文件***的空间管理，和基于容量池的空间管理两类，分别如图3和图4所示。
在图3中，NAS接口软件将网络文件访问请求映射为本地文件***中 文件对象的访问，文件对象提供的字节粒度的访问服务与网络文件服务完全匹配；而SAN接口软件将对逻辑卷的访问请求映射为本地文件***中文件对象的访问，逻辑卷对象和文件对象一一对应，逻辑卷对象的访问服务要求的粒度为扇区，而文件对象提供了比扇区更细的访问粒度，也能够满足逻辑卷的服务需求。在图4中，NAS接口软件处理与图3所示的相同，即NAS接口软件都是一侧对外提供NAS服务，一侧基于文件***，也就是说NAS接口软件的外部接口和内部逻辑都是一样的；而SAN接口软件则将对逻辑卷的访问请求映射为容量池上的卷对象的访问，逻辑卷对象和容量池卷对象一一对应，容量池上卷的访问服务粒度为扇区，与逻辑卷的服务要求完全匹配。
文件***和容量池都具备管理存储空间的能力，基于的方法有类似和相通的地方，图5和图6示意了两种比较常见的方式，用户可用的存储空间划分为相同或者不同粒度的数据页面，这些页面的信息(例如页面地址)存放在元数据中。在图5所示的B树方式中，树的叶节点就是用户可用的数据页面，而树的枝节点是元数据；在图6所示的iNode页面指针方式中，iNode和页面指针是元数据。无论采用何种空间管理方式，都必须基于元数据来分配、释放、查找页面。其中，元数据带来一定的空间开销，数据页面越小，同样大小的物理空间划分的页面数越多，则需要的元数据量就越大。
从上述描述可以看出，基于文件***的空间管理，对空间实现了细粒度页面如4KB的管理，完善的文件***使得各类业务功能可以很方便的得到支持，比如瘦供给功能(Thin Provision，也称为自动精简功能)，重复数据删除(De-duplication)等，但是，这种方式的元数据量相对较大，无法全部放在内存中，虽然可以针对具体某种输入输出(IO)流进行优化，但一般来说，对比传统的SAN设备，其性能并不高。
基于容量池的空间管理，如果对空间实现细粒度页面的管理，则会与统一存储设备内部存在的文件***逻辑重复，NAS服务数据路径处理更加复杂冗余，相比传统的NAS设备，其性能不高；如果对空间实现粗粒度 页面如1MB的管理，可能将所有元数据都放在内存中处理，性能不错，但是，这种粗粒度页面会使得一些SAN服务的高级功能难以实施，或者实施效果不好，比如Thin Provision，如果粒度超过64k，就很难在一般的应用场景下取得比较好的效果。
综上所述，现有存储软件架构的空间管理方式，空间管理粒度细，利于功能开展，但是性能会稍差；而空间管理粒度粗，性能比较好，但是功能支持不好。也就是说，现有的空间管理方式，存在性能和功能难以兼顾的问题，不能满足各类应用对于性能和功能上的不同需求。
发明内容
为了解决上述技术问题，本发明实施例提供一种实现存储空间管理的架构及方法，能够解决性能和功能难以兼顾的问题，从而灵活满足各类应用对于性能和功能上的不同需求。
为了达到本发明实施例目的，本发明实施例提供了一种实现存储空间管理的架构，包括控制器；控制器至少包括：设置为仅提供对数据块以粗粒度进行管理的容量池；设置为对数据块以细粒度进行管理的文件***；设置为完成NAS接口协议栈的处理并提供NAS服务的NAS接口组件；以及，设置为完成SAN接口协议栈的处理并提供SAN服务的SAN接口组件；其中，
SAN接口组件还设置为：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；采用包括文件***和容量池的空间管理方式提供SAN服务。
可选地，所述SAN接口组件设置为：对所述确定出的性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷，将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务；其中，
所述创建的文件对象为：***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留；从所述保留文件系 统中创建文件对象，其中，文件对象的大小等于用户创建的逻辑卷大小。
可选地，所述SAN接口组件还设置为：对于性能要求高于或等于所述预先设置的阈值的逻辑卷，采用所述容量池的空间管理方式提供SAN服务。
可选地，所述NAS接口组件还设置为：采用包括所述文件***和所述容量池的空间管理方式提供NAS服务。
可选地，所述架构还包括：数据迁移模块；
数据迁移模块设置为：当检测到SAN服务的指定数据路径上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知所述SAN接口组件执行数据迁移至SAN服务的另一数据路径上。
可选地，所述数据迁移模块设置为：
定期检测所述指定数据路径上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件，当满足自动迁移条件时，启动数据迁移；
创建所述另一数据路径上的逻辑卷LUN-A’，该逻辑卷LUN-A’的逻辑卷标识LUN ID和所述逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；
向所述SAN接口组件发送第一通知；从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从所述另一数据路径写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向所述SAN接口组件发送第二通知；删除逻辑卷LUN-A，释放相关资源；
相应地，
所述SAN接口组件还设置为：接收到第一通知，后续将对逻辑卷LUN-A的写入送给数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
可选地，
在所述数据迁移模块从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从所述另一数据路径写入逻辑卷LUN-A’对应的文件对象的过程中，如果所述逻辑卷LUN-A持续提供SAN服务，则，
所述SAN接口组件还设置为：将主机写入所述逻辑卷LUN-A的数据发送给所述数据迁移模块；
所述数据迁移模块还设置为：将数据从所述指定数据路径写入逻辑卷LUN-A对应的虚拟卷，同时通过所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象。
本发明实施例还提供了一种实现存储空间管理的方法，包括：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；
采用包括文件***和容量池的空间管理方式提供SAN服务。
可选地，所述采用包括文件***和容量池的空间管理方式提供SAN服务包括：
***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留；
从保留文件***中创建文件对象，文件对象的大小等于用户创建的逻辑卷大小；
SAN接口组件将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务。
可选地，该方法还包括：对于性能要求高于或等于所述预先设置的阈值的逻辑卷，采用所述容量池的空间管理方式提供SAN服务。
可选地，该方法还包括：采用包括所述文件***和所述容量池的空间管理方式提供NAS服务。
可选地，当数据迁移模块检测到SAN服务的指定数据路径上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知SAN接口组件执行数据迁移至SAN服务的另一数据路径上。
可选地，所述执行数据迁移至SAN服务的另一数据路径上包括：
所述数据迁移模块定期检测所述指定数据路径上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件，当满足自动迁移条件时，启动数据迁移；创建所述另一数据路径上的逻辑卷LUN-A’，该逻辑卷LUN-A’的逻辑卷标识LUN ID和所述逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；向所述SAN接口组件发送第一通知；从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从所述另一数据路径写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向所述SAN接口组件发送第二通知；删除逻辑卷LUN-A，释放相关资源；
相应地，
所述SAN接口组件接收到第一通知，后续将对逻辑卷LUN-A的写入送给数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
可选地，所述自动迁移条件包括：逻辑卷在一段时间内的IO访问低于某个预先设置的IO访问阈值，或者逻辑卷的创建时间长度大于预先设置的时间阈值。
可选地，在所述数据迁移模块从所述指定数据路径读取所述逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象的过程中，如果所述逻辑卷LUN-A持续提供SAN服务，该方法还包括：
所述SAN接口组件将主机写入所述逻辑卷LUN-A的数据发送给数据迁移模块；
所述数据迁移模块将数据从所述指定数据路径写入所述逻辑卷LUN-A对应的虚拟卷，同时通过所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象。
与相关技术相比，本申请技术方案包括：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；采用包括文件***和容量池两个层次的空间管理方式提供SAN服务。本发明实施例提供的技术方案，通过包括有文件***和容量池两个层次的空间管理对主机提供SAN服务，解决了性能和功能难以兼顾的问题，从而灵活满足了各类应用对于性能和功能上的不同需求。
可选地，本发明实施例还包括：对于性能要求高于或等于预先设置的阈值的逻辑卷，采用容量池单层次的空间管理方式提供SAN服务。以及，采用包括文件***和容量池两个层次的空间管理方式提供NAS服务。本发明实施例真正解决了性能和功能难以兼顾的问题，从而灵活满足了各类应用对于性能和功能上的不同需求。
可选地，本发明实施例还包括：当数据迁移模块检测到SAN服务的指定数据路径如数据路径3上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知SAN接口组件执行数据迁移。通过本发明实施例的SAN接口组件实现了将对***中不同逻辑卷的访问请求分别映射为对文件对象或者对于容量池上卷对象的访问请求，这种映射关系实现了在不影响对逻辑卷访问的情况下，在逻辑卷的生命周期中进行变化。
本发明实施例的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本发明而了解。本发明实施例的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
附图说明
此处所说明的附图用来提供对本发明实施例的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成 对本发明的不当限定。在附图中：
图1为现有控制器合一的统一存储设备的示意图；
图2为现有控制器分离的统一存储设备的示意图；
图3为现有基于文件***的空间管理的控制器合一的统一存储设备软件架构示意图；
图4为现有基于容量池的空间管理的控制器合一的统一存储设备软件架构示意图；
图5为现有空间管理方式中B树方式的示意图；
图6为现有空间管理方式中iNode页面指针方式的示意图；
图7为本发明实施例实现存储空间管理的架构的示意图；
图8为本发明实施例SAN存储服务的迁移的示意图；
图9为本发明实施例实现存储空间管理的方法的流程图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚明白，下文中将结合附图对本发明的实施例进行详细说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互任意组合。
图7为本发明实施例实现存储空间管理的架构的示意图，如图7所示，包括控制器，控制器至少包括容量池，文件***，NAS接口组件，SAN接口组件；其中，
NAS接口组件，设置为完成NAS接口协议栈，如NFS/CIFS/FTP等的处理，NAS服务通过NAS接口组件提供；
SAN接口组件，设置为完成SAN接口协议栈，如FC/iSCSI+SCSI等的处理，SAN服务通过SAN接口组件提供。
文件***，设置为对数据块以比较细的粒度如KB粒度进行管理。基于这种细粒度管理，压缩、加密、Thin Porvision、Deduplication等高级存 储功能很容易实现。不同的文件***有自己的空间管理的方法，比如EXT3(是一种Linux内核原生的文件***的名称)采用类似图6的结构，ZFS(是SUN公司开发的一个文件***)采用类似图5的结构；
容量池，设置为仅提供对数据块以较粗粒度如MB以上粒度进行管理。基于这种粗粒度管理的容量池，提供磁盘阵列(RAID，Redundant Arrays of Independent Disks)、数据自动分级等基本功能。一种可行的容量池空间管理方式如专利US6823442所述。
SAN接口组件还设置为：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；采用包括文件***和容量池两个层次的空间管理方式提供SAN服务。可选地，设置为：
对于确定出的性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷，将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务；其中，
所述创建的文件对象为：***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留；从所述保留文件***中创建文件对象，其中，文件对象的大小等于用户创建的逻辑卷大小。
如图7中形成了3条可行的数据路径，其中，NAS服务通过如图7中粗虚线所示的数据路径1提供，SAN服务通过粗单点划线所示的数据路径2和粗双点划线所示的数据路径3提供。
与图3、图4所示的现有解决方案相比，不同的是，在本发明实施例图7中，SAN服务不只拥有单条路径，而是拥有两条不同的数据路径。现有技术中的单个数据路径上提供单一的服务质量，在IO性能和空间效率的权衡上只能选择一种策略，难以满足多种不同应用的需求。但是，在本发明实施例图7所示的实现空间管理的架构中，通过引入两条不同的数据路径即数据路径2和数据路径3，提供了不同的策略，从而更好的满足了不同应用的不同需求。
下面详细介绍基于本发明实施例实现空间管理的架构的三种不同数 据路径针对不同应用的不同需求提供的服务过程。
图7中的数据路径1提供NAS服务，在数据路径1上：
磁盘空间加入容量池，容量池将磁盘空间划分为MB以上粒度的页面，生成对应的元数据。容量池完成RAID、在线扩容、数据自动分级等容量管理功能，将磁盘空间虚拟化为容量池空间。
用户选择创建文件***时，用户指定或者***自动设定文件***的初始容量配额，***从容量池上分配对应空间大小的虚拟卷，如当开启自动分级功能时虚拟卷的属性采用默认属性；在虚拟卷上创建文件***；文件***将虚拟卷的容量划分为4～128KB的页面进行管理，在其上创建目录、文件等存储对象。
NAS接口组件将主机的网络文件访问请求，映射到对文件***对象的访问，对主机提供NAS服务。
在本发明实施例中，SAN接口组件可以使用文件***提供的服务，也可以使用容量池提供的服务。在创建逻辑卷时，用户指定逻辑卷的性能要求，***根据用户的这个选项选择实际的数据路径即可。其中，
对于性能要求高的逻辑卷如高于或等于预先设置的阈值，SAN服务可以选择如图7的数据路径3，在数据路径3上：容量池空间管理方式和上述NAS数据路径所述相同，即：根据用户选择创建逻辑卷属性，***从容量池上分配对应空间大小的虚拟卷，其属性满足用户设定(例如自动分级属性)；SAN接口组件将主机对于逻辑卷的访问请求映射到对容量池上虚拟卷的访问，对主机提供SAN服务。如图7所示，数据路径3只有容量池一层粗粒度的空间管理，元数据可以实现常驻内存中，IO性能最佳。
而对于性能要求不高如低于预先设置的阈值，或者需要开启高级数据管理功能的逻辑卷，SAN服务可以选择如图7中的数据路径2，在数据路径2上：容量池空间管理方式和上述NAS数据路径所述相同，即：***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留，对于用户不可见；从上述保留文件***中创建文件对 象，文件对象的大小等于用户创建的逻辑卷大小；SAN接口组件将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务。
可见，数据路径2有文件***和容量池两个层次的空间管理，其中，文件***的元数据无法全部常驻内存，性能相比数据路径3要低；但是，由于空间管理粒度较小，可以很好的支持各类数据管理功能。
需要说明的是，在图7所示的实施方式中，SAN接口组件外部将***中不同逻辑卷的访问请求映射为：对文件对象或者对于容量池上卷对象的访问请求，但是，每个卷只能有一种映射方式，这种映射关系一经形成，在逻辑卷的整个生命周期内不变。
图8为本发明实施例SAN存储服务的迁移的示意图，与图7相比，本发明实施例实现空间管理的架构还包括：图8中增加的用于完成***内部的数据迁移的数据迁移功能软件组件。
随着时间推移，SAN服务访问的逻辑卷上的数据性能要求可能降低，同时需要开启高级功能(如压缩、Deduplication、Thin Provision等)来提升空间使用效率。用户根据对于逻辑卷负载的理解，可以设置一定的自动迁移条件控制某项高级功能的开启条件，设置的自动迁移条件可以包括但不限于：逻辑卷在一段时间内的IO访问低于某个预先设置的IO访问阈值，或者逻辑卷的创建时间长度大于预先设置的时间阈值等。
根据用户设置的策略，***内部的数据迁移模块执行逻辑卷数据从容量池向文件***的自动迁移，包括：
如图8中，数据迁移模块设置为：当检测到数据路径3上的逻辑卷LUN-A满足预先设置的自动迁移条件时，执行数据迁移并通知SAN接口组件。可选地，设置为：
定期检测数据路径3上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件。当满足自动迁移条件时，启动数据迁移；创建如图7中的数据路径2上的逻辑卷LUN-A’。该逻辑卷LUN-A’的逻辑卷标识(LUN ID) 和数据路径3上的逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；向SAN接口组件发送第一通知；从数据路径3读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从数据路径2写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向SAN接口组件发送第二通知；删除LUN-A，释放相关资源；
SAN接口组件还设置为：接收到来自数据迁移模块的通知，执行数据迁移。具体设置为：
接收到第一通知，后续将对逻辑卷LUN-A的写入送给数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
可选地，
在数据迁移模块从数据路径3读取逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从数据路径2写入逻辑卷LUN-A’对应的文件对象的过程中，如果，逻辑卷LUN-A持续提供SAN服务，那么，
SAN接口组件还设置为：将主机写入逻辑卷LUN-A的数据发送给数据迁移模块；
数据迁移模块还设置为：将数据从数据路径3写入逻辑卷LUN-A对应的虚拟卷，同时也通过数据路径2写入逻辑卷LUN-A’对应的文件对象。
可选地，用户也可以设置策略来控制逻辑卷数据进行反向的迁移。具体实现在本发明实施例8所示的实施方式的基础上是容易实现的，这里不再赘述。
通过图8所示的实施方式，本发明实施例的SAN接口组件实现了将对***中不同逻辑卷的访问请求分别映射为对文件对象或者对于容量池上卷对象的访问请求，这种映射关系可以在不影响对逻辑卷访问的情况下，在逻辑卷的生命周期中进行变化。
图9为本发明实施例实现存储空间管理的方法的流程图，如图9所示， 包括：
步骤900：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷。
步骤901：采用包括文件***和容量池两个层次的空间管理方式提供SAN服务。本步骤具体包括：
***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留，对于用户不可见；
从上述保留文件***中创建文件对象，文件对象的大小等于用户创建的逻辑卷大小；
SAN接口组件将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务。
本发明实施例方法，通过包括有文件***和容量池两个层次的空间管理对主机提供SAN服务，虽然文件***的元数据无法全部常驻内存，性能相比采用容量池单层次的空间管理方式提供SAN服务的要低，但是，由于空间管理粒度较小，很好的支持了各类数据管理功能，解决了性能和功能难以兼顾的问题，从而灵活满足了各类应用对于性能和功能上的不同需求。
本发明实施例方法还包括：对于性能要求高于或等于预先设置的阈值的逻辑卷，采用容量池单层次的空间管理方式提供SAN服务。具体包括：
根据用户选择创建逻辑卷属性，***从容量池上分配对应空间大小的虚拟卷，其属性满足用户设定(例如自动分级属性)；SAN接口组件将主机对于逻辑卷的访问请求映射到对容量池上虚拟卷的访问，对主机提供SAN服务。
本发明实施例方法还包括：采用包括文件***和容量池两个层次的空间管理方式提供NAS服务。具体包括：
磁盘空间加入容量池，容量池将磁盘空间划分为MB以上粒度的页面， 生成对应的元数据；容量池完成RAID、在线扩容、数据自动分级等容量管理功能，将磁盘空间虚拟化为容量池空间；
用户选择创建文件***时，用户指定或者***自动设定文件***的初始容量配额，***从容量池上分配对应空间大小的虚拟卷，如当开启自动分级功能时虚拟卷的属性采用默认属性；在虚拟卷上创建文件***；文件***将虚拟卷的容量划分为4～128KB的页面进行管理，在其上创建目录、文件等存储对象；
NAS接口组件将主机的网络文件访问请求，映射到对文件***对象的访问，对主机提供NAS服务。
可选地，对于SAN服务，本发明实施例方法还包括：
当数据迁移模块检测到SAN服务的指定数据路径如数据路径3上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知SAN接口组件执行数据迁移至SAN服务的另一数据路径上。
可选地，包括：
数据迁移模块定期检测SAN服务的指定数据路径如数据路径3上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件。当满足自动迁移条件时，启动数据迁移；创建SAN服务的另一数据路径如数据路径2上的逻辑卷LUN-A’。该逻辑卷LUN-A’的逻辑卷标识(LUN ID)和SAN服务的指定数据路径如数据路径3上的逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；向SAN接口组件发送第一通知；从SAN服务的指定数据路径如数据路径3读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从SAN服务的另一数据路径如数据路径2写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向SAN接口组件发送第二通知；删除逻辑卷LUN-A，释放相关资源。
相应地，
SAN接口组件接收到第一通知，后续将对逻辑卷LUN-A的写入送给 数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
其中，自动迁移条件可以包括但不限于：逻辑卷在一段时间内的IO访问低于某个预先设置的IO访问阈值，或者逻辑卷的创建时间长度大于预先设置的时间阈值等。
这样，通过本发明实施例的SAN接口组件实现了将对***中不同逻辑卷的访问请求分别映射为对文件对象或者对于容量池上卷对象的访问请求，这种映射关系实现了在不影响对逻辑卷访问的情况下，在逻辑卷的生命周期中进行变化。
可选地，
在数据迁移模块从SAN服务的指定数据路径如数据路径3读取逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从SAN服务的另一数据路径如数据路径2写入逻辑卷LUN-A’对应的文件对象的过程中，如果，逻辑卷LUN-A持续提供SAN服务，那么，
SAN接口组件将主机写入逻辑卷LUN-A的数据发送给数据迁移模块；
数据迁移模块将数据从SAN服务的指定数据路径如数据路径3写入逻辑卷LUN-A对应的虚拟卷，同时也通过SAN服务的另一数据路径如数据路径2写入逻辑卷LUN-A’对应的文件对象。
以上所述，仅为本发明的较佳实例而已，并非用于限定本发明的保护范围。凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
工业实用性
如上所述，通过上述实施例及优选实施方式，通过SAN接口组件实现了将对***中不同逻辑卷的访问请求分别映射为对文件对象或者对于容量池上卷对象的访问请求，这种映射关系实现了在不影响对逻辑卷访问的情况下，在逻辑卷的生命周期中进行变化。能够解决性能和功能难以兼 顾的问题，从而灵活满足各类应用对于性能和功能上的不同需求。

Claims (15)

  1. 一种实现存储空间管理的架构，包括控制器；控制器至少包括：设置为仅提供对数据块以粗粒度进行管理的容量池；设置为对数据块以细粒度进行管理的文件***；设置为完成NAS接口协议栈的处理并提供NAS服务的NAS接口组件；以及，设置为完成SAN接口协议栈的处理并提供SAN服务的SAN接口组件；其中，
    SAN接口组件还设置为：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；采用包括文件***和容量池的空间管理方式提供SAN服务。
  2. 根据权利要求1所述的架构，其中，所述SAN接口组件设置为：对所述确定出的性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷，将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务；其中，
    所述创建的文件对象为：***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留；从所述保留文件***中创建文件对象，其中，文件对象的大小等于用户创建的逻辑卷大小。
  3. 根据权利要求1所述的架构，其中，所述SAN接口组件还设置为：对于性能要求高于或等于所述预先设置的阈值的逻辑卷，采用所述容量池的空间管理方式提供SAN服务。
  4. 根据权利要求1所述的架构，其中，所述NAS接口组件还设置为：采用包括所述文件***和所述容量池的空间管理方式提供NAS服务。
  5. 根据权利要求1～4任一项所述的架构，其中，所述架构还包括：数据迁移模块；
    数据迁移模块设置为：当检测到SAN服务的指定数据路径上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知所述SAN接口组件执行数据迁移至SAN服务的另一数据路径上。
  6. 根据权利要求5所述的架构，其中，所述数据迁移模块设置为：
    定期检测所述指定数据路径上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件，当满足自动迁移条件时，启动数据迁移；
    创建所述另一数据路径上的逻辑卷LUN-A’，该逻辑卷LUN-A’的逻辑卷标识LUN ID和所述逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；
    向所述SAN接口组件发送第一通知；从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从所述另一数据路径写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向所述SAN接口组件发送第二通知；删除逻辑卷LUN-A，释放相关资源；
    相应地，
    所述SAN接口组件还设置为：接收到第一通知，后续将对逻辑卷LUN-A的写入送给数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
  7. 根据权利要求6所述的架构，其中，
    在所述数据迁移模块从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从所述另一数据路径写入逻辑卷 LUN-A’对应的文件对象的过程中，如果所述逻辑卷LUN-A持续提供SAN服务，则，
    所述SAN接口组件还设置为：将主机写入所述逻辑卷LUN-A的数据发送给所述数据迁移模块；
    所述数据迁移模块还设置为：将数据从所述指定数据路径写入逻辑卷LUN-A对应的虚拟卷，同时通过所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象。
  8. 一种实现存储空间管理的方法，包括：确定性能要求低于预先设置的阈值的逻辑卷，或需要开启高级数据管理功能的逻辑卷；
    采用包括文件***和容量池的空间管理方式提供SAN服务。
  9. 根据权利要求8所述的方法，其中，所述采用包括文件***和容量池的空间管理方式提供SAN服务包括：
    ***自动在容量池上创建虚拟卷，并在该虚拟卷上创建文件***，该虚拟卷和文件***内部保留；
    从保留文件***中创建文件对象，文件对象的大小等于用户创建的逻辑卷大小；
    SAN接口组件将主机对于逻辑卷的访问请求映射到对创建的文件对象的访问，对主机提供SAN服务。
  10. 根据权利要求8所述的方法，其中，该方法还包括：对于性能要求高于或等于所述预先设置的阈值的逻辑卷，采用所述容量池的空间管理方式提供SAN服务。
  11. 根据权利要求8所述的方法，其中，该方法还包括：采用包括所述文件***和所述容量池的空间管理方式提供NAS服务。
  12. 根据权利要求8～11任一项所述的方法，其中，当数据迁移模块检测到SAN服务的指定数据路径上的逻辑卷LUN-A满足预先设置的自动迁移条件时，通知SAN接口组件执行数据迁移至SAN服务的另一数据路径上。
  13. 根据权利要求12所述的方法，其中，所述执行数据迁移至SAN服务的另一数据路径上包括：
    所述数据迁移模块定期检测所述指定数据路径上的逻辑卷LUN-A当前是否满足预先设置的自动迁移条件，当满足自动迁移条件时，启动数据迁移；创建所述另一数据路径上的逻辑卷LUN-A’，该逻辑卷LUN-A’的逻辑卷标识LUN ID和所述逻辑卷LUN-A的逻辑卷标识相同，但不提供SAN服务；向所述SAN接口组件发送第一通知；从所述指定数据路径读取逻辑卷LUN-A对应的虚拟卷的数据；将读出的数据，从所述另一数据路径写入逻辑卷LUN-A’对应的文件对象，直到逻辑卷LUN-A’上的数据和逻辑卷LUN-A上的数据完全相同；向所述SAN接口组件发送第二通知；删除逻辑卷LUN-A，释放相关资源；
    相应地，
    所述SAN接口组件接收到第一通知，后续将对逻辑卷LUN-A的写入送给数据迁移模块；接收到第二通知，停止对逻辑卷LUN-A的服务，开始对逻辑卷LUN-A’提供SAN服务，将原来对逻辑卷LUN-A的服务映射到对文件***上的对逻辑卷LUN-A’对象的访问。
  14. 根据权利要求13所述的方法，其中，所述自动迁移条件包括：逻辑卷在一段时间内的IO访问低于某个预先设置的IO访问阈值，或者逻辑卷的创建时间长度大于预先设置的时间阈值。
  15. 根据权利要求13所述的方法，其中，在所述数据迁移模块 从所述指定数据路径读取所述逻辑卷LUN-A对应的虚拟卷的数据，并将读出的数据从所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象的过程中，如果所述逻辑卷LUN-A持续提供SAN服务，该方法还包括：
    所述SAN接口组件将主机写入所述逻辑卷LUN-A的数据发送给数据迁移模块；
    所述数据迁移模块将数据从所述指定数据路径写入所述逻辑卷LUN-A对应的虚拟卷，同时通过所述另一数据路径写入所述逻辑卷LUN-A’对应的文件对象。
PCT/CN2016/105075 2016-01-25 2016-11-08 一种实现存储空间管理的架构及方法 WO2017128802A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610049745.6 2016-01-25
CN201610049745.6A CN106997274B (zh) 2016-01-25 2016-01-25 一种实现存储空间管理的架构及方法

Publications (1)

Publication Number Publication Date
WO2017128802A1 true WO2017128802A1 (zh) 2017-08-03

Family

ID=59397363

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105075 WO2017128802A1 (zh) 2016-01-25 2016-11-08 一种实现存储空间管理的架构及方法

Country Status (2)

Country Link
CN (1) CN106997274B (zh)
WO (1) WO2017128802A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112000634B (zh) * 2020-07-28 2023-11-14 中国建设银行股份有限公司 Nas存储文件***的容量管理方法、***、设备和存储介质
CN113094756A (zh) * 2021-05-13 2021-07-09 统信软件技术有限公司 一种数据加密方法及计算设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003162439A (ja) * 2001-11-22 2003-06-06 Hitachi Ltd ストレージシステム及びその制御方法
CN1648842A (zh) * 2004-01-29 2005-08-03 株式会社日立制作所 具有多个接口的存储装置
CN1688982A (zh) * 2002-08-09 2005-10-26 网络装置公司 为文件和块存取协议提供综合支持的多协议存储设备
CN101986651A (zh) * 2010-08-26 2011-03-16 上海网众信息技术有限公司 远程存储的方法及其***及客户端
CN105068771A (zh) * 2015-09-17 2015-11-18 浪潮(北京)电子信息产业有限公司 一种统一存储方法及***

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104539746B (zh) * 2015-01-30 2018-10-26 浪潮(北京)电子信息产业有限公司 一种访问数据的方法、nas机头和san
CN104994135B (zh) * 2015-05-25 2018-09-21 华为技术有限公司 存储***中融合san及nas存储架构的方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003162439A (ja) * 2001-11-22 2003-06-06 Hitachi Ltd ストレージシステム及びその制御方法
CN1688982A (zh) * 2002-08-09 2005-10-26 网络装置公司 为文件和块存取协议提供综合支持的多协议存储设备
CN1648842A (zh) * 2004-01-29 2005-08-03 株式会社日立制作所 具有多个接口的存储装置
CN101986651A (zh) * 2010-08-26 2011-03-16 上海网众信息技术有限公司 远程存储的方法及其***及客户端
CN105068771A (zh) * 2015-09-17 2015-11-18 浪潮(北京)电子信息产业有限公司 一种统一存储方法及***

Also Published As

Publication number Publication date
CN106997274B (zh) 2021-04-30
CN106997274A (zh) 2017-08-01

Similar Documents

Publication Publication Date Title
US10324832B2 (en) Address based multi-stream storage device access
US9229826B2 (en) Volatile memory representation of nonvolatile storage device set
US11409448B2 (en) Selectively storing data into allocation areas using streams
US10055420B1 (en) Method to optimize random IOS of a storage device for multiple versions of backups using incremental metadata
US20180011657A1 (en) Use of predefined block pointers to reduce duplicate storage of certain data in a storage subsystem of a storage server
US9176677B1 (en) Virtual provisioning space reservation
US10289496B1 (en) Parallel proxy backup methodology
US8751547B2 (en) Multiple file system and/or multi-host single instance store techniques
TW201935243A (zh) 固態驅動器、分散式資料儲存系統和利用鍵值儲存的方法
US20170199707A1 (en) Methods and systems for efficiently storing data
WO2019015479A1 (zh) 在固态硬盘的ftl实现数据拷贝的方法、***及固态硬盘
US11269547B2 (en) Reusing overwritten portion of write buffer of a storage system
US20180267713A1 (en) Method and apparatus for defining storage infrastructure
JP5547814B2 (ja) 計算機システム、仮想サーバへのボリューム割り当て方法及び計算機読み取り可能な記憶媒体
WO2022262381A1 (zh) 一种数据压缩方法及装置
US11816359B2 (en) Scalable solid-state storage system and methods thereof
WO2017128802A1 (zh) 一种实现存储空间管理的架构及方法
TW202203033A (zh) 儲存元件以及管理儲存元件中命名空間之系統與方法
US10152234B1 (en) Virtual volume virtual desktop infrastructure implementation using a primary storage array lacking data deduplication capability
US11157198B2 (en) Generating merge-friendly sequential IO patterns in shared logger page descriptor tiers
US20140188952A1 (en) Reading data without an indirection logical reference identifier in a system that uses indirection access
US11947803B2 (en) Effective utilization of different drive capacities
US20240028262A1 (en) Scalable Solid-State Storage System and Methods Thereof
US9811259B2 (en) Read I/O completions outside file operating system domain
JP2024525170A (ja) データ圧縮方法及び装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16887674

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16887674

Country of ref document: EP

Kind code of ref document: A1