CN104583930A - Method of data migration, controller and data migration apparatus - Google Patents
Method of data migration, controller and data migration apparatus Download PDFInfo
- Publication number
- CN104583930A CN104583930A CN201480002086.XA CN201480002086A CN104583930A CN 104583930 A CN104583930 A CN 104583930A CN 201480002086 A CN201480002086 A CN 201480002086A CN 104583930 A CN104583930 A CN 104583930A
- Authority
- CN
- China
- Prior art keywords
- failure domain
- level
- level failure
- data block
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a method of data migration, a controller and a data migration apparatus. The method comprises that: the controller counts access frequency of a data block in a first level failure domain, the data block comprising ith target data, wherein i is an integer larger than or equal to zero, and i<N; when the access frequency of the data block reaches up to a preset threshold value, the controller determines a second level failure domain to store the data block in a second level disk set, the determined second level failure domain being free of saving the target data; and the controller migrates the data block to the second level failure domain. Multiple target data are prevented from being migrated to the same failure domain during hierarchical storage operation, and data reliability is ensured.
Description
Technical field
The present invention relates to memory technology, particularly relate to a kind of method of Data Migration, controller and data migration device.
Background technology
Memory device is when storing data, and for the data that importance is higher, the mode usually by storing many parts of data trnascriptions simultaneously ensures the reliability of data.For example, can be stored in different failure domains respectively by many parts of data trnascriptions, when the storage space at the place of a data trnascription damages, other data trnascriptions still can be used.Failure domain refers to the storage area of a certain setting range, and the corrupted data in this region can not have influence on the data in other regions.
Meanwhile, Bedding storage technology has been widely used in memory device.Bedding storage technology refers to, when storage medium in memory device also exists larger speed and price variance, by current accessed frequently data (also known as dsc data) be stored on the storage medium of high speed high price, by current accessed not frequently data (also known as cold data) be stored on the storage medium of low speed low price.
When above two kinds of technology are applied to same memory device, this situation may be there is, such as: in memory device, save two parts of data trnascriptions, wherein, a data trnascription is kept in a failure domain of the storage medium of low speed low price, another part of data trnascription is kept in a failure domain of the storage medium of low speed low price, when the data block at a copy of it data trnascription place becomes dsc data block, need this data block to migrate on the storage medium of high speed high price to improve its access efficiency, just likely this data trnascription and another part of data trnascription are stored in same failure domain, so when the storage space of this failure domain is damaged, data will be lost, reliability can not be protected.
Summary of the invention
Embodiments provide a kind of method of Data Migration, controller and data migration device.Avoid and when performing classification and storing operation, many parts of target datas are migrated in same failure domain, ensure that the reliability of data.
First aspect, embodiments provide a kind of method of Data Migration, described method is performed by memory device, described memory device at least comprises controller, the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain; Described method comprises:
The access frequency of the data block in the described first level failure domain of described controller statistics, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
When the access frequency of described data block reaches predetermined threshold value, the second level failure domain of described data block to be stored determined by described controller in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
In the second level failure domain determined described in described data block moves to by described controller.
In conjunction with first aspect, in the implementation that the first is possible, described controller determines that in described second level disc-pack the second level failure domain of described data block to be stored comprises:
Described controller is according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
In conjunction with the first possible implementation of first aspect or first aspect, in the implementation that the second is possible, described method also comprises:
The corresponding relation between described first level failure domain and described the second level failure domain determined preserved by described controller;
When the access frequency of described data block is lower than described predetermined threshold value, described data block moves to described first level failure domain from described the second level failure domain determined according to described corresponding relation by described controller.
In conjunction with the first of first aspect or first aspect to any one possible implementation of the second, in the implementation that the third is possible, described method also comprises:
Described controller receives described target data, and the rule according to presetting determines that the number that described target data is preserved in described memory device is N;
Described controller copies described target data, generates the described target data of N part.
Second aspect, embodiments provides a kind of controller, and described controller comprises processor and communication interface;
Described communication interface, for communicating with disk array, described disk array comprises the first level disc-pack and the second level disc-pack, and described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described disk array, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain;
Described processor, for adding up the access frequency of the data block in described first level failure domain, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
When the access frequency of described data block reaches predetermined threshold value, the second level failure domain of described data block to be stored determined by described processor in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
In the second level failure domain determined described in described data block moves to by described processor.
In conjunction with second aspect, in the implementation that the first is possible, described processor is specifically for according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
In conjunction with the first possible implementation of second aspect or second aspect, in the implementation that the second is possible, described processor is also for preserving the corresponding relation between described first level failure domain and described the second level failure domain determined;
When the access frequency of described data block is lower than described predetermined threshold value, described processor is also for moving to described data block described first level failure domain from described the second level failure domain determined according to described corresponding relation.
In conjunction with the first of second aspect or second aspect to any one possible implementation of the second, in the implementation that the third is possible, described processor is also for receiving described target data, and the rule according to presetting determines that the number that described target data is preserved in described memory device is N; Described controller copies described target data, generates the described target data of N part.
The third aspect, embodiments provide a kind of data migration device, described data migration device is arranged in the controller of memory device, described memory device at least comprises described controller, the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain; Described data migration device comprises:
Statistical module, for adding up the access frequency of the data block in described first level failure domain, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
Determination module, for when the access frequency of described data block reaches predetermined threshold value, determines the second level failure domain of described data block to be stored in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
Transferring module, in the second level failure domain of determining described in described data block being moved to.
In conjunction with the third aspect, in the implementation that the first is possible, described determination module is specifically for according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
In conjunction with the first possible implementation of the third aspect or the third aspect, in the implementation that the second is possible, also comprise:
Preserve module, for preserving the corresponding relation between described first level failure domain and described the second level failure domain determined;
Described transferring module, also for described data block being moved to described first level failure domain from described the second level failure domain determined according to described corresponding relation.
In conjunction with the first of the third aspect or the third aspect to any one possible implementation of the second, in the implementation that the third is possible, also comprise: receiver module, for receiving described target data, the rule according to presetting determines that the number that described target data is preserved in described memory device is N;
Replication module, for copying described target data, generates the described target data of N part.
Embodiments provide a kind of method of Data Migration, controller and data migration device, add up the access frequency of the data block in described first level failure domain, described data block comprises a copy of it in many parts of target datas, when the access frequency of the data block in the first level failure domain reaches predetermined threshold value, described data block is migrated to the second level failure domain, and described second level failure domain does not preserve described target data, avoiding problems and when performing classification and storing operation, many parts of target datas are migrated in same failure domain, ensure that the reliability of data.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the application scenarios figure provided according to the embodiment of the present invention;
Fig. 2 is the structural representation of the memory device provided according to the embodiment of the present invention;
Fig. 3 is the exemplary plot of the disk array provided according to the embodiment of the present invention;
Fig. 4 is the failure domain distribution schematic diagram in the disk array provided according to the embodiment of the present invention;
Fig. 5 is the schematic flow sheet of the method for the Data Migration provided according to the embodiment of the present invention;
Fig. 6 is the structural representation of the data migration device provided according to the embodiment of the present invention.
Embodiment
For making the object of the embodiment of the present invention, technical scheme and advantage clearly, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
The system architecture of the embodiment of the present invention
As shown in Figure 1, Fig. 1 is a kind of application scenarios figure of the embodiment of the present invention, and in a kind of application scenarios described in Fig. 1, this storage system comprises main frame 100, connection device 105 and memory device 110.
Main frame 100 can comprise the known any computing equipment of current techniques, and as server, desk-top computer, application server etc., in main frame 100, be provided with operating system and other application programs, main frame 100 can have multiple.
Connection device 105 can comprise any interface between the known memory device of current techniques and main frame, as optical fiber switch, or other existing switches.
Memory device 110 can comprise the known memory device of current techniques, as disk array, disk cluster (Just a Bunch Of Disks, JBOD), direct access storage device (Direct Access StorageDevice, the disc driver of one or more interconnection DASD), wherein, direct access storage device can comprise the fc tape storage device FC of such as tape library, one or more storage unit.
The structural representation of a kind of memory device 110 that Fig. 2 provides for the embodiment of the present invention, the memory device shown in Fig. 2 is a kind of disk array.As shown in Figure 2, memory device 110 can comprise controller 115 and redundant array of inexpensive disks 1 25, wherein, here disk array refers to Redundant Array of Independent Disks (RAID) (Redundant Arrays of Independent Disks, RAID), redundant array of inexpensive disks 1 25 can have multiple, and redundant array of inexpensive disks 1 25 is made up of multiple disk 130.The communication protocols such as small computer system interface (Small Computer System Interface, SCSI) agreement can be passed through between redundant array of inexpensive disks 1 25 and controller 115 communicate to connect, not limit at this.
Controller 115 is memory devices 110 " brain ", mainly comprises processor (processor) 118, buffer memory (cache) 120, storer (memory) 122, communication bus (abbreviation bus) 126 and communication interface (Communication Interface) 128.Processor 118, buffer memory 120, storer 122 and communication interface 128 complete mutual communication by communication bus 126.
Communication interface 128, for communicating with main frame 100 or redundant array of inexpensive disks 1 25.
Storer 122, for depositing program 124, storer 124 may comprise high-speed RAM storer, still may comprise nonvolatile memory (non-volatile memory), such as at least one magnetic disk memory.Be understandable that, storer 124 can be random access memory (Random-AccessMemory, RAM), magnetic disc, hard disk, CD, solid state hard disc (Solid State Disk, SSD) or nonvolatile memory etc. various can be program code stored (non-transitory) machine readable media of non-transience.
Program 124 can comprise program code, and described program code comprises computer-managed instruction.
The data that buffer memory 120 (Cache) receives from main frame 100 for buffer memory or the data read from redundant array of inexpensive disks 1 25, to improve the Performance And Reliability of array.Buffer memory 120 can be RAM, ROM, flash memory (Flash memory) or solid state hard disc (Solid State Disk, various (non-transitory) machine readable media that can store the non-transience of data such as SSD), does not limit at this.
Processor 118 may be a central processor CPU, or specific integrated circuit ASIC (Application Specific Integrated Circuit), or is configured to the one or more integrated circuit implementing the embodiment of the present invention.In processor 118, be provided with operating system and other software programs, different software programs can regard as a processing module, has different functions.Such as, process and the I/O (Input/Output, I/O) of disk 130 is asked, other process etc. are carried out to the data in disk.Thus make that controller 115 can realize I/O operation, data trnascription copies and the various data management function such as Bedding storage.In embodiments of the present invention, processor 118, for executive routine 124, specifically can perform the correlation step in following embodiment of the method.
Redundant array of inexpensive disks 1 25 is for storing data, in embodiments of the present invention, redundant array of inexpensive disks 1 25 can comprise polytype disk 130, such as, solid state hard disc (Solid State Drive, or Serial Attached SCSI (SAS) (Serial Attached SCSI SSD), or optical-fibre channel (Fiber Channel SAS), FC) hard disk drive (Hard Disk Drive, HDD), wherein, the english abbreviation that SCSI (Small Computer SystemInterface) is attached small computer system interface or Serial Advanced Technology Attachment (SerialAdvanced Technology Attachment, or near line (Near Line SATA), NL) Serial Attached SCSI (SAS) (Serial Attached SCSI, SAS) HDD, do not limit at this.For convenience, follow-up by solid state hard disc referred to as SSD, SAS HDD, FC HDD, SATA HDD is unified referred to as HDD.
Fig. 3 is the exemplary plot of redundant array of inexpensive disks 1 25, and as shown in Figure 3, redundant array of inexpensive disks 1 25 can comprise multiple disk 130, wherein, multiple disk 130 can be divided into high-speed equipment layer and low-speed device layer, and high-speed equipment layer can comprise the disk 130 of multiple high speed high price, such as SSD; Low-speed device layer can comprise the disk 130 of multiple low speed low price, such as HDD.Be understandable that, multiple disk 130 can also be divided into other levels according to the type of disk by the embodiment of the present invention, only illustrates with high-speed equipment layer, low-speed device layer two level here.
In addition, the storage space of redundant array of inexpensive disks 1 25 can also be divided into multiple failure domain.Failure domain refers to the logical storage of a segment fault isolation, and the corrupted data in this region can not have influence on the data in other regions.In embodiments of the present invention, failure domain is by memory device 110 sets itself, and a failure domain can be a RAID, and also can be a disk, can also be the storage area according to other model split, the embodiment of the present invention limit.For convenience, the embodiment of the present invention is that a disk is described for a failure domain, as shown in Figure 4:
Redundant array of inexpensive disks 1 25 comprises high-speed equipment layer and low-speed device layer, high-speed equipment layer can comprise multiple SSD (for a SSD in figure, but be not limited to a SSD), low-speed device layer can comprise multiple HDD (for two HDD in figure, but being not limited to two HDD).A SSD can be a failure domain, and two HDD are respectively a failure domain.
In the embodiment of the present invention, need to be numbered failure domain, concrete coding rule can be, in each mechanical floor inside, the failure domain that it comprises is numbered, such as, suppose that high-speed equipment layer comprises three failure domains, so, for high-speed equipment layer, the numbering of each failure domain is failure domain 0, failure domain 1, failure domain 2 respectively; Suppose that low-speed device layer comprises four failure domains, so, for low-speed device layer, the numbering of each failure domain is failure domain 0, failure domain 1, failure domain 2, failure domain 3 respectively.In embodiments of the present invention, the numbering of failure domain numbered referred to as failure domain, failure domain numbering refers to the numbering for distinguishing each failure domain, such as 0,1 etc.Be understandable that, failure domain numbering can also adopt other forms of expression such as letter, and the embodiment of the present invention does not limit.
Memory device 110 is when storing data, and for the data that importance is higher, such as metadata, the mode usually by storing many parts of data trnascriptions simultaneously ensures the reliability of data.Such as, as shown in Figure 4, saving two parts of data trnascriptions in low-speed device layer, is copy 0 and copy 1 respectively.Wherein, copy 0 is kept in failure domain 0, and copy 1 is kept in failure domain 1.There is the faults such as disk failures in assumed fault territory 0, can also obtain data trnascription from failure domain 1, is unlikely to loss of data occurs.It will be appreciated by persons skilled in the art that be no matter failure domain 0 or failure domain 1 except preserving above-described data trnascription, other data can also be preserved.In embodiments of the present invention, copy numbering refers to the numbering for distinguishing each data trnascription, such as 0,1.Be understandable that, copy numbering can also adopt other forms of expression such as letter, and the embodiment of the present invention does not limit.
Composition graphs 1, any one memory device 110 all can receive the data write request of one or more main frame 100 transmission by connection device 105, determine that data that described data write request carries by the mechanical floor that is written into (such as according to the Bedding storage strategy preset, low-speed device layer), determine that these data need backup and the data trnascription number stored according to the data importance judgment rule preset simultaneously.Then by the different failure domain of each data trnascription write low-speed device layer.Any one memory device 110 also can receive the data read request of one or more main frame 100 transmission by connection device 105, from the address that described data read request is carried, read data.
In addition, any one memory device 110 in the embodiment of the present invention can also support Bedding storage technology.Be explained below in conjunction with Fig. 4.Fig. 4 is the failure domain distribution schematic diagram in the disk array provided according to the embodiment of the present invention.
The storage space of redundant array of inexpensive disks 1 25 can be divided into several data blocks (not shown in Fig. 4), and data block refers to the access frequency integrally can monitoring wherein data, and the data cell of integrally moving.A data block may be arranged in a disk, also may be arranged in multiple disk, but in embodiments of the present invention, a data block must be arranged in a failure domain.Be understandable that, when a data block is arranged in multiple disk, the failure domain at described data block place just refers to a RAID.The access frequency of each data block in redundant array of inexpensive disks 1 25 monitored by memory device 110, for example, when the access frequency of the data block at discovery copy 0 place is higher than predetermined threshold value, can determine that the data in this data block become dsc data, therefore according to the incidence relation preset, this data block can will be migrated to the failure domain of high-speed equipment layer, such as, in the failure domain 0 of high-speed equipment layer.It should be noted that, the statistics of access frequency is in units of a data block, and the capacity of data block is greater than the size of a data trnascription usually, and this data block can also comprise other data except comprising copy 0.Therefore, the data block heating at copy 0 place may be that copy 0 is accessed frequently and causes, and also may be that other data in this data block except copy 0 are accessed frequently and cause.When the data block heating at copy 0 place be by copy 0 be accessed frequently cause time, so the data block at copy 1 place also can heating, needs to be migrated in the failure domain of high-speed equipment layer; When the data block heating at copy 0 place be by this data block in other data except copy 0 be accessed frequently cause time, so the data block at copy 1 place may heating, does not need to be migrated in high-speed equipment layer.Copy 0 is temporarily only discussed here and is migrated to the situation of the failure domain 0 of high-speed equipment layer.
Memory device 110 continues to monitor the access frequency of each data block in redundant array of inexpensive disks 1 25, when finding that the data block of having moved to the failure domain 0 of high speed mechanical floor turns cold again, needs the data block at copy 0 place to move back to again in low-speed device layer.In order to prevent copy 0 and copy 1 to be stored in same failure domain, the failure domain of low-speed device layer will be able to moved back to according to the data block at incidence relation determination copy 0 place of presetting.
The method of Data Migration
Introduce the method for the Data Migration that the embodiment of the present invention provides below, as shown in Figure 5, the process flow diagram of the method for the Data Migration provided for the embodiment of the present invention, described method can be applied in the memory device shown in the storage system shown in Fig. 1 and Fig. 2; Described memory device at least comprises the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, described second level disc-pack comprises at least two the second level failure domains, wherein, described first level failure domain refers to the logical storage of the segment fault isolation in described first level disc-pack, and described second level failure domain refers to the logical storage of the segment fault isolation in described second level disc-pack; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain.It should be noted that, the first level disc-pack in the present embodiment is also the low-speed device layer shown in Fig. 3, and the second level disc-pack in the present embodiment is also the high-speed equipment layer shown in Fig. 3.So, the first level failure domain is the failure domain that low-speed device layer comprises, and may be a HDD, also may be a RAID, or the logical storage of other fault isolation; Second level failure domain is the failure domain that high-speed equipment layer comprises, and may be a SSD, also may be a RAID, or the logical storage of other fault isolation.
Described method comprises the steps:
Step S201: the access frequency of the data block in described first level failure domain added up by processor 118, and described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N.
In the present embodiment, N part target data is saved in memory device altogether, namely previously described data trnascription.And need to be numbered N part target data, such as the 0th part of target data (copy 0), the 1st part of target data (copy 1), the 2nd part of target data (copy 2) ... N-1 part target data (copy N-1).Wherein, i-th part of target data is kept in a data block in the first level failure domain.
Processor 118 regularly can add up the access frequency of each data block in the first level failure domain, when the access frequency of the data block finding described i-th part of target data place reaches predetermined threshold value, performs step S202.It should be noted that, for the access frequency of each data block of statistics, can be triggered by timer, also can be manual activation, or other modes trigger, the present embodiment limit this.
Step S202: when the access frequency of described data block reaches predetermined threshold value, the second level failure domain of described data block to be stored determined by processor 118 in described second level disc-pack, described in the second level failure domain of determining do not preserve a copy of it of described N part target data.
When the access frequency of the described data block that processor in step S201 118 counts reaches predetermined threshold value, illustrate that the data comprised in this data block become dsc data, in order to improve access efficiency, can be migrated in the second level disc-pack (such as, multiple SSD).
Further, owing to comprising multiple second level failure domain in the second level disc-pack, processor 118 needs to determine described data block to be migrated in which failure domain in multiple second level failure domain.
A kind of embodiment judges whether each the second level failure domain in the second level disc-pack preserves target data successively, if do not have, then this second level failure domain can be defined as the second level failure domain of described data block to be stored; If have, then continue to judge whether next second level failure domain preserves target data, until find the second level failure domain not preserving described target data.It should be noted that, for this embodiment, after determining the second level failure domain of described data block to be stored, the corresponding relation between the first level failure domain that described second level failure domain and described data block move out can be preserved.When the access frequency of described data block is lower than predetermined threshold value, illustrate that described data block turns cold, according to the corresponding relation preserved, described data block can be moved back to the first original level failure domain.
Another kind of embodiment is, utilizes the incidence relation of setting to determine the second level failure domain of described data block to be stored.The incidence relation of described setting refers to that described target data is kept at the relation between the numbering " i " of the target data of preserving in the number " N " in memory device 110, described data block and the numbering of the second level failure domain.Specifically, the product that the numbering of described second level failure domain equals a described N and random number adds described i, thus obtain and, wherein, the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.Described incidence relation will be illustrated below:
Suppose N=3, i=0, that is, described target data is kept in memory device 110 and saves 3 parts, and the numbering of every part of target data is 0,1,2 respectively.Wherein, the target data of preserving in described data block is the 0th part of target data.So, the numbering of the second level failure domain of described data block to be stored adds 0 with regard to equaling 3 with the product of random number, thus obtain and.When random number gets 0, the second level failure domain of described data block to be stored be numbered 0; When random number gets 1, the second level failure domain of described data block to be stored be numbered 3; When random number gets 2, the second level failure domain of described data block to be stored be numbered 6 ... but, random number can not more than the number of the second level failure domain divided by described N thus obtain business.Suppose that the number of the second level failure domain is 7, so 7 is 2 divided by the business of 3 gained, and therefore random number must be less than or equal to 2.
Therefore, for the 0th part of target data, can migrate to and be numbered in the second level failure domain of 0 or 3 or 6.
Optionally, for this embodiment, after the second level failure domain determining described data block to be stored, the corresponding relation between the first level failure domain that described second level failure domain and described data block move out can not be preserved.When the access frequency of described data block is lower than predetermined threshold value (described data block turns cold), according to incidence relation similar above, described data block can be moved back to in a first level failure domain of the first level disc-pack.Now, the product that the numbering of the first level failure domain equals a described N and random number adds described i, thus obtain and, wherein, the number that described random number is no more than the first level failure domain that described memory device comprises is divided by described N thus the business obtained.Such as, the numbering of described target data is constant, remain the 0th part of target data, the number that described target data is kept in memory device 110 remains 3, i.e. i=0, N=3, the numbering of the first level failure domain so obtained according to described incidence relation can be 0 or 3 or 6 (supposing also to comprise 7 first level failure domains in the first level disc-pack).That is, when described data block needs to move back to the first level disc-pack, can migrate to be numbered 0 the first level failure domain or be numbered the first level failure domain of 3 or be numbered the first level failure domain of 0.
Optionally, for above-mentioned embodiment also can described determine the second level failure domain of described data block to be stored after, preserve the corresponding relation between the first level failure domain that described second level failure domain and described data block move out.When the access frequency of described data block is lower than predetermined threshold value, according to the corresponding relation preserved, described data block can be moved back to the first original level failure domain.Be understandable that, described data block can only be moved back to the first original level failure domain by the corresponding relation according to preserving, and compared with the first level failure domain redefined with utilizing incidence relation, its scope is narrower.
Step S203: in the second level failure domain determined described in described data block moves to by processor 118.
According to the example of step S202, described data block can migrate to and be numbered in the second level failure domain of 0 or 3 or 6 by processor 118.
When failure domain is a disk, the second level failure domain being numbered 0 or 3 or 6 can be the disk being numbered 0 or 3 or 6 respectively; When failure domain is a RAID, the the second level failure domain being numbered 0 or 3 or 6 can be the RAID being numbered 0 or 3 or 6 respectively, now, described data block is moved to described in can be specifically that described data block striping is write in described RAID in the second level failure domain of determining.
The access frequency of the data block in the described first level failure domain of embodiment of the present invention statistics, described data block comprises a copy of it in many parts of target datas, when the access frequency of the data block in the first level failure domain reaches predetermined threshold value, described data block is migrated to the second level failure domain, and described second level failure domain does not preserve described target data, avoiding problems and when performing classification and storing operation, many parts of target datas are migrated in same failure domain, ensure that the reliability of data.
In the above-described embodiments, can also comprise:
Step 204: processor 118 receives described target data, the rule according to presetting determines that the number that described target data is preserved in described memory device is N; Described target data is copied, generates the described target data of N part.
Processor 118 can Receiving Host 100 send data write request, carry described target data in described data write request.Here the rule preset refers to the corresponding relation between the number that the importance of described target data and described target data need to preserve.Such as, if described target data is business datum, so its importance is not high, can preserve 2 parts in memory device 110; If described target data is the metadata of business datum, so its importance is higher, can preserve 3 parts in memory device 110, etc.
After determining the number N that described target data is preserved in described memory device, memory device 110 can copy described target data, generates N part target data.
The device of the embodiment of the present invention
The embodiment of the present invention provides a kind of data migration device, described data migration device is arranged in the controller of memory device, described memory device at least comprises described controller, the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain; As shown in Figure 6, described data migration device comprises:
Statistical module 401, for adding up the access frequency of the data block in described first level failure domain, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
Determination module 402, for when the access frequency of described data block reaches predetermined threshold value, in described second level disc-pack, determine the second level failure domain of described data block to be stored, described in the second level failure domain of determining do not preserve described target data;
Transferring module 403, in the second level failure domain of determining described in described data block being moved to.
Optionally, described determination module 402 specifically for:
According to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
Optionally, described data migration device also comprises:
Preserve module 404, for preserving the corresponding relation between described first level failure domain and described the second level failure domain determined;
Accordingly, described transferring module 403, also for described data block being moved to described first level failure domain from described the second level failure domain determined according to described corresponding relation.
Optionally, described data migration device also comprises:
Receiver module 405, for receiving described target data, the rule according to presetting determines that the number that described target data is preserved in described memory device is N;
Replication module 406, for copying described target data, generates the described target data of N part.
The device that the embodiment of the present invention provides, can be arranged in the controller described by preceding embodiment, for performing the date storage method described by preceding embodiment, the detailed description of modules function see the description in embodiment of the method, can not repeat them here.
One of ordinary skill in the art will appreciate that the possible implementation of various aspects of the present invention or various aspects can be embodied as system, method or computer program.Therefore, the possible implementation of each aspect of the present invention or various aspects can adopt complete hardware embodiment, completely software implementation (comprising firmware, resident software etc.), or the form of the embodiment of integration software and hardware aspect, is all referred to as " circuit ", " module " or " system " here.In addition, the possible implementation of each aspect of the present invention or various aspects can adopt the form of computer program, and computer program refers to the computer readable program code be stored in computer-readable medium.
Computer-readable medium can be computer-readable signal media or computer-readable recording medium.Computer-readable recording medium is including but not limited to electronics, magnetic, optics, electromagnetism, infrared or semiconductor system, equipment or device, or aforesaid appropriately combined arbitrarily, as random access memory
(RAM), ROM (read-only memory) (ROM), Erasable Programmable Read Only Memory EPROM (EPROM or flash memory), optical fiber, portable ROM (read-only memory) (CD-ROM).
Processor in computing machine reads the computer readable program code be stored in computer-readable medium, makes processor can perform the function action specified in the combination of each step or each step in flow charts; Generate the device implementing the function action specified in the combination of each block of block diagram or each piece.
Computer readable program code can perform completely on the computing machine of user, part performs on the computing machine of user, as independent software package, part on the computing machine of user and part on the remote computer, or to perform on remote computer or server completely.Also it should be noted that in some alternate embodiment, in flow charts in each step or block diagram each piece the function that indicates may not according to occurring in sequence of indicating in figure.Such as, depend on involved function, in fact two steps illustrated in succession or two blocks may be executed substantially concurrently, or these blocks sometimes may be performed by with reverse order.
Claims (12)
1. the method for a Data Migration, it is characterized in that, described method is performed by memory device, described memory device at least comprises controller, the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain; Described method comprises:
The access frequency of the data block in the described first level failure domain of described controller statistics, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
When the access frequency of described data block reaches predetermined threshold value, the second level failure domain of described data block to be stored determined by described controller in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
In the second level failure domain determined described in described data block moves to by described controller.
2. method according to claim 1, is characterized in that, described controller determines that in described second level disc-pack the second level failure domain of described data block to be stored comprises:
Described controller is according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
3. method according to claim 1 and 2, is characterized in that, also comprises:
The corresponding relation between described first level failure domain and described the second level failure domain determined preserved by described controller;
When the access frequency of described data block is lower than described predetermined threshold value, described data block moves to described first level failure domain from described the second level failure domain determined according to described corresponding relation by described controller.
4. the method according to the arbitrary claim of claim 1-3, is characterized in that, also comprise:
Described controller receives described target data, and the rule according to presetting determines that the number that described target data is preserved in described memory device is N;
Described controller copies described target data, generates the described target data of N part.
5. a controller, is characterized in that, described controller comprises processor and communication interface;
Described communication interface, for communicating with disk array, described disk array comprises the first level disc-pack and the second level disc-pack, and described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described disk array, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain;
Described processor, for adding up the access frequency of the data block in described first level failure domain, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
When the access frequency of described data block reaches predetermined threshold value, the second level failure domain of described data block to be stored determined by described processor in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
In the second level failure domain determined described in described data block moves to by described processor.
6. controller according to claim 5, it is characterized in that, described processor is specifically for according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
7. the controller according to claim 5 or 6, is characterized in that,
Described processor is also for preserving the corresponding relation between described first level failure domain and described the second level failure domain determined;
When the access frequency of described data block is lower than described predetermined threshold value, described processor is also for moving to described data block described first level failure domain from described the second level failure domain determined according to described corresponding relation.
8. the controller according to the arbitrary claim of claim 5-7, is characterized in that,
Described processor is also for receiving described target data, and the rule according to presetting determines that the number that described target data is preserved in described memory device is N; Described controller copies described target data, generates the described target data of N part.
9. a data migration device, it is characterized in that, described data migration device is arranged in the controller of memory device, described memory device at least comprises described controller, the first level disc-pack and the second level disc-pack, described first level disc-pack comprises the first level failure domain, and described second level disc-pack comprises at least two the second level failure domains; Preserve N part target data in described memory device, N be greater than 1 positive integer, described N is less than the number of the first level failure domain, and described N is less than the number of the second level failure domain; Described data migration device comprises:
Statistical module, for adding up the access frequency of the data block in described first level failure domain, described data block comprises i-th part of target data, wherein i be more than or equal to zero integer, and, i<N;
Determination module, for when the access frequency of described data block reaches predetermined threshold value, determines the second level failure domain of described data block to be stored in described second level disc-pack, described in the second level failure domain of determining do not preserve described target data;
Transferring module, in the second level failure domain of determining described in described data block being moved to.
10. device according to claim 9, it is characterized in that, described determination module is specifically for according to described i and N, and the incidence relation preset, determine the numbering of described second level failure domain, described incidence relation refers to that the product that the numbering of described second level failure domain equals described N and random number adds described i, thus obtain and, wherein, described random number be more than or equal to 1 positive integer, and the number that described random number is no more than the second level failure domain that described memory device comprises is divided by described N thus the business obtained.
11. devices according to claim 9, is characterized in that, also comprise:
Preserve module, for preserving the corresponding relation between described first level failure domain and described the second level failure domain determined;
Described transferring module, also for described data block being moved to described first level failure domain from described the second level failure domain determined according to described corresponding relation.
12. devices according to the arbitrary claim of claim 9-11, is characterized in that, also comprise:
Receiver module, for receiving described target data, the rule according to presetting determines that the number that described target data is preserved in described memory device is N;
Replication module, for copying described target data, generates the described target data of N part.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2014/084526 WO2016023230A1 (en) | 2014-08-15 | 2014-08-15 | Data migration method, controller and data migration device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104583930A true CN104583930A (en) | 2015-04-29 |
| CN104583930B CN104583930B (en) | 2017-09-08 |
Family
ID=53097339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480002086.XA Active CN104583930B (en) | 2014-08-15 | 2014-08-15 | Method, controller and the data migration device of Data Migration |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN104583930B (en) |
| WO (1) | WO2016023230A1 (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107085546A (en) * | 2016-02-16 | 2017-08-22 | 深圳市深信服电子科技有限公司 | Data managing method and device based on failure field technique |
| CN107704212A (en) * | 2017-10-31 | 2018-02-16 | 紫光华山信息技术有限公司 | A kind of data processing method and device |
| CN108255413A (en) * | 2016-12-29 | 2018-07-06 | 华为技术有限公司 | Method, the storage system of Data Migration after a kind of storage system dilatation |
| CN108306748A (en) * | 2017-01-12 | 2018-07-20 | 阿里巴巴集团控股有限公司 | Network failure locating method, device and interactive device |
| CN110321067A (en) * | 2018-03-30 | 2019-10-11 | 卡巴斯基实验室股份制公司 | The system and method that estimation and management storage equipment are degenerated |
| CN110737924A (en) * | 2018-07-20 | 2020-01-31 | 中移(苏州)软件技术有限公司 | method and equipment for data protection |
| CN111104056A (en) * | 2018-10-25 | 2020-05-05 | 华为技术有限公司 | Data recovery method, system and device in storage system |
| CN112068976A (en) * | 2019-06-10 | 2020-12-11 | 北京京东尚科信息技术有限公司 | Data backup storage method and device, electronic equipment and storage medium |
| CN114466030A (en) * | 2021-12-27 | 2022-05-10 | 天翼云科技有限公司 | Management method and device for data distribution storage strategy and distribution storage system |
| CN114675791A (en) * | 2022-05-27 | 2022-06-28 | 苏州浪潮智能科技有限公司 | Disk processing method and system and electronic equipment |
| CN116027990A (en) * | 2023-03-29 | 2023-04-28 | 苏州浪潮智能科技有限公司 | RAID card, data access method, system and storage medium thereof |
| CN116910016A (en) * | 2023-09-14 | 2023-10-20 | 交通运输部北海航海保障中心天津通信中心 | AIS data processing method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070208788A1 (en) * | 2006-03-01 | 2007-09-06 | Quantum Corporation | Data storage system including unique block pool manager and applications in tiered storage |
| CN101223498A (en) * | 2005-07-15 | 2008-07-16 | 国际商业机器公司 | Virtualisation engine and method, system, and computer program product for managing the storage of data |
| CN103106151A (en) * | 2011-11-15 | 2013-05-15 | Lsi公司 | Apparatus to manage efficient data migration between tiers |
| CN103914516A (en) * | 2014-02-25 | 2014-07-09 | 深圳市中博科创信息技术有限公司 | Method and system for layer-management of storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102156738B (en) * | 2011-04-13 | 2012-12-19 | 成都市华为赛门铁克科技有限公司 | Method for processing data blocks, and data block storage equipment and system |
-
2014
- 2014-08-15 CN CN201480002086.XA patent/CN104583930B/en active Active
- 2014-08-15 WO PCT/CN2014/084526 patent/WO2016023230A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101223498A (en) * | 2005-07-15 | 2008-07-16 | 国际商业机器公司 | Virtualisation engine and method, system, and computer program product for managing the storage of data |
| US20070208788A1 (en) * | 2006-03-01 | 2007-09-06 | Quantum Corporation | Data storage system including unique block pool manager and applications in tiered storage |
| CN103106151A (en) * | 2011-11-15 | 2013-05-15 | Lsi公司 | Apparatus to manage efficient data migration between tiers |
| CN103914516A (en) * | 2014-02-25 | 2014-07-09 | 深圳市中博科创信息技术有限公司 | Method and system for layer-management of storage system |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107085546A (en) * | 2016-02-16 | 2017-08-22 | 深圳市深信服电子科技有限公司 | Data managing method and device based on failure field technique |
| CN107085546B (en) * | 2016-02-16 | 2020-05-01 | 深信服科技股份有限公司 | Data management method and device based on fault domain technology |
| US10908833B2 (en) | 2016-12-29 | 2021-02-02 | Huawei Technologies Co., Ltd. | Data migration method for a storage system after expansion and storage system |
| CN108255413A (en) * | 2016-12-29 | 2018-07-06 | 华为技术有限公司 | Method, the storage system of Data Migration after a kind of storage system dilatation |
| CN108306748A (en) * | 2017-01-12 | 2018-07-20 | 阿里巴巴集团控股有限公司 | Network failure locating method, device and interactive device |
| CN108306748B (en) * | 2017-01-12 | 2021-03-30 | 阿里巴巴集团控股有限公司 | Network fault positioning method and device and interaction device |
| CN107704212A (en) * | 2017-10-31 | 2018-02-16 | 紫光华山信息技术有限公司 | A kind of data processing method and device |
| CN110321067A (en) * | 2018-03-30 | 2019-10-11 | 卡巴斯基实验室股份制公司 | The system and method that estimation and management storage equipment are degenerated |
| CN110321067B (en) * | 2018-03-30 | 2023-04-14 | 卡巴斯基实验室股份制公司 | System and method for estimating and managing storage device degradation |
| CN110737924A (en) * | 2018-07-20 | 2020-01-31 | 中移(苏州)软件技术有限公司 | method and equipment for data protection |
| CN110737924B (en) * | 2018-07-20 | 2021-07-27 | 中移(苏州)软件技术有限公司 | Data protection method and equipment |
| CN111552435A (en) * | 2018-10-25 | 2020-08-18 | 华为技术有限公司 | Data recovery method, system and device in storage system |
| CN111104056B (en) * | 2018-10-25 | 2021-12-31 | 华为技术有限公司 | Data recovery method, system and device in storage system |
| CN111104056A (en) * | 2018-10-25 | 2020-05-05 | 华为技术有限公司 | Data recovery method, system and device in storage system |
| CN112068976A (en) * | 2019-06-10 | 2020-12-11 | 北京京东尚科信息技术有限公司 | Data backup storage method and device, electronic equipment and storage medium |
| CN114466030A (en) * | 2021-12-27 | 2022-05-10 | 天翼云科技有限公司 | Management method and device for data distribution storage strategy and distribution storage system |
| CN114466030B (en) * | 2021-12-27 | 2024-03-12 | 天翼云科技有限公司 | Management method and device of data distributed storage strategy and distributed storage system |
| CN114675791A (en) * | 2022-05-27 | 2022-06-28 | 苏州浪潮智能科技有限公司 | Disk processing method and system and electronic equipment |
| CN114675791B (en) * | 2022-05-27 | 2022-10-28 | 苏州浪潮智能科技有限公司 | Disk processing method and system and electronic equipment |
| CN116027990A (en) * | 2023-03-29 | 2023-04-28 | 苏州浪潮智能科技有限公司 | RAID card, data access method, system and storage medium thereof |
| CN116910016A (en) * | 2023-09-14 | 2023-10-20 | 交通运输部北海航海保障中心天津通信中心 | AIS data processing method |
| CN116910016B (en) * | 2023-09-14 | 2024-06-11 | 交通运输部北海航海保障中心天津通信中心 | AIS data processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104583930B (en) | 2017-09-08 |
| WO2016023230A1 (en) | 2016-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104583930A (en) | Method of data migration, controller and data migration apparatus | |
| US10001947B1 (en) | Systems, methods and devices for performing efficient patrol read operations in a storage system | |
| US10346241B2 (en) | Preemptive relocation of failing data | |
| US9996435B2 (en) | Reliability scheme using hybrid SSD/HDD replication with log structured management | |
| US7975168B2 (en) | Storage system executing parallel correction write | |
| US10019362B1 (en) | Systems, devices and methods using solid state devices as a caching medium with adaptive striping and mirroring regions | |
| US20170139605A1 (en) | Control device and control method | |
| US9529674B2 (en) | Storage device management of unrecoverable logical block addresses for RAID data regeneration | |
| US20130103902A1 (en) | Method and apparatus for implementing protection of redundant array of independent disks in file system | |
| US20170123699A1 (en) | Storage control device | |
| US9836223B2 (en) | Changing storage volume ownership using cache memory | |
| US9563524B2 (en) | Multi level data recovery in storage disk arrays | |
| US10365845B1 (en) | Mapped raid restripe for improved drive utilization | |
| US10095585B1 (en) | Rebuilding data on flash memory in response to a storage device failure regardless of the type of storage device that fails | |
| US20170308321A1 (en) | Storage apparatus and storage system | |
| CN103534688A (en) | Data recovery method, storage equipment and storage system | |
| US10324794B2 (en) | Method for storage management and storage device | |
| US10579540B2 (en) | Raid data migration through stripe swapping | |
| US9378092B2 (en) | Storage control apparatus and storage control method | |
| US10621059B2 (en) | Site recovery solution in a multi-tier storage environment | |
| KR20210076834A (en) | Enhanced ssd reliability | |
| US20140297988A1 (en) | Storage device, allocation release control method | |
| US9323630B2 (en) | Enhanced data recovery from data storage devices | |
| US20150067285A1 (en) | Storage control apparatus, control method, and computer-readable storage medium | |
| US20140372672A1 (en) | System and method for providing improved system performance by moving pinned data to open nand flash interface working group modules while the system is in a running state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |