CN108710474A - A kind of date storage method, device and computer storage media - Google Patents
A kind of date storage method, device and computer storage media Download PDFInfo
- Publication number
- CN108710474A CN108710474A CN201810481557.XA CN201810481557A CN108710474A CN 108710474 A CN108710474 A CN 108710474A CN 201810481557 A CN201810481557 A CN 201810481557A CN 108710474 A CN108710474 A CN 108710474A
- Authority
- CN
- China
- Prior art keywords
- data
- storage device
- read
- write
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/068—Hybrid storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of date storage method, device and computer storage media, including：Logical device receives input and output (IO) request of data；Reading and writing data is carried out according to the I/O Request received；Wherein, the logical device is made of two or more storage medium including the first storage device and the second storage device；The read and write rate of first storage device is more than the read and write rate of second storage device.The embodiment of the present invention carries out the storage of data by the first storage device and the second storage device, improves the application efficiency of storage resource.
Description
Technical field
Present document relates to but be not limited to computer technology, espespecially a kind of date storage method, device and computer storage media.
Background technology
With the development of electronic information technology, the processing speed of central processing unit (CPU) and memory is getting faster；Mechanical magnetic
Component part of the disk as computer, rate is promoted has larger gap compared with the former.Solid state disk (SSD) is compared to mechanical hard disk
The time (time that i.e. magnetic head navigates to magnetic track and disc is waited for turn to correct position) is not addressed, addressing speed carries significantly
Height, especially under the more application environment of random read-write, performance is substantially improved；But the unit of solid state disk stores price phase
For mechanical hard disk cost higher, how mechanical hard disk and solid state disk is efficiently balancedly applied, be manufacturer server and user
All the problem of paying close attention to.
Currently, the storage product of some enterprise-levels reads and writes the action message of data by statistics, by the data information of read-write
Different accumulation layers is written according to action message；Such as the data of read-write are divided by dsc data and cold number by action message
According to；Dsc data is migrated to the highest accumulation layer of performance, cold data is migrated to the high accumulation layer of at low cost, capacity；This
According to the method that action message writes data into different accumulation layers, major function needs to be integrated in inside storage device, needs to store
The support of equipment firmware, it is expensive, and it is not suitable for server product.
Fig. 1 is the relevant technologies kernel composition schematic diagram, as shown in Figure 1, in linux kernel version, device map
(Device Mapper) layer is located on device drives, and after input/output (IO) data reach DM layers, DM layers based on mapping
Target device in table and target type, are filtered it and remap, and relay to corresponding device drives, final to be written
The physical disk that bottom is made of SSD or mechanical hard disk；If the data accessed are located at mechanical hard disk, need to pass through mechanical arm
After navigating to corresponding magnetic track progress addressing, then disc is waited for turn to corresponding position, data could be operated, be needed
Long period；If the data accessed are located at SSD, addressing can not had to, it can faster complete data manipulation；It accesses and is located at not
With the data of medium, there are larger performance differences, are become apparent in the data of random access large amount of small documents.
To sum up, how reasonable application memory resource carries out the storage of data as problem to be solved.
Invention content
It is the general introduction of the theme to being described in detail herein below.This general introduction is not to limit the protection model of claim
It encloses.
A kind of date storage method of offer of the embodiment of the present invention, device and computer storage media can promote storage money
The application efficiency in source.
An embodiment of the present invention provides a kind of date storage methods, including：
Logical device receives the input and output I/O Request of data；
Reading and writing data is carried out according to the I/O Request received；
Wherein, the logical device is by two or more including the first storage device and the second storage device
Storage medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
Optionally, the I/O Request that the basis receives carries out reading and writing data：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than
Or when equal to preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge
When going out the data being had been written into first storage device more than the first memory capacity threshold value, it will be counted according to the write request
According to write-in second storage device；
When the I/O Request received is read request, first read corresponding to the read request from first storage device
Data；When not reading the data corresponding to the read request from first storage device, from second storage device
The middle data read corresponding to the read request.
Optionally, the date storage method further includes：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second and store
Capacity threshold；
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to pre-
If Prune Policies delete more than the memory capacity threshold value data.
Optionally, described to include more than the data of the second memory capacity threshold value according to the deletion of preset Prune Policies：
According to the read-write state information of data, the data more than the memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or do not read and write it is lasting when
It is long.
Optionally, when the read-write state information includes the read-write frequency and the duration, the basis is default
Prune Policies delete and include more than the data of the second memory capacity threshold value：
The data of first storage device will be written, is classified according to preset frequency threshold value according to the read-write frequency；
To the data less than the frequency threshold value, it is ranked up according to the time length of the duration；Most from the duration
Long data start, and delete the data being had been written into first storage device according to sequence to less than or equal to described second
Memory capacity threshold value；
It is described to be deleted more than described according to preset Prune Policies when the read-write state information includes the duration
The data of memory capacity threshold value include：
The data of first storage device will be written, is ranked up according to the time length of the duration；From institute
It states the longest data of duration to start, the data being had been written into first storage device be deleted to less than or equal to described
Second memory capacity threshold value；
It is described to be deleted greatly according to preset Prune Policies when the read-write state information includes two or more information
Include in the data of the memory capacity threshold value：
It is that corresponding weight coefficient is arranged in each read-write state information according to preset Weight Algorithm；The first storage of write-in is set
Standby data calculate according to each read-write state information and corresponding weight coefficient and determine the deletion corresponding to each data with reference to letter
Breath；According to calculate obtain the deletion reference information, by the data being had been written into first storage device delete to less than
Or it is equal to the second memory capacity threshold value.
Optionally, described when reading the data corresponding to the read request from second storage device, the data
Storage method further includes：
First storage device is written into the data read from second storage device.
Optionally, before the I/O Request of the logical device reception data, the date storage method further includes：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；
The Hash information that addition generates is added to the I/O Request.
Optionally, first storage device is solid state disk, and second storage device is mechanical hard disk.
On the other hand, the embodiment of the present invention also provides a kind of data storage device, including：Receiving unit and read-write cell；
Wherein,
Receiving unit is used for：Receive the input and output I/O Request of data；
Read-write cell is used for：Reading and writing data is carried out according to the I/O Request received；
Wherein, the logical device is by two or more including the first storage device and the second storage device
Storage medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
Optionally, the read-write cell is specifically used for：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than
Or when equal to preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge
When going out the data being had been written into first storage device more than the first memory capacity threshold value, it will be counted according to the write request
According to write-in second storage device；
When the I/O Request received is read request, according to sequencing：Determine whether to set from first storage
It is standby to hit and read the data corresponding to the read request；Data not corresponding to the read request described in first storage device
When, it is hit from second storage device and reads the data corresponding to the read request.
Optionally, the data storage device further includes cleaning unit, is used for：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second and store
Capacity threshold；
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to pre-
If Prune Policies delete more than the memory capacity threshold value data.
Optionally, the cleaning unit is specifically used for：
According to the read-write state information of data, the data more than the second memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or do not read and write it is lasting when
It is long.
Optionally, the cleaning unit is specifically used for：
When the read-write state information includes the read-write frequency and the duration, first storage will be written and set
Standby data are classified according to the read-write frequency according to preset frequency threshold value；To the data less than the frequency threshold value, according to
The time length of the duration is ranked up；Since the longest data of the duration, according to sequence by described the
The data being had been written into one storage device delete to less than or equal to the second memory capacity threshold values；
When the read-write state information includes the duration, the data of first storage device will be written, according to
The time length of the duration is ranked up；Since the longest data of the duration, first storage is set
The data being had been written into standby delete to less than or equal to the second memory capacity threshold values；
It is each read-write shape according to preset Weight Algorithm when the read-write state information includes two or more information
Corresponding weight coefficient is arranged in state information；To the data of the first storage device are written, according to each read-write state information and corresponding
Weight coefficient calculates the deletion reference information determined corresponding to each data；It, will according to the deletion reference information obtained is calculated
The data being had been written into first storage device delete to less than or equal to the second memory capacity threshold values.
Optionally, the read-write cell is additionally operable to：It is described to read the reading in second storage device of hit and ask
When seeking corresponding data, first storage device is written into the data read from second storage device.
Optionally, the data storage device further includes map processing unit, is used for：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；
The Hash information that addition generates is added to the I/O Request.
Optionally, first storage device is solid state disk, and second storage device is mechanical hard disk.
On the other hand, the embodiment of the present invention also provides a kind of computer storage media, is deposited in the computer storage media
Computer executable instructions are contained, the computer executable instructions are for executing above-mentioned date storage method.
Compared with the relevant technologies, technical scheme includes：Logical device receives input and output (IO) request of data；
Reading and writing data is carried out according to the I/O Request received；Wherein, the logical device is by including the first storage device and the second storage
Two or more storage medium composition including equipment；The read and write rate of first storage device is deposited more than described second
Store up the read and write rate of equipment.The embodiment of the present invention carries out the storage of data by the first storage device and the second storage device, carries
The application efficiency of storage resource is risen.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The purpose of the present invention and other advantages can be by specification, rights
Specifically noted structure is realized and is obtained in claim and attached drawing.
Description of the drawings
Attached drawing is used for providing further understanding technical solution of the present invention, and a part for constitution instruction, with this
The embodiment of application technical solution for explaining the present invention together, does not constitute the limitation to technical solution of the present invention.
Fig. 1 is the relevant technologies kernel composition schematic diagram；
Fig. 2 is the flow chart of date storage method of the embodiment of the present invention；
Fig. 3 is the structure diagram of data storage device of the embodiment of the present invention；
Fig. 4 is schematic diagram of the present invention using sample data read-write.
Specific implementation mode
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention
Embodiment be described in detail.It should be noted that in the absence of conflict, in the embodiment and embodiment in the application
Feature mutually can arbitrarily combine.
Step shown in the flowchart of the accompanying drawings can be in the computer system of such as a group of computer-executable instructions
It executes.Also, although logical order is shown in flow charts, and it in some cases, can be with suitable different from herein
Sequence executes shown or described step.
Fig. 2 is the flow chart of date storage method of the embodiment of the present invention, as shown in Fig. 2, including：
Step 201, logical device receive input and output (IO) request of data；
Wherein, logical device is by two or more storage including the first storage device and the second storage device
Medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
It should be noted that the capacity and number of the first storage device and the second storage device can be according to costs and reading
Efficiency carries out analysis by those skilled in the art and determines.The embodiment of the present invention can pass through two pieces or polylith high-speed processing apparatus
(SSD), low speed storage device (mechanical hard disk) generates above-mentioned logical device.
Step 202 carries out reading and writing data according to the I/O Request received；
Optionally, the embodiment of the present invention includes according to the I/O Request progress reading and writing data received：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than
Or when equal to preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge
When going out the data being had been written into first storage device more than the first memory capacity threshold value, it will be counted according to the write request
According to write-in second storage device；
When the I/O Request received is read request, first read corresponding to the read request from first storage device
Data；When not reading the data corresponding to the read request from first storage device, from second storage device
The middle data read corresponding to the read request.
Optionally, date storage method of the embodiment of the present invention further includes：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second and store
Capacity threshold；Here, predetermined period can be written into according to the frequency, the data that size, the data of the first storage device are read and write
Speed, the second memory capacity threshold value etc., carry out analysis by those skilled in the art and determine.
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to pre-
If Prune Policies delete more than the memory capacity threshold value data.
Optionally, the embodiment of the present invention deletes the number more than the second memory capacity threshold value according to preset Prune Policies
According to including：
According to the read-write state information of data, the data more than the memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or do not read and write it is lasting when
It is long.
It should be noted that when data are deleted from the first storage device, the embodiment of the present invention can be by the data of deletion
It is written to the second storage device；Alternatively, being based on the relevant technologies, it would be desirable to which the data of deletion are written to second in a manner of migrating and are deposited
Store up equipment.
Optionally, the embodiment of the present invention：
It is described according to preset cleaning plan when the read-write state information includes the read-write frequency and the duration
It slightly deletes and includes more than the data of the second memory capacity threshold value：
The data of first storage device will be written, is classified according to preset frequency threshold value according to the read-write frequency；
To the data less than the frequency threshold value, it is ranked up according to the time length of the duration；Most from the duration
Long data start, and delete the data being had been written into first storage device according to sequence to less than or equal to described second
Memory capacity threshold value；
It is described to be deleted more than described according to preset Prune Policies when the read-write state information includes the duration
The data of memory capacity threshold value include：
The data of first storage device will be written, is ranked up according to the time length of the duration；From institute
It states the longest data of duration to start, the data being had been written into first storage device be deleted to less than or equal to described
Second memory capacity threshold value；
It is described to be deleted greatly according to preset Prune Policies when the read-write state information includes two or more information
Include in the data of the memory capacity threshold value：
It is that corresponding weight coefficient is arranged in each read-write state information according to preset Weight Algorithm；The first storage of write-in is set
Standby data calculate according to each read-write state information and corresponding weight coefficient and determine the deletion corresponding to each data with reference to letter
Breath；According to calculate obtain the deletion reference information, by the data being had been written into first storage device delete to less than
Or it is equal to the second memory capacity threshold value.
Here, by read-write state information include read-write the frequency and duration for, can the storage based on data it is important
The weight coefficient of degree and application scenarios, the setting read-write frequency is the first weight coefficient；The weight coefficient of duration is second
Weight coefficient；Each data to the first storage device is written calculate separately：Current data reads and writes the frequency and the first weight coefficient
Be multiplied obtain the first product, duration be multiplied with the second weight coefficient acquisition the second product；By the first sum of products second
After product addition, above-mentioned deletion reference information is obtained；When the embodiment of the present invention assumes that deletion reference information value is bigger, identification is got over
Should be deleted, then according to delete reference information is descending be ranked up after, since deleting reference information maximum value one by one
Data deletion is carried out, until the remaining data having been written into are less than or equal to the memory capacity threshold in first storage device
Stop when value.
Optionally, when reading the data corresponding to the read request from second storage device, the data storage
Method further includes：
First storage device is written into the data read from second storage device.
It should be noted that with reference to the relevant technologies, when data are written, corresponding write request can be generated, according to generation
Write request carries out data write-in.
Optionally, before the I/O Request of logical device reception data, date storage method of the embodiment of the present invention further includes：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；
The Hash information that addition generates is added to I/O Request.
Here, with reference to the relevant technologies, hit of the Hash information for data is handled, i.e., is set in the first storage according to read request
It is that hit is determined whether by Hash information when carrying out the hit processing of data in standby or the second storage device.
Optionally, the first storage device of the embodiment of the present invention is solid state disk, and the second storage device is mechanical hard disk.
Compared with the relevant technologies, technical scheme includes：Logical device receives input and output (IO) request of data；
Reading and writing data is carried out according to the I/O Request received；Wherein, the logical device is by including the first storage device and the second storage
Two or more storage medium composition including equipment；The read and write rate of first storage device is deposited more than described second
Store up the read and write rate of equipment.The embodiment of the present invention carries out the storage of data by the first storage device and the second storage device, carries
The application efficiency of storage resource is risen.
Fig. 3 is the structure diagram of data storage device of the embodiment of the present invention, as shown in figure 3, including：Receiving unit and read-write
Unit；Wherein,
Receiving unit is used for：Receive the input and output I/O Request of data；
Read-write cell is used for：Reading and writing data is carried out according to the I/O Request received；
Wherein, the logical device is by two or more including the first storage device and the second storage device
Storage medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
It should be noted that data storage device of the embodiment of the present invention can be by the first storage device and the second storage device
Form logical device.
Optionally, read-write cell of the embodiment of the present invention is specifically used for：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than
Or when equal to preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge
When going out the data being had been written into first storage device more than the first memory capacity threshold value, it will be counted according to the write request
According to write-in second storage device；
When the I/O Request received is read request, first read corresponding to the read request from first storage device
Data；When not reading the data corresponding to the read request from first storage device, from second storage device
The middle data read corresponding to the read request.
Optionally, the data storage device further includes cleaning unit, is used for：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second and store
Capacity threshold；
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to pre-
If Prune Policies delete more than the memory capacity threshold value data.
Optionally, the cleaning unit is specifically used for：
According to the read-write state information of data, the data more than the second memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or do not read and write it is lasting when
It is long.
Optionally, the cleaning unit is specifically used for：
When the read-write state information includes the read-write frequency and the duration, first storage will be written and set
Standby data are classified according to the read-write frequency according to preset frequency threshold value；To the data less than the frequency threshold value, according to
The time length of the duration is ranked up；Since the longest data of the duration, according to sequence by described the
The data being had been written into one storage device delete to less than or equal to the second memory capacity threshold values；
When the read-write state information includes the duration, the data of first storage device will be written, according to
The time length of the duration is ranked up；Since the longest data of the duration, first storage is set
The data being had been written into standby delete to less than or equal to the second memory capacity threshold values；
It is each read-write shape according to preset Weight Algorithm when the read-write state information includes two or more information
Corresponding weight coefficient is arranged in state information；To the data of the first storage device are written, according to each read-write state information and corresponding
Weight coefficient calculates the deletion reference information determined corresponding to each data；It, will according to the deletion reference information obtained is calculated
The data being had been written into first storage device delete to less than or equal to the second memory capacity threshold values.
Optionally, the read-write cell is additionally operable to：It is described to read the reading in second storage device of hit and ask
When seeking corresponding data, first storage device is written into the data read from second storage device.
It should be noted that with reference to the relevant technologies, when data are written, corresponding write request can be generated, according to generation
Write request carries out data write-in.
Optionally, the data storage device further includes map processing unit：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；What addition generated
Hash information is added to the I/O Request.
With reference to the relevant technologies, here, hit of the Hash information for data is handled, i.e., is set in the first storage according to read request
It is that hit is determined whether by Hash information when carrying out the hit processing of data in standby or the second storage device.
Optionally, first storage device is solid state disk, and second storage device is mechanical hard disk.
The embodiment of the present invention also provides a kind of computer storage media, and computer is stored in the computer storage media
Executable instruction, the computer executable instructions are for executing above-mentioned date storage method.
Present invention method is carried out to understand detailed description below by way of using example, is only used for using example old
The present invention is stated, is not intended to limit the scope of protection of the present invention.
Using example
The present invention is based on (SuSE) Linux OS, by the first storage device (such as SSD) of one piece or one piece or more high speed,
The second storage device (such as mechanical hard disk) of one piece or one piece or more low speed generates logical device；DM layers receive input and output IO
Request (also referred to as read-write requests), is sent to the logical device, rather than the physical disk of bottom；Logical device receives IO and asks
After asking, reading and writing data is carried out according to I/O Request；Including but not limited to (include but not limited to according to read-write state information by data：
The read-write frequency in preset duration, and/or the duration that do not read and write) the read-write temperature that determines data, according to determining reading
It writes temperature and carries out reading and writing data；Assuming that judging the cold and hot of data according to the read-write frequency, then the present invention can be by hot number using example
According to the first storage device is written, the second storage device is written into cold data, to obtain better storage performance.
Present invention application example process may include：
Initialization process is carried out to the installation environment of logical device；
The installation path of the first storage device and the second storage device is respectively set；
According to the installation path of the first storage device and the second storage device, system is carried out to the logical device that needs generate
Matching for attribute postpones, and generates logical device；After completing system property configuration generation logical device, present invention application example can be true
Determine the logical address of logical device；
According to the mapping relations of existing hash algorithm calculating logic address and Hash information in the related technology, reflected with basis
It penetrates relationship and the Hash information for being used for hiting data is added in I/O Request；
Present invention application example is if it is write request, and i.e. feedback is completed after SSD is written in data, when the first storage of write-in is set
When standby data reach certain amount, then write the data to mechanical hard disk.May include specifically：
When the I/O Request received is write request, it is pre- to judge that the data being had been written into the first storage device are less than or equal to
If the first memory capacity threshold value when, the first storage device is write data into according to write request；Judge in the first storage device
When the data having been written into are more than the first memory capacity threshold value, the second storage device is write data into according to write request.
In the I/O Request that DM layers receive, whether logical device according to the request type of IO and can hit, and carry out different places
Reason；By taking read request as an example, read request is first dealt into SSD, if required data in SSD, i.e., it is right to hit read request in SSD
The data answered；The miss such as in SSD, then send read request to mechanical hard disk, and mechanical hard disk is fed back after reading data；This
Using example after mechanical hard disk reads data, SSD is written in the data read from mechanical hard disk by invention, in case next time is interviewed
It asks；Here, the data in present invention application example mechanical hard disk can't be deleted.
The data being written in SSD can be cleared up from two dimensions：Quantity dimension is caching a certain amount of data block
Afterwards, it is cleared up；Time dimension is cleared up according to the time of the data block free time of caching, and the data in being cached with guarantee are
Often accessed dsc data, maintains higher reading hit rate.If it is write request, i.e. feedback is completed after reaching SSD, when writing
When entering data and reaching certain storage capacity threshold value, then write the data to mechanical hard disk.Fig. 4 is that the present invention reads and writes using sample data
Schematic diagram, as shown in figure 4, data according to dsc data and cold data definition divide be written and read；Present invention application example is more
Data manipulation is completed by the SSD of speed, therefore, compared with existing system in the related technology, is preferably utilized and is deposited
The characteristics of storing up equipment, can not only obtain high performance, but also can be with effectively save cost.
The embodiment of the present invention is when the data being had been written into SSD are greater than or equal to preset memory capacity threshold value, root
The data more than memory capacity threshold value are deleted according to preset Prune Policies, can specifically include：
When read-write state information includes the read-write frequency and the duration, the data of the first storage device, root will be written
Classify according to preset frequency threshold value according to the read-write frequency；To the data less than frequency threshold value, according to the time of duration
Length is ranked up；Since the longest data of duration, the data being had been written into the first storage device are deleted according to sequence
It removes to less than or equal to the second memory capacity threshold value；
When read-write state information includes the duration, the data of the first storage device will be written, according to duration
Time length be ranked up；Since the longest data of duration, the data being had been written into the first storage device are deleted
To less than or equal to the second memory capacity threshold value；
When read-write state information includes two or more information, believe for each read-write state according to preset Weight Algorithm
Corresponding weight coefficient is arranged in breath；To the data of the first storage device are written, according to each read-write state information and corresponding weight
Coefficient calculates the deletion reference information determined corresponding to each data；According to the deletion reference information obtained is calculated, by the first storage
The data being had been written into equipment delete to less than or equal to the second memory capacity threshold values.
The embodiment of the present invention is used for data buffer storage, makes by the way that SSD and mechanical hard disk to be generated as to new logical device, SSD
More read-write is completed in SSD, to improve the readwrite performance of system.
One of ordinary skill in the art will appreciate that all or part of step in the above method can be instructed by program
Related hardware (such as processor) is completed, and described program can be stored in computer readable storage medium, as read-only memory,
Disk or CD etc..Optionally, all or part of step of above-described embodiment can also be come using one or more integrated circuits
It realizes.Correspondingly, the form realization of hardware may be used in each module/unit in above-described embodiment, such as passes through integrated electricity
Its corresponding function is realized on road, can also be realized in the form of software function module, such as is stored in by processor execution
Program/instruction in memory realizes its corresponding function.The present invention is not limited to the hardware and softwares of any particular form
In conjunction with.
Although disclosed herein embodiment it is as above, the content only for ease of understanding the present invention and use
Embodiment is not limited to the present invention.Technical staff in any fields of the present invention is taken off not departing from the present invention
Under the premise of the spirit and scope of dew, any modification and variation, but the present invention can be carried out in the form and details of implementation
Scope of patent protection, still should be subject to the scope of the claims as defined in the appended claims.
Claims (17)
1. a kind of date storage method, which is characterized in that including：
Logical device receives the input and output I/O Request of data；
Reading and writing data is carried out according to the I/O Request received；
Wherein, the logical device is by two or more storage including the first storage device and the second storage device
Medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
2. date storage method according to claim 1, which is characterized in that the I/O Request that the basis receives is into line number
Include according to read-write：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than or wait
When preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge institute
When stating the data being had been written into the first storage device more than the first memory capacity threshold value, data are write according to the write request
Enter second storage device；
When the I/O Request received is read request, the number corresponding to the read request is first read from first storage device
According to；When not reading the data corresponding to the read request from first storage device, read from second storage device
Take the data corresponding to the read request.
3. date storage method according to claim 2, which is characterized in that the date storage method further includes：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second memory capacity
Threshold value；
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to preset
Prune Policies delete the data more than the memory capacity threshold value.
4. date storage method according to claim 3, which is characterized in that described to be deleted greatly according to preset Prune Policies
Include in the data of the second memory capacity threshold value：
According to the read-write state information of data, the data more than the memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or the duration that do not read and write.
5. date storage method according to claim 4, which is characterized in that
It is described to be deleted according to preset Prune Policies when the read-write state information includes the read-write frequency and the duration
Except the data more than the second memory capacity threshold value include：
The data of first storage device will be written, is classified according to preset frequency threshold value according to the read-write frequency；
To the data less than the frequency threshold value, it is ranked up according to the time length of the duration；Most from the duration
Long data start, and delete the data being had been written into first storage device according to sequence to less than or equal to described second
Memory capacity threshold value；
When the read-write state information includes the duration, described deleted according to preset Prune Policies is more than the storage
The data of capacity threshold include：
The data of first storage device will be written, is ranked up according to the time length of the duration；It is held from described
The continuous longest data of duration start, and the data being had been written into first storage device are deleted to less than or equal to described second
Memory capacity threshold value；
When the read-write state information includes two or more information, described deleted according to preset Prune Policies is more than institute
The data for stating memory capacity threshold value include：
It is that corresponding weight coefficient is arranged in each read-write state information according to preset Weight Algorithm；To the first storage device of write-in
Data calculate the deletion reference information determined corresponding to each data according to each read-write state information and corresponding weight coefficient；Root
According to the deletion reference information obtained is calculated, the data being had been written into first storage device are deleted to less than or are equal to
The second memory capacity threshold value.
6. according to claim 2~5 any one of them date storage method, which is characterized in that described from second storage
When reading the data corresponding to the read request in equipment, the date storage method further includes：
First storage device is written into the data read from second storage device.
7. according to Claims 1 to 5 any one of them date storage method, which is characterized in that the logical device receives number
According to I/O Request before, the date storage method further includes：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；
The Hash information that addition generates is added to the I/O Request.
8. according to Claims 1 to 5 any one of them date storage method, which is characterized in that
First storage device is solid state disk, and second storage device is mechanical hard disk.
9. a kind of data storage device, which is characterized in that including：Receiving unit and read-write cell；Wherein,
Receiving unit is used for：Receive the input and output I/O Request of data；
Read-write cell is used for：Reading and writing data is carried out according to the I/O Request received；
Wherein, the logical device is by two or more storage including the first storage device and the second storage device
Medium forms；The read and write rate of first storage device is more than the read and write rate of second storage device.
10. data storage device according to claim 9, which is characterized in that the read-write cell is specifically used for：
When the I/O Request received is write request, judge that the data being had been written into first storage device are less than or wait
When preset first memory capacity threshold value, first storage device is write data into according to the write request；Judge institute
When stating the data being had been written into the first storage device more than the first memory capacity threshold value, data are write according to the write request
Enter second storage device；
When the I/O Request received is read request, the number corresponding to the read request is first read from first storage device
According to；When not reading the data corresponding to the read request from first storage device, read from second storage device
Take the data corresponding to the read request.
11. data storage device according to claim 10, which is characterized in that the data storage device further includes cleaning
Unit is used for：
According to predetermined period, judge whether the data being had been written into first storage device are more than preset second memory capacity
Threshold value；
When judging that the data being had been written into first storage device are more than the second memory capacity threshold value, according to preset
Prune Policies delete the data more than the memory capacity threshold value.
12. data storage device according to claim 11, which is characterized in that the cleaning unit is specifically used for：
According to the read-write state information of data, the data more than the second memory capacity threshold value are deleted；
Wherein, the read-write state information includes：The read-write frequency in preset duration, and/or the duration that do not read and write.
13. data storage device according to claim 12, which is characterized in that the cleaning unit is specifically used for：
When the read-write state information includes the read-write frequency and the duration, first storage device will be written
Data are classified according to the read-write frequency according to preset frequency threshold value；To the data less than the frequency threshold value, according to
The time length of the duration is ranked up；Since the longest data of the duration, according to sequence by described the
The data being had been written into one storage device delete to less than or equal to the second memory capacity threshold values；
When the read-write state information includes the duration, the data of first storage device will be written, according to described
The time length of duration is ranked up；It, will be in first storage device since the longest data of the duration
The data having been written into delete to less than or equal to the second memory capacity threshold values；
When the read-write state information includes two or more information, believe for each read-write state according to preset Weight Algorithm
Corresponding weight coefficient is arranged in breath；To the data of the first storage device are written, according to each read-write state information and corresponding weight
Coefficient calculates the deletion reference information determined corresponding to each data；It, will be described according to the deletion reference information obtained is calculated
The data being had been written into first storage device delete to less than or equal to the second memory capacity threshold values.
14. according to claim 10~13 any one of them data storage device, which is characterized in that the read-write cell is also used
In：When reading the data corresponding to the read request from second storage device, it will be read from second storage device
First storage device is written in the data taken.
15. according to claim 9~13 any one of them data storage device, which is characterized in that the data storage device
Further include map processing unit, is used for：
The mapping relations between Hash information and the logical address of the logical device are determined by preset hash algorithm；
When carrying out data write-in, Hash information is generated according to the determining mapping relations and the logical address of write-in；
The Hash information that addition generates is added to the I/O Request.
16. according to claim 9~13 any one of them data storage device, which is characterized in that
First storage device is solid state disk, and second storage device is mechanical hard disk.
17. a kind of computer storage media, computer executable instructions, the calculating are stored in the computer storage media
Machine executable instruction requires the date storage method described in any one of 1~8 for perform claim.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810481557.XA CN108710474A (en) | 2018-05-18 | 2018-05-18 | A kind of date storage method, device and computer storage media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810481557.XA CN108710474A (en) | 2018-05-18 | 2018-05-18 | A kind of date storage method, device and computer storage media |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108710474A true CN108710474A (en) | 2018-10-26 |
Family
ID=63869106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810481557.XA Pending CN108710474A (en) | 2018-05-18 | 2018-05-18 | A kind of date storage method, device and computer storage media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108710474A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110196689A (en) * | 2019-06-03 | 2019-09-03 | 无锡华云数据技术服务有限公司 | Read method, device and the distributed memory system of data |
| CN110286858A (en) * | 2019-06-26 | 2019-09-27 | 北京奇艺世纪科技有限公司 | A kind of data processing method and relevant device |
| CN111158609A (en) * | 2019-12-31 | 2020-05-15 | 广州鼎甲计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN112947844A (en) * | 2019-12-11 | 2021-06-11 | 北京金山云网络技术有限公司 | Data storage method and device, electronic equipment and medium |
| WO2021120137A1 (en) * | 2019-12-19 | 2021-06-24 | 华为技术有限公司 | Data storage method and storage apparatus |
| CN113885797A (en) * | 2021-09-24 | 2022-01-04 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
| CN116069261A (en) * | 2023-03-03 | 2023-05-05 | 苏州浪潮智能科技有限公司 | Data processing method, system, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678149A (en) * | 2013-12-19 | 2014-03-26 | 华为技术有限公司 | Data processing method and device |
| US20160147482A1 (en) * | 2014-11-26 | 2016-05-26 | Hyoungsuk JANG | Storage device including nonvolatile memory device and method of programming the same |
| CN105630699A (en) * | 2015-04-15 | 2016-06-01 | 上海磁宇信息科技有限公司 | Solid-state drive using MRAM and read-write cache management method |
| CN106598495A (en) * | 2016-12-07 | 2017-04-26 | 深圳市深信服电子科技有限公司 | Mixed storage service quality control method and control apparatus |
| CN107632784A (en) * | 2017-09-14 | 2018-01-26 | 郑州云海信息技术有限公司 | The caching method of a kind of storage medium and distributed memory system, device and equipment |
-
2018
- 2018-05-18 CN CN201810481557.XA patent/CN108710474A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678149A (en) * | 2013-12-19 | 2014-03-26 | 华为技术有限公司 | Data processing method and device |
| US20160147482A1 (en) * | 2014-11-26 | 2016-05-26 | Hyoungsuk JANG | Storage device including nonvolatile memory device and method of programming the same |
| CN105630699A (en) * | 2015-04-15 | 2016-06-01 | 上海磁宇信息科技有限公司 | Solid-state drive using MRAM and read-write cache management method |
| CN106598495A (en) * | 2016-12-07 | 2017-04-26 | 深圳市深信服电子科技有限公司 | Mixed storage service quality control method and control apparatus |
| CN107632784A (en) * | 2017-09-14 | 2018-01-26 | 郑州云海信息技术有限公司 | The caching method of a kind of storage medium and distributed memory system, device and equipment |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110196689A (en) * | 2019-06-03 | 2019-09-03 | 无锡华云数据技术服务有限公司 | Read method, device and the distributed memory system of data |
| CN110286858A (en) * | 2019-06-26 | 2019-09-27 | 北京奇艺世纪科技有限公司 | A kind of data processing method and relevant device |
| CN112947844A (en) * | 2019-12-11 | 2021-06-11 | 北京金山云网络技术有限公司 | Data storage method and device, electronic equipment and medium |
| WO2021120137A1 (en) * | 2019-12-19 | 2021-06-24 | 华为技术有限公司 | Data storage method and storage apparatus |
| CN111158609A (en) * | 2019-12-31 | 2020-05-15 | 广州鼎甲计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN111158609B (en) * | 2019-12-31 | 2020-11-20 | 广州鼎甲计算机科技有限公司 | Data storage method and device, computer equipment and storage medium |
| CN113885797A (en) * | 2021-09-24 | 2022-01-04 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
| CN113885797B (en) * | 2021-09-24 | 2023-12-22 | 济南浪潮数据技术有限公司 | Data storage method, device, equipment and storage medium |
| CN116069261A (en) * | 2023-03-03 | 2023-05-05 | 苏州浪潮智能科技有限公司 | Data processing method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108710474A (en) | A kind of date storage method, device and computer storage media | |
| KR101246982B1 (en) | Using external memory devices to improve system performance | |
| CN106548789B (en) | Method and apparatus for operating stacked tile type magnetic recording equipment | |
| US8112603B2 (en) | Methods, systems, and computer program products for file relocation on a data storage device | |
| US9959054B1 (en) | Log cleaning and tiering in a log-based data storage system | |
| US20140229657A1 (en) | Readdressing memory for non-volatile storage devices | |
| CN107025070B (en) | Versioned stores device and method | |
| CN105843753B (en) | Logical block address for hard disk drive maps | |
| TW201017405A (en) | Improved hybrid drive | |
| WO2017148242A1 (en) | Method for accessing shingled magnetic recording (smr) hard disk, and server | |
| CN109804359A (en) | For the system and method by write back data to storage equipment | |
| US9183127B2 (en) | Sequential block allocation in a memory | |
| CN108334284A (en) | Tail delay perception foreground garbage collection algorithm | |
| WO2017063495A1 (en) | Data migration method and apparatus | |
| US20240086332A1 (en) | Data processing method and system, device, and medium | |
| CN110795363A (en) | Hot page prediction method and page scheduling method for storage medium | |
| US8151068B2 (en) | Data copy management for faster reads | |
| US11321002B2 (en) | Converting a virtual volume between volume types | |
| CN116755625A (en) | Data processing method, device, equipment and readable storage medium | |
| US11010091B2 (en) | Multi-tier storage | |
| US10185660B2 (en) | System and method for automated data organization in a storage system | |
| CN107102900B (en) | Management method of shared memory space | |
| US20140359228A1 (en) | Cache allocation in a computerized system | |
| CN108984117B (en) | Data reading and writing method, medium and equipment | |
| KR20120035070A (en) | Methods of managing buffer cache using solid state disk as an extended buffer and apparatuses for using solid state disk as an extended buffer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181026 |