CN111124303B - Data storage method, device and system - Google Patents
Data storage method, device and system Download PDFInfo
- Publication number
- CN111124303B CN111124303B CN201911310870.8A CN201911310870A CN111124303B CN 111124303 B CN111124303 B CN 111124303B CN 201911310870 A CN201911310870 A CN 201911310870A CN 111124303 B CN111124303 B CN 111124303B
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- hot
- access frequency
- hard disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000013500 data storage Methods 0.000 title claims abstract description 45
- 238000013523 data management Methods 0.000 claims abstract description 96
- 230000003287 optical effect Effects 0.000 claims abstract description 69
- 230000015654 memory Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000002035 prolonged effect Effects 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000003139 buffering effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a data storage method, a device and a system, wherein the method comprises the following steps: acquiring data to be stored, and storing attribute information of the data to be stored into a data management area; judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, and when the access frequency is greater than or equal to the preset access frequency, modifying attribute information of the data in a data management area, and storing hot data into a hard disk; acquiring a storage position of thermal data in a hard disk, and storing storage position information into a data management area; writing all data to be stored into an optical disc library; and acquiring storage positions of all data to be stored in the optical disc library, and storing the storage position information into a data management area. By implementing the invention, the data is stored by adopting different steps in combination with different attribute information of the data to be stored, the problem that the data must be magnetically buffered and/or electrically buffered before being stored in the optical disk in the existing data storage technology is solved, and the service life of media such as magnetic disk and the like is prolonged.
Description
Technical Field
The present invention relates to the field of data storage, and in particular, to a method, an apparatus, and a system for storing data.
Background
Under the big data age, many enterprises have a large amount of data, and aiming at the application frequency and the application scene of different data, the data can be stored in different media in a classified way so as to save the cost. In the existing storage technology, data storage by combining a disk array with a tape library is still a mainstream technology which can be applied to data storage of a data center, and the existing storage method is to buffer data to a memory, store the data into a disk, cool the data, and store the data into an optical disk. After a series of processing, all the original data are finally stored in the optical disc for storage. Because the media such as the magnetic disk has the limit of read-write times, when a large amount of data is stored, the service life of the magnetic disk is seriously influenced by multiple times of read-write, that is, the prior storage technology must firstly magnetically buffer and/or electrically buffer the data and then store the data into the optical disk, and the storage mode can influence the service life of the media such as the magnetic disk.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is to overcome the defect that the service life of media such as magnetic discs is shortened by firstly carrying out magnetic buffering and/or electric buffering on data and then storing the data into the optical discs in the existing data storage technology, thereby providing a data storage method, a data storage device and a data storage system.
According to a first aspect, an embodiment of the present invention discloses a method for storing data, the method comprising: acquiring data to be stored, wherein the attribute of the data to be stored defaults to cold data, and the attribute of the data comprises hot data or cold data; storing the attribute information of the data to be stored into a data management area, wherein the data management area is a storage space arranged in a memory; judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, wherein the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data; when the data to be stored is hot data, storing the hot data into a hard disk, and modifying attribute information of the data to be stored in the data management area; acquiring a storage position of the thermal data in a hard disk, and storing storage position information of the thermal data in the hard disk into the data management area; writing all the data to be stored into an optical disc library; and acquiring storage positions of all the data to be stored in the optical disc library, and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
With reference to the first aspect, in a first implementation manner of the first aspect, after the obtaining the data to be stored, the method for storing data further includes: and identifying the data to be stored, storing the identification information of the data to be stored into the data management area, wherein the identification information of the data to be stored is unique.
With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the method for reading data further includes: judging whether the current access frequency of the thermal data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
According to a second aspect, an embodiment of the present invention discloses a method for reading data, including: acquiring data to be read, and judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in a hard disk by the data storage method as described in the first aspect or any implementation manner of the first aspect; when the data to be read is hot data, directly acquiring hard disk storage position information of the hot data in the data management area; and reading the thermal data according to the hard disk storage position information.
With reference to the second aspect, in a first implementation manner of the second aspect, the method for reading data further includes: when the data is cold data, acquiring optical disc library storage position information of the cold data in a data management area; acquiring the cold data according to the storage position information of the optical disc library; and reading the obtained cold data into a hard disk, and reading and processing.
With reference to the first implementation manner of the second aspect, in a second implementation manner of the second aspect, the method for reading data further includes: judging whether the current access frequency of the thermal data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
According to a third aspect, an embodiment of the present invention discloses a data storage device, including: the first acquisition module is used for acquiring data to be stored, wherein the attribute of the data to be stored defaults to cold data, the attribute of the data comprises hot data or cold data, and the first acquisition module is also used for storing the attribute information of the data to be stored into a data management area, and the data management area is a storage space arranged in a memory; the first judging module is used for judging whether the access frequency of the data to be stored is greater than or equal to the data with the preset access frequency, wherein the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data; the first storage module is used for storing the hot data into a hard disk when the data to be stored are hot data and modifying attribute information of the data to be stored in the data management area; the second acquisition module is used for acquiring the storage position of the thermal data in the hard disk and storing the storage position information of the thermal data in the hard disk into the data management area; the second storage module is used for writing all the data to be stored into the optical disc library; and the third acquisition module is used for acquiring storage positions of all the data to be stored in the optical disc library and storing the storage position information of all the data to be stored in the optical disc library into the data management area.
According to a fourth aspect, an embodiment of the present invention discloses a data reading apparatus, including: a fourth acquisition module, configured to acquire data to be read; the second judging module is used for judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in a hard disk by a storage device for data as described in the third aspect; a fifth obtaining module, configured to directly obtain hard disk storage location information of the hot data in the data management area when the data to be read is hot data; and the reading module is used for reading the thermal data according to the hard disk storage position information.
According to a fifth aspect, an embodiment of the present invention discloses a data storage system, including: at least one control device, the control device is used for executing the data storage method according to the first aspect or any implementation manner of the first aspect, and storing the received data to be stored.
According to a sixth aspect, an embodiment of the present invention discloses a data reading system, including: at least one control device for performing the method for reading data according to the second aspect or any of the embodiments of the second aspect.
According to a seventh aspect, the present embodiments disclose a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of storing data according to the first aspect or any of the embodiments of the first aspect or the method of reading data according to the second aspect or any of the embodiments of the second aspect.
The technical scheme of the invention has the following advantages:
1. According to the data storage method provided by the embodiment of the invention, the access frequency of the data to be stored is judged, the attribute information of the data to be stored is correspondingly modified, when the access frequency is larger than the preset access frequency, the hot data is stored in the hard disk, the position information of the hot data in the hard disk is stored in the data management area, and finally all the data are stored in the optical disc library, so that the data storage work is completed, the problem that the data must be firstly magnetically buffered and/or electrically buffered, namely, the data is firstly stored in the magnetic storage medium, and then the data in the magnetic storage medium is slowly recorded in the optical disc is solved, the data storage step is simplified, and the service life of the storage medium such as a magnetic disc is prolonged.
2. According to the data reading method provided by the embodiment of the invention, firstly, the data to be read is obtained, whether the attribute of the data to be read is hot data is judged, and when the data to be read is hot data, the hot data can be directly read by directly obtaining the hard disk storage position information of the hot data in the data management area; when the data is judged to be cold data, the cold data can be directly obtained and read and processed by obtaining the storage position information of the optical disc library of the cold data in the data management area, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, and the consumption of storage media such as magnetic discs is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a data management area, an optical disc library, and a data management area according to the present invention;
FIG. 2 is a flowchart of a specific example of a data storage method in embodiment 1 of the present invention;
FIG. 3 is a flowchart of a specific example of a method for reading data in embodiment 2 of the present invention;
FIG. 4 is a block diagram showing a data storage device according to embodiment 3 of the present invention;
fig. 5 is a block diagram showing a configuration of a specific example of a data reading apparatus in embodiment 4 of the present invention;
FIG. 6 is a schematic block diagram of a control device according to the present invention;
Fig. 7 is a schematic block diagram of a first controller in the control apparatus of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; the two components can be directly connected or indirectly connected through an intermediate medium, or can be communicated inside the two components, or can be connected wirelessly or in a wired way. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
The method for storing data provided by the embodiment of the invention, as shown in fig. 1, can be applied to an application environment composed of a data management area 10, an optical disc library 20 and a hard disc storage area 30, wherein the data management area 10 is used for storing attribute information and position information of data and managing the data stored in the optical disc library and the hard disc; the optical disc library 20 is used for storing data, and can automatically expand the capacity of the optical disc library by realizing the read-write operation of the optical disc library through an optical disc driver and realizing the automatic access of the optical disc through a mechanical device; the hard disk storage area is used for storing data with higher access frequency, and can also be used for cleaning the data, and particularly can be a hard disk array taking a Solid state disk (Solid STATE DRIVES, SSD) or a mechanical hard disk (HARD DISK DRIVE, HDD) as a medium.
Example 1
The embodiment of the invention provides a data storage method, which can be applied to a specific application scene of large institutions such as banks, governments, companies and the like, which need to store data with different access frequencies, and the data storage method in the embodiment, as shown in fig. 2, comprises the following steps:
Step S11: acquiring data to be stored, defaulting the attribute of the data to be stored into cold data, wherein the attribute of the data comprises hot data or cold data, storing attribute information of the data to be stored into a data management area, and the data management area is a storage space arranged in a memory; in this embodiment, the data to be stored may be data information such as assets, receipts, details, etc. of a customer in a banking system, may be data information such as meeting records, policy changes, etc. in a government system, may also be data information such as purchasing, paying out, etc. in a company system, where all the data needs to be stored as soon as possible after data generation, specifically, the manner of acquiring the data to be stored may be that data information is acquired through a universal serial bus (Universal Serial Bus, USB), a data line may be connected to the system, the acquired data content may be transmitted through a USB data line, or may also be that the acquired data content may be directly acquired through a Wi-Fi system, and a person skilled in the art may determine the manner of acquiring the data according to actual needs.
Step S12: judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, wherein the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data; in this embodiment, the access frequency is the number of times the user calls the data in the target time period, the attribute information of the data may be hot data or cold data, specifically, when the attribute information of the data is hot data, the access frequency of the data is higher than a preset access frequency threshold, that is, in a specific application scenario, the data may be called for multiple times in the target time period, for example, when in a banking system, the data storing the balance of the user is called for multiple times because the user views the balance information for multiple times in the target time period, so the data storing the balance of the user is hot data; when the attribute of the data is cold data, it means that the access frequency of the data in the target period is lower than a preset access frequency threshold, that is, the number of times the data is used in the target period is lower than a preset number of times.
Step S13: when the data to be stored is hot data, the hot data is stored in the hard disk, and the attribute information of the data to be stored in the data management area is modified.
For example, attribute information that is to be determined as hot data by judging the access frequency is modified from cold data to hot data in the data management area; the data management area 10, which may also be referred to as a storage management area, is a management area provided in a memory for managing already stored data; when the data to be stored is hot data, the attribute information indicating the identity of the hot data is stored in the data management area 10, the data information carried by the hot data is stored in the hard disk, specifically, the hard disk can be a nonvolatile memory used on the mobile terminal, and the data is written in the hard disk by changing the polarity of electromagnetic flow carried by the magnetic head close to the magnetic surface, so that the performance of the hard disk for storing the data is very stable, the data is not easy to lose, and the data can be stably stored for a long time.
Step S14: and acquiring the storage position of the thermal data in the hard disk, and storing the storage position information of the thermal data in the hard disk into a data management area. For example, when the hot data is stored in the hard disk, the storage location of the hot data in the hard disk is acquired, and since the hard disk can store a large amount of data, when the hot data is stored in the hard disk, the storage location of the hot data in the hard disk needs to be acquired, and the storage location information of the hot data in the hard disk is stored in the data management area 10, so that the subsequent management operation of the data management area 10 on the stored data is facilitated.
Specifically, the storage location in the hard disk may be an n-track m-sector, for example, the hard disk is composed of a plurality of disks, and both the upper and lower surfaces of the disks may store information, so that the surfaces of the disks are called recording surfaces, and the recording surfaces have a series of concentric circles thereon, which are called tracks. Each recording surface typically has tens to hundreds of tracks, each of which is divided into a plurality of sectors. The addresses of the tracks are numbered sequentially from outside to inside, the outermost concentric circle is called a 0 track, and the innermost concentric circle is called an n track. Each track is divided into sectors. There are various methods for numbering the sectors, and they may be consecutive numbers or interval numbers. After the recording surface is numbered by the process, the storage position in the hard disk of the n-track m-sector can be used, namely the hard disk address can be used for finding the corresponding data storage position in the actual hard disk.
Step S15: and writing all data to be stored into the optical disc library. For example, all the data to be stored includes hot data and cold data, the optical disc library 20 may be an optical disc library capable of expanding capacity, the expansion of capacity may be achieved by sequentially placing a plurality of optical discs in the same optical disc library, specifically, the automatic access operation of the plurality of optical discs may be completed by a mechanical device in the optical disc library, and the high-speed optical disc storage and reading operations may be achieved by a large number of optical disc drives.
Step S16: and acquiring storage positions of all the data to be stored in the optical disc library, and storing the storage position information of all the data to be stored in the optical disc library into a data management area. For example, when all data is stored in the optical disc library 20, since a plurality of optical discs are sequentially placed in the optical disc library, the optical disc storage position of the data at this time can be represented by the n-number optical disc, and when the storage position information of each piece of data is obtained, the storage position information of the data is stored in the data management area 10, so that the management operation of the data management area 10 on all the stored data is facilitated.
According to the data storage method provided by the embodiment of the invention, the attribute information of the data to be stored, namely the access frequency of the data to be stored is judged, the attribute judgment result information of the data to be stored is stored in the data management area, when the access frequency is larger than the preset access frequency, the data is determined to be hot data, the hot data is stored in the hard disk, the position information of the hot data in the hard disk is stored in the data management area, and finally all the data are stored in the optical disc library, so that the data storage work is completed, the problem that the data must be subjected to magnetic buffering and/or electric buffering in the existing data storage technology, namely the data is stored in the magnetic storage medium first, then the data in the magnetic storage medium is slowly recorded in the optical disc is solved, and different storage paths are selected in combination with the attribute information of the data, so that the steps of data storage are simplified, and the service life of the storage medium such as a magnetic disc is prolonged.
As an optional implementation manner of the present application, the method for storing data provided by the embodiment of the present application further includes: after the data to be stored is acquired, the data to be stored is identified, and the identification information of the data to be stored is stored in the data management area, wherein the identification information of the data to be stored is unique. For example, the data may be identified by an Identification (ID), and each data may correspond to a unique identification code, which may be a data number or an alphabetic code. According to the data storage method provided by the embodiment of the application, the unique coding information is used for identifying the data to be stored, so that unified management of the data in subsequent data operation is facilitated. As an optional implementation manner of the present application, the method for storing data provided by the embodiment of the present application further includes:
Judging whether the current access frequency of the thermal data stored in the hard disk is lower than the preset access frequency or not according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, the hot data in the hard disk is cleaned, cleaning information of the hot data is fed back to the data management area 10, attribute information of the hot data is modified, and hard disk storage position information of the hot data in the data management area 10 is deleted.
The present access frequency of the thermal data in the hard disk is determined according to a preset time interval, which may be 5 seconds or 10 seconds, and the present invention is not limited to this, and those skilled in the art can determine the present invention according to specific situations.
According to the data storage method provided by the embodiment of the invention, the access frequency of the hot data stored in the hard disk is regularly judged, and when the access frequency is lower than the preset access frequency, the data is cleaned. And modifying the attribute information of the data in the data management area, deleting the position information of the data corresponding to the hard disk in the data management area, and deleting the data which does not accord with the hot data standard in the hard disk, thereby achieving the purpose of saving the space of the hard disk and reducing the cost of data storage.
Example 2
The embodiment of the invention provides a data reading method, which can be applied to mobile terminals or service personnel in large institutions such as banks, government, companies and the like, and is applied to a specific application scene in which the stored data is required to be read, and the data reading method in the embodiment, as shown in fig. 3, comprises the following steps:
Step S21: acquiring data to be read; in this embodiment, the data to be read may be the data stored in the foregoing embodiment, for example, when the user queries the balance information, the user needs to read the pre-stored balance data of the user to obtain the current balance of the user.
Step S22: judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in a hard disk by the data storage method according to any one of the embodiments described above. For example, the attribute information of the data may be hot data or cold data, and when the data is hot data, it means that the data is data stored in the hard disk due to the access frequency being greater than or equal to a preset access frequency; when the data is cold data, it means that the data is data stored in the optical disc library 20 due to the access frequency being less than the preset access frequency.
Step S23: when the data to be read is hot data, the hard disk storage position information of the hot data in the data management area is directly acquired. In this embodiment, when the data to be read is hot data, according to the data storage method described in any of the above embodiments, the hot data is stored in the hard disk, and the hard disk storage location information of the hot data is stored in the data management area 10, and at this time, the data information of the hot data is to be acquired, which is to be done by acquiring the storage location information of the hot data in the data management area 10.
Step S24: and reading the hot data according to the storage position information of the hard disk. In this embodiment, the storage location information of the hard disk may be n-track m-sector, which represents a specific location of the hot data in the hard disk, and the data information of the corresponding location in the hard disk may be directly obtained according to the storage location information of the hard disk of the hot data stored in the data management area 10.
According to the data reading method provided by the embodiment of the invention, firstly, the data to be read is obtained, whether the attribute of the data to be read is hot data is judged, and when the data to be read is hot data, the hot data can be directly read by directly obtaining the hard disk storage position information of the hot data in the data management area; when the data is judged to be cold data, the cold data can be directly obtained and read and processed by obtaining the storage position information of the optical disc library of the cold data in the data management area, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, the data processing is convenient, and the consumption of storage media such as a magnetic disc is reduced.
As an optional implementation manner of the present application, the method for reading data provided by the embodiment of the present application further includes:
Step S25: when the data is cold data, the storage position information of the optical disc library of the cold data in the data management area is acquired. In this embodiment, when the data to be read is cold data, according to the method for storing data described in any of the above embodiments, the cold data is stored in the optical disc library, and the optical disc library storage location information of the cold data is stored in the data management area.
Step S26: acquiring cold data according to the storage position information of the optical disc library;
Step S27: and reading the obtained cold data into a hard disk, and reading and processing.
In the method for reading data provided by the embodiment of the invention, in the cold data storage optical disc library, the optical disc library storage position information of the cold data is stored in the data management area, when the data to be read is judged to be the cold data, the data information of the corresponding position of the optical disc library can be directly obtained according to the optical disc library storage position information, and the cold data can be read into the hard disc after the data information of the cold data is obtained, so that the data processing is performed. The optical disc library mechanical arm has slower reading speed, so that the data in the optical disc library is buffered into the hard disc, and then the data is read and processed, so that the reading speed of the data is improved, and the reading efficiency of the data is also improved.
As an optional implementation manner of the present application, the method for reading data provided by the embodiment of the present application further includes:
Judging whether the current access frequency of the thermal data stored in the hard disk is lower than the preset access frequency or not according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, the hot data in the hard disk is cleaned, cleaning information of the hot data is fed back to the data management area, attribute information of the hot data is modified, and hard disk storage position information of the hot data in the data management area is deleted.
According to the data reading method provided by the embodiment of the invention, the access frequency of the hot data stored in the hard disk is regularly judged, and when the access frequency is lower than the preset access frequency, the data is cleaned. And modifying the attribute information of the data in the data management area, deleting the position information of the data corresponding to the hard disk in the data management area, and deleting the data which does not accord with the hot data standard in the hard disk, thereby achieving the purpose of saving the space of the hard disk and reducing the cost of data storage.
Example 3
The embodiment of the invention provides a data storage device, which can be applied to large institutions such as banks, governments, companies and the like, and is used for storing data with different access frequencies in a specific application scene, as shown in fig. 4, and comprises the following steps:
The first obtaining module 31 is configured to obtain data to be stored, where an attribute of the data to be stored defaults to cold data, and an attribute of the data includes hot data or cold data, and is configured to store attribute information of the data to be stored in a data management area, where the data management area is a storage space provided in a memory, and details of implementation can be seen from the description related to step S11 of the foregoing method embodiment.
The first determining module 32 is configured to determine whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, where the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data, and details of implementation can be found in the related description of step S12 of the foregoing method embodiment.
The first storage module 33 is configured to store the hot data in the hard disk, and modify attribute information of the data to be stored in the data management area when the data to be stored is the hot data, and details of implementation can be found in the related description of step S13 of the above method embodiment.
The second obtaining module 34 is configured to obtain a storage location of the thermal data in the hard disk, store the storage location information of the thermal data in the hard disk in the data management area, and refer to the related description of step S14 in the above method embodiment for details.
The second storage module 35 is configured to write all the data to be stored into the optical disc library, and details of implementation can be found in the description of step S15 of the above method embodiment.
The third obtaining module 36 is configured to obtain storage locations of all data to be stored in the optical disc library, store storage location information of all data to be stored in the optical disc library in the data management area, and refer to the description of step S16 in the above method embodiment for details.
According to the data storage method provided by the embodiment of the invention, the attribute information of the data to be stored, namely the access frequency of the data to be stored is judged, the attribute judgment result information of the data to be stored is stored in the data management area, when the access frequency is smaller than the preset access frequency, the cold data is determined, the attribute information of the cold data is stored in the data management area, the problem that the data can be stored after being magnetically/electrically buffered in the existing storage technology is solved, and the data can be stored in different storage paths by combining the new attribute information of the data, so that the data storage steps are simplified, the data storage cost is saved, and the read-write times of storage media such as magnetic discs are reduced.
Example 4
The embodiment of the invention provides a data reading device, which can be applied to large-scale institutions such as banks, governments, companies and the like, and is used for reading stored data in a specific application scene, as shown in fig. 5, and comprises the following steps:
the fourth obtaining module 41 is configured to obtain the data to be read, and details of implementation can be found in the related description of step S21 of the above method embodiment.
A second judging module 42, configured to judge whether the attribute of the data to be read is hot data; the thermal data refers to data stored in the data management area by the storage means of the data as in embodiment 3, and the detailed implementation is described in connection with step S22 of the above-described method embodiment.
The fifth obtaining module 43 is configured to directly obtain the hard disk storage location information of the hot data in the data management area when the data to be read is the hot data, and details of implementation can be found in the related description of step S23 of the above method embodiment.
The reading module 44 is configured to read the thermal data according to the hard disk storage location information, and details of implementation can be found in the description related to step S24 of the above method embodiment.
According to the data reading method provided by the embodiment of the invention, firstly, the data to be read is obtained, whether the attribute of the data to be read is hot data is judged, and when the data to be read is hot data, the hot data can be directly read by directly obtaining the hard disk storage position information of the hot data in the data management area; when the data is judged to be cold data, the cold data can be directly obtained and read and processed by obtaining the storage position information of the optical disc library of the cold data in the data management area, and the data can be directly obtained through different position information of the data by combining the reading mode of the data and the access frequency of the data to be read, so that the data reading speed is improved, the data processing is convenient, and the consumption of storage media such as a magnetic disc is reduced.
Example 5
The present embodiment provides a control apparatus, as shown in fig. 6, including:
The first communication module 511: the data transmission device is used for transmitting data to be stored or data to be read; the first communication module can be a Bluetooth module and a Wi-Fi module, and then communicates through a set wireless communication protocol.
First controller 512: is connected to the first communication module 511, as shown in fig. 7, and includes: at least one processor 61; and a memory 62 communicatively coupled to the at least one processor 61; the memory 62 stores instructions executable by the at least one processor 61, and when receiving data information, the at least one processor 61 executes the data storage method shown in fig. 2 or the data reading method shown in fig. 3, and in fig. 7, one processor is taken as an example, the processor 61 and the memory 62 are connected through the bus 60, and in this embodiment, the first communication module may be a wireless communication module, for example, a bluetooth module, a Wi-Fi module, or the like, or may be a wired communication module. The transmission between the first controller 512 and the first communication module 511 is wireless transmission.
The processor 61 may be a central processing unit (Central Processing Unit, CPU). The Processor 61 may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, DSPs), application SPECIFIC INTEGRATED Circuits (ASICs), field-Programmable gate arrays (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 62 is used as a non-transitory computer readable storage medium, and may be used to store a non-transitory software program, a non-transitory computer executable program, and a module, such as program instructions/modules corresponding to a data storage method or a data reading method in an embodiment of the present application. The processor 61 executes various functional applications of the server and data processing, that is, implements the data storage method or the data reading method in the above-described embodiment, by running the non-transitory software programs, instructions, and modules stored in the memory 62.
Memory 62 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of a processing device operated by the server, or the like. In addition, the memory 62 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 62 may optionally include memory located remotely from processor 61, which may be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more modules are stored in the memory 62 that, when executed by the one or more processors 61, perform the methods described in any of the above embodiments.
The embodiment of the invention also provides a non-transitory computer readable medium storing computer instructions for causing a computer to execute the method for storing data or the method for reading data described in any of the above embodiments, where the storage medium may be a magnetic disk, an optical disc, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), a Flash Memory (Flash Memory), a hard disk (HARD DISK DRIVE, abbreviated as HDD), a Solid-state disk (Solid-state-STATE DRIVE, SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While still being apparent from variations or modifications that may be made by those skilled in the art are within the scope of the invention.
Claims (9)
1. A method of storing data, comprising the steps of:
acquiring data to be stored, wherein the attribute of the data to be stored defaults to cold data, and the attribute of the data comprises hot data or cold data;
storing the attribute information of the data to be stored into a data management area, wherein the data management area is a storage space arranged in a memory;
Judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, wherein the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data;
When the data to be stored is hot data, storing the hot data into a hard disk, and modifying attribute information of the data to be stored in the data management area, wherein the method comprises the following steps: modifying attribute information, which is determined to be hot data by judging an access frequency, from cold data to hot data in a data management area;
acquiring a storage position of the thermal data in a hard disk, and storing storage position information of the thermal data in the hard disk into the data management area;
writing all the data to be stored into an optical disc library;
acquiring storage positions of all the data to be stored in the optical disc library, and storing storage position information of all the data to be stored in the optical disc library into the data management area;
the method further comprises the steps of:
Judging whether the current access frequency of the thermal data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval;
when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
2. The method for storing data according to claim 1, further comprising, after acquiring the data to be stored: and identifying the data to be stored, storing the identification information of the data to be stored into the data management area, wherein the identification information of the data to be stored is unique.
3. A method of reading data, comprising:
acquiring data to be read;
Judging whether the attribute of the data to be read is hot data or not; the thermal data refers to data stored in a hard disk by the data storage method according to claim 1 or 2;
When the data to be read is hot data, directly acquiring hard disk storage position information of the hot data in the data management area;
reading the thermal data according to the hard disk storage position information;
the method further comprises the steps of:
Judging whether the current access frequency of the thermal data stored in the hard disk is lower than a preset access frequency or not according to a preset time interval;
when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
4. A method of reading data according to claim 3, further comprising:
When the data is cold data, acquiring optical disc library storage position information of the cold data in a data management area;
acquiring the cold data according to the storage position information of the optical disc library;
And reading the obtained cold data into a hard disk, and reading and processing.
5. A data storage device comprising
The first acquisition module is used for acquiring data to be stored, wherein the attribute of the data to be stored defaults to cold data, the attribute of the data comprises hot data or cold data, and the first acquisition module is also used for storing the attribute information of the data to be stored into a data management area, and the data management area is a storage space arranged in a memory;
The first judging module is used for judging whether the access frequency of the data to be stored is greater than or equal to a preset access frequency, wherein the data with the access frequency greater than or equal to the preset access frequency is hot data, and the data with the access frequency less than the preset access frequency is cold data;
The first storage module is configured to store the hot data in a hard disk when the data to be stored is hot data, and modify attribute information of the data to be stored in the data management area, where the first storage module includes: modifying attribute information, which is determined to be hot data by judging an access frequency, from cold data to hot data in a data management area;
The second acquisition module is used for acquiring the storage position of the thermal data in the hard disk and storing the storage position information of the thermal data in the hard disk into the data management area;
The second storage module is used for writing all the data to be stored into the optical disc library;
the third acquisition module is used for acquiring storage positions of all the data to be stored in the optical disc library, and storing storage position information of all the data to be stored in the optical disc library into the data management area;
The apparatus further comprises:
The data updating module is used for judging whether the current access frequency of the thermal data stored in the hard disk is lower than the preset access frequency according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
6. A data reading apparatus, comprising:
a fourth acquisition module, configured to acquire data to be read;
The second judging module is used for judging whether the attribute of the data to be read is hot data or not; the thermal data is data stored in a hard disk by the data storage device according to claim 5;
A fifth obtaining module, configured to directly obtain hard disk storage location information of the hot data in the data management area when the data to be read is hot data;
the reading module is used for reading the thermal data according to the hard disk storage position information;
The apparatus further comprises:
The data updating module is used for judging whether the current access frequency of the thermal data stored in the hard disk is lower than the preset access frequency according to the preset time interval; when the current access frequency of the hot data is lower than the preset access frequency, clearing the hot data in the hard disk, feeding back clearing information of the hot data to a data management area, modifying attribute information of the hot data, and deleting hard disk storage position information of the hot data in the data management area.
7. A data storage system, comprising:
at least one control device for performing the method for storing data according to claim 1 or 2, storing the received data to be stored.
8. A data reading system, comprising:
At least one control device for performing the method of reading data according to claim 3 or 4.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, carries out the steps of the method of storing data according to claim 1 or 2 or the method of reading data according to claim 3 or 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310870.8A CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911310870.8A CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111124303A CN111124303A (en) | 2020-05-08 |
| CN111124303B true CN111124303B (en) | 2024-06-07 |
Family
ID=70499742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911310870.8A Active CN111124303B (en) | 2019-12-18 | 2019-12-18 | Data storage method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111124303B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782148B (en) * | 2020-06-30 | 2023-09-05 | 中国工商银行股份有限公司 | Data storage control method and device, electronic equipment and storage medium |
| CN112379842A (en) * | 2020-11-18 | 2021-02-19 | 深圳安捷丽新技术有限公司 | Method and device for predicting cold and hot properties of data |
| CN114816232B (en) * | 2021-07-27 | 2022-11-08 | 云南省地质环境监测院(云南省环境地质研究院) | Method and device for efficiently accessing geological disaster big data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
| CN106547474A (en) * | 2015-09-22 | 2017-03-29 | 辛力彬 | A kind of box mixing data storage center |
| CN108958663A (en) * | 2018-07-23 | 2018-12-07 | 南京叠嘉信息科技有限公司 | Data read-write method based on hard disk and CD mixing storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10782888B2 (en) * | 2018-02-27 | 2020-09-22 | Cisco Technology, Inc. | Method and device for improving file system write bandwidth through hard disk track management |
-
2019
- 2019-12-18 CN CN201911310870.8A patent/CN111124303B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
| CN106547474A (en) * | 2015-09-22 | 2017-03-29 | 辛力彬 | A kind of box mixing data storage center |
| CN108958663A (en) * | 2018-07-23 | 2018-12-07 | 南京叠嘉信息科技有限公司 | Data read-write method based on hard disk and CD mixing storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111124303A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111124303B (en) | Data storage method, device and system | |
| US9552174B2 (en) | Method and system for preventing unreliable data operations at cold temperatures | |
| KR102134694B1 (en) | Method, computer-readable storage media and storage controller module for managing storage regions on a storage device | |
| US9373354B2 (en) | Method and system for preventing unreliable data operations at cold temperatures | |
| CN102215268A (en) | Method and device for transferring file data | |
| RU2010130774A (en) | METHOD FOR SELECTING MULTIPLE MEMORY CELLS (OPTIONS) | |
| CN104516471A (en) | Method and device for managing power supply of storage system | |
| CN107291399B (en) | Back-end storage method, device and system based on SPDK | |
| US20140286150A1 (en) | Data archive system and quality inspection method | |
| CN103076993A (en) | Storage system and method for concentration type system | |
| CN109491589A (en) | A kind of delamination process and device based on file content | |
| US20070168603A1 (en) | Information recording apparatus and control method thereof | |
| CN105183374A (en) | Data read and write method and mainboard | |
| US20170212711A1 (en) | Disk apparatus and control method | |
| CN102222033B (en) | A kind of method and device for preserving small computer system interface access error | |
| US20170329553A1 (en) | Storage control device, storage system, and computer-readable recording medium | |
| US20160070648A1 (en) | Data storage system and operation method thereof | |
| CN106708445B (en) | Link selecting method and device | |
| JP2016134050A (en) | Data library system | |
| US8576675B2 (en) | Data library apparatus, data archive system and server | |
| JP2016009510A (en) | Techniques for reducing time required for erasing recorded data by utilizing read head and write head | |
| CN104156327A (en) | Method for recognizing object power failure in write back mode in distributed file system | |
| US9588898B1 (en) | Fullness control for media-based cache operating in a steady state | |
| US11682428B2 (en) | Hard disk format conversion method and apparatus, and storage device | |
| CN105242985B (en) | Data recovery method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |