CN113655963B - Data storage system, method and computer equipment based on memory bridging - Google Patents
Data storage system, method and computer equipment based on memory bridging Download PDFInfo
- Publication number
- CN113655963B CN113655963B CN202110952956.1A CN202110952956A CN113655963B CN 113655963 B CN113655963 B CN 113655963B CN 202110952956 A CN202110952956 A CN 202110952956A CN 113655963 B CN113655963 B CN 113655963B
- Authority
- CN
- China
- Prior art keywords
- disk
- memory
- data
- module
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012544 monitoring process Methods 0.000 claims abstract description 71
- 238000007726 management method Methods 0.000 claims abstract description 28
- 238000001514 detection method Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 5
- 230000000903 blocking effect Effects 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0664—Virtualisation aspects at device level, e.g. emulation of a storage device or system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a data storage system, a method and computer equipment based on memory bridging, wherein the system comprises: the memory bridging read-write subsystem comprises: the virtual disk management system comprises a virtual memory disk, a virtual disk management module and a second storage decision module; a disk read-write subsystem, comprising: the system comprises a disk monitoring module and a first storage decision module, wherein the disk monitoring module is used for acquiring disk monitoring information and data statistics information; the first storage decision module is used for determining a target address of a write disk according to the statistical information, writing data into a disk, or writing the data into a virtual memory disk; the virtual disk management module is used for acquiring memory monitoring information of the virtual memory disk; and the second storage decision module is used for executing a scheduling instruction according to the memory monitoring information and the statistical information and writing the data in the virtual memory disk into the disk. The invention adopts the memory as the virtual disk to execute data read-write, response feedback and disk write operation, thereby improving the read-write speed and the disk storage reliability of the storage device.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a data storage system, method and computer device based on memory bridging.
Background
With the development of intelligent technology, the application scenario related to the intelligent device is greatly expanded, the service logics of system suppliers and application developers of the intelligent device are continuously enriched, and simultaneously, higher requirements are put on the performance of the system and application programs, wherein the I/O (input/output, generally understood as read-write operation of a disk) performance is an important index for measuring the performance of the system.
At present, the ways of improving IO performance are mainly limited by the access rate of a storage medium, the system cost and the restriction of the IO architecture of a traditional operating system, and include: the IO operation is reduced, and the storage hardware with faster reading and writing speed is configured, however, a larger gap still exists between the speed improving effect of the storage hardware and the increasing requirement of IO performance, so that the operation of reading and writing the disk is reduced as much as possible on the basis of improving the storage hardware.
In the prior art, in order to ensure the consistency of the response state of the IO operation and the real writing of the IO content into the storage device, the realization of the upper layer disk storage interface usually adopts a synchronous mode, and the state is returned to the upper layer user after the real writing of the IO content into the storage device, which has the problems that the data reading and writing speed is slow, the IO blocking of the upper layer application program is easy to cause, the IO performance is influenced, even the normal operation of the business logic is influenced, and the user experience is poor.
Disclosure of Invention
The invention provides a data storage system, a method and computer equipment based on memory bridging, which are used for realizing that a memory is used as a virtual disk to execute data writing and response feedback, and simultaneously, the memory can also write the stored data into the disk, so that the reading and writing speed of the storage equipment is improved, and the reliability is high.
In a first aspect, an embodiment of the present invention provides a data storage system based on memory bridging, including: the system comprises a disk read-write subsystem, a memory bridging read-write subsystem and a disk, wherein the disk read-write subsystem comprises a disk monitoring module and a first storage decision module, and the memory bridging read-write subsystem comprises a virtual memory disk, a virtual disk management module and a second storage decision module; the disk monitoring module is used for acquiring disk monitoring information of the disk and statistical information of upper application program data, and issuing scheduling instructions to the first storage decision module and the second storage decision module according to the disk monitoring information and the statistical information; the first storage decision module is used for determining a target address of a write disk according to the statistical information, writing data into a disk, or writing the data into the virtual memory disk; the virtual disk management module is used for acquiring memory monitoring information of the virtual memory disk; and the second storage decision module is used for executing a scheduling instruction according to the memory monitoring information and the statistical information and writing the data in the virtual memory disk into the disk.
Optionally, the memory bridging read-write subsystem further includes: and the auxiliary power supply module is used for supplying power to the disk and the power utilization module in the memory bridging read-write subsystem when the main power supply is powered off, so that the memory bridging read-write subsystem writes data into the disk.
Optionally, the virtual disk management module is connected to an electric quantity query port of the auxiliary power module, and is configured to obtain a real-time residual electric quantity of the auxiliary power module, determine a maximum capacity threshold of the virtual memory disk according to the real-time residual electric quantity, and determine an available data storage area of the virtual memory disk.
Optionally, the disk monitoring information includes a disk residual capacity, and the memory monitoring information includes a memory disk occupation amount; the disk monitoring module is used for acquiring the occupation amount of the memory disk, and issuing a disk writing scheduling instruction to the second storage decision module according to the residual capacity of the disk and the occupation amount of the memory disk, so that the second storage decision module writes the data in the virtual memory disk into the disk.
Optionally, the first storage decision module is configured to determine whether the current operation is a high-frequency operation according to the statistical information, and determine a write target address according to a determination result.
Optionally, the memory monitoring information further includes: one or more of the number of writing of the memory disk buffer area, the power-down detection information of the main power supply of the memory disk and the scheduling instruction issued by the disk monitoring module.
Optionally, the second storage decision module stores a preset write disc policy, where the preset write disc policy includes: acquiring statistical information of data to be written in the virtual memory disk, wherein the statistical information comprises one or more of file read-write time, file size, file read-write times, file last closing time or user-defined priority; sorting the priority of the data to be written based on the statistical information; and according to the priority ordering result, writing the data to be written into the disk from high to low in sequence.
Optionally, in the data reading stage, the first storage decision module is configured to perform a data reading operation on the virtual memory disk; and when the data is not stored in the virtual memory disk, the first storage decision module is also used for executing data reading operation on the disk and writing the read data into the virtual memory disk.
In a second aspect, an embodiment of the present invention further provides a data storage method based on memory bridging, for implementing operation data storage by using a disk and a virtual memory disk, where the method includes the following steps: acquiring disc monitoring information of the disc and statistical information of the operation data; issuing a scheduling instruction to the first storage decision module and the second storage decision module according to the disk monitoring information and the statistical information; determining a write-disc target address according to the statistical information by adopting the first storage decision module, and writing data into a magnetic disc according to the write-disc target address, or writing the data into the virtual memory disc; a virtual disk management module is adopted to acquire memory monitoring information of the virtual memory disk; and writing the data in the virtual memory disk into the disk by adopting the second memory decision module according to the memory monitoring information and the statistical information.
In a third aspect, an embodiment of the present invention further provides a computer device, including the memory bridge-based data storage system described above.
The system is provided with a disk read-write subsystem, a memory bridge read-write subsystem and a disk, wherein the disk read-write subsystem is used for acquiring disk monitoring information of the disk and statistical information of upper application program operation data, and issuing a scheduling instruction to a first storage decision module of the disk read-write subsystem and a second storage decision module of the memory bridge read-write subsystem, wherein the first storage decision module determines a target address of the disk according to the statistical information and writes the data into the disk or writes the data into a virtual internal memory disk; the memory monitoring information of the virtual memory disk is obtained through the virtual disk management module, the second memory decision module writes the data in the virtual memory disk into the disk according to the memory monitoring information and the statistical information, the memory is used as the virtual disk to execute data reading and writing, response feedback and disk writing operation, the problem that an IO port of an upper application is blocked due to low reading and writing speed of the existing storage device is solved, the reading and writing speed of the storage device is improved, the reliability of the memory disk is improved, the running effect of an upper application is improved, and the user experience is improved.
Drawings
FIG. 1 is a schematic diagram of a general memory architecture in the related art;
FIG. 2 is a schematic diagram of a data storage system based on memory bridging according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of another memory bridge-based data storage system according to an embodiment of the present invention;
fig. 4 is a flowchart of a data storage method based on memory bridging according to a second embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Fig. 1 is a schematic structural diagram of a general memory architecture in the related art.
As shown in fig. 1, in the general storage architecture, an upper layer application 01 may write data into a file system 03 via a buffer unit 02 or in a direct write manner, where the file system 03 writes data into the disk D one by one through a DMA (Direct Memory Access ) module 04 according to a preset write disk rule. When the DMA module 04 performs a disc writing operation, the response speed of the disc D is relatively slow, if an abnormality occurs in the process of writing data into the disc D (for example, power failure or insufficient disc space), the data writing will fail, and in order to return a correct state to the user, the upper layer call will wait until the data writing is completed, resulting in slow data writing speed and an IO blocking problem of the application program.
Based on the above, the embodiment of the invention provides a data storage system, a method and computer equipment based on memory bridging, and the data storage system realizes data reading and writing based on a general storage architecture and a memory bridging mode, thereby achieving the technical effects of improving the data reading and writing speed and improving the system reliability.
Example 1
Fig. 2 is a schematic structural diagram of a data storage system based on memory bridging according to an embodiment of the present invention, where the embodiment is applicable to an application scenario with high IO requirements, where the scenario with high IO requirements includes a Web (World Wide Web) and an FTP (File Transfer Protocol, file transfer) server system.
As shown in fig. 2, the memory bridge-based data storage system 00 includes: the system comprises a disk read-write subsystem 100, a memory bridging read-write subsystem 200 and a disk D, wherein the disk read-write subsystem 100 comprises a disk monitoring module 101 and a first storage decision module 102; the memory bridge read/write subsystem 200 includes a virtual memory disk 201, a virtual disk management module 202, and a second storage decision module 203.
In this embodiment, the first storage decision module 102 may write data into the disk D through the first DMA module 103, or write data into the virtual memory disk 201 through the first DMA module 103, and feed back status information to the upper layer application 01, where the status information includes information such as insufficient disk space, abnormal power failure, or complete data writing, and the second storage decision module 203 may write data stored in the virtual memory disk 201 into the disk D through the second DMA module 204.
Referring to fig. 2, a disk monitoring module 101 is configured to obtain disk monitoring information of a disk D and statistical information of operation data of an upper application program 01, and issue scheduling instructions to a first storage decision module 102 and a second storage decision module 203 according to the disk monitoring information and the statistics; the first storage decision module 102 is configured to determine a write-disk target address according to the statistical information, and write data to the disk D according to the write-disk target address, or write data to the virtual memory disk 201; a virtual disk management module 202, configured to obtain memory monitoring information of the virtual memory disk 201; the second storage decision module 203 is configured to execute a scheduling instruction according to the memory monitoring information and the statistics information, and write the data in the virtual memory disc 201 into the disc.
Optionally, the statistical information may include one or more of file read-write time, file size, number of file read-write times, last closing time of the file, or user-defined priority; the disk monitoring information may include disk remaining capacity, which is used to reflect the available space in disk D.
Specifically, for the write operation of the upper application IO interface, the disk monitoring module 101 monitors the physical disk D in real time, and counts the write operation data, where the disk monitoring module 101 may determine the priority of the write data according to the statistics information, write the data with higher priority into the virtual memory 201 preferentially, and identify the data as W (indicating that the write is needed), write the data with lower priority directly into the disk D, determine whether there is a risk of insufficient disk space according to the disk monitoring information, schedule the second storage decision module 203 to trigger the write operation according to the determination result, and issue a scheduling instruction to the first storage decision module 102, so that the first storage decision module 102 invokes a notification of insufficient disk space for the IO interface of the upper application 01, thereby implementing the execution of data read-write, response feedback and write operation with the memory as the virtual disk, solving the problem of the blockage of the upper application IO interface caused by the slow read-write speed of the existing storage device, which is beneficial to improving the read-write speed of the storage device, improving the reliability of the upper application, and improving the user experience.
Optionally, fig. 3 is a schematic structural diagram of another memory bridge-based data storage system according to the first embodiment of the present invention, and on the basis of fig. 2, a design mechanism of an auxiliary power supply is added to the memory bridge read/write subsystem 200.
As shown in fig. 3, the memory bridge read/write subsystem 200 further includes: the auxiliary power module 205, when the main power is turned off, the auxiliary power module 205 is configured to supply power to the disk D and the power module in the memory bridge read/write subsystem 200, so that the memory bridge read/write subsystem 200 writes data into the disk D.
In this embodiment, the main power supply is used to supply power to the disk read-write subsystem 100, the memory bridge read-write subsystem 200 and the disk D,
alternatively, the auxiliary power module 205 may include a backup battery assembly and a battery management unit, where the battery management unit may be configured to provide a power inquiry interface for the backup battery assembly and trigger a power change notification, and the auxiliary power module 205 may switch the backup battery assembly into the system when the main power supply is powered down.
Specifically, if the auxiliary power module 205 determines that the main power is powered down, the auxiliary power module 205 starts and supplies power to the necessary modules for performing the data writing operation, where the necessary modules for performing the data writing operation may include modules such as the virtual memory disc 201, the second memory decision module 203, the CPU, the second DMA module 204, and the disk D. Therefore, by introducing the auxiliary power module, the reliability of the bottom-layer disk storage operation of the storage system is improved, and the problem of IO interface blocking is further improved.
Optionally, as shown in fig. 3, the virtual disk management module 202 is connected to the power query interface of the auxiliary power module 205, and the virtual disk management module 202 is configured to obtain the real-time remaining power Q of the auxiliary power module 205 0 And according to the real-time residual quantity Q 0 Determining a maximum capacity threshold S of virtual disk 201 V The maximum capacity threshold S V Available data storage area of virtual disk 201 may be characterized at the current time.
When performing a data writing operation, the virtual disk management module 202 acquires the real-time remaining power Q of the auxiliary power module 205 in real time 0 And according to the real-time residual quantity Q 0 Dynamically calculating a maximum capacity threshold S V The virtual disk management module 202 sorts out the upper bits in the memory address not to exceed the maximum capacity threshold S V Is mounted as a virtual disk for storing data transmitted by the upper layer application 01.
Specifically, to complete the disk writing operation in the power-down scenario of the main power supply, the auxiliary power module 205 needs to continuously supply power to the virtual memory disk 201, the second storage decision module 203, the CPU, the second DMA module 204 and the disk D until the data in the virtual memory disk 201 is completely written to the disk.
If the maximum power consumption of disk D is defined to be equal to Q D The maximum power consumption of virtual disk 201 is equal to Q C The maximum power consumption of CPU is equal to Q CPU The maximum power consumption of the second DMA module 204 is equal to Q DMA The writing speed of the magnetic disk D is equal to T R The virtual disk management module 202 may calculate the maximum capacity threshold S according to equation one as follows V ：
Wherein the maximum capacity threshold S V Available data storage area that can be used to update virtual disk 201.
In this embodiment, the virtual disk management module 202 is further configured to monitor the virtual memory disk 201 and send the memory monitoring information to the second storage decision module 203.
Optionally, the memory monitoring information includes a memory disk occupation amount; the disk monitoring module 101 is configured to obtain a memory disk occupation amount, and issue a disk writing scheduling instruction to the second storage decision module 203 according to the remaining capacity of the disk and the memory disk occupation amount, so that the second storage decision module 203 writes data in the virtual memory disk 201 into the disk.
Specifically, when the remaining capacity of the disk approaches the occupation amount of the memory disk, the disk monitoring module 101 determines that there is a risk of insufficient disk space, issues a disk writing scheduling instruction to the second storage decision module 203, and the second storage decision module 203 writes the data in the virtual memory disk 201 into the disk D through the second DMA module 204, which is beneficial to avoiding failure of disk writing due to insufficient disk space and improving reliability of the system.
Optionally, the first storage decision module 102 is configured to determine whether the current operation is a high frequency operation according to the statistical information, and determine the write target address according to the determination result.
Specifically, in the data writing stage, the first storage decision module 102 obtains the sorting result of the statistical information by the disk monitoring module 101, if the data to be written is located in the first 50% of the sorting, the first storage decision module 102 determines that the current operation is a high-frequency operation, determines the writing target address as the address of the virtual memory disk 201, writes the data into the virtual memory disk 201 by the first DMA module 103, and directly identifies the data as W (indicating that the writing of the disk is required); otherwise, the first storage decision module 102 determines that the current operation is a low frequency operation, determines the write target address as the address of the disk D, and writes the data directly to the disk D by the first DMA module 103. Therefore, the data with higher priority is responded quickly through the memory, the storage result is fed back to the upper interface, IO waiting of a user is avoided, the reading and writing speed is improved, meanwhile, the data with lower priority is directly written into the disk, memory resources are prevented from being occupied, and reliability is improved.
Optionally, the memory monitoring information includes: one or more of memory disk occupation amount, memory disk buffer write times, power down detection information of a main power supply of a memory disk, and scheduling instructions issued by the disk monitoring module 101.
Specifically, the second storage decision module 203 executes a scheduling instruction according to the memory monitoring information, and writes the data in the virtual memory disc to the disc D, which includes at least one of the following policies:
(1) The virtual disk 201 write operation is triggered according to the memory disk occupation amount.
The second storage decision module 203 obtains the memory disk occupation amount of the virtual memory disk 201 through the virtual disk management module 202, determines whether the memory disk occupation amount reaches a preset threshold (for example, the preset may be 70%), and if the memory disk occupation amount reaches the preset threshold, the second storage decision module 203 triggers the writing operation of the virtual memory disk 201.
(2) The virtual disk 201 write operation is triggered according to the number of memory disk buffer writes.
The second storage decision module 203 obtains the buffer write times of each buffer area of the virtual memory disc 201 through the virtual disc management module 202, judges whether the buffer write times reach a preset time threshold, and if the buffer write times reach the preset time threshold, the second storage decision module 203 triggers the virtual memory disc 201 to write a disc.
(3) The virtual memory disc 201 is triggered to write according to the power-down detection information of the main power supply of the memory disc.
The second storage decision module 203 obtains the main power failure detection information through the virtual disk management module 202, judges whether the main power failure occurs, and if the main power failure occurs, the second storage decision module 203 triggers the virtual memory disk 201 to write.
(4) The virtual memory disc 201 writing operation is triggered according to the scheduling instruction.
The second storage decision module 203 obtains a scheduling instruction issued by the disk monitoring module 101, and if the scheduling instruction is a disk writing instruction, the second storage decision module 203 triggers a disk writing operation of the virtual memory disk 201.
Thus, by the above-described at least one write disk scheduling policy, a write disk operation of the virtual disk 201 can be realized, and system reliability can be improved.
Optionally, the second storage decision module 203 stores a preset write disc policy, where the preset write disc policy includes: acquiring statistical information of data to be written in the virtual memory disk 201; sorting the priorities of the data to be written based on the statistical information; and according to the priority ordering result, writing the data to be written into the disk D from high to low in sequence.
Wherein, the statistics information of the data to be written is consistent with the statistics information of the operation data of the upper layer application program 01 in the disk monitoring module 101, including: one or more of file read-write time, file size, file read-write times, file last closing time or user-defined priority.
Optionally, the priority of the data to be written is ordered based on the statistical information, including the following strategies:
a) And acquiring file read-write time of data to be written, wherein the earlier the file read-write time is, the higher the priority of the corresponding data is.
b) And acquiring user-defined priority of the data to be written, wherein the higher the user-defined priority is, the higher the priority of the corresponding data is.
c) And acquiring the file size of the data to be written, wherein the higher the file size is, the higher the disk writing cost is, and the lower the priority of the corresponding data is.
d) Acquiring the read-write times of the file of the data to be written, wherein the more the file is opened, the more frequent the file is operated,
e) And acquiring the last closing time of the file to be written with the data, wherein the closer the last closing time of the file is to the current time, the lower the probability of reusing the file is, and the lower the priority of the corresponding data is.
Further, the second storage decision module 203 may set different weight values for the different priority ranking policies, calculate the final data priority according to the priority weights, and sequentially write the data to be written into the disk D from high to low according to the priority. Therefore, by optimizing the write disc strategy, the write disc cost can be reduced, and the read-write efficiency can be improved.
Optionally, during the data reading phase, the first storage decision module 102 is configured to perform a data reading operation on the virtual memory disc 201; the first storage decision module 102 is further configured to perform a data reading operation on the disk when no data is stored in the virtual memory disk 201, and write the read data to the virtual memory disk 201.
Specifically, for the read operation of the upper application IO interface, the first storage decision module 102 first queries from the virtual memory disk 201, if no data is stored in the virtual memory disk 201, the first storage decision module 102 directly reads the data from the disk D through the first DMA module 103, and simultaneously writes the data into the virtual memory disk 201 through the first DMA module 103 and identifies the data as R (indicating that the data does not need to be written into the disk, only for querying), and after the main power is turned off, the data identified as R in the virtual memory disk 201 does not need to be written into the disk D.
Optionally, the upper layer application 01 may further update the data in the virtual memory disc 201 through the disc monitoring module 101, the first storage decision module 102 and the first DMA module 103, and if the identifier of the updated data is R, the updated data is identified as W.
Therefore, the embodiment of the invention adopts the virtual memory disc 201 to execute data read-write, thereby being beneficial to improving the read-write speed of the storage device, improving the reliability of the memory disc, improving the running effect of an upper application program and improving the user experience.
Example two
The second embodiment of the invention also provides a data storage method based on memory bridging, which adopts a magnetic disk and a virtual memory disk to realize operation data storage.
Fig. 4 is a flowchart of a data storage method based on memory bridging according to a second embodiment of the present invention.
As shown in fig. 4, the memory bridging-based data storage method specifically includes the following steps:
step S1: and acquiring the disc monitoring information and the statistical information of the operation data of the disc.
Step S2: and issuing a scheduling instruction to the first storage decision module and the second storage decision module according to the disk monitoring information and the statistical information.
Step S3: and judging whether the target address of the write disk is a virtual disk or not according to the statistical information by adopting a first storage decision module.
If the target address of the writing disk is a magnetic disk, executing the step S4; if the write disc target address is the virtual disc, step S5 is executed.
Step S4: the first storage decision module writes the operational data to the disk.
Step S5: the first storage decision module writes the operation data to the virtual disk.
Step S6: and acquiring memory monitoring information of the virtual memory disk by adopting a virtual disk management module.
Step S7: and writing the data in the virtual memory disc into the disc by adopting a second memory decision module according to the memory monitoring information and the statistical information.
Optionally, the memory bridging-based data storage method further includes the following steps: judging whether the main power supply is powered down, if so, adopting an auxiliary power supply module to supply power to the disk and a power utilization module in the memory bridging read-write subsystem so as to enable the memory bridging read-write subsystem to write data into the disk.
Optionally, the memory bridging-based data storage method further includes: the method comprises the steps that a virtual disk management module is adopted to obtain the real-time residual electric quantity of an auxiliary power module, and the maximum capacity threshold value of a virtual memory disk is determined according to the real-time residual electric quantity; the maximum capacity threshold is used to determine the available data storage area in the virtual memory disk.
Optionally, the disk monitoring information includes a disk residual capacity, and the memory monitoring information includes a memory disk occupation amount; the data storage method based on memory bridging further comprises the following steps: and acquiring the occupation amount of the memory disk, and issuing a disk writing scheduling instruction to the second storage decision module according to the residual capacity of the disk and the occupation amount of the memory disk so that the second storage decision module writes the data in the virtual memory disk into the disk.
Optionally, when executing step S2, the memory bridge-based data storage method further includes: judging whether the current operation is high-frequency operation according to the statistical information, and determining a writing target address according to the judgment result.
Optionally, the memory monitoring information further includes: one or more of the number of writing of the memory disk buffer area, the power-down detection information of the main power supply of the memory disk and the scheduling instruction issued by the disk monitoring module.
Optionally, the memory bridging-based data storage method further includes: performing a write disc operation of the second storage decision module based on a preset write disc measurement, the preset write disc strategy comprising: acquiring statistical information of data to be written in the virtual memory disk, wherein the statistical information comprises one or more of file read-write time, file size, file read-write times, file last closing time or user-defined priority; sorting the priorities of the data to be written based on the statistical information; and according to the priority ordering result, writing the data to be written into the disk from high to low in sequence.
Optionally, during the data reading stage, the memory bridging-based data storage method further includes: executing data reading operation on the virtual memory disk by adopting a first storage decision module; when the data is not stored in the virtual memory disk, the method further comprises: and executing data reading operation on the disk by adopting a first storage decision module, and writing the read data into the virtual memory disk.
Therefore, according to the data storage method based on memory bridging, the disk monitoring information of the disk and the statistical information of the operation data of the upper application program are obtained through the disk read-write subsystem, the scheduling instruction is issued to the first storage decision module and the second storage decision module according to the disk monitoring information and the statistical information, and the first storage decision module determines a disk write target address according to the statistical information, writes data into the disk, or writes the data into the virtual memory disk; the memory monitoring information of the virtual memory disk is obtained through the virtual disk management module, the second memory decision module writes the data in the virtual memory disk into the disk according to the memory monitoring information and the statistical information, the memory is used as the virtual disk to execute data reading and writing, response feedback and disk writing operation, the problem that an IO port of an upper application is blocked due to low reading and writing speed of the existing storage device is solved, the reading and writing speed of the storage device is improved, the reliability of the memory disk is improved, the running effect of an upper application is improved, and the user experience is improved.
Example III
Based on the above embodiments, a third embodiment of the present invention provides a computer device.
Fig. 5 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
As shown in fig. 5, the computer device 1 includes the memory bridge-based data storage system 00 described above.
Optionally, the computer device includes a Web server and an FTP server.
The computer equipment provided by the embodiment of the invention is provided with the data storage system based on memory bridging in any embodiment, has the same functional modules and beneficial effects of the system, realizes that the memory is used as a virtual disk to execute data read-write, response feedback and disk write operation, solves the problem of upper application IO port blocking caused by low read-write speed of the existing storage equipment, is beneficial to improving the read-write speed of the storage equipment, improves the reliability of the storage disk, improves the operation effect of upper application programs, improves the operation performance of the computer equipment and improves the user experience.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (9)
1. A memory bridging-based data storage system, comprising: the system comprises a disk read-write subsystem, a memory bridging read-write subsystem and a disk, wherein the disk read-write subsystem comprises a disk monitoring module and a first storage decision module, and the memory bridging read-write subsystem comprises a virtual memory disk, a virtual disk management module and a second storage decision module;
the disk monitoring module is used for acquiring disk monitoring information of the disk and statistical information of upper application program data, and issuing scheduling instructions to the first storage decision module and the second storage decision module according to the disk monitoring information and the statistical information;
the first storage decision module is used for determining a target address of a write disk according to the statistical information, writing data into a disk, or writing the data into the virtual memory disk;
the virtual disk management module is used for acquiring memory monitoring information of the virtual memory disk;
the second storage decision module is used for executing a scheduling instruction according to the memory monitoring information and the statistical information and writing the data in the virtual memory disk into the disk;
the memory bridging read-write subsystem further comprises: the auxiliary power supply module is used for continuously supplying power to the virtual memory disk, the second memory decision module, the CPU, the second DMA module and the magnetic disk when the main power supply is powered off;
the virtual disk management module is connected with the electric quantity inquiry port of the auxiliary power supply module, and is used for acquiring the real-time residual electric quantity of the auxiliary power supply module, determining a maximum capacity threshold of the virtual memory disk according to the real-time residual electric quantity, the maximum electric quantity of the magnetic disk, the maximum electric quantity of the virtual memory disk, the maximum electric quantity of the CPU, the maximum electric quantity of the second DMA module and the disk writing speed of the magnetic disk, and determining an available data storage area of the virtual memory disk.
2. The memory bridge based data storage system of claim 1, wherein the auxiliary power module is configured to power the disk and the power module in the memory bridge read/write subsystem when the main power is turned off, so that the memory bridge read/write subsystem writes data to the disk.
3. The memory bridge-based data storage system of claim 1, wherein the disk monitoring information comprises disk remaining capacity, and the memory monitoring information comprises memory disk occupancy;
the disk monitoring module is used for acquiring the occupation amount of the memory disk, and issuing a disk writing scheduling instruction to the second storage decision module according to the residual capacity of the disk and the occupation amount of the memory disk, so that the second storage decision module writes the data in the virtual memory disk into the disk.
4. The memory bridging-based data storage system according to claim 1, wherein the first storage decision module is configured to determine whether a current operation is a high frequency operation according to the statistics, and determine a write target address according to a determination result.
5. The memory bridge based data storage system of claim 1, wherein the memory monitoring information further comprises: one or more of the number of writing of the memory disk buffer area, the power-down detection information of the main power supply of the memory disk and the scheduling instruction issued by the disk monitoring module.
6. The memory bridging-based data storage system according to any one of claims 1-5, wherein the second storage decision module stores a preset write disk policy, the preset write disk policy comprising:
acquiring statistical information of data to be written in the virtual memory disk, wherein the statistical information comprises one or more of file read-write time, file size, file read-write times, file last closing time or user-defined priority;
sorting the priority of the data to be written based on the statistical information;
and according to the priority ordering result, writing the data to be written into the disk from high to low in sequence.
7. The memory bridging-based data storage system according to any one of claims 1-5, wherein during a data read phase, the first storage decision module is configured to perform a data read operation on the virtual memory disk;
and when the data is not stored in the virtual memory disk, the first storage decision module is also used for executing data reading operation on the disk and writing the read data into the virtual memory disk.
8. The data storage method based on memory bridging is characterized in that a disk and a virtual memory disk are adopted to realize operation data storage, and the method comprises the following steps:
acquiring disc monitoring information of the disc and statistical information of the operation data;
issuing a scheduling instruction to a first storage decision module and a second storage decision module according to the disk monitoring information and the statistical information;
determining a target address of a disk to be written according to the statistical information by adopting the first storage decision module, and writing data into a disk, or writing the data into the virtual memory disk;
a virtual disk management module is adopted to acquire memory monitoring information of the virtual memory disk;
writing the data in the virtual memory disk into the disk by adopting the second memory decision module according to the memory monitoring information and the statistical information;
when the main power supply is powered off, an auxiliary power module is adopted to continuously supply power to the virtual memory disk, the second memory decision module, the CPU, the second DMA module and the disk;
and acquiring the real-time residual electric quantity of the auxiliary power supply module, and determining a maximum capacity threshold of the virtual memory disk according to the real-time residual electric quantity, the maximum electric quantity of the magnetic disk, the maximum electric quantity of the virtual memory disk, the maximum electric quantity of the CPU, the maximum electric quantity of the second DMA module and the disk writing speed of the magnetic disk, wherein the maximum capacity threshold is used for determining an available data storage area of the virtual memory disk.
9. A computer device comprising the memory bridge-based data storage system of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952956.1A CN113655963B (en) | 2021-08-19 | 2021-08-19 | Data storage system, method and computer equipment based on memory bridging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952956.1A CN113655963B (en) | 2021-08-19 | 2021-08-19 | Data storage system, method and computer equipment based on memory bridging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113655963A CN113655963A (en) | 2021-11-16 |
| CN113655963B true CN113655963B (en) | 2024-03-01 |
Family
ID=78492312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110952956.1A Active CN113655963B (en) | 2021-08-19 | 2021-08-19 | Data storage system, method and computer equipment based on memory bridging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113655963B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298619A (en) * | 2014-09-26 | 2015-01-21 | 北京控制工程研究所 | High-speed two-stage storage system based on Ramdisk and solid state disk and data storage method |
| CN104965793A (en) * | 2014-09-26 | 2015-10-07 | 浙江大华技术股份有限公司 | Cloud storage data node apparatus |
| CN107168657A (en) * | 2017-06-15 | 2017-09-15 | 深圳市云舒网络技术有限公司 | It is a kind of that cache design method is layered based on the virtual disk that distributed block is stored |
| CN109062505A (en) * | 2018-07-13 | 2018-12-21 | 南瑞集团有限公司 | A kind of write performance optimization method under cache policy write-in layering hardware structure |
| CN109388221A (en) * | 2018-10-09 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of storage system and its standby fulgurite reason method, apparatus and equipment |
| CN112000574A (en) * | 2020-08-18 | 2020-11-27 | 中国建设银行股份有限公司 | Pressure testing method, device, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9075528B2 (en) * | 2011-11-17 | 2015-07-07 | Jade Quantum Technologies, Inc | High-efficiency virtual disk management system |
-
2021
- 2021-08-19 CN CN202110952956.1A patent/CN113655963B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298619A (en) * | 2014-09-26 | 2015-01-21 | 北京控制工程研究所 | High-speed two-stage storage system based on Ramdisk and solid state disk and data storage method |
| CN104965793A (en) * | 2014-09-26 | 2015-10-07 | 浙江大华技术股份有限公司 | Cloud storage data node apparatus |
| CN107168657A (en) * | 2017-06-15 | 2017-09-15 | 深圳市云舒网络技术有限公司 | It is a kind of that cache design method is layered based on the virtual disk that distributed block is stored |
| CN109062505A (en) * | 2018-07-13 | 2018-12-21 | 南瑞集团有限公司 | A kind of write performance optimization method under cache policy write-in layering hardware structure |
| CN109388221A (en) * | 2018-10-09 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of storage system and its standby fulgurite reason method, apparatus and equipment |
| CN112000574A (en) * | 2020-08-18 | 2020-11-27 | 中国建设银行股份有限公司 | Pressure testing method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113655963A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8146092B2 (en) | System and method for selecting and executing an optimal load distribution processing in a storage system | |
| CN102741775B (en) | For the methods, devices and systems changed the system power states of computer platform | |
| TW201942738A (en) | Electronic device, computer system, and control method | |
| US7536229B2 (en) | Information processing apparatus, information processing method, and computer program | |
| US20070101059A1 (en) | Storage control system and control method for storage control which suppress the amount of power consumed by the storage control system | |
| CN1624628A (en) | Method and system for power management including device use evaluation and power-state control | |
| US20110246720A1 (en) | Storage system with multiple controllers | |
| CN111475012B (en) | Dynamic power routing to hardware accelerators | |
| US11914894B2 (en) | Using scheduling tags in host compute commands to manage host compute task execution by a storage device in a storage system | |
| JP2019153248A (en) | Migration program, migration method, and server indicating migration | |
| Rahmani et al. | Burst‐aware virtual machine migration for improving performance in the cloud | |
| WO2021164163A1 (en) | Request processing method and apparatus, device and storage medium | |
| CN115809072A (en) | Method, device and equipment for upgrading process and storage medium | |
| CN113655963B (en) | Data storage system, method and computer equipment based on memory bridging | |
| US20090265569A1 (en) | Power control method for computer system | |
| CN104460938A (en) | System-wide power conservation method and system using memory cache | |
| JP2007272721A (en) | Storage device and its control method | |
| US11983578B2 (en) | Provisioning a computing subsystem including disaggregated hardware resources that comply with a power domain requirement for a workload | |
| US11372560B2 (en) | Utilization-efficient resource allocation | |
| US11422702B2 (en) | Managing utilization of storage class memory (SCM) resources | |
| WO2024114728A1 (en) | Heterogeneous processor and related scheduling method | |
| US11972148B2 (en) | Proactive storage operation management using thermal states | |
| CN112306781B (en) | Thread fault processing method, device, medium and equipment | |
| WO2024001994A1 (en) | Energy-saving management method and apparatus, and computing device and computer-readable storage medium | |
| CN117112180B (en) | Task-based cluster automation control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: No.15 Xinghuo Road, Jiangbei new district, Nanjing, Jiangsu Province, 210031 Applicant after: Dongji Technology Co.,Ltd. Address before: No.15 Xinghuo Road, Jiangbei new district, Nanjing, Jiangsu Province, 210031 Applicant before: JIANGSU SEUIC TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |