CN111399753A - Method and device for writing pictures - Google Patents
Method and device for writing pictures Download PDFInfo
- Publication number
- CN111399753A CN111399753A CN201910786664.8A CN201910786664A CN111399753A CN 111399753 A CN111399753 A CN 111399753A CN 201910786664 A CN201910786664 A CN 201910786664A CN 111399753 A CN111399753 A CN 111399753A
- Authority
- CN
- China
- Prior art keywords
- target
- picture
- storage node
- memory
- target picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The disclosure provides a method and a device for writing in pictures, and belongs to the technical field of storage machines. The method is applied to the storage node in the cloud storage system, and the storage node is configured with a memory, a disk and an SSD. The method comprises the following steps: the target storage node receives a write-in request of a target picture, judges whether a picture resource pool to which the target storage node belongs supports accelerated write-in or not, acquires a configured cache type in configuration information of the storage node if the data volume of the target picture is smaller than a target numerical value when the picture resource pool to which the target storage node belongs supports accelerated write-in, writes the target picture into a memory of the target storage node if the cache type is memory acceleration, and writes the target picture into an SSD of the target storage node if the cache type is SSD acceleration. By adopting the method and the device, the writing efficiency can be improved.
Description
Technical Field
The present disclosure relates to the field of storage technologies, and in particular, to a method and an apparatus for writing pictures.
Background
With the development of computer technology and network technology, more and more application scenes for extracting pictures are needed to store the pictures. For example, a camera installed on a road needs to be stored after taking a picture, and needs to be extracted when a manager views the picture.
In the related art, after a picture is taken and stored, the picture is generally stored in a magnetic disk and then read from the magnetic disk.
The writing speed of the disk is slow, so that the efficiency of writing pictures is low.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present disclosure provide a method and an apparatus for writing a picture. The technical scheme is as follows:
in a first aspect, a method for writing pictures is provided, where the method is applied to a storage node in a cloud storage system, and the storage node is configured with a disk, a memory, and a Solid State Drive (SSD); the method comprises the following steps:
a target storage node receives a write-in request of a target picture and judges whether a picture resource pool to which the target storage node belongs supports accelerated write-in or not;
when the image resource pool to which the target storage node belongs supports accelerated writing, if the data volume of the target image is smaller than a target numerical value, acquiring a configured cache type in configuration information of the storage node, wherein the cache type is the cache type of the target image when the target storage node writes;
if the cache type is memory acceleration, the target storage node writes the target picture into a memory of the target storage node;
and if the cache type is SSD acceleration, the target storage node writes the target picture into the SSD of the target storage node.
Optionally, the method further includes:
and writing the target picture into a disk of the storage node.
In this way, the picture loss can be prevented.
Optionally, the writing the target picture into the memory of the target storage node includes:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node;
if the current memory allowance of the target picture is smaller than the data volume of the target picture, deleting the picture with the writing time point longest from the current time point in the memory;
and if the memory allowance after the picture is deleted is larger than or equal to the data volume of the target picture, writing the picture into the memory, and if the memory allowance after the picture is deleted is smaller than the data volume of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
In this way, storage in memory can be guaranteed as much as possible.
Optionally, the method further includes:
the target storage node receives a reading request of the target picture sent by a requester, and judges whether a picture resource pool to which the target storage node belongs supports accelerated writing or not, wherein the reading request carries index information of the target picture stored in a disk;
if the picture resource pool to which the target storage node belongs supports accelerated writing, reading the target picture according to the cache type corresponding to the target picture and the reading request;
and sending the target picture to the requester.
Thus, the picture can be quickly read.
Optionally, the reading the target picture according to the cache type corresponding to the target picture and the read request includes:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information;
if the first index is found in the index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from a disk;
if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, and if the second index is found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index;
and if the second index is not found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from a disk according to the reading request.
In this way, since the picture can be read in the memory or the SSD, the picture can be quickly read.
In a second aspect, an embodiment of the present disclosure provides an apparatus for writing a picture, where the apparatus is applied to a storage node in a cloud storage system, and the storage node is configured with a disk, a memory, and an SSD; the device comprises:
the receiving module is used for receiving a writing request of a target picture;
the judging module is used for judging whether the image resource pool to which the target storage node belongs supports accelerated writing;
a determining module, configured to, when a picture resource pool to which the target storage node belongs supports accelerated writing, if a data amount of the target picture is smaller than a target numerical value, obtain a configured cache type in configuration information of the storage node, where the cache type is a cache type of the target picture when the target storage node writes;
a write-in module, configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration;
and if the cache type is SSD acceleration, writing the target picture into an SSD of the target storage node.
Optionally, the writing module is further configured to:
and writing the target picture into a disk of the storage node.
Optionally, the writing module is configured to:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node;
if the current memory allowance of the target picture is smaller than the data volume of the target picture, deleting the picture with the writing time point longest from the current time point in the memory;
and if the memory allowance after the picture is deleted is larger than or equal to the data volume of the target picture, writing the picture into the memory, and if the memory allowance after the picture is deleted is smaller than the data volume of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
Optionally, the receiving module is further configured to receive a reading request of a target picture sent by a requester, where the reading request carries index information that the target picture is stored in a disk;
the judging module is further configured to judge whether the picture resource pool to which the target storage node belongs supports accelerated writing;
the device further comprises:
a reading module, configured to, if the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type corresponding to the target picture and the reading request;
and the sending module is used for sending the target picture to the requester.
Optionally, the reading module is configured to:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information;
if the first index is found in the index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from a disk;
if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, and if the second index is found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index;
and if the second index is not found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from a disk according to the reading request.
In a third aspect, a storage node is provided, where the storage node includes a processor and a memory, and the memory stores at least one instruction, where the instruction is loaded and executed by the processor to implement the method for writing pictures according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, the instruction being loaded and executed by a processor to implement the method for writing pictures as described in the first aspect above.
The beneficial effects brought by the technical scheme provided by the embodiment of the disclosure at least comprise:
in the embodiment of the disclosure, a storage node in a cloud storage system is configured with a disk, a memory and an SSD. When a target picture is written in the cloud storage system, the target storage node can receive a write-in request of the target picture, judge whether a picture resource pool to which the target storage node belongs supports accelerated write-in, judge the data volume of the target picture and the size of a target numerical value if the picture resource pool to which the storage node belongs supports accelerated write-in, and acquire a configured cache type in configuration information of the storage node if the data volume of the target picture is smaller than the target numerical value. And if the cache type is memory acceleration, the target storage node writes the target picture into the memory of the target storage node, and if the cache type is SSD acceleration, the target storage node writes the target picture into an SSD configured by the target storage node. Therefore, when the data volume of the picture is small, the picture can be stored in the memory or the SSD, and the writing speed of the memory or the SSD is high, so that when the data volume of the picture is small, the writing speed of the picture can be relatively increased, and the writing efficiency can be further improved.
Drawings
Fig. 1 is a schematic diagram of a cloud storage system provided by an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for writing pictures according to an embodiment of the disclosure;
fig. 3 is a schematic diagram of setting a cache type according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a target value setting provided by an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a set acceleration strategy provided by an embodiment of the present disclosure;
fig. 6 is a flowchart of a method for extracting a picture according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an apparatus for writing pictures according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an apparatus for writing pictures according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a storage node according to an embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
The embodiment of the disclosure provides a method for writing pictures, and an execution main body of the method can be a storage node, and the storage node can be a server.
The storage node may be provided with a processor, a memory, a transceiver, and the like, the processor may be used for processing of a process of writing a picture, the memory may be used for data required and generated in the process of writing a picture, and the transceiver may be used for receiving and transmitting data, for example, may be used for transmitting or receiving a picture.
Before implementation, an application scenario related to the embodiment of the present disclosure is first introduced:
the method for writing pictures, which is provided by the embodiment of the disclosure, is applied to a plurality of storage nodes of a cloud storage system. The storage nodes in the cloud storage system can be divided into different picture resource pools, and the same storage node does not exist in the different picture resource pools. In this way, it can be considered that a plurality of picture resource pools are included in the cloud storage system, and each picture resource pool may be configured to support accelerated writing or not support accelerated writing. The storage node in the picture resource pool supporting accelerated writing is configured with a disk, a memory and an SSD, and the storage node in the picture resource pool not supporting accelerated writing may be configured with a disk, a memory and an SSD, or may be configured with only a disk and a memory. It should be noted here that the picture resource pool is a virtual storage space.
As shown in fig. 1, the cloud storage system may include an application layer and a device layer. The application layer comprises a management node and storage nodes, the storage nodes form a picture resource pool, and the management node can be used for managing the storage nodes in the picture resource pool. The device layer includes external storage components of the storage node, such as disks and SSDs. In addition, a virtual file system layer is also included between the application layer and the device layer and is used for connecting the device layer and the application layer.
As shown in fig. 2, an embodiment of the present disclosure provides a method for writing a picture, which is described by taking a picture taken by a camera as an example when the picture is stored, and a processing flow of the method may be as follows:
The target picture may be any one of the pictures.
In an implementation, after the camera takes the target picture, a write request may be generated and the target picture is added thereto, and the camera determines the connected management node. The write request is then sent to the management node. After receiving the write-in request, the management node can check the picture resource pool connected with the identification of the camera, and determine the storage capacity of each storage node in the storage nodes included in the picture resource pool and the total storage space of each storage node. For any storage node, comparing the storage capacity of the storage node with the total storage space capacity of the storage node to obtain the storage occupancy rate of the storage node. The management node may select a storage node with the least occupancy rate (hereinafter, may be referred to as a target storage node), and send a write request of the target picture to the target storage node.
The target storage node may receive a write request sent by the management node, and parse the write request to obtain a target picture.
The target storage node may obtain a corresponding relationship between the storage node and the picture resource pool, and determine the picture resource pool to which the target storage node belongs from the corresponding relationship, or the configuration information of the target storage node includes an identifier of the picture resource pool to which the target storage node belongs, and the target storage node may directly obtain the identifier of the picture resource pool to which the target storage node belongs from the configuration information. And then the target storage node acquires a configuration file of the picture resource pool, acquires a writing identification bit of the picture resource pool from the configuration file, and determines whether the picture resource pool supports accelerated writing according to the numerical value of the writing identification bit.
Wherein, a target value can be set and stored in the storage node, the target value is generally 0-500 k, and can be set to 100 k. The cache types are generally classified into no acceleration, memory acceleration, SSD acceleration, and the like.
In implementation, when the value of the write identification bit of the picture resource pool to which the target storage node belongs is the first value, the target storage node may determine that the picture resource pool supports accelerated writing. When the value of the write identification bit of the picture resource pool to which the target storage node belongs is the second value, the target storage node may determine that the picture resource pool does not support accelerated write. The first value and the second value are different values. For example, the first value may be 1 and the second value may be 0.
Then, the target storage node may determine a size relationship between the data volume of the target picture and the target numerical value, and if the data volume of the target picture is smaller than the target numerical value, the configured cache type may be acquired in the configuration information of the target storage node, and then the cache type is set as the cache type corresponding to the target picture. Therefore, when the data volume of the picture is smaller than the target value, the speed can be increased in a memory or SSD mode, and the writing speed is increased. Therefore, due to the fact that the target value is set, when the data volume of the picture is small, the picture can be stored in the SSD or the memory, and when the data volume of the picture is large, the picture can be stored in the magnetic disk, and the reading speed of the picture with the small data volume is improved.
Optionally, in order to prevent the data loss of the picture, the following processing may be performed:
and writing the target picture into a disk of the storage node.
In implementation, after receiving the picture write request, the target storage node may first directly write the target picture into the disk, and then determine whether the picture resource pool to which the target storage node belongs supports accelerated write. Therefore, the pictures can be written into the disk when the picture resource pool does not support acceleration or supports acceleration, and the loss of picture data can be prevented.
In addition, in step 202, if the data size of the target picture is greater than or equal to the target value, since the target picture is already written in the disk of the target storage node, no subsequent processing is required, and the writing processing of the target picture is completed.
In implementation, if the cache type of the target picture is memory acceleration, the target storage node may write the target picture into the memory of the target storage node and create an index value of the target picture for subsequently reading the target picture from the memory of the target storage node. Here the index value may be an identification of the target picture. And corresponding to the memory, maintaining an index table by the target storage node, wherein the index table comprises a corresponding relation between an index value and storage position information, and recording a corresponding relation between the index value and an identifier of a block of the target picture stored in the disk by the target storage node, so that the storage position information of the target picture in the memory can be found by using the identifier of the block in the disk subsequently.
If the cache type is SSD acceleration, the destination storage node may determine its own SSD, write the destination picture into its own SSD, and create an index value of the destination picture for subsequent reading of the destination picture from the SSD. Here the index value may be an identification of the target picture. Corresponding to the SSD, the target storage node maintains an index table, the index table comprises a corresponding relation between an index value and storage position information, and the target storage node also records a corresponding relation between the index value and the identification of the block of the picture stored in the disk, so that the storage position information of the target picture stored in the SSD can be found by using the identification of the block in the disk subsequently.
Thus, due to the memory storage and the SSD storage, the speed of writing pictures and reading pictures can be faster for pictures with smaller data volume.
Optionally, when the target picture is written into the memory of the target storage node, the data size of the target picture and the size of the memory margin are also determined, and the processing may be as follows:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node; if the current memory allowance of the target picture is smaller than the data amount of the target picture, deleting the picture with the writing time point longest from the current time point in the memory; and if the residual amount of the memory after the picture is deleted is larger than or equal to the data amount of the target picture, writing the picture into the memory, and if the residual amount of the memory after the picture is deleted is smaller than the data amount of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
In an implementation, the target storage node may determine a memory margin of the current memory, where the memory margin refers to a space in the current memory where data is not stored yet. The target storage node may then compare the amount of memory remaining with the size of the data volume of the target picture. And if the memory margin is greater than or equal to the data size of the target picture, the target picture can be directly written into the memory of the target storage node.
If the remaining amount of the memory is smaller than the data size of the target picture, the target storage node may determine that the picture with the longest time point from the current time point is written in the memory, and then delete the picture. And then judging the current memory allowance, and if the memory allowance is larger than or equal to the data volume of the target picture, directly writing the target picture into the memory. If the current memory allowance is still smaller than the data volume of the target picture, the picture with the longest distance from the current time point to the current time point at the time point in the current memory can be continuously deleted until the memory allowance is larger than or equal to the data volume of the target picture, and the target picture is written into the memory of the target storage node.
In addition, in the embodiment of the present disclosure, after the target storage node writes the target picture into the disk of the target storage node, the target storage node may return index information to the management node, where the index information is an identifier of the disk and an identifier of the block, where the target picture is written into the disk.
It should be noted that the flow shown in fig. 2 may be executed by an application program on the target storage node. After the application program is started, the configuration information of the target storage node can be read, and configuration items such as a cache type, a memory size, a target numerical value and the like can be obtained from the configuration information. If the application program detects that the memory cache in the configuration item is opened, a memory for caching the picture can be created according to the size of the memory, the memory is initialized, and the picture can be cached by using the memory subsequently. If the memory allocation fails, the application program fails to start, and the memory cache picture cannot be used subsequently.
In addition, in the flow shown in fig. 2, the process may be executed by an application on the target storage node, where the application has two processes, one process is an a process, and the other process is a B process, where the a process and the B process are connected by a unix domain socket link, and in the flow shown in fig. 2, the a process may receive a write request from a requester, and notify the B process through the unix domain socket link, and the B process may execute the above steps 202 to 203. Then, after the target picture is written, index information (the index information is the identification of the disk and the identification of the block stored in the disk) is returned to the process A through a unix domain socket word link. The a-process may be fed back to the requestor.
In addition, in the embodiment of the present disclosure, a process of acquiring a cache type by a target storage node is further provided:
as shown in fig. 3, a user may click an option of setting a cache type in a cloud storage system management page displayed by a terminal, trigger the terminal to display a storage node without the cache type set, find a target storage node therein, and select a cache type for the target storage node. The user may then click on the confirmation option, trigger generation of setting information of the cache type (the setting information carries the set cache type), and send the setting information of the cache type to the target storage node (a 1). The target storage node may update the cache type set by the user into the configuration file (a2), and then return success to the display terminal of the cloud storage system management page (a 3).
In addition, in the embodiment of the present disclosure, a process of acquiring a target value by a target storage node is further provided:
as shown in fig. 4, a user may click a threshold setting option on a cloud storage system management page displayed by the terminal, the terminal may display a threshold input box, and the user may input a target value. The user may then click on the determination option, and trigger the terminal to send setting information of the threshold value (which carries the target value) to the management node (b 1). The management node writes the target value into the root node and inserts the target value into the database of the management node (b2), and updates the target value to the zookeeper node (b 3). The target storage node may listen to the zookeeper node, obtain a target value, write to the memory, and write to its own configuration file (b 4). In addition, other management nodes can synchronize the target value to the memory and the database of the management node by monitoring the target value of the management node.
In addition, in the embodiment of the present disclosure, a process of acquiring, by a target storage node, an indication message indicating whether an image resource pool supports acceleration is further provided:
as shown in fig. 5, a user may click an acceleration policy setting option in a cloud storage system management page of a terminal, trigger the terminal to display a picture resource pool to which an acceleration policy is not set, find a picture resource pool to which a target storage node belongs, and select an acceleration policy for the picture resource pool to which the target storage node belongs. And then the user can click a confirmation option to trigger the terminal to generate acceleration indication information, wherein the acceleration indication information can be used for indicating any one of acceleration supporting and acceleration not supporting strategies. The terminal then sends the acceleration indication information to the management node (c1), and the management node may write the acceleration policy in the acceleration indication information into the root node, insert the acceleration policy into the database of the management node (c2), and update the acceleration policy to the zookeeper node (c 3). The target storage node may listen to the zookeeper node, obtain the acceleration policy, write to memory, and write to its own configuration file (c 4). In addition, other management nodes can synchronize the acceleration strategy to the memory and the database of the other management nodes by monitoring the acceleration strategy of the management node.
Optionally, the acceleration indication information and the target value received by the management node are both received by using a command-line interface (C L I) interface, and similarly, when the target storage node receives the cache type, the cache type is received by using a C L I interface.
Optionally, as shown in fig. 6, in the embodiment of the present disclosure, processing for reading a target picture is further provided, and a corresponding processing flow may be as follows:
In implementation, when the target picture is to be viewed, the target picture can be read, and a user viewing the target picture can operate a terminal (i.e., a requester) to send a read request of the target picture to the target storage node. The read request carries index information of the target picture stored in the disk, which may be an identifier of the disk and an identifier of a block in the disk.
The target storage node may receive a read request of the target picture, and parse the read request to the index information of the target picture stored in the disk, that is, the identifier of the stored disk and the identifier of the block in the disk.
In implementation, the target storage node may obtain a corresponding relationship between the storage node and the picture resource pool, and determine, from the corresponding relationship, the picture resource pool to which the target storage node belongs, or the configuration information of the target storage node includes an identifier of the picture resource pool to which the target storage node belongs, and the target storage node may directly obtain, from the configuration information, the picture resource pool to which the target storage node belongs. And then obtaining a configuration file of the picture resource pool, obtaining a writing identification bit of the picture resource pool from the configuration file, determining that the picture resource pool supports accelerated writing if the value is a first value according to the value of the writing identification bit, and determining that the picture resource pool does not support accelerated writing if the value is a second value.
If the image resource pool to which the target storage node belongs supports accelerated writing, the target storage node may obtain a configuration item of a cache type in the configuration file, determine the cache type according to a value of the configuration item, where the cache type is any one of no acceleration, memory acceleration, and SSD acceleration, and determine the cache type in the configuration file as the cache type corresponding to the target image (this process has been discussed previously, and is not described here again). For example, if the value of the configuration entry is 0, the cache type is determined not to be accelerated, if the value of the configuration entry is 1, the cache type is determined to be SSD accelerated, and if the value of the configuration entry is 2, the cache type is determined to be memory accelerated.
The target storage node may read the target picture from the target storage node by using the cache type corresponding to the target picture and the index information of the target picture stored in the disk.
Optionally, the target picture may be read as follows:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information, and if the first index is found in an index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from the disk, if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, if the second index is found in the index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index, and if the second index is not found in the index table corresponding to the SSD configured by the target storage node, reading the target picture from the disk according to the reading request.
In implementation, if the cache type corresponding to the target picture is memory acceleration, the index information may be used to find a corresponding relationship between an index value and an identifier of a block of the picture stored in the disk, determine a first index of the target picture stored in the memory of the target storage node, and then find whether the first index exists in an index table corresponding to the memory of the target storage node, and if the first index exists, the target picture may be read at a position corresponding to the first index in the memory of the target storage node.
If the first index does not exist in the index table corresponding to the memory of the target storage node, the target picture can be directly read from the disk by using the identifier of the disk and the identifier of the block (in this case, the target picture is usually deleted if the storage time of the target picture is longer).
If the cache type corresponding to the target picture is not accelerated, the target picture can be directly read from the disk by using the identification of the disk and the identification of the block.
If the cache type corresponding to the target picture is SSD acceleration, the corresponding relation between the index value and the identification of the block of the picture stored in the disk can be searched according to the index information, the second index of the target picture stored in the SSD of the target storage node is determined, and if the second index can be found in the index table of the SSD of the target storage node, the target picture can be read at the position corresponding to the second index in the SSD of the target storage node.
If the second index can not be found in the index table of the SSD of the target storage node, the target picture can be read out by using the identification of the disk and the identification of the block in the index information.
It should be noted that MYSQ L (Structured Query L angle) database index (which may be an identifier of a disk and an identifier of a block) is carried in the read request to locate the location of the data of the picture in the disk.
In an implementation, after the target storage node reads the target picture, the target picture may be sent to the requester.
In addition, in the flow shown in fig. 6, the process may be executed by an application on the target storage node, where the application has two processes, one process may be an a process, and the other process may be a B process, where the a process and the B process are connected by a unix domain socket word link, and in the flow shown in fig. 6, the a process may receive a read request from a requestor and notify the read request to the B process by the unix domain socket word link, and the B process may execute the above steps 602 to 603, and then send the read target picture to the a process by the unix domain socket word link, and the a process may feed back the read target picture to the requestor.
In the embodiment of the disclosure, a storage node in a cloud storage system is configured with a disk, a memory and an SSD. When a target picture is written in the cloud storage system, the target storage node can receive a write-in request of the target picture, judge whether a picture resource pool to which the target storage node belongs supports accelerated write-in, judge the data volume of the target picture and the size of a target numerical value if the picture resource pool to which the storage node belongs supports accelerated write-in, and acquire a configured cache type in configuration information of the storage node if the data volume of the target picture is smaller than the target numerical value. And if the cache type is memory acceleration, the target storage node writes the target picture into the memory of the target storage node, and if the cache type is SSD acceleration, the target storage node writes the target picture into an SSD configured by the target storage node. Therefore, when the data volume of the picture is small, the picture can be stored in the memory or the SSD, and the writing speed of the memory or the SSD is high, so that when the data volume of the picture is small, the writing speed of the picture can be relatively increased, and the writing efficiency can be further improved.
Based on the same technical concept, the embodiment of the disclosure also provides a device for writing in pictures, the device is applied to a storage node of a cloud storage system, and the storage node is configured with a disk, a memory and an SSD. As shown in fig. 7, the apparatus includes:
a receiving module 710, configured to receive a write request of a target picture;
a judging module 720, configured to judge whether the picture resource pool to which the target storage node belongs supports accelerated writing;
a determining module 730, configured to, when the picture resource pool to which the target storage node belongs supports accelerated writing, obtain a configured cache type in configuration information of the storage node if the data size of the target picture is smaller than a target numerical value, where the cache type is a cache type of the target picture when the target storage node writes;
a write-in module 740, configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration; and if the cache type is SSD acceleration, writing the target picture into an SSD of the target storage node.
Optionally, the writing module 740 is further configured to:
and writing the target picture into a disk of the storage node.
Optionally, the writing module 740 is configured to:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node;
if the current memory allowance of the target picture is smaller than the data volume of the target picture, deleting the picture with the writing time point longest from the current time point in the memory;
and if the memory allowance after the picture is deleted is larger than or equal to the data volume of the target picture, writing the picture into the memory, and if the memory allowance after the picture is deleted is smaller than the data volume of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
Optionally, the receiving module 710 is further configured to receive a reading request of a target picture sent by a requester, where the reading request carries index information that the target picture is stored in a disk;
the determining module 720 is further configured to determine whether the picture resource pool to which the target storage node belongs supports accelerated writing;
as shown in fig. 8, the apparatus further includes:
a reading module 750, configured to, if the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type corresponding to the target picture and the reading request;
a sending module 760, configured to send the target picture to the requester.
Optionally, the reading module 750 is configured to:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information;
if the first index is found in the index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from a disk;
if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, and if the second index is found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index;
and if the second index is not found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from a disk according to the reading request.
In the embodiment of the disclosure, a storage node in a cloud storage system is configured with a disk, a memory and an SSD. When a target picture is written in the cloud storage system, the target storage node can receive a write-in request of the target picture, judge whether a picture resource pool to which the target storage node belongs supports accelerated write-in, judge the data volume of the target picture and the size of a target numerical value if the picture resource pool to which the storage node belongs supports accelerated write-in, and acquire a configured cache type in configuration information of the storage node if the data volume of the target picture is smaller than the target numerical value. And if the cache type is memory acceleration, the target storage node writes the target picture into the memory of the target storage node, and if the cache type is SSD acceleration, the target storage node writes the target picture into an SSD configured by the target storage node. Therefore, when the data volume of the picture is small, the picture can be stored in the memory or the SSD, and the writing speed of the memory or the SSD is high, so that when the data volume of the picture is small, the writing speed of the picture can be relatively increased, and the writing efficiency can be further improved.
It should be noted that: in the device for writing pictures provided in the above embodiment, only the division of the above functional modules is taken as an example when pictures are written, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus for writing pictures and the method for writing pictures provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.
Fig. 9 is a schematic structural diagram of a storage node according to an embodiment of the present invention, where the storage node 900 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 901 and one or more memories 902, where the memory 902 stores at least one instruction, and the at least one instruction is loaded and executed by the processors 901 to implement the method steps for determining a written picture.
The embodiment of the present disclosure further provides a storage node, where the storage node includes a processor and a memory, where the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method for writing pictures as described above.
The embodiment of the present disclosure also provides a computer-readable storage medium, where at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the method for writing pictures as described above.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present disclosure and is not intended to limit the present disclosure, so that any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (10)
1. The method for writing in the picture is characterized in that the method is applied to a storage node in a cloud storage system, and the storage node is provided with a disk, a memory and a Solid State Disk (SSD); the method comprises the following steps:
a target storage node receives a write-in request of a target picture and judges whether a picture resource pool to which the target storage node belongs supports accelerated write-in or not;
when the image resource pool to which the target storage node belongs supports accelerated writing, if the data volume of the target image is smaller than a target numerical value, acquiring a configured cache type in configuration information of the storage node, wherein the cache type is the cache type of the target image when the target storage node writes;
if the cache type is memory acceleration, the target storage node writes the target picture into a memory of the target storage node;
and if the cache type is SSD acceleration, the target storage node writes the target picture into the SSD of the target storage node.
2. The method of claim 1, further comprising:
and writing the target picture into a magnetic disk of the target storage node.
3. The method of claim 1, wherein writing the target picture to the memory of the target storage node comprises:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node;
if the current memory allowance of the target picture is smaller than the data volume of the target picture, deleting the picture with the writing time point longest from the current time point in the memory;
and if the memory allowance after the picture is deleted is larger than or equal to the data volume of the target picture, writing the picture into the memory, and if the memory allowance after the picture is deleted is smaller than the data volume of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
4. The method of any of claims 1 to 3, further comprising:
the target storage node receives a reading request of the target picture sent by a requester, and judges whether a picture resource pool to which the target storage node belongs supports accelerated writing or not, wherein the reading request carries index information of the target picture stored in a disk;
if the picture resource pool to which the target storage node belongs supports accelerated writing, reading the target picture according to the cache type corresponding to the target picture and the reading request;
and sending the target picture to the requester.
5. The method according to claim 4, wherein reading the target picture according to the cache type corresponding to the target picture and the read request comprises:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information;
if the first index is found in the index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from a disk;
if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, and if the second index is found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index;
and if the second index is not found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from a disk according to the reading request.
6. The device for writing in the picture is characterized in that the device is applied to a storage node in a cloud storage system, and the storage node is provided with a disk, a memory and a Solid State Disk (SSD); the device comprises:
the receiving module is used for receiving a writing request of a target picture;
the judging module is used for judging whether the image resource pool to which the target storage node belongs supports accelerated writing;
a determining module, configured to, when a picture resource pool to which the target storage node belongs supports accelerated writing, if a data amount of the target picture is smaller than a target numerical value, obtain a configured cache type in configuration information of the storage node, where the cache type is a cache type of the target picture when the target storage node writes;
a write-in module, configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration; and if the cache type is SSD acceleration, writing the target picture into an SSD of the target storage node.
7. The apparatus of claim 6, wherein the write module is further configured to:
and writing the target picture into a disk of the storage node.
8. The apparatus of claim 7, wherein the write module is configured to:
if the current memory allowance of the target storage node is larger than or equal to the data volume of the target picture, writing the target picture into the memory of the target storage node;
if the current memory allowance of the target picture is smaller than the data volume of the target picture, deleting the picture with the writing time point longest from the current time point in the memory;
and if the memory allowance after the picture is deleted is larger than or equal to the data volume of the target picture, writing the picture into the memory, and if the memory allowance after the picture is deleted is smaller than the data volume of the target picture, executing the processing of deleting the picture with the longest writing time point and the longest current time point in the memory.
9. The apparatus according to any one of claims 6 to 8, wherein the receiving module is further configured to receive a read request of a target picture sent by a requestor, where the read request carries index information that the target picture is stored in a disk;
the judging module is further configured to judge whether the picture resource pool to which the target storage node belongs supports accelerated writing;
the device further comprises:
a reading module, configured to, if the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type corresponding to the target picture and the reading request;
and the sending module is used for sending the target picture to the requester.
10. The apparatus of claim 9, wherein the reading module is configured to:
if the cache type corresponding to the target picture is memory acceleration, determining a first index of the target picture stored in the memory of the target storage node according to the index information;
if the first index is found in the index table corresponding to the memory of the target storage node, reading the target picture from the memory of the target storage node according to the first index; if the first index is not found in the index table corresponding to the memory of the target storage node, reading the target picture from a disk;
if the cache type corresponding to the target picture is SSD acceleration, determining a second index of the target picture stored in the SSD according to the index information, and if the second index is found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from the SSD configured by the target storage node according to the second index;
and if the second index is not found in an index table corresponding to the SSD configured by the target storage node, reading the target picture from a disk according to the reading request.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910786664.8A CN111399753B (en) | 2019-08-23 | 2019-08-23 | Method and device for writing pictures |
| PCT/CN2020/110330 WO2021036909A1 (en) | 2019-08-23 | 2020-08-20 | Picture write-in method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910786664.8A CN111399753B (en) | 2019-08-23 | 2019-08-23 | Method and device for writing pictures |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111399753A true CN111399753A (en) | 2020-07-10 |
| CN111399753B CN111399753B (en) | 2022-08-05 |
Family
ID=71430246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910786664.8A Active CN111399753B (en) | 2019-08-23 | 2019-08-23 | Method and device for writing pictures |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111399753B (en) |
| WO (1) | WO2021036909A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036909A1 (en) * | 2019-08-23 | 2021-03-04 | 杭州海康威视***技术有限公司 | Picture write-in method and apparatus |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111898A (en) * | 2014-05-26 | 2014-10-22 | 中国能源建设集团广东省电力设计研究院 | Hybrid storage system based on multidimensional data similarity and data management method |
| US20150058553A1 (en) * | 2013-08-22 | 2015-02-26 | Acer Incorporated | Data writing method, hard disc module, and data writing system |
| US9075754B1 (en) * | 2011-12-31 | 2015-07-07 | Emc Corporation | Managing cache backup and restore |
| US20150199269A1 (en) * | 2014-01-14 | 2015-07-16 | Lsi Corporation | Enhanced ssd caching |
| CN107301090A (en) * | 2016-04-14 | 2017-10-27 | 华为技术有限公司 | It is using the method and device for setting data handling path in storage server |
| CN108008912A (en) * | 2016-11-02 | 2018-05-08 | 华为技术有限公司 | A kind of memory buffers method and intelligent terminal based on DRAM |
| CN108268219A (en) * | 2018-02-01 | 2018-07-10 | 杭州宏杉科技股份有限公司 | A kind of method and device for handling I/O request |
| CN108845768A (en) * | 2018-06-19 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of date storage method, device, equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9251064B2 (en) * | 2014-01-08 | 2016-02-02 | Netapp, Inc. | NVRAM caching and logging in a storage system |
| CN107463627B (en) * | 2017-07-11 | 2020-08-11 | 深圳市云领天下科技有限公司 | Picture loading method and terminal |
| CN107861891A (en) * | 2017-11-14 | 2018-03-30 | 郑州天迈科技股份有限公司 | Audio/video data access method for public transport vehicle-mounted hard disk |
| CN111399753B (en) * | 2019-08-23 | 2022-08-05 | 杭州海康威视***技术有限公司 | Method and device for writing pictures |
-
2019
- 2019-08-23 CN CN201910786664.8A patent/CN111399753B/en active Active
-
2020
- 2020-08-20 WO PCT/CN2020/110330 patent/WO2021036909A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9075754B1 (en) * | 2011-12-31 | 2015-07-07 | Emc Corporation | Managing cache backup and restore |
| US20150058553A1 (en) * | 2013-08-22 | 2015-02-26 | Acer Incorporated | Data writing method, hard disc module, and data writing system |
| US20150199269A1 (en) * | 2014-01-14 | 2015-07-16 | Lsi Corporation | Enhanced ssd caching |
| CN104111898A (en) * | 2014-05-26 | 2014-10-22 | 中国能源建设集团广东省电力设计研究院 | Hybrid storage system based on multidimensional data similarity and data management method |
| CN107301090A (en) * | 2016-04-14 | 2017-10-27 | 华为技术有限公司 | It is using the method and device for setting data handling path in storage server |
| CN108008912A (en) * | 2016-11-02 | 2018-05-08 | 华为技术有限公司 | A kind of memory buffers method and intelligent terminal based on DRAM |
| CN108268219A (en) * | 2018-02-01 | 2018-07-10 | 杭州宏杉科技股份有限公司 | A kind of method and device for handling I/O request |
| CN108845768A (en) * | 2018-06-19 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of date storage method, device, equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036909A1 (en) * | 2019-08-23 | 2021-03-04 | 杭州海康威视***技术有限公司 | Picture write-in method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111399753B (en) | 2022-08-05 |
| WO2021036909A1 (en) | 2021-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11296940B2 (en) | Centralized configuration data in a distributed file system | |
| WO2022063322A1 (en) | Hard drive migration method, distributed storage cluster system and storage medium | |
| WO2017050064A1 (en) | Memory management method and device for shared memory database | |
| US20190199794A1 (en) | Efficient replication of changes to a byte-addressable persistent memory over a network | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| WO2022083287A1 (en) | Storage space management method and apparatus, device, and storage medium | |
| US20230305724A1 (en) | Data management method and apparatus, computer device, and storage medium | |
| CN111198845A (en) | Data migration method, readable storage medium and computing device | |
| CN107181773A (en) | Data storage and data managing method, the equipment of distributed memory system | |
| CN106095331B (en) | Control method for internal resources of fixed large file | |
| CN113467719B (en) | Data writing method and device | |
| CN111399760A (en) | NAS cluster metadata processing method and device, NAS gateway and medium | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN110706148A (en) | Face image processing method, device, equipment and storage medium | |
| CN111399753B (en) | Method and device for writing pictures | |
| CN117520278A (en) | Multi-client high-precision directory quota control method for distributed file system | |
| CN113051221A (en) | Data storage method, device, medium, equipment and distributed file system | |
| CN117435569A (en) | Dynamic capacity expansion method, device, equipment, medium and program product for cache system | |
| CN116049306A (en) | Data synchronization method, device, electronic equipment and readable storage medium | |
| WO2018028321A1 (en) | Method and apparatus for managing virtual external storage device, and terminal | |
| CN114490540A (en) | Data storage method, medium, device and computing equipment | |
| CN114647658A (en) | Data retrieval method, device, equipment and machine-readable storage medium | |
| CN108376104B (en) | Node scheduling method and device and computer readable storage medium | |
| CN117539409B (en) | Query acceleration method and device based on data cache, medium and electronic equipment | |
| CN112434001B (en) | Multi-data source configurable MongoDB data synchronization system supporting data preprocessing, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |