CN115878584A - Data access method, storage system and storage node - Google Patents

Abstract

The embodiment of the application provides a data access method, a storage system and a storage node. In the method, the first storage node receives a first read request for reading first file data, and determines that the first file data is stored in a second storage node, wherein the storage system at least comprises the first storage node and the second storage node. The first storage node reads the first file data and the metadata of the first file data from the second storage node, determines the first file data to be read-only data according to the metadata of the first file data, and responds to the determination that the first file data is read-only data, and stores the first file data locally to serve as a copy of the first file data stored in the second storage node. The scheme provided by the embodiment of the application improves the access efficiency of the storage system to the hot spot data.

Description

Data access method, storage system and storage node

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data access method, a storage system, and a storage node.

Background

Current storage systems, such as Network Attached Storage (NAS) systems, may provide storage services for file data to hosts, where the storage services include directory and file-based rights management, read, write, lookup, delete, and other functions. To support the management of very large directories and files, a storage system may be deployed in multiple storage nodes using a distributed architecture. Specifically, files in a file system or data blocks in the files (hereinafter collectively referred to as file data) are stored in a distributed manner in a plurality of storage nodes by a distributed file technology. When a plurality of hosts are connected to a storage system with a distributed architecture, there is a possibility that the plurality of hosts may access the same file at the same time, and a storage node storing the file cannot process a large number of access requests at the same time, thereby causing performance degradation of the storage system.

Disclosure of Invention

The embodiment of the application aims to provide a data access scheme, and after the storage node determines that the read data is read-only data, the storage node locally stores a copy of the read data, so that the access efficiency of hot spot data is improved.

A first aspect of the application provides a data access method performed by a first storage node in a storage system, according to which the first storage node receives a first read request for reading first file data, wherein the storage system comprises at least the first storage node and a second storage node. After determining that the first file data is stored in the second storage node, the first storage node reads the first file data and metadata of the first file data from the second storage node. The first storage node determines that the first file data is read-only data according to the metadata of the first file data, and responds to the determination that the first file data is read-only data, and the first file data is stored locally to be used as a copy of the first file data stored in the second storage node.

After the first file data is read by the first storage node, the copy of the first file data is stored in the case that the first file data is determined to be read-only data, and other storage nodes in the storage system can similarly generate the copy of the first file data, so that the storage system can include multiple copies of the first file data, and can provide a reading service for the first file data at multiple storage nodes, thereby avoiding the forwarding of a data reading request by the storage nodes, and improving the access efficiency of the hot data and avoiding the access bottleneck problem of the hot data in the case that the first file data is the hot data. In addition, under the condition of mass files, a large number of file accesses are read accesses, and the reading performance of the read-only files can be greatly improved by storing the copies of the read-only files according to the read-only attributes of the files.

In a possible implementation manner of the first aspect, the determining that the first file data is read-only data according to the metadata of the first file data includes: determining, from the metadata of the first file data, that the first file data satisfies at least one of the following conditions: the first file data is a file in the read-only snapshot; the first file data is a write-once read-many WORM file; and the file attribute of the first file data is a read-only attribute.

In a possible implementation manner of the first aspect, the method further includes: when the available storage space of the first storage node is determined to be smaller than a preset threshold value, deleting at least part of copy data stored in the first storage node according to the access frequency of the plurality of copy data stored in the first storage node, wherein the plurality of copy data stored in the first storage node comprise copies of the first file data.

By reducing the copy data stored in the storage nodes when the available storage space is small, the copy data can be dynamically adjusted when the available storage space is small, the occupation of the space is reduced, and the requirement of a user on the storage capacity is met.

In a possible implementation manner of the first aspect, the method further includes: locally storing metadata of the first file data, wherein the metadata comprises an expiration time of a read-only attribute of the first file data; deleting the first file data and the metadata of the first file data stored in the first storage node after determining that the first file data is changed into non-read-only file data according to the expiration time.

In a possible implementation manner of the first aspect, the method further includes: receiving a second read request used by the host for reading the first file data; determining that a copy of the first file data is stored in the first storage node; and sending the first file data to the host.

A second aspect of the present application provides a storage system, including a first storage node and a second storage node, where the first storage node is configured to receive a first read request, where the first read request is used to read first file data; determining that the first file data is stored in the second storage node; sending a second read request for the first file data to the second storage node; the second storage node is used for responding to the second read request and returning the first file data and the metadata of the first file data to the first storage node; the first storage node is further used for determining the first file data to be read-only data according to the metadata of the first file data; in response to determining that the first file data is read-only data, storing the first file data locally as a copy of the first file data stored in the second storage node.

In a possible implementation manner of the second aspect, the determining, by the first storage node according to the metadata of the first file data, that the first file data is read-only data specifically includes: the first storage node is further configured to determine, according to the metadata of the first file data, that the first file data satisfies at least one of the following conditions: the first file data is a file in the read-only snapshot; the first file data is a write-once read-many WORM file; and the file attribute of the first file data is a read-only attribute.

In a possible implementation manner of the second aspect, the first storage node is further configured to: when the available storage space of the first storage node is determined to be smaller than a preset threshold value, deleting at least part of copy data stored in the first storage node according to the access frequency of the plurality of copy data stored in the first storage node, wherein the plurality of copy data stored in the first storage node comprise copies of the first file data.

In a possible implementation manner of the second aspect, the first storage node is further configured to: storing metadata of the first file data, wherein the metadata comprises the expiration time of a read-only attribute of the first file data; deleting the first file data and the metadata of the first file data stored in the first storage node after determining that the first file data is changed into non-read-only file data according to the expiration time.

In a possible implementation manner of the second aspect, the first storage node is further configured to: receiving a second read request used by the host to read the first file data; determining that a copy of the first file data is stored in the first storage node; and sending the first file data to the host.

A third aspect of the present application provides a first storage node in a storage system, where the storage system further includes a second storage node, and the first storage node includes a functional module for executing the first aspect or any one of the possible implementation manners of the first aspect.

A fourth aspect of the present application provides a storage node, comprising a processor and a memory, wherein the memory stores executable computer program instructions, and the processor executes the executable computer program instructions to implement the method according to the first aspect and possible implementation manners thereof.

A fifth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed in a computer or processor, cause the computer or processor to implement the method of the first aspect and its possible implementations.

A sixth aspect of the present application provides a computer program product comprising computer program instructions which, when executed on a computer or processor, cause the computer or processor to perform the method according to the first aspect and possible implementation manner of the first aspect.

Drawings

The embodiments of the present application can be made more clear by describing the embodiments with reference to the attached drawings:

fig. 1 is an architecture diagram of a memory system provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a distributed storage method according to an embodiment of the present disclosure;

fig. 3 is a flowchart of a method for writing a file in a storage node according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for reading file data from a storage system according to an embodiment of the present application;

fig. 5 is a structural diagram of a first storage node in a storage system according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 1 is an architecture diagram of a memory system according to an embodiment of the present application. As shown in FIG. 1, the storage system 200 includes a plurality of storage nodes, schematically illustrated as storage node 20a, storage node 20b, and storage node 20c, each of which is interconnected. Each host 100 can access data to any storage node through an Application program or Application (APP), and is also called an "Application server". The host 100 may be a physical machine or a virtual machine. Physical machines include, but are not limited to, desktop computers, servers, notebook computers, and mobile devices. Each storage node may be a physical machine or a virtual machine. The host may be directly connected to each storage node, or may be connected to a plurality of storage nodes through a switch (not shown in fig. 1).

Taking the storage node 20a as an example, the storage node 20a includes a processor 201, a memory 202, and a hard disk 203. The processor 201 may be a Central Processing Unit (CPU), and is configured to process an operation request from a host or an operation request from another storage node, and also configured to process a request generated inside the storage node. The processor 201 may also be a controller with computing capabilities.

The memory 202 is an internal memory that directly exchanges data with the processor 201, can read and write data at any time, and is fast enough to be used as a temporary data storage for an operating system or other programs that are in operation. The Memory 202 includes at least two types of memories, for example, the Memory may be a Random Access Memory (RAM) or a Read Only Memory (ROM). For example, the RAM may include Dynamic Random Access Memory (DRAM), storage Class Memory (SCM), or the like.

The hard disk 203 is used to provide a non-volatile storage resource, which is typically higher in access latency and lower in cost than memory. The Hard Disk 203 may be, for example, a Solid State Disk (SSD), a Hard Disk Drive (HDD), or the like.

The existing distributed storage system can adopt the following data dispersed storage modes:

in the first mode, a plurality of directories are dispersedly attached and stored to a plurality of storage nodes by taking the directory as a unit, and subfiles under the directories are attached to the same storage node as the directory. Although the granularity of the dispersion is not large, the operation such as searching, traversing, deleting and the like based on the directory and the read-write performance of the small files are good, and the method is suitable for the scenes of massive small files. Wherein the directory being attributed to a storage node means that the storage node is determined to store the directory.

In the second mode, a plurality of files are dispersedly attributed and stored to a plurality of storage nodes by taking the files as units, so that the dispersed granularity is smaller, the concurrency performance under the scene of a plurality of files under a directory can be improved, and the method is suitable for the scene with moderate file quantity and size.

And in the third mode, the file is sliced into data blocks (blocks), and the data blocks are distributed and attributed and stored in a plurality of storage nodes, so that the distributed granularity is smaller, the method is suitable for wide scenes with small number of files and large files, and the processing performance can be provided by fully using a plurality of storage nodes.

For any of the above-described ways of storing data in a decentralized manner, each storage node may store respective metadata after storing directories and files therein. The metadata of a directory includes, for example, an identifier and a name of the directory, an identifier of an upper directory (i.e., a parent directory) to which the directory belongs, an identifier of a lower subdirectory included in the directory, an identifier of a storage node to which each subdirectory belongs, a name of a file included in the directory, an identifier of a storage node to which each file belongs, an access attribute of the directory, creation time of the directory, and other information. The metadata of the file includes, for example, an identifier of a parent directory of the file, an identifier of the file, a file name, a file size, a file creation time, a file access attribute, and other information, where the file access attribute includes, for example, read-only, non-read, and other attributes. The scheme provided by the embodiment of the present application will be mainly described below by taking the second dispersed storage manner as an example. It is understood that the scheme provided by the embodiment of the present application can be similarly applied to other two ways of decentralized storage.

Fig. 2 is a schematic diagram illustrating the second distributed storage method. As shown in fig. 2, in the storage system, a specific directory (e.g., directory/a) in a preset storage node (e.g., storage node 20 a) may be set as a root directory, and metadata of the root directory/a, which includes identifications of a plurality of subdirectories of the directory/a, is recorded in the storage node 20a. As shown in fig. 2, the root directory/a includes, for example, subdirectories/E,/F, and/G, so that the metadata of the directory/a also records the identification of each subdirectory and the information of the storage node to which each subdirectory belongs, specifically, the directory/E belongs (i.e., is stored in) the storage node 20a, the directory/F belongs to the storage node 20b, and the directory/G belongs to the storage node 20c. In addition, the metadata of the directory/a also records information such as the access attribute of the directory/a.

Taking the directory/E included in the directory/a as an example, since the directory/E belongs to (i.e., is stored in) the storage node 20a, the storage node 20a stores metadata of the directory/E, and the metadata records an identifier (i.e.,/a) of a parent directory of the directory/E, identifiers of files such as file I, file J, and file K included in the directory/E, information of storage nodes to which the respective files belong, and information of access attributes of the directory/E.

The storage node 20a also stores therein metadata of each file belonging to the storage node 20a, including metadata of the file I. The metadata of the file I records, for example, information such as a parent directory identifier (i.e.,/E) of the file I, a size of the file I, a creation time of the file I, and an access attribute of the file I.

When accessing any file, the host 100 may generate an access request for the file according to the directory path of the file. For example, when a host wishes to read a file I, the directory path of the file I may be determined to be/a/E/I from a locally installed storage system client, such that a read request for the file I may be generated based on the directory path, including the directory path "/a/E/I" in the read request. It is possible for host 100 to send the read request to storage node 20b. In this case, in the related art, the storage node 20b may first read the metadata of the root directory/a from the storage node 20a, determine that the directory/E is stored in the storage node 20a, and then read the metadata of the directory/E, determine that the file I is stored in the storage node 20a. Thereafter, the storage node 20b sends the read request to the storage node 2a. The storage node 2a, after receiving the read request, reads the file I locally and returns the file I to the storage node 20b, so that the storage node 20b returns the file I to the host. In the reading process, the storage node 20b forwards the access request to the storage node 20a, and then receives the access result from the storage node 20a, thereby causing an increase in data access delay.

In order to solve this problem, in a related art, a metadata server is provided in a storage system, and metadata of directories and files stored in respective storage nodes are collectively stored in the metadata server. Before accessing a file in a storage system, a host firstly queries a storage node to which the file belongs in a metadata server, so that the host can directly access the storage node to which the file belongs, and the forwarding of an access request by the storage node is avoided. In the related art, the host needs to query the metadata server each time the host accesses the file, and the requirement on the access performance of the metadata server is high. In addition, if a file accessed by a host is a hot file, the related art still cannot solve the problem of system performance degradation when multiple hosts simultaneously access the same hot file.

In another related technology, a storage node to which a file belongs may be calculated on the host side through a distributed file algorithm, for example, the storage node to which the file belongs may be calculated through a specific algorithm according to information such as a file identifier of the file, so that the host side may directly send an access request for the file to the storage node to which the file belongs, and avoid forwarding of the access request by the storage node. The related technology needs to install software for calculating the file attribution storage node on the host side, and the stability and compatibility of the software can affect the experience of customers. In addition, if the file accessed by the host is a hot file, the related art still cannot solve the problem of system performance degradation when multiple hosts simultaneously access the same hot file.

For the read-only file, the storage node generates a copy of the file after reading the file from another storage node to which the file belongs, and the multiple storage nodes can similarly generate multiple copies of the file, so that a read service for the file can be provided at the multiple storage nodes, and performance degradation caused by forwarding of a read request by the storage nodes is avoided. Meanwhile, for the hot file, as a plurality of copies of the file are stored in a plurality of storage nodes by the method, reading service can be provided by the plurality of copies, and the problem of system performance reduction caused by the hot file is avoided.

Fig. 3 is a flowchart of a method for writing a file in a storage node according to an embodiment of the present disclosure. The method may be performed collectively by any of hosts 100, storage nodes 20a, and storage nodes 20b in FIG. 1, for example. It is to be understood that, although the storage node 20a and the storage node 20b are described as performing the method as an example, the embodiment of the present application is not limited thereto, and the method may be applied to any two storage nodes included in the storage system.

As shown in fig. 3, first, in step S301, the host 100 generates a write request for a file.

The host 100 may obtain, through a client of the distributed storage system, an overall directory of a file system in the storage system, which, for example, as shown in fig. 2, includes a root directory/a, sub-directories/E,/F,/G under the root directory/a, and files included under the respective directories.

The host 100, for example, wishes to newly write a file L in the folder/E, and thus the host 100 may generate a write request for the file L including the directory path to be written, i.e., "/a/E/L", and all data of the file L to write the file L into the directory E. Optionally, the write request may further include an access attribute (e.g., read-only attribute) of the file L.

In step S302, the host 100 transmits a write request of file data to the server 20b.

Assume that the host 100 currently establishes a connection with the storage node 20b, and therefore the host 100 sends a write request for the file data to the server 2b.

At step S303, the storage node 20b determines that the file L belongs to the server 20a.

Referring to FIG. 2, according to directory path "/A/E/L" of file L, storage node 20b may determine that file L belongs to storage node 20a according to a predetermined algorithm. Specifically, the storage node 20b may calculate a hash value for the directory path "/a/E/L" or the name of the file L, and modulo the number of storage nodes with the hash value, so as to determine that the file L belongs to the storage node 20a according to the modulo result.

At step S304, the storage node 20b, after determining that the file L belongs to the storage node 20a, sends a write request for the file L to the storage node 20a.

In step S305, the storage node 20a writes the file L.

In one embodiment, the storage node 20a writes all data of the file L into the storage node 20a, and the storage node 20a may allocate a physical address in the hard disk 203 to the file L, store the file L in the physical address, and record a correspondence between a unique identifier of the file L and the physical address in the hard disk for reading the file L.

The unique identifier of the file L may be a file handle (file handle) of the file L. Specifically, the storage System in the embodiment of the present application may support multiple protocols, such as a Server Message Block (SMB) protocol, a Network File System (Network File System, NFS), and the like. Under the SMB protocol, the storage node 20b may directly query the file handle of the file L based on the directory path "/a/E/L" of the file L corresponding to the file data through the query function. The file handle is unique in the entire file system, and includes, for example, a volume identifier (corresponding to the file system identifier), an inode number (corresponding to the storage node identifier), and a generation number (corresponding to the logical address in the storage node).

Under the NFS protocol, the storage node 20b needs to perform multi-layer query according to the directory path "/a/E/L" of the file L through a query function to obtain the file handle of the file L. Specifically, the storage node 20b first queries the directory file handle of the directory/E according to the directory file handle of the root directory/a and the directory name of the directory/E preset in the storage node 20b. After that, the storage node 20b refers to the file handle of the file L based on the directory file handle of the directory/E and the file name of the file L.

In another embodiment, the storage node 20a may disperse the file L into a plurality of data blocks, and determine a storage node to which each data block belongs, so as to store the plurality of data blocks of the file L to the belonging storage nodes respectively. When storing each data block, each storage node for storing each data block may similarly allocate a physical address to the data block, calculate a handle of the data block based on a directory path of the data block, and record a corresponding relationship between the handle and the physical address for reading the data block.

In step S306, the storage node 20a updates the metadata.

After writing the file L as described above, the storage node 20a first updates the directory metadata of the folder/E, records the identification of the file L in the directory metadata, and records that the file L belongs to the storage node 20a. Then, the storage node 20a generates metadata of the file L, and the metadata includes related information of the access attribute of the file L, and the related information may include information in the following cases.

In the first case, the write request defines the file L as a read-only file, so that the storage node 20a records the access attribute of the file L as a read-only attribute in the metadata of the file L according to the information in the write request.

In the second case, the file L is a file in the read-only snapshot, and the file name or the specific file number in the file identifier of the file L indicates that the file L is a read-only snapshot file, so that the file L can be indicated as a read-only file by recording the file name or the file identifier of the file L in the metadata of the file L.

In a third case, file L is a Write Once Read Many (WORM) file, and the Write request includes an effective time and an invalid time for the read-only attributes of file L, so that storage node 20a records the effective time and the invalid time in the metadata of file L to indicate the read-only attributes of file L in a particular period.

In the fourth case, the directory (e.g., directory/E) to which the file L belongs has a read-only attribute, and thus the storage node 20a records that the file L has a read-only attribute in the metadata of the file L according to the read-only attribute of the directory/E.

In the case where the storage node 20a stores the plurality of data blocks of the file L in a distributed manner in the plurality of storage nodes, the storage node 20a further records, in the metadata of the file L, the identifier of each data block and the identifier of the storage node to which each data block belongs.

In step S307, the storage node 20a returns write request completion information to the storage node 20b after completing the writing of the file L and its metadata.

In step S308, the storage node 20b returns the write request completion information to the host 100 after receiving the write request completion information of the storage node 20a. After the write is completed, the general directory of the file system in the storage system available in the storage system client of the host 100, in which the identification or name of the newly written file L is included, is updated accordingly.

Fig. 4 is a flowchart of a method for reading file data from a storage system according to an embodiment of the present application. The method may be performed by any of hosts 100, storage nodes 20a, and storage nodes 20b in FIG. 2. It is to be understood that, although the storage node 20a and the storage node 20b are described as performing the method as an example, the embodiment of the present application is not limited thereto, and the method may be applied to any two storage nodes included in the storage system.

As shown in fig. 4, the host 100 generates a read request for reading file data in step S401.

Referring to fig. 2, the host 100 generates a read request for data in the file L when it desires to read the file L or a part of the data in the file L after writing the file L to the storage system. The read request includes a directory path of the file L, i.e., "/a/E/L", if the data to be read is part of the read file L, an address of the data to be read in the file L is also included in the read request, and the address includes, for example, an offset start address of the data to be read in the file L and a length (e.g., a number of bytes) of the data.

In the case that the file L is stored as a whole in the storage system, the read request is used to enable the storage system to locate a physical address of the file L, so as to read the data to be read from the file L. In the case where the file L is stored as a plurality of data blocks in a distributed manner in the storage system, the read request is used to enable the storage system to locate a physical address to be read of a data block M including data to be read, so as to read the data to be read from the data block M. The file L and the data block M are hereinafter collectively referred to as file data.

At step S402, the host 100 sends a read request for reading file data to the storage node 20b.

Assume that host 100 currently establishes a connection with storage node 20b, and thus, host 100 sends the read request to storage node 20b.

At step S403, the storage node 20b determines that the file data is attributed to the server 20a.

The storage node 20b first determines whether the file L or the data block M corresponding to the data to be read is stored locally.

Specifically, in the case where the file L is stored as a whole in the storage system, the storage node 20b may query the file handle of the file L according to the directory path of the file L as described above, and the storage node 20b may determine whether the physical address corresponding to the file handle of the file L is recorded locally, and if not, it indicates that the file L is not stored locally.

Storage node 20b, upon determining that file L is not stored locally, may determine that file L belongs to storage node 20a based on directory path "/A/E/L" of file L. Specifically, the storage node 20b first reads the metadata of the directory/a from the storage node 20a to which the root directory/a belongs, and determines that the directory/E belongs to the storage node 20a from the metadata of the directory/a. Thereafter, storage node 20b may read the metadata of directory/E from storage node 20a, from which it may be determined that file L belongs to storage node 20a.

In a case that the file L is stored in each storage node in the form of a plurality of data blocks, the storage node 20b may first read the metadata of the file L from the storage node 20a by reading the directory path of the file L in the request, and determine the identifier of the data block M corresponding to the data to be read, so as to obtain the directory path of the data block M. The handle to data block M may then be similarly queried according to its directory path to determine whether data block M is stored locally. In the event that it is determined that data block M is not stored locally, storage node 20b may similarly determine, for example, from the metadata of file L that data block M belongs to storage node 20a.

At step S404, the storage node 20b transmits a read request for the file data to the storage node 20a.

After determining that the file L or the data block M belongs to the storage node 20a, the storage node 20b sends a read request for the file L or the data block M to the storage node 20a.

In step S405, the storage node 20a reads the file data.

In the case of reading the file L, the storage node 20a may query the file handle of the file L similarly as above, and read the file L based on the file handle. Specifically, under the SMB protocol, the storage node 20a may directly query the file handle of the file L through the directory path "/a/E/L" of the file L, so as to read the file L according to the file handle of the file L. Under the NFS protocol, the storage node 20a may first query the file handle of the directory/E according to the file handle of the directory/a and then query the file handle of the file L according to the file handle of the directory/E according to the directory path "/a/E/L" as described above, so as to read the file L according to the file handle of the file L.

In the case of reading data block M, similarly, storage node 20a may query the handle of data block M according to its directory path and read the data block according to its handle.

After reading file L or data block M, storage node 20a may also read the metadata of file L according to the directory path of file L.

In step S406, the storage node 20a returns the file data and the metadata of the file L to the storage node 20b.

In the case where the file data is file L, the storage node 20a returns file L and metadata of file L to the storage node 20b.

In the case where the file data is data block M, storage node 20a returns metadata of data block M and file L to storage node 20b.

In step S407, the storage node 20b, after receiving the file data from the storage node 20a, reads the data from the file data, and returns the read data to the host 100.

Specifically, the storage node 20b may read data from the file L or the data block M based on the specific address of the data to be read in the read request received from the host 100, and return the read data to the host 100.

In step S408, the storage node 20b determines the file data as read-only data from the metadata of the file L.

Referring to the above description, in one case, the metadata of the file L records therein that the access attribute of the file L is a read-only attribute, so that it can be determined that the file L or the data M in the file L is read-only data.

In another case, a specific number is included in the name of the file L recorded in the metadata of the file L, so that the file L can be determined to be a file in the read-only snapshot according to the file name of the file L, and thus the file L or the data block M in the file L can be determined to be read-only data.

In still another case, the validation time and the aging time of the read-only attribute of the file L are recorded in the metadata of the file L, so that it can be determined that the file L or the data block M in the file L is currently read-only data according to the time information.

In step S409, the storage node 20b stores the file data as a copy.

Storage node 20b may determine its file handle from the directory path of file L or data block M and assign a physical address to the file handle where file L or data block M is stored as a copy.

In the case of storing the file L, the storage node 20b stores the file L in a local hard disk as a copy of the file L stored in the storage node 20a. To distinguish from the file L in the storage node 20a, "file L" stored in this storage node 20b is hereinafter denoted as "file L'". In addition, the storage node 20b also stores metadata of the file L ' in the hard disk, where the metadata includes information in the metadata of the file L received from the storage node 20a, and the metadata of the file L ' also records that the file L ' is a copy of the file L. The storage node 20b transmits storage completion information to the home node of the file L (i.e., the storage node 20 a) after storing the file L ', so that the storage node 20a records the duplicate file L ' of the file L in the parent directory/E of the file L and records that the file L ' belongs to the storage node 20b.

In the case of storing the data block M, the storage node 20b stores the data block M (hereinafter, referred to as data block M ') in the local hard disk as a copy of the data block M stored in the storage node 20a, and causes the storage node 20a to record the storage of the data block M' by the storage node 20b in the metadata of the file L.

After the storage node 20b stores the copy of the file L or the copy of the data block M, and after the host sends a read request for the file L or the data included in the data block M to the storage node 20b again, the storage node 20b may query the local physical address of the file L 'or the data block M' as described above, directly read the file L or the data block M from the physical address, and read corresponding data from the physical address and return the read data to the host 100, so that the storage node 20a does not need to read the corresponding data, thereby saving reading time and improving access efficiency of the storage system.

In the case where the storage node 20b stores the file L 'and its metadata, when the host sends an overwrite request for the file L to the storage node 20b, the storage node 20b may determine that the file L is a read-only file based on the metadata of the file L', thereby returning write failure information to the host.

In the case where the data block M' is stored in the storage node 20b, when the host sends a rewrite request for the data block M to the storage node 20b, the storage node 20b may read the metadata of the file L from the storage node 20a to determine that the data block M is read-only data, thereby returning write failure information to the host.

In the solution of the embodiment of the present application, the storage node 20b stores the file L or the data block M as a copy only after determining that the file L or the data block M is read-only data. In some cases, the read-only properties of the file L or the data block M may change. For example, the host instructs to change file L to a non-read-only file, or file L to a WORM file, the current time has elapsed since the read-only attribute failed time, and file L is changed to a non-read-only file.

After determining that the file L is changed to the non-read-only file, the storage node 20a may send metadata change information of the file L to the storage node 20b according to a storage node identifier (for example, the storage node 20 b) to which the replica file L ' belongs recorded in the parent directory/E of the file L, so that the storage node 20b determines that the read-only attribute of the file L is changed to the non-read-only attribute according to the information, and accordingly deletes the replica file L ' and the metadata thereof or deletes the data block M '.

Alternatively, in the case where file L is a WORM file, the metadata of file L' includes the expiration time of the read-only property of file L. After the storage node 20b determines that the file L becomes a non-read-only file based on the metadata of the file L ', the file L' and the metadata thereof may be deleted.

In step S410, the storage node 20b deletes the copy of the file data after the available storage space is smaller than the preset threshold.

After the storage node 20b stores the copy of the file L or the data block M, an identifier (for example, a directory path) of the file L or the data block M may be recorded in a copy data table, and the number of times of access to the copy in a preset time period may be recorded in the copy data block table, so as to calculate the access frequency of the copy.

The storage node 20b may monitor a local available storage space (e.g., an available hard disk space), and after determining that the available storage space is smaller than a preset threshold, may determine an access frequency of each copy according to the copy data table, and delete at least a portion of the copies with a lower access frequency, thereby saving the storage space. When the storage node 20b monitors that the local available storage space is smaller than the smaller threshold, all the copies stored in the storage node 20b may be deleted according to the copy data table.

Specifically, when it is determined to delete, for example, a duplicate of the file L (i.e., the file L ') from the duplicate data table, the storage node 20b refers to the physical address of the file L ' according to the directory path of the file L, deletes the file L ' from the physical address, deletes the metadata of the file L ' stored locally, and notifies the storage node 20a of the deletion information for the file L ', so that the storage node 20a correspondingly modifies the metadata of the parent directory/E of the file L ', i.e., the information of the file L ' is deleted at the metadata of the parent directory/E.

When it is determined to delete, for example, data block M ' from the duplicate data table, storage node 20b refers to the physical address of data block M ' according to the directory path of data block M, deletes data block M ' from the physical address, and notifies storage node 20a of the deletion information for file M ', so that storage node 20a correspondingly modifies the metadata of file L, i.e., deletes the information of data block M ' at the metadata of the file.

Fig. 5 is a structural diagram of a first storage node in a storage system according to an embodiment of the present application, where the storage system further includes a second storage node, and the first storage node is configured to execute the method shown in fig. 3 or fig. 4, and includes:

a receiving unit 51, configured to receive a first read request, where the first read request is used to read first file data;

a determining unit 52, configured to determine that the first file data is stored in the second storage node;

a reading unit 53, configured to read the first file data and the metadata of the first file data from the second storage node;

the determining unit 52 is further configured to determine the first file data as read-only data according to the metadata of the first file data;

a storage unit 54, configured to store the first file data locally as a copy of the first file data stored in the second storage node in response to a determination that the first file data is read-only data.

In a possible implementation manner, the determining unit 52 is specifically configured to: determining, from the metadata of the first file data, that the first file data satisfies at least one of the following conditions: the first file data is a file in the read-only snapshot; the first file data is a write-once read-many WORM file; and

and the file attribute of the first file data is a read-only attribute.

In one possible implementation manner, the first storage node further includes: a deleting unit 55, configured to delete at least part of the duplicate data stored in the first storage node according to an access frequency of multiple duplicate data stored in the first storage node when it is determined that an available storage space of the first storage node is smaller than a preset threshold, where the multiple duplicate data stored in the first storage node includes a duplicate of the first file data.

In a possible implementation manner, the storage unit 54 is further configured to: locally storing metadata of the first file data, wherein the metadata comprises the expiration time of a read-only attribute of the first file data;

the deleting unit 55 is further configured to: deleting the first file data and the metadata of the first file data stored in the first storage node after determining that the first file data is changed into non-read-only file data according to the expiration time.

In a possible implementation manner, the receiving unit 51 is further configured to receive a second read request used by the host to read the first file data; the determining unit 52 is further configured to determine that a copy of the first file data is stored in the first storage node; the first storage node further comprises a sending unit 56 for sending the first file data to the host.

An embodiment of the present application further provides a storage node, which includes a processor and a memory, where the memory stores executable computer program instructions, and the processor executes the executable computer program instructions to implement the method shown in fig. 3 or fig. 4.

The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed in a computer or a processor, the computer or the processor is enabled to implement the method shown in fig. 3 or fig. 4.

Embodiments of the present application also provide a computer program product comprising computer program instructions, which when run in a computer or processor, cause the computer or processor to perform the method as shown in fig. 3 or fig. 4.

It is to be understood that the terms "first," "second," and the like, herein are used for descriptive purposes only and not for purposes of limitation, to distinguish between similar concepts.

It is clear to a person skilled in the art that the descriptions of the embodiments provided in the present application may be referred to each other, and for convenience and simplicity of description, for example, the functions of each apparatus, the devices, and the steps performed in relation to the embodiments provided in the present application may be referred to the relevant descriptions of the method embodiments of the present application, and the method embodiments and the apparatus embodiments may be referred to each other.

Those skilled in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs all or part of the steps comprising the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as a read-only memory (ROM), a random-access memory (RAM), a magnetic disk, and an optical disk.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to be performed in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, e.g., from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, fiber optic, digital subscriber line, or wireless (e.g., infrared, wireless, microwave, etc.) manner.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways without exceeding the scope of this application. For example, the above-described embodiments are merely illustrative, and for example, the division of the modules or units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement without inventive effort.

Additionally, the apparatus and methods described, as well as the illustrations of various embodiments, may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present application. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electronic, mechanical or other form.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A method of data access, the method being performed by a first storage node in a storage system, the method comprising:

receiving a first read request for reading first file data, wherein the storage system at least comprises a first storage node and a second storage node;

determining that the first file data is stored in the second storage node;

reading the first file data and metadata of the first file data from the second storage node;

determining the first file data as read-only data according to the metadata of the first file data;

in response to determining that the first file data is read-only data, storing the first file data locally as a copy of the first file data stored in the second storage node.

2. The method of claim 1, wherein the determining that the first file data is read-only data according to the metadata of the first file data comprises:

determining, from the metadata of the first file data, that the first file data satisfies at least one of the following conditions:

the first file data is a file in the read-only snapshot;

the first file data is a write-once read-many WORM file; and

and the file attribute of the first file data is a read-only attribute.

3. The method of claim 1 or 2, further comprising:

when the available storage space of the first storage node is smaller than a preset threshold value, deleting at least part of copy data stored in the first storage node according to the access frequency of the plurality of copy data stored in the first storage node, wherein the plurality of copy data stored in the first storage node comprise copies of the first file data.

4. The method of claim 2, further comprising:

locally storing metadata of the first file data, wherein the metadata comprises the expiration time of a read-only attribute of the first file data;

deleting the first file data and the metadata of the first file data stored in the first storage node after determining that the first file data is changed into non-read-only file data according to the expiration time.

5. The method of claim 1 or 2, further comprising:

receiving a second read request used by the host for reading the first file data;

determining that a copy of the first file data is stored in the first storage node;

and sending the first file data to the host.

6. A storage system includes a first storage node and a second storage node,

the first storage node is used for receiving a first read request, wherein the first read request is used for reading first file data; determining that the first file data is stored in the second storage node; sending a second read request for the first file data to the second storage node;

the second storage node is used for responding to the second read request and returning the first file data and the metadata of the first file data to the first storage node;

the first storage node is further used for determining the first file data as read-only data according to the metadata of the first file data; in response to determining that the first file data is read-only data, storing the first file data locally as a copy of the first file data stored in the second storage node.

7. The storage system of claim 6, wherein the first storage node is further configured to determine, according to the metadata of the first file data, that the first file data is read-only data specifically includes:

the first storage node is further configured to determine, according to the metadata of the first file data, that the first file data satisfies at least one of the following conditions:

the first file data is a file in the read-only snapshot;

the first file data is a write-once read-many WORM file; and

and the file attribute of the first file data is a read-only attribute.

8. The storage system according to claim 6 or 7,

the first storage node is further configured to: when the available storage space of the first storage node is determined to be smaller than a preset threshold value, deleting at least part of copy data stored in the first storage node according to the access frequency of the plurality of copy data stored in the first storage node, wherein the plurality of copy data stored in the first storage node comprise copies of the first file data.

9. The storage system of claim 7,

the first storage node is further configured to: storing metadata of the first file data, wherein the metadata comprises the failure time of the read-only attribute of the first file data; deleting the first file data and the metadata of the first file data stored in the first storage node after determining that the first file data is changed into non-read-only file data according to the expiration time.

10. The storage system according to claim 6 or 7,

the first storage node is further configured to: receiving a second read request used by the host for reading the first file data; determining that a copy of the first file data is stored in the first storage node; and sending the first file data to the host.

11. A storage node comprising a processor and a memory, the memory having stored therein executable computer program instructions, the processor executing the executable computer program instructions to implement the method of any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that it stores computer program instructions which, when executed in a computer or processor, cause the computer or processor to perform the method of any of claims 1-5.

Images