CN108241469B - Data storage method, device and system - Google Patents

Data storage method, device and system Download PDF

Info

Publication number
CN108241469B
CN108241469B CN201611224468.4A CN201611224468A CN108241469B CN 108241469 B CN108241469 B CN 108241469B CN 201611224468 A CN201611224468 A CN 201611224468A CN 108241469 B CN108241469 B CN 108241469B
Authority
CN
China
Prior art keywords
storage
node
target
storage management
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611224468.4A
Other languages
Chinese (zh)
Other versions
CN108241469A (en
Inventor
赵龙
金辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd, Beijing Kingsoft Cloud Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN201611224468.4A priority Critical patent/CN108241469B/en
Publication of CN108241469A publication Critical patent/CN108241469A/en
Application granted granted Critical
Publication of CN108241469B publication Critical patent/CN108241469B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The embodiment of the invention provides a data storage method, a device and a system, wherein the method comprises the following steps: after receiving a storage request sent by a client, a control node; determining a target storage management cluster according to the storage request; then obtaining state information of the target storage cluster from the target storage cluster corresponding to the target storage management cluster, and determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; and the request target storage management node determines a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node. By adopting the scheme provided by the embodiment of the invention to store data, the control node can obtain the state information corresponding to the storage cluster after receiving the storage request without waiting, thereby saving the time for storing data.

Description

Data storage method, device and system
Technical Field
The present invention relates to the field of resource management technologies, and in particular, to a data storage method, apparatus, and system.
Background
At present, a common data storage system generally includes: the storage management cluster is a set of a plurality of storage nodes, and the storage management cluster is connected with the storage cluster.
The existing data storage process is roughly: the control node receives a storage request aiming at target data sent by a client, then allocates a storage management node aiming at the storage request, and stores the target data into a storage node corresponding to the storage management node. In the foregoing process, when allocating a storage management node for the storage request, the storage management cluster needs to be probed to find a storage management node in a surviving state, where the storage management node in the surviving state is a node capable of providing a storage management service. The existing activity detection mechanism is generally a periodic activity detection mechanism, that is, a control node periodically issues an activity detection command to a storage management cluster, the storage management cluster requests a database to update after receiving the activity detection command, after the update is completed, the control node determines a storage management node in a survival state in the storage management cluster according to data storage state information of the database, then the control node selects one storage management node from the storage management nodes in the survival state as a target storage management node, and the target storage management node allocates one storage node for the storage request, so that a client stores the target data into the selected storage node; that is, the control node can determine the alive state of each storage management node only after the cycle start time of issuing the probe alive command arrives, and can allocate the storage management node in the alive state to the storage request.
However, if the storage request is received between two adjacent cycle start times, the control node may allocate the storage management node in the alive state to the storage request only after the next cycle start time arrives, and therefore, the data storage method is used to store data, although the storage of the data can be completed, the allocation of the storage management node to the storage request needs to wait until the cycle start time arrives, which wastes excessive time on waiting, and thus the time for storing the data becomes long.
Disclosure of Invention
The embodiment of the invention aims to provide a data storage method, a data storage device and a data storage system, so as to save the time for storing data.
In order to achieve the above object, an embodiment of the present invention discloses a data storage method, which is applied to a control node in a data storage system, and the method includes:
receiving a storage request sent by a client for storing target data;
determining a target storage management cluster from the storage management clusters of the data storage system according to the storage request;
acquiring state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster;
determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information;
determining a target storage management node from the storage management nodes in the survival state;
requesting the target storage management node to determine a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node.
Preferably, the determining, according to the obtained state information, a storage management node in an alive state in the target storage management cluster includes:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
Preferably, the determining a target storage management node from the storage management nodes in the alive state includes:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
Preferably, when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
Preferably, the method further comprises:
monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
if so, requesting the target storage management node to determine a second storage node so that the client stores the target data to the second storage node, wherein the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
Preferably, the method further comprises:
monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data to the first storage node;
if so, marking the state of the target storage management node as a dead state, and returning to execute the step of determining the target storage management node from the storage management nodes in the survival state.
In order to achieve the above object, an embodiment of the present invention further discloses a data storage apparatus, which is applied to a control node in a data storage system, and the apparatus includes:
the receiving module is used for receiving a storage request which is sent by a client and used for storing target data;
the first determining module is used for determining a target storage management cluster from the storage management clusters of the data storage system according to the storage request;
an obtaining module, configured to obtain state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster;
a second determining module, configured to determine, according to the obtained state information, a storage management node in a surviving state in the target storage management cluster;
a third determining module, configured to determine a target storage management node from the storage management nodes in the surviving state;
a first storage module, configured to request the target storage management node to determine, according to the storage request, a first storage node for storing the target data, so that the client stores the target data in the first storage node.
Preferably, the second determining module is specifically configured to:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
Preferably, the third determining module is specifically configured to:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
Preferably, when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
Preferably, the apparatus further comprises:
the first monitoring module is used for monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
a second storage module, configured to, when the first monitoring module monitors that the first storage node fails, request the target storage management node to determine a second storage node, so that the client stores the target data in the second storage node, where the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
Preferably, the apparatus further comprises:
the second monitoring module is used for monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data in the first storage node;
and the triggering module is used for marking the state of the target storage management node as a death state and triggering the third determining module under the condition that the second monitoring module monitors that the target storage management node requests upgrading maintenance.
In order to achieve the above object, an embodiment of the present invention further discloses a data storage system, where the system includes: the storage management cluster comprises at least one storage management node, and the storage cluster comprises at least one storage node;
the control node is used for receiving a storage request sent by a client and used for storing target data; determining a target storage management cluster from the storage management clusters according to the storage request; acquiring state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster; determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; requesting the target storage management node to determine a first storage node for storing the target data according to the storage request;
the target storage management node is used for determining a first storage node for storing the target data according to the storage request;
and the first storage node is used for receiving and storing the target data sent by the client.
As can be seen from the above, in the solution provided in the embodiment of the present invention, after receiving a storage request sent by a client, a control node; determining a target storage management cluster according to the storage request; then obtaining state information of the target storage cluster from the target storage cluster corresponding to the target storage management cluster, and determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; and the request target storage management node determines a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node. By adopting the scheme provided by the embodiment of the invention to store data, the control node can obtain the state information corresponding to the storage cluster after receiving the storage request without waiting, thereby saving the time for storing data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data storage method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a specific process of data storage according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating another specific process of data storage according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating another data storage method according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating another data storage method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of another data storage device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another data storage device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present invention will be described in detail below with reference to specific examples.
Fig. 1 is a schematic flowchart of a data storage method according to an embodiment of the present invention, where the method includes:
s101: and receiving a storage request sent by the client for storing the target data.
With the popularization of networks and the rise of big data technologies, people are no longer limited to storing data generated in production, life, study and work into a local storage space or a storage medium such as a mobile hard disk and a flash disk, but more and more store the data into a network storage space.
Due to the development of networks and the increase of data storage requirements of users, a general data storage system may include a control node, a plurality of storage management clusters and a plurality of storage management clusters, each storage management cluster may include a plurality of storage management nodes, each storage cluster may include a plurality of storage nodes, there is often a correspondence between a storage management cluster and a storage cluster, for convenience of data storage and resource management, generally, one storage management cluster corresponds to one storage cluster, and a storage management node in a storage management cluster may be in communication connection with one or more storage nodes in a storage cluster.
For a user, the user may send a storage request for storing the data through a client used by the user, and in the scheme provided in the embodiment of the present invention, data that needs to be stored is referred to as target data. The client may be, but is not limited to, a cloud client, such as a jinshan enterprise cloud disk, for example, assuming that a user wants to store a local video compression packet a with a data size of 50GB into a network storage space, at this time, the user may send a corresponding storage request to a control node in the network-side data storage system through the client "jinshan enterprise cloud disk", and the control node may trigger a process to process the storage request after receiving the storage request, where in general, one process may process one storage request. The storage request may contain attribute information of the target data, such as: name, data size, data type, etc. of the target data, and the storage request may include attribute information of the target data, for example: video compression package A, 50GB, ZIP compression file.
S102: and determining a target storage management cluster from the storage management clusters of the data storage system according to the storage request.
Those skilled in the art will appreciate that different storage management clusters may implement management of different types of data resources, for example, assuming that a data storage system includes 3 storage management clusters, respectively: cluster1, cluster2 and cluster3, wherein cluster1 may be responsible for managing data resources of document and image types, cluster2 may be responsible for managing data resources of compressed file types, and cluster3 may be responsible for managing data resources of audio and video types. Of course, each storage management cluster is not necessarily responsible for managing only the corresponding data resource, and may share a part of the data resource of the type when the load of other storage management clusters is too large.
As described above, the control node can receive a storage request for target data sent by a user through a client, and the storage request received by the control node includes attribute information of the target data, so that the control node can specify a target storage management node for managing the storage request from among a plurality of storage management clusters based on the attribute information of the target data included in the storage request. For example, the storage request may include attribute information of the target data: the video compression package a, 50GB, and ZIP compressed file, then the control node may determine that the target data type corresponding to the storage request is a compressed file according to the attribute information, and as can be seen from the above, in the 3 storage management clusters cluster1, cluster2, and cluster3, the cluster2 may be responsible for managing data resources of the compressed file type, and therefore, the control node determines the cluster2 as the target storage management cluster from the 3 storage management clusters cluster1, cluster2, and cluster 3.
S103: and acquiring the state information of the target storage cluster from the target storage cluster corresponding to the target storage management cluster.
As can be seen from the above, the storage management clusters correspond to the storage clusters, generally, one storage management cluster corresponds to one storage cluster, and after determining the target storage management cluster, the control node may determine, from the plurality of storage clusters, the storage cluster corresponding to the target storage management cluster, which is referred to as the target storage cluster.
As will be understood by those skilled in the art, in a data storage system, a storage cluster is an ultimate carrier for storing data, and a storage cluster can be understood as a large storage space, the total amount of the storage space can reach the TB (Terabyte, or Terabyte), and a specific storage task will be specifically implemented by a storage node therein, and the storage node can have various forms, for example: ceph, sheetlog, emc, etc. The storage nodes in the same storage cluster can be in the same form or different forms, if the storage nodes in the storage cluster are in the same form, the storage cluster of the type is called an independent mode, and if the storage nodes in the storage cluster are not in the same form, the storage cluster of the type is called a mixed insertion mode. Of course, the embodiment of the present invention does not explicitly limit the specific form of each storage node in the storage cluster.
As shown in fig. 2, for a specific process of data storage provided by the embodiment of the present invention, the control node shown in fig. 2 is communicatively connected to a cluster2 (upper dashed box), it may be understood that the control node has a communication connection with storage management nodes 1 to 4 included in the cluster2, the cluster2 is communicatively connected to the storage cluster 11 (lower dashed box), it may be understood that each storage management node in the cluster2 is communicatively connected to the storage cluster 11, or it may be understood that each storage management node in the cluster2 is communicatively connected to each storage node 1 to 6 in the storage cluster 11. The control node shown in fig. 2 receives the storage request, and determines that the target storage management cluster is cluster2 according to the storage request, and cluster2 is correspondingly communicatively connected to the storage cluster 11, so that the storage cluster 11 is referred to as the target storage cluster. At this time, the control node may send a liveliness command to the storage cluster 11, and the storage cluster 11 feeds back corresponding state information to the control node after receiving the liveliness command. In order to obtain the state of data storage truly and accurately, in a specific implementation manner provided in the embodiment of the present invention, the control node may obtain the state information of the target storage cluster through the following two situations.
The following description will take fig. 2 as an example to acquire status information of a target storage cluster.
One situation is: the control node may issue a liveness detection command to each storage node in the storage cluster 11 (target storage cluster) for the received storage request, where the liveness detection command is used to trigger each storage node to feed back its own state information to the control node; the state information of the target storage cluster can be accurately obtained through the state information of each storage node.
In general, after receiving a liveness detection command issued by a control node, each storage node in a target storage cluster immediately feeds back its own state information to the control node, and the whole process has almost no or very short time delay. In order to avoid that the waiting time of the control node is too long due to the failure of the storage node, after the control node issues a keep alive command to each storage node in the target storage cluster, whether state information fed back by each storage node can be received within a preset time period can be judged, if the state information fed back by a certain storage node is not received within the preset time period, the storage node is considered to be failed at the moment and is in an unavailable state, and the control node can obtain the state information corresponding to the target storage cluster according to the received state information of the storage node.
The other situation is as follows: assuming that a service node dedicated for recording state information of each storage node or state information of a storage cluster is arranged in the storage cluster 11, at this time, the control node may also issue a probe command to the service node in the target storage cluster for the received storage request, and after the service node receives the probe command, the service node feeds back the recorded state information of each storage node or the state information of the storage cluster 11 to the control node, and according to different specific contents of the state information recorded by the service node, introduction may be performed in the following two scenarios; scene one: the service node is dedicated to record the status information of the storage cluster 11, and the status information of the storage cluster 11 may include: the total amount of storage space remaining in the storage cluster 11, the number of storage nodes in the storage cluster 11 having a storage space greater than 50GB remaining, the total number of storage nodes, and the number of storage nodes in a writable state, and so on. In this scenario, the service node feeds back the state information of the storage cluster 11 recorded at the current time to the control node, or the service node may feed back the state information of the storage cluster 11 recorded in a previous recording period to the control node.
In a second scenario, the service node is specially configured to record state information of each storage node, and then, after receiving the foregoing activation detection command, the service node feeds back the state information of each storage node recorded at the current time to the control node, or the service node may feed back the state information of each storage node recorded in a previous recording period to the control node. The specific form of the state information of each storage node recorded in the service node may be many, and the most common form is a table format, for example, the state information corresponding to the storage nodes 1 to 6 recorded in the service node is shown in table 1 below, specifically:
TABLE 1
Storing node names Whether or not it is in a writable state Remaining amount of storage space Total amount of storage space
Storage node 1 Whether or not 20GB 240GB
Storage node 2 Is that 100GB 240GB
Storage node
3 Is that 75GB 240GB
Storage node
4 \ \ \
Storage node
5 Is that 80GB 240GB
Storage node 6 Is that 128GB 240GB
It should be noted that table 1 is only a specific example provided by the embodiment of the present invention, and the application does not explicitly limit the number of storage nodes included in the storage cluster 11 (target storage cluster) and the specific state information of each storage node. The recording period mentioned above is to count the state information corresponding to the target storage cluster or the state information corresponding to each storage node in the target storage cluster periodically for the service node in the target storage cluster, for example, the set recording period is 0.5 second, that is, the state information of the target storage cluster will be recorded once in 0.5 second. In order to not increase the operation load of the data storage system, and because the data storage state change of the storage cluster in a short time is small, after the service node receives the activation probing command, the state information of the target storage cluster does not need to be recorded in real time, and the state data recorded in the previous recording period is used as the state information at the current moment and is fed back to the control node.
As can be seen from the above, no matter what manner is used, the control node can obtain the state information about the target storage cluster from the target storage cluster, but in a normal case, the state information corresponding to each storage node can more accurately reflect the data storage state of the whole target storage cluster, and therefore, the service node can feed back the state information corresponding to each storage node in the target storage cluster to the control node, but the embodiment of the present invention does not explicitly limit the specific content of the state information.
S104: and determining the storage management nodes in the survival state in the target storage management cluster according to the obtained state information.
As will be understood by those skilled in the art, each storage management node in the storage management cluster is used to manage a corresponding data resource in the storage cluster, and therefore, the storage state of the storage cluster will directly affect the survival state of each storage management node in the storage management cluster, so that the survival state corresponding to each storage management node in the storage management cluster can be directly determined according to the state information of the storage cluster.
Referring to the foregoing embodiment, the state information of the target storage cluster received by the control node may include the total amount of remaining storage space of the target storage cluster, the number of storage nodes whose remaining storage space in the target storage cluster is greater than 50GB, the total number of storage nodes, and the number of storage nodes in a writable state, and it is assumed that the state information received by the control node is that the total amount of remaining storage space of the storage cluster 11 (target storage cluster) is 45GB, the number of storage nodes whose remaining storage space is greater than 50GB is 0, the total number of storage nodes is 6, the number of storage nodes in a writable state is 4, and the target data is: as can be seen from the above description, the total amount of the storage space remaining in the storage cluster 11 is only 45GB, and although there are 4 storage nodes in the storage cluster 11 that are in a writable state, since the data size of the target data is 50GB, the storage nodes in the writable state cannot store the target data, it can be determined that all storage management nodes in the cluster2 (target storage management cluster) corresponding to the storage cluster 11 cannot provide management services, that is, each storage management node in the target storage management cluster is in a dead state.
As can be seen from the above, there is a corresponding relationship between the storage management node in cluster2 and each storage node in the storage cluster 11, in a general case, the storage management node in cluster2 can be in communication connection with a plurality of storage nodes in the storage cluster 11, the storage management node can manage data resources of the storage nodes connected to the storage management node, and in order to further determine the survival status of the storage management node, in a specific implementation manner provided by the embodiment of the present invention, the storage management node in the survival status in the target storage management cluster is determined according to the obtained status information, and may:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
As shown in table 1 above, the target storage cluster shown in table 1 has 6 storage nodes in total, where the storage node 1 is in an unwritable state, and the total amount of storage space corresponding to the storage nodes 1 to 6 is 240GB, but no status information corresponding to the storage node 4 is recorded in table 1, which indicates that there is a high possibility that the storage node 4 fails at this time, the storage node 2 is in a writable state, and the remaining amount of storage space is 100GB, the storage node 3 is in a writable state, and the remaining amount of storage space is 75GB, the storage node 5 is in a writable state, and the remaining amount of storage space is 80GB, and the storage node 6 is in a writable state, and the remaining amount of storage space is 128 GB; assuming that there are 4 storage management nodes in cluster2, named as storage management nodes 1-4, respectively, the preset correspondence may be:
the storage management node 1 is correspondingly in communication connection with the storage nodes 1 and the storage nodes 4 in the storage cluster 11;
the storage management node 2 is correspondingly in communication connection with the storage nodes 2, the storage nodes 3 and the storage nodes 6 in the storage cluster 11;
the storage management node 3 is correspondingly in communication connection with the storage nodes 1, 2 and 5 in the storage cluster;
the storage management node 4 is correspondingly connected with the storage nodes 5 in the storage cluster in a communication mode.
As can be seen from the state information of each storage node in table 1 and the preset corresponding relationship, the storage node 1 communicatively connected to the storage management node 1 is in an unwritable state, and the storage node 4 communicatively connected to the storage management node 1 fails, so that it can be determined that the storage management node 1 is in a dead state at this time;
the storage node 2, the storage node 3 and the storage node 6 which are in communication connection with the storage management node 2 are all in a writable state, and the rest storage space is more than 50GB, so that the storage management node 2 can be determined to be in a survival state at the moment;
the storage nodes which are in communication connection with the storage management node 3 are a storage node 1, a storage node 2 and a storage node 5, wherein the storage node 1 is in a non-writable state, but the storage node 2 and the storage node 5 are both in a writable state and the remaining storage space is greater than 50GB, then if the storage management node 3 receives a storage request at this time, the target data can be stored for the storage request, and therefore, it can be determined that the storage management node 3 is in a live state at this time;
the storage node communicatively connected to the storage management node 4 is only the storage node 5, and the storage node 5 is in a writable state and the remaining storage space is greater than 50GB, so that it can be determined that the storage management node 4 is also in a surviving state at this time.
In summary, it can be determined that the storage management nodes in the cluster2 (target storage management cluster) in the storage state are: a storage management node 2, a storage management node 3, and a storage management node 4. It should be noted that the preset corresponding relationship is only a specific example provided by the embodiment of the present invention, and the specific form of the corresponding relationship is not explicitly limited in the present application.
S105: and determining a target storage management node from the storage management nodes in the survival state.
After determining the storage management nodes in the cluster2 that are in the alive state, the control node may determine one of the storage management nodes in the alive state as the target storage management node. For example, a random manner may be adopted to randomly select one of all storage management nodes in the cluster2 in a living state as a target storage management node, but selecting the target storage management node in the random manner may cause some storage management nodes to be frequently hit, and other storage management nodes to be idle for a long time, which may cause uneven resource allocation, and thus may cause a network congestion phenomenon.
In view of the above phenomenon, in a specific implementation manner provided in an embodiment of the present invention, the determining a target storage management node from storage management nodes in a surviving state may be:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
In order to enable the entire data storage system to uniformly and stably operate, generally, after receiving a storage request sent by a client, a control node may determine a target storage management node according to a load balancing principle. For example, assuming that the storage management nodes determined to be in the surviving state in cluster2 include storage management node 2, storage management node 3, and storage management node 4, where the remaining storage spaces corresponding to storage node 2, storage node 3, and storage node 6 corresponding to storage management node 2 are 100GB, 75GB, and 128GB, respectively, then the total amount of the remaining storage space corresponding to storage management node 2 is 303 GB; the remaining storage spaces corresponding to the storage node 1, the storage node 2 and the storage node 5 corresponding to the storage management node 3 are 20GB, 100GB and 80GB, respectively, but the storage node 1 is in a non-writable state, and then the remaining storage space corresponding to the storage node 1 is not considered, so that the total amount of the remaining storage space corresponding to the storage management node 3 is 180 GB; the remaining storage space of the storage node 5 corresponding to the storage management node 4 is 80GB, and then the total amount of the remaining storage space corresponding to the storage management node 4 is 80 GB; in general, the larger the total amount of the remaining storage space is, the more data can be stored, that is, the stronger the load capacity is, and in summary, the total amount of the remaining storage space corresponding to the storage management node 2 is the largest, so that the storage management node 2 may be determined as the target storage management node.
The target storage management nodes may also be determined according to a principle that the request number is uniform, and it is understood by those skilled in the art that the storage management nodes in the target storage management cluster are structurally peer-to-peer except for different names, so that the target storage management nodes may be determined according to the number of requests received by each storage management node in the target storage management cluster in a surviving state, for example, the storage management node in the target storage management cluster in the surviving state that receives the least number of requests may be determined as the target storage management node; the control node may also count the number of received requests in a period of time, and determine that the received storage request is the first, for example, if the control node receives the storage request of 91 th, and there are 3 storage management nodes in the cluster2 that are in a live state at this time, and the remainder of 91/3 is 1, then it will determine that the first of the 3, that is, the storage management node 2, is the target storage management node.
Or the target storage management node can be determined according to a load balancing principle and a request quantity uniformity principle, that is, the target storage management node is finally determined from the storage management nodes in the survival state in the target storage management cluster by combining the two principles.
It should be noted that the above methods for determining a target storage management node from storage management nodes in an alive state are only some specific examples provided by the embodiments of the present invention, and do not constitute a limitation on the specific manner for determining the target storage management node, and any modifications, equivalents, improvements, and the like, which are within the spirit and principle of the present invention, are included in the scope of the present invention.
S106: requesting the target storage management node to determine a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node.
It can be known from the above that, the storage management node in the cluster2 may be communicatively connected to one or more storage nodes in the storage cluster 11, and assuming that the determined target storage management node is the storage management node 2, then the control node will request the storage management node 2 to allocate a specific storage space for the target data, the storage nodes corresponding to the storage management node 2 are the storage node 2, the storage node 3, and the storage node 6, and the storage management node determines the first storage node for storing the target data in what manner, which is not explicitly described in this application, and the specific manner may be determined according to a specific data storage structure, for example, the data storage structure may be a distributed storage structure, a multi-copy storage structure, and the like.
For example, assuming that the storage management node 2 determines the first storage node according to the remaining space sizes corresponding to the storage node 2, the storage node 3 and the storage node 6, it may determine that the storage node 6 is the first storage node. After the storage node 6 is determined to be the first storage node, the target data can be stored, and the storage process of the target data can be at least as follows:
in the mode 1, after the storage management node 2 determines that the storage node 6 is the first storage node, the storage management node feeds back information such as a network address corresponding to the storage node 6 to the control node, and after receiving the network address of the storage node 6, the control node can send a response instruction to the client, and simultaneously send the network address of the storage node 6 to the client, so that the client can directly access the storage node 6 and store target data in the storage node 6 through the network address corresponding to the storage node 6, thereby completing storage of the target data.
In the mode 2, after the storage management node 2 determines that the storage node 6 is the first storage node, the message that the storage node 6 is the first storage node is fed back to the control node, and after the control node determines the network address corresponding to the storage node 6, the control node sends a response instruction to the client and sends the network address of the storage node 6 to the client, so that the client can directly access the storage node 6 and store the target data in the storage node 6 after receiving the network address, and the storage of the target data is completed.
In the mode 3, when the storage management node 2 determines the storage node 6 as the first storage node, an active storage request instruction is issued to the storage node 6, and the storage node 6 establishes connection with the client according to the active storage request instruction, so that the storage of the target data is realized.
It should be noted that, the above description only lists several common data storage manners, and is not an exhaustive list of storage manners in the embodiments of the present invention, and therefore, the above description does not limit the specific manner in which the target data is stored in the first storage node.
It is worth mentioning that, in the above process, only the storage management node belongs to a certain storage management cluster alone as an example, but in order to save the cost of building the data storage system, the number of the storage management nodes is often set to be small, which may cause the phenomenon that the same storage management node belongs to multiple storage management clusters.
In view of the foregoing situation, in a specific implementation manner provided in the embodiment of the present invention, when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
When the target storage management node belongs to a plurality of storage management clusters at the same time, when the first storage node for storing the target data is determined, the storage node corresponding to the target storage management node needs to be selected from the storage management clusters corresponding to the determined target storage management cluster.
As shown in FIG. 3, a schematic diagram of another specific process for data storage provided by the embodiment of the present invention is shown in FIG. 3, where the control node is communicatively connected to cluster2 and cluster3, it can be understood that the control node can be communicatively connected to each of cluster2 and cluster3, the storage management cluster to which the control node is communicatively connected is peer-to-peer in form, as shown in FIG. 3, the storage cluster 11 is communicatively connected to cluster2, and the storage cluster 21 is communicatively connected to cluster3, then the storage management node 2 belongs to both cluster2 and cluster3, if the control node determines that cluster2 is the target storage management cluster according to the storage request, this also means that the storage request corresponding to the target data will be managed by the storage management node 2 in cluster2, therefore, the storage management node 2 will also correspond to the storage management node 2 in the storage cluster2 when determining the first storage node, The storage node 3 and the storage node 6. If the target storage management cluster determined by the control node according to the storage request is cluster3, then the storage request corresponding to the target data is managed by the storage management node 2 in cluster3 (assuming that the storage management node 2 in cluster2 is also named as storage management node 2 in cluster 3), so that the storage management node 2 can be determined in the storage node corresponding to the storage management node 2 in the storage cluster 21 corresponding to cluster3 when determining the first storage node, and since the process is similar to the process of determining the first storage node in cluster2, the description will be omitted here.
In general, in order to further save the construction cost of the data storage system, the control node generally has the characteristic of high parallelism degree, that is, the control node can simultaneously and parallelly carry out a plurality of processes. Taking fig. 3 as an example, assuming that a control node receives two storage requests at the same time, where a storage request 1 is a storage request for target data, and a storage request 2 is a storage request for another data, at this time, the control node will concurrently execute two processes, where the process 1 is used to process the storage request 1, and the process 2 is used to process the storage request 2, and assuming that the control node determines that a target storage management cluster corresponding to the storage request 1 is cluster2, and a target storage cluster corresponding to the storage request 2 is cluster3, since the process 1 and the process 2 are concurrently executed, an operation process of the process 1 and an operation process of the process 2 do not affect each other.
Illustratively, assuming that the target management node for storage request 1 determined in process 1 is storage management node 2 in cluster2, and the target storage management node for storage request 2 determined in process 2 is storage management node 2 in cluster3 (assuming that storage management node 2 in cluster2 is also named storage management node 2 in cluster 3), and for process 1, the storage request 1 corresponding to the target data is managed by storage management node 2 in cluster2, then storage management node 2 will also determine, in cluster2, the storage node 2, storage node 3 and storage node 6 corresponding to storage management node 2 in storage cluster 11 corresponding to cluster2 when determining the first storage node. For process 2, the storage request 2 corresponding to another data is managed by the storage management node 2 in cluster3 (assuming that the storage management node 2 in cluster2 is also named as storage management node 2 in cluster 3), so when determining the first storage node, the storage management node 2 can be determined in the storage cluster 21 corresponding to cluster3 and the storage node corresponding to the storage management node 2.
Because the process 1 and the process 2 run in parallel, the running state of the process 1 and the running state of the process 2 do not affect each other, for example, the process 2 fails before completing the storage of other data corresponding to the storage request 2, so that the storage management node 2 corresponding to the process 2 loses the data management capability, i.e., is in a dead state, and because the running state of the process 1 and the running state of the process 2 do not affect each other, even if the storage management node 2 belongs to both cluster2 and cluster3, the storage management node 2 belonging to the cluster3 is in a dead state and does not affect the survival state of the storage management node 2 belonging to the cluster 2; similarly, assuming that the storage management node 2 belonging to cluster2 is in a dead state, the surviving state of the storage management node 2 in cluster3 is not affected.
It should be noted that, the above description is only given by taking an example that the storage management node 2 in the cluster2 belongs to two storage management clusters, but the embodiment of the present invention does not limit the specific number of storage management nodes belonging to a storage management cluster, for example, the storage management node 2 may belong to cluster1, cluster2 and cluster3, and the target storage management node may also be the storage management node 3 and the storage management node 4, and therefore, the embodiment of the present invention also does not further limit the specific content of the target storage management node.
In summary, applying the embodiment provided in fig. 1, after receiving a storage request sent by a client, a control node; determining a target storage management cluster according to the storage request; then obtaining state information of the target storage cluster from the target storage cluster corresponding to the target storage management cluster, and determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; and the request target storage management node determines a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node. By adopting the scheme provided by the embodiment of the invention to store data, the control node can obtain the state information corresponding to the storage cluster after receiving the storage request without waiting, thereby saving the time for storing data. Meanwhile, the data storage state of the data storage system is reflected more accurately in real time due to the fact that the storage clusters on the bottom layer are directly explored, one-time acceptance of storage requests is guaranteed, the load uniformity principle and the request quantity uniformity principle are adopted when the target storage management nodes are determined, data resources can be uniformly distributed, and the phenomenon of network congestion is relieved to a certain extent.
Fig. 4 is a schematic flow chart of another data storage method provided in the embodiment of the present invention, which includes, in addition to steps S101 to S106 in the embodiment provided in fig. 1, the method further includes the steps of:
s107: monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
s108: in the case that it is monitored in step S107 that the first storage node fails, the target storage management node is requested to determine a second storage node, so that the client stores the target data in the second storage node, where the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
As can be seen from the above, there may be multiple ways to store the target data, and in consideration of a situation that a failure may occur in the process of storing the target data in the first storage node, in the scheme provided in the embodiment of the present invention, the control node may monitor a storage process of the target data when the target data is stored in the first storage node, and if it is monitored that the first storage node fails before the target data is stored incompletely, the control node may determine one storage node from other storage nodes in the target storage cluster, which are in communication connection with the target storage management node, as the second storage node, and store the target data in the second storage node again according to the foregoing way.
For example, it is assumed that the storage management node 2 in the cluster2 is in communication connection with the storage nodes 2, the storage nodes 3, and the storage nodes 6 in the storage cluster 11, where the storage node 6 is a first storage node, during the process of storing the target data in the storage node 6, the storage node 6 fails, after the control node monitors that the storage node 6 fails, it will determine again one storage node from the storage nodes 2 and the storage node 3 as a second storage node, and assuming that the storage node 2 is determined as the second storage node, since the storage node 2 and the storage node 6 are structurally completely equivalent, the process of storing the target data in the storage node 2 is identical to the foregoing process of storing the target data in the storage node 6, and details thereof are omitted here.
It can be seen that, by applying the embodiment provided in fig. 4, the control node monitors the process of storing the target data in the first storage node, and if it is monitored that the first storage node fails in the process, the control node determines one storage node from other storage nodes in the target storage cluster, which are in communication connection with the target storage management node, as the second storage node again, and stores the target data in the second storage node again, so that the phenomenon of failure in storing the target data due to failure of a certain storage node in the process of storing the target data is avoided, and the stability of the data storage system is improved.
Fig. 5 is a schematic flow chart of another storage method provided in the embodiment of the present invention, which includes, in addition to steps S101 to S106 in the embodiment provided in fig. 1, the steps of the method further include:
s109: monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data to the first storage node;
s110: in the case that it is monitored in step S109 that the target storage management node requests upgrade maintenance, the state of the target storage management node is marked as a dead state, and step S105 is executed again.
As can be understood by those skilled in the art, in a normal situation, as long as a target storage management node requests upgrade maintenance, a control node interrupts all tasks running on the target storage management node, so that the target control node can complete its own upgrade maintenance task, in consideration of the situation that the target storage management node performs upgrade maintenance during the process of storing target data in a first storage node, in the solution provided in the embodiment of the present invention, the control node may monitor whether the target storage management node requests upgrade maintenance or not when the target data is stored in the first storage node, and if it is monitored that the target storage management node requests upgrade maintenance, the control node marks the state of the target storage management node as a dead state, returns to step S105, and re-determines a storage management node to take over the data management task on the target storage management node, so as to realize the free upgrade maintenance of the target storage management node.
In summary, by applying the embodiment provided in fig. 5, the target storage management node can be freely upgraded and maintained in the target data storage process, the later maintenance and repair work on the data storage system is facilitated, and the stability of the data storage system is further improved.
Fig. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present invention, where the data storage device includes: a receiving module 201, a first determining module 202, an obtaining module 203, a second determining module 204, a third determining module 205 and a first storing module 206.
A receiving module 201, configured to receive a storage request sent by a client for storing target data;
a first determining module 202, configured to determine, according to the storage request, a target storage management cluster from storage management clusters of the data storage system;
an obtaining module 203, configured to obtain state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster;
a second determining module 204, configured to determine, according to the obtained state information, a storage management node in a surviving state in the target storage management cluster;
further, the second determining module 204 is specifically configured to:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
A third determining module 205, configured to determine a target storage management node from the storage management nodes in the alive state;
further, the third determining module 205 is specifically configured to:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
A first storage module 206, configured to request the target storage management node to determine, according to the storage request, a first storage node for storing the target data, so that the client stores the target data in the first storage node.
Further, when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
Application fig. 6 provides an embodiment, after the control node receives the storage request sent by the client; determining a target storage management cluster according to the storage request; then obtaining state information of the target storage cluster from the target storage cluster corresponding to the target storage management cluster, and determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; and the request target storage management node determines a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node. By adopting the scheme provided by the embodiment of the invention to store data, the control node can obtain the state information corresponding to the storage cluster after receiving the storage request without waiting, thereby saving the time for storing data. Meanwhile, the data storage state of the data storage system is reflected more accurately in real time due to the fact that the storage clusters on the bottom layer are directly explored, one-time acceptance of storage requests is guaranteed, the load uniformity principle and the request quantity uniformity principle are adopted when the target storage management nodes are determined, data resources can be uniformly distributed, and the phenomenon of network congestion is relieved to a certain extent.
Fig. 7 is a schematic structural diagram of another data storage apparatus provided in an embodiment of the present invention, where the apparatus includes, in addition to the receiving module 201, the first determining module 202, the obtaining module 203, the second determining module 204, the third determining module 205, and the first storing module 206 provided in the embodiment of fig. 4: a first monitoring module 207 and a second storage module 208.
A first monitoring module 207, configured to monitor whether the first storage node fails in a process that the client stores the target data in the first storage node;
a second storage module 208, configured to, when the first monitoring module 207 monitors that the first storage node fails, request the target storage management node to determine a second storage node, so that the client stores the target data in the second storage node, where the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
By applying the embodiment provided by fig. 7, the control node monitors the process of storing the target data in the first storage node, and if it is monitored that the first storage node fails in the process, the control node determines one storage node from other storage nodes in the target storage cluster, which are in communication connection with the target storage management node, as the second storage node again, and stores the target data in the second storage node again, so that the phenomenon of failure in storing the target data due to failure of one storage node in the process of storing the target data is avoided, and the stability of the data storage system is improved.
Fig. 8 is a schematic structural diagram of another data storage apparatus provided in an embodiment of the present invention, where the apparatus includes, in addition to the receiving module 201, the first determining module 202, the obtaining module 203, the second determining module 204, the third determining module 205, and the first storing module 206 provided in the embodiment of fig. 4: a second monitoring module 209 and a triggering module 210.
A second monitoring module 209, configured to monitor whether the target storage management node requests upgrade maintenance in a process that the client stores the target data in the first storage node;
a triggering module 210, configured to mark the state of the target storage management node as a dead state and trigger a third determining module when the second monitoring module 209 monitors that the target storage management node requests upgrade maintenance.
By applying the embodiment provided by the figure 8, the target storage management node can be freely upgraded and maintained in the target data storage process, the later overhauling and maintenance work of the data storage system is facilitated, and the stability of the data storage system is further improved.
An embodiment of the present invention further provides a data storage system, where the system includes: the storage management cluster comprises at least one storage management node, and the storage cluster comprises at least one storage node;
the control node is used for receiving a storage request sent by a client and used for storing target data; determining a target storage management cluster from the storage management clusters according to the storage request; acquiring state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster; determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; requesting the target storage management node to determine a first storage node for storing the target data according to the storage request;
the target storage management node is used for determining a first storage node for storing the target data according to the storage request;
and the first storage node is used for receiving and storing the target data sent by the client.
The control node is specifically configured to:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
Further, the control node is specifically configured to:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
It should be noted that, when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
By applying the data storage system provided by the embodiment of the invention to store data, the state information corresponding to the storage cluster can be obtained after the storage request is received, waiting is not needed, and the data storage time is saved. Meanwhile, the data storage state of the data storage system can be reflected more accurately in real time by directly detecting the activity of the storage cluster at the bottom layer, and a guarantee is provided for one-time acceptance of the storage request.
In another data storage system provided in an embodiment of the present invention, the control node is further configured to:
monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
if so, requesting the target storage management node to determine a second storage node so that the client stores the target data to the second storage node, wherein the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
In the data storage system provided in this embodiment, the control node monitors a process of storing the target data in the first storage node, and if it is monitored that the first storage node fails in the process, the control node determines one storage node from other storage nodes in the target storage cluster, which are in communication connection with the target storage management node, as the second storage node again, and stores the target data in the second storage node again, so that a phenomenon that the storage of the target data fails due to a failure of a certain storage node in a target data storage process is avoided, and the stability of the data storage system is improved.
In another data storage system provided in an embodiment of the present invention, the control node is further configured to:
monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data to the first storage node;
if so, marking the state of the target storage management node as a dead state, and returning to execute the step of determining the target storage management node from the storage management nodes in the survival state.
In the data storage system provided by the embodiment, the target storage management node can be freely upgraded and maintained in the target data storage process, the later-stage overhaul and maintenance work on the data storage system is facilitated, and the stability of the data storage system is further improved.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. As for the above device embodiment and system embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (13)

1. A data storage method is applied to a control node in a data storage system, and is characterized by comprising the following steps:
receiving a storage request sent by a client for storing target data;
determining a target storage management cluster from the storage management clusters of the data storage system according to the storage request;
acquiring state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster;
determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information;
determining a target storage management node from the storage management nodes in the survival state;
requesting the target storage management node to determine a first storage node for storing the target data according to the storage request, so that the client stores the target data to the first storage node.
2. The method of claim 1, wherein determining storage management nodes in a surviving state in the target storage management cluster according to the obtained state information comprises:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
3. The method of claim 1 or 2, wherein the determining the target storage management node from the storage management nodes in the surviving state comprises:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
4. The method according to claim 1 or 2,
when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
5. The method according to claim 1 or 2, characterized in that the method further comprises:
monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
if so, requesting the target storage management node to determine a second storage node so that the client stores the target data to the second storage node, wherein the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
6. The method of claim 1, further comprising:
monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data to the first storage node;
if so, marking the state of the target storage management node as a dead state, and returning to execute the step of determining the target storage management node from the storage management nodes in the survival state.
7. A data storage device for use in a control node in a data storage system, the device comprising:
the receiving module is used for receiving a storage request which is sent by a client and used for storing target data;
the first determining module is used for determining a target storage management cluster from the storage management clusters of the data storage system according to the storage request;
an obtaining module, configured to obtain state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster;
a second determining module, configured to determine, according to the obtained state information, a storage management node in a surviving state in the target storage management cluster;
a third determining module, configured to determine a target storage management node from the storage management nodes in the surviving state;
a first storage module, configured to request the target storage management node to determine, according to the storage request, a first storage node for storing the target data, so that the client stores the target data in the first storage node.
8. The apparatus of claim 7, wherein the second determining module is specifically configured to:
determining a storage management node in a survival state in the target storage management cluster according to the obtained state information and a preset corresponding relationship, wherein the corresponding relationship is as follows: and the corresponding relation between each storage management node in the target storage management cluster and each storage node in the target storage cluster.
9. The apparatus according to claim 7 or 8, wherein the third determining module is specifically configured to:
and determining a target storage management node from the storage management nodes in the survival state according to a load balancing principle and/or a request quantity uniform principle.
10. The apparatus according to claim 7 or 8,
when the target storage management node belongs to at least two storage management clusters including the target storage management cluster, the first storage node is: and the storage node in the target storage cluster corresponding to the target storage management node.
11. The apparatus of claim 7 or 8, further comprising:
the first monitoring module is used for monitoring whether the first storage node fails in the process that the client stores the target data in the first storage node;
a second storage module, configured to, when the first monitoring module monitors that the first storage node fails, request the target storage management node to determine a second storage node, so that the client stores the target data in the second storage node, where the second storage node is: and the storage nodes except the first storage node in the storage nodes corresponding to the target storage management node in the target storage cluster.
12. The apparatus of claim 7, further comprising:
the second monitoring module is used for monitoring whether the target storage management node requests upgrading maintenance in the process that the client stores the target data in the first storage node;
and the triggering module is used for marking the state of the target storage management node as a death state and triggering the third determining module under the condition that the second monitoring module monitors that the target storage management node requests upgrading maintenance.
13. A data storage system, the system comprising: the storage management cluster comprises at least one storage management node, and the storage cluster comprises at least one storage node;
the control node is used for receiving a storage request sent by a client and used for storing target data; determining a target storage management cluster from the storage management clusters according to the storage request; acquiring state information of the target storage cluster from a target storage cluster corresponding to the target storage management cluster; determining storage management nodes in a survival state in the target storage management cluster according to the obtained state information; determining a target storage management node from the storage management nodes in the survival state; requesting the target storage management node to determine a first storage node for storing the target data according to the storage request;
the target storage management node is used for determining a first storage node for storing the target data according to the storage request;
and the first storage node is used for receiving and storing the target data sent by the client.
CN201611224468.4A 2016-12-27 2016-12-27 Data storage method, device and system Active CN108241469B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611224468.4A CN108241469B (en) 2016-12-27 2016-12-27 Data storage method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611224468.4A CN108241469B (en) 2016-12-27 2016-12-27 Data storage method, device and system

Publications (2)

Publication Number Publication Date
CN108241469A CN108241469A (en) 2018-07-03
CN108241469B true CN108241469B (en) 2021-04-30

Family

ID=62701577

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611224468.4A Active CN108241469B (en) 2016-12-27 2016-12-27 Data storage method, device and system

Country Status (1)

Country Link
CN (1) CN108241469B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110267060B (en) * 2019-07-24 2021-03-26 北京视界云天科技有限公司 Video file storage injection method and device, computer equipment and storage medium
CN114691640A (en) * 2020-12-28 2022-07-01 网联清算有限公司 Database survival detection method and device
CN114553747A (en) * 2022-02-22 2022-05-27 度小满科技(北京)有限公司 Method, device, terminal and storage medium for detecting abnormality of redis cluster
CN116932505A (en) * 2022-03-31 2023-10-24 北京沃东天骏信息技术有限公司 Data query method, data writing method, related device and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103139302A (en) * 2013-02-07 2013-06-05 浙江大学 Real-time copy scheduling method considering load balancing
CN105205160A (en) * 2015-09-29 2015-12-30 浙江宇视科技有限公司 Data write-in method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8819317B1 (en) * 2013-06-12 2014-08-26 International Business Machines Corporation Processing input/output requests using proxy and owner storage systems

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103139302A (en) * 2013-02-07 2013-06-05 浙江大学 Real-time copy scheduling method considering load balancing
CN105205160A (en) * 2015-09-29 2015-12-30 浙江宇视科技有限公司 Data write-in method and device

Also Published As

Publication number Publication date
CN108241469A (en) 2018-07-03

Similar Documents

Publication Publication Date Title
CN108241469B (en) Data storage method, device and system
CN108829352B (en) User quota method and system for distributed storage system
CN106406983B (en) Task scheduling method and device in cluster
CN112532675B (en) Method, device and medium for establishing network edge computing system
CN113037794B (en) Method, device and system for computing resource allocation scheduling
CN113806066A (en) Big data resource scheduling method, system and storage medium
CN105095317A (en) Distributive database service management system
CN110908774B (en) Resource scheduling method, equipment, system and storage medium
CN103078933A (en) Method and device for determining data migration time
US20080288951A1 (en) Method, Device And System For Allocating A Media Resource
CN110569302A (en) method and device for physical isolation of distributed cluster based on lucene
JP2023504469A (en) NODE MANAGEMENT METHOD, APPARATUS, DEVICE, STORAGE MEDIUM AND SYSTEM
CN114745358A (en) IP address management method, system and controller in load balancing service
CN111580951B (en) Task allocation method and resource management platform
KR20100073151A (en) Asymetric cluster filesystem
CN112860386A (en) Method for switching nodes in distributed master-slave system
US20220345521A1 (en) Network edge computing method, apparatus, device and medium
CN111404978A (en) Data storage method and cloud storage system
CN111177160A (en) Service updating method, device, server and medium
US20230063541A1 (en) Determining computer resource usage at multiple levels of a container orchestration system hierarchy
CN104468674B (en) Data migration method and device
CN115865942A (en) Cloud platform resource monitoring method, electronic device and computer readable storage medium
WO2021248972A1 (en) Default gateway management method, gateway manager, server, and storage medium
CN105282203A (en) Method and device for establishing concentrated-directory P2P network
CN114327784A (en) Kubernetes storage resource binding method, system, equipment and medium based on position information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant