CN107729185A - A kind of fault handling method and device - Google Patents
A kind of fault handling method and device Download PDFInfo
- Publication number
- CN107729185A CN107729185A CN201711015125.1A CN201711015125A CN107729185A CN 107729185 A CN107729185 A CN 107729185A CN 201711015125 A CN201711015125 A CN 201711015125A CN 107729185 A CN107729185 A CN 107729185A
- Authority
- CN
- China
- Prior art keywords
- osd
- target storage
- storage node
- node
- threshold value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention, which provides a kind of fault handling method and device, this method, to be included:Obtain the free memory capacity of memory node and the information of object storage device OSD quantity;When it is determined that the free memory capacity of target storage node and the ratio of OSD quantity are less than the first predetermined threshold value, OSD increase operation of the refusal for the target storage node, and the part OSD of the target storage node is arranged to first kind inoperative Down states, so that the free memory capacity of the target storage node and the ratio of remaining OSD quantity are more than or equal to first predetermined threshold value;Wherein, remaining described OSD refers to the available OSD in addition to the part OSD for being arranged to first kind Down states in the OSD of the target storage node.The risk for being likely to occur OSD performance bottlenecks and data recovery failure on memory node after memory failure can be evaded using the present invention.
Description
Technical field
The present invention relates to network communication technology field, more particularly to a kind of fault handling method and device.
Background technology
Ceph (distributed memory system) is an open source projects, there is provided software definition, unified storage solution,
Possess can extend on a large scale, high-performance, without Single Point of Faliure the advantages of.
One typical Ceph clustered deploy(ment), can be that each piece of physical hard disk creates an OSD in clustered node
(Object Storage Device, object storage device).
The failure domain of Ceph clusters generally includes disk, node (i.e. server), frame, power circuit etc..Work as failure domain
After interior any component failure, the corresponding OSD that can cause to be deployed in thereon can break down, and Ceph clusters can be by these
OSD is labeled as Down (inoperative) state, and carries out initialization operation, reorganizes number impacted on the node to break down
According to.
But practice is found, in existing Ceph clusters implementation, because the partial memory of memory node is unavailable simultaneously
OSD state will not be directly affected, therefore, when the partial memory of memory node is unavailable, Ceph clusters can't be to OSD's
State is adjusted, if but now data recovery or releveling event occur for Ceph clusters, it can have a strong impact on Ceph clusters
Process performance, even result in data recovery failure situation occur.
The content of the invention
The present invention provides a kind of fault handling method and device, may to solve memory node memory failure in the prior art
Ceph cluster process performances can be reduced, even result in the problem of Ceph company-datas recover failure.
First aspect according to embodiments of the present invention, there is provided a kind of fault handling method, this method include:
Obtain the free memory capacity of memory node and the information of object storage device OSD quantity;
When it is determined that the free memory capacity of target storage node and the ratio of OSD quantity are less than first in the memory node
During predetermined threshold value, refusal for the target storage node OSD increase operations, and by the part of the target storage node
OSD is arranged to first kind inoperative Down states, so that the free memory capacity of the target storage node and remaining OSD number
The ratio of amount is more than or equal to first predetermined threshold value;Wherein, remaining described OSD refers in the OSD of the target storage node
Available OSD in addition to the part OSD for being arranged to first kind Down states.
Second aspect according to embodiments of the present invention, there is provided a kind of fault handling method, this method include:
Obtain the information of itself free memory capacity and object storage object equipment OSD quantity;
The information of itself free memory capacity and OSD quantity is reported to monitor.
The third aspect according to embodiments of the present invention, there is provided a kind of fault treating apparatus, the device include:
Acquiring unit, for obtaining the free memory capacity of memory node and the information of object storage device OSD quantity;
Determining unit, for determining that memory size and object in the memory node with the presence or absence of target storage node are deposited
The ratio of storage equipment OSD quantity is less than the first predetermined threshold value;
Processing unit, for determining that the free memory of target storage node in the memory node holds when the determining unit
When the ratio of amount and OSD quantity is less than the first predetermined threshold value, refusal increases operation for the OSD of the target storage node, and
The part OSD of the target storage node is arranged to first kind inoperative Down states, so that the target storage node
Free memory capacity and the ratio of remaining OSD quantity be more than or equal to first predetermined threshold value;Wherein, remaining described OSD is
Refer to the available OSD in addition to the part OSD for being arranged to first kind Down states in the OSD of the target storage node.
Fourth aspect according to embodiments of the present invention, there is provided a kind of fault treating apparatus, the device include:
Acquiring unit, for obtaining the information of itself free memory capacity and object storage object equipment OSD quantity;
Transmitting element, for reporting the information of itself free memory capacity and OSD quantity to monitor.
Using the embodiment of the present invention, the first default threshold of the memory source deficiency by being provided for indicating memory node
Value, when it is determined that the free memory capacity of target storage node and the ratio of OSD quantity in memory node are less than the first default threshold
During value, refusal is operated for the OSD increases of target storage node, and the part OSD of target storage node is arranged into the first kind
Type Down states, preset so that the free memory capacity of target storage node and the ratio of remaining OSD quantity are more than or equal to first
Threshold value, the risk for being likely to occur OSD performance bottlenecks and data recovery failure on memory node after memory failure is evaded.
Brief description of the drawings
Fig. 1 is a kind of schematic flow sheet of fault handling method provided in an embodiment of the present invention;
Fig. 2 is a kind of schematic flow sheet of fault handling method provided in an embodiment of the present invention;
Fig. 3 is a kind of structural representation of fault treating apparatus provided in an embodiment of the present invention;
Fig. 4 is the structural representation of another fault treating apparatus provided in an embodiment of the present invention;
Fig. 5 is a kind of structural representation of fault treating apparatus provided in an embodiment of the present invention;
Fig. 6 is the hardware configuration that a kind of monitor and memory node provided in an embodiment of the present invention are located at same physical host
Schematic diagram.
Embodiment
In order that those skilled in the art more fully understand the technical scheme in the embodiment of the present invention, and make of the invention real
Apply the above-mentioned purpose of example, feature and advantage can be more obvious understandable, below in conjunction with the accompanying drawings to technical side in the embodiment of the present invention
Case is described in further detail.
Fig. 1 is referred to, is a kind of schematic flow sheet of fault handling method provided in an embodiment of the present invention, wherein, the event
Barrier processing method can apply to the memory node of Ceph clusters, as shown in figure 1, the fault handling method can include following step
Suddenly:
It should be noted that in embodiments of the present invention, if non-specified otherwise, memory size is in units of G, memory node
OSD quantity be that the OSD of UP (work) state is in memory node, the embodiment of the present invention subsequently no longer repeats.
Step 101, the information for obtaining itself free memory capacity and OSD quantity.
Step 102, the information for reporting to monitor itself free memory capacity and OSD quantity.
In the embodiment of the present invention, in order to ensure the OSD of each memory node in Ceph clusters energy normal works, it is necessary to ensure to deposit
Each OSD is configured with enough internal memories in storage node, for memory node generally speaking, it is necessary to ensures the available interior of memory node
The ratio for depositing capacity and OSD quantity is sufficiently large.
When memory node, which initializes, to be run, it will usually sufficient memory source is reserved for OSD, still, works as memory node
When inside points internal memory breaks down, such as memory bar loosen, memory bar interface fault, the free memory capacity of memory node and
The ratio of OSD quantity can decline, and even result in low free memory to meet the needs of OSD normal works.
Therefore, in embodiments of the present invention, memory node can report itself available interior according to preset strategy to monitor
The information of capacity and OSD quantity is deposited, such as free memory capacity and the ratio of OSD quantity.
For example, memory node regularly can report itself free memory capacity and OSD numbers (as periodically) to monitor
The ratio of amount;Or memory node can when the ratio of itself free memory capacity and OSD quantity meets specified requirements, to
Monitor reports itself free memory capacity and the ratio of OSD quantity.
It is above-mentioned to report itself free memory capacity and OSD quantity to monitor in the one of embodiment of the present invention
Information, it can include:
When itself free memory capacity and the ratio of OSD quantity are less than the first predetermined threshold value, first is sent to monitor
Type of report.
In this embodiment it is possible to a threshold value (the referred to herein as first default threshold is set in each memory node in advance
Value), the threshold value is used to be judged as whether the reserved memory sources of OSD meet that OSD is capable of the Minimum requirements of normal work, i.e., ought deposit
When the free memory capacity of storage node and the ratio of OSD quantity are more than or equal to first predetermined threshold value, show that the memory node is
Memory source reserved OSD meets that OSD is capable of the Minimum requirements of normal work, otherwise, shows what the memory node was reserved for OSD
Memory source is unsatisfactory for the Minimum requirements that OSD is capable of normal work.Wherein, first predetermined threshold value can be set according to actual scene
It is fixed, such as determined according to the actual hard-disk capacity of memory node.
Alternatively, first predetermined threshold value can be the arbitrary value in the range of 0.5~1.5.
Correspondingly, in this embodiment, memory node can obtain itself free memory capacity and the ratio of OSD quantity in real time
Value, to determine whether the ratio of itself free memory capacity and OSD quantity is less than the first predetermined threshold value, and when determination itself is available
When the ratio of memory size and OSD quantity is less than the first predetermined threshold value, report (the referred to herein as first kind is sent to monitor
Report), to notify the ratio of the free memory capacity of the monitor memory node and OSD quantity to be less than the first predetermined threshold value, i.e.,
The free memory capacity of the memory node performs step to be not enough to the Minimum requirements for meeting OSD normal works, to trigger monitor
Rapid 202.
For example, memory node can obtain itself free memory capacity and the ratio of OSD quantity in real time, and by OSD
Finger daemon periodically (cycle can set according to actual scene, such as 300 seconds) poll free memory capacity and OSD numbers
The ratio of amount, and whether the ratio of the free memory capacity and OSD quantity is judged more than the first predetermined threshold value, and when OSD is guarded
When process determines that the ratio of the free memory capacity and OSD quantity is less than the first predetermined threshold value, the first kind is sent to monitor
Report.
Wherein, it is contemplated that memory node usually there will be multiple OSD finger daemons (consistent with OSD quantity), in order to avoid
Multiple OSD finger daemons in same memory node repeat to send first kind report to monitor, and each OSD finger daemons can be with
The poll of free memory capacity and the ratio of OSD quantity is carried out with different cycle, or, with the identical cycle but it is different when
Carve and start poll.
For example, it is assumed that memory node create have 32 OSD, can correspondingly, in the memory node run 32 OSD guard into
Journey (OSD finger daemons 1~32), each OSD finger daemons can use with the cycle of 300 seconds to memory node poll in memory node
The ratio of memory size and OSD quantity, for example, OSD finger daemons 2 can be at 5 seconds of the Poll starting time of OSD finger daemons 1
Start poll afterwards, OSD finger daemons 3 can start poll ... OSD after 5 seconds of the Poll starting time of OSD finger daemons 2
Finger daemon 32 can start poll after 5 seconds of the Poll starting time of OSD finger daemons 31.
When any OSD finger daemons find that the free memory capacity of memory node and the ratio of OSD quantity are pre- less than first
If during threshold value, the OSD finger daemons can send first kind report to monitor;Monitor receives first kind report
When, a confirmation message can be responded, such as ACK (confirmation) message;OSD finger daemons receive the confirmation message of monitor return
Afterwards, can be sent a notification message to other OSD finger daemons so that other OSD finger daemons receive the notification message it
Afterwards, silent status can be in certain time length, i.e., without free memory capacity and the ratio of OSD quantity in preset duration
Poll.Wherein, the silent duration of each OSD finger daemons needs to be more than each OSD finger daemons progress free memory capacity and OSD
The polling cycle of the ratio of quantity, such as when polling cycle is 300 seconds, silent duration can be 900 seconds or 1800 seconds etc..When
OSD finger daemons receive the notification message that other OSD finger daemons are sent in the duration of silence, then OSD finger daemons can
It is zeroed with the timing to the silence period, restarts the timing of silence period;Otherwise, i.e. OSD finger daemons are in silent duration
When not receiving the notification message that other OSD finger daemons are sent, OSD finger daemons can start reach poll next time
During start time, the poll of free memory capacity and the ratio of OSD quantity is carried out.
In the embodiment of the present invention, when monitor receives the first kind report that memory node reports, on the one hand, can refuse
Absolutely for the OSD increase operations of the memory node;On the other hand, the part OSD on memory node can be carried out at work
Reason, i.e., the part OSD on memory node is arranged to first kind Down states and (is referred to as managing Down shapes herein
State) so that the free memory capacity of memory node and the ratio of remaining OSD quantity are more than or equal to the first predetermined threshold value, its is specific
The associated description that may refer in method flow shown in Fig. 2 is realized, the embodiment of the present invention will not be described here.
Wherein, remaining OSD refers to remove the part OSD for being arranged to first kind Down states in the OSD of memory node
Outside available OSD (i.e. the OSD of UP states);First kind Down and OSD Down in existing Ceph clusters difference are:
The former OSD not failure, i.e. OSD has in harness ability, but is set to off working state;The latter OSD breaks down, i.e.,
OSD does not have the ability of work, and is arranged to Down states.
It can be seen that in method flow shown in Fig. 1, itself free memory capacity and OSD quantity are obtained by memory node
Ratio, and when reporting itself free memory capacity and the ratio of OSD quantity to monitor, so that monitor is it is determined that memory node
Free memory capacity and the ratio of OSD quantity when being less than the first predetermined threshold value, the part OSD on memory node is arranged to
One type Down states, the free memory capacity of memory node and the ratio of remaining OSD quantity is set to be more than or equal to the first default threshold
Value, that is, it is that OSD is reserved with enough free memory capacity to ensure memory node, and having evaded may after memory failure on memory node
There is the risk of OSD performance bottlenecks and data recovery failure.
Fig. 2 is referred to, is a kind of schematic flow sheet of fault handling method provided in an embodiment of the present invention, wherein, the party
The monitor (Monitor) that method can apply in Ceph clusters, as shown in Fig. 2 the fault handling method can include it is following
Step:
It should be noted that in embodiments of the present invention, monitor can be deployed in different physical hosts from memory node
On, same physical host can also be deployed in.Memory node can be understood as comprising at least the collection for the OSD for playing store function
Close, certain memory node can also include the processor for possessing control function.
The information of step 201, the free memory capacity for obtaining memory node and OSD quantity.
In the embodiment of the present invention, in order to avoid being likely to occur OSD performance bottlenecks and data after memory failure on memory node
Recover the risk of failure, monitor can obtain the free memory capacity of memory node and the information of OSD quantity, so as to it is determined that
When the free memory capacity of memory node and the too low ratio of OSD quantity, corresponding strategy is taken to improve the available of memory node
The ratio of memory size and OSD quantity.Wherein, the information of the available storage of the memory node of above-mentioned acquisition and OSD quantity
It is directed to whole cluster.But the process of above- mentioned information is obtained, obtained, that is, obtained in units of physical host
Parameter is for being same physical host, that is, (the total of OSD set can for the free memory capacity of the memory node obtained
With memory size) and OSD quantity be same physical host parameter.
In the one of embodiment of the present invention, the free memory capacity of above-mentioned acquisition memory node and the letter of OSD quantity
Breath, can include:
The free memory capacity and the information of OSD quantity that reception memory node reports.
In this embodiment, memory node reports the specific reality of the information of free memory capacity and OSD quantity to monitor
The associated description in embodiment of the method shown in Fig. 1 is now may refer to, the embodiment of the present invention will not be repeated here.
In another embodiment, the information of the free memory capacity of above-mentioned acquisition memory node and OSD quantity,
It can include:
Detect the active volume of the memory node and the information of OSD quantity.
In this embodiment, monitor can be with the active volume of active probe memory node and the information of OSD quantity, example
Such as, monitor can the active volume of timing (as periodically) detection memory node and the information of OSD quantity.
It should be noted that in this embodiment it is possible to institute in Ceph clusters is detected by any monitor in Ceph clusters
Have the free memory capacity of memory node and the ratio of OSD quantity, or, can according to the quantity of monitor in Ceph clusters,
All memory nodes in Ceph clusters are divided into multiple packets, the quantity of the packet can be consistent with the quantity of monitor, and one
Individual monitor detects the free memory capacity of memory node and the ratio of OSD quantity in a packet, and it is implemented herein
Do not repeat.
Step 202, when it is determined that the free memory capacity of target storage node in memory node and the ratio of OSD quantity are small
When the first predetermined threshold value, refusal for target storage node OSD increase operations, and by the part OSD of target storage node
First kind inoperative Down states are arranged to, so that the free memory capacity of target storage node and the ratio of remaining OSD quantity
Value is more than or equal to the first predetermined threshold value.
In the embodiment of the present invention, target storage node and the memory node for being not specific to a certain fixation, but may refer to
Any memory node in Ceph clusters.
In the one of embodiment of the present invention, the free memory capacity of target storage node and the ratio of OSD quantity are determined
Value is less than the first predetermined threshold value, can include:
Receive the first kind report of target storage node transmission.
Wherein, target storage node may refer to side shown in Fig. 1 to the specific implementation of monitor transmission first kind report
Associated description in method flow, the embodiment of the present invention will not be repeated here.
In this embodiment, when monitor receives the first kind report of target storage node transmission, it may be determined that
The free memory capacity of target storage node and the ratio of OSD quantity are less than the first predetermined threshold value.
In another embodiment, the free memory capacity of target storage node and the ratio of OSD quantity are determined
Less than the first predetermined threshold value, can include:
Detect the free memory capacity of target storage node and the ratio of OSD quantity is less than the first predetermined threshold value.
In this embodiment, monitor can with the free memory capacity of each memory node in detecting periodically Ceph clusters and
The ratio of OSD quantity, and judge whether the free memory capacity of each memory node and the ratio of OSD quantity are less than the first default threshold
Value.
In the embodiment of the present invention, the free memory capacity of target storage node and the ratio of OSD quantity are determined when monitor
During less than the first predetermined threshold value, on the one hand, monitor needs to refuse the OSD increase operations for the target storage node, that is, works as
Monitor, which receives, to be needed, when increasing OSD notification message on target storage node, to forbid this OSD increase operations, avoid
The increase of quantity further reduces the free memory capacity of target storage node and the ratio of OSD quantity on target storage node,
There is the risk of OSD performance bottlenecks and data recovery failure in aggravation target storage node.
On the other hand, monitor can carry out work disposal to the part OSD of target storage node, i.e., store target
The part OSD of node is arranged to first kind Down states, so that the free memory capacity of target storage node and remaining OSD number
The ratio of amount is more than or equal to the first predetermined threshold value, that is, ensures that the memory source of target storage node disclosure satisfy that in UP states
The Minimum requirements of OSD normal work.
As an example it is assumed that the first predetermined threshold value is h1, the initial free memory capacity of target storage node is M0, initially
OSD quantity is N0, M0/N0 >=h1;If a certain moment, target storage node inside points memory failure, cause free memory capacity
Change turns to M1, and M1/N0 < h1, and target storage node can report the first kind to report to monitor, and now, monitor can be from
N1 OSD is selected to be arranged to first kind Down states in target storage node so that M1/ (N0-N1) >=h1.Optionally, select
The quantity that selects out be arranged to the OSD of first kind Down states can be the minimum value for the N1 for meeting M1/ (N0-N1) >=h1.
Further, in embodiments of the present invention, after monitor has carried out above-mentioned processing to memory node, if storage
The memory failure of node is repaired, then the free memory capacity of memory node can increase, and correspondingly, can use for memory node is interior
Depositing capacity and the ratio of OSD quantity can also increase, and now, monitor can be arranged to by the memory node upper part or all
The OSD of first kind Down states reverts to UP states.
And in order to avoid monitor is extensive by the OSD for being partly or entirely arranged to first kind Down states in memory node
It is that the free memory capacity of memory node is again lowered to less than the first default threshold with the ratio of OSD quantity after UP states again
Value, results in the need for that the part OSD of memory node is arranged into first kind Down states again, can be pre- so as to form concussion
Another threshold value (referred to herein as the second predetermined threshold value) is first set, second predetermined threshold value is used to determine whether to store
The OSD for being arranged to first kind Down states on node reverts to UP states.Wherein, it is pre- to be more than first for second predetermined threshold value
If threshold value.
Preferably, second predetermined threshold value is free memory capacity and OSD ratio ideally, you can uses internal memory
Optimal or preferred proportion between capacity and OSD.
Optionally, the second predetermined threshold value can be the arbitrary value in the range of 1.5~2.0, wherein, when the first predetermined threshold value is
When 1.5, the second predetermined threshold value needs to be more than 1.5.
Correspondingly, it is above-mentioned that the part OSD of target storage node is arranged to first in the one of embodiment of the present invention
After type Down states, it can also include:
When it is determined that the free memory capacity of target storage node and the ratio of OSD quantity are more than the second predetermined threshold value, permit
Perhaps the OSD increase operations of target storage node are directed to, and be more than with the free memory capacity after recovery and the ratio of OSD quantity etc.
In the principle of the second predetermined threshold value, the portion in the part OSD of first kind Down states will be arranged in target storage node
Divide or whole OSD revert to UP states.
In the embodiment, monitor determines that the free memory capacity of target storage node and the ratio of OSD quantity are more than the
Two predetermined threshold values can include:
The Second Type report that target storage node reports is received, or, detects the free memory of target storage node
Capacity and the ratio of OSD quantity are more than the second predetermined threshold value.
Wherein, Second Type report is that target storage node determines itself free memory capacity and the ratio of OSD quantity
Turn to what is sent during more than the second predetermined threshold value by becoming less than the first predetermined threshold value, allow to be directed to target to trigger monitor and perform
The OSD increase operations of memory node, and be more than or equal to second with the free memory capacity after recovery and the ratio of OSD quantity and preset
The principle of threshold value, the part or all of OSD in the part OSD of first kind Down states will be arranged in target storage node
Revert to the operation of UP states.
Wherein, target storage node sends specific implementation and the target storage node of Second Type report to prison to monitor
Realizing for visual organ transmission first kind report is similar, and the embodiment of the present invention will not be described here.
In this embodiment, when monitor determine target storage node free memory capacity and OSD quantity ratio it is big
When the second predetermined threshold value, monitor can determine that target storage node is reserved with enough memory sources for OSD, now, can
To allow target storage node to increase OSD quantity.
Correspondingly, on the one hand, monitor can allow the OSD increase operations for target storage node;On the other hand, supervise
The OSD for being partly or entirely arranged to first kind Down states in target storage node can be reverted to UP states by visual organ.
Wherein, monitor is carried out when above-mentioned recovery is handled to target storage node, it is necessary to ensure that target storage node recovers
The ratio of free memory capacity and OSD quantity after processing is more than or equal to Second Threshold.
For example, still exemplified by one example of the above, it is assumed that the second predetermined threshold value is h2, if a certain moment, target storage section
Point free memory capacity restoration is M2 (M1 < M2≤M0), and M2/ (N0-N1) > h2, and now, monitor can store from target
It is arranged to select N2 (0 < N2 < N1) individual OSD to revert to UP states in the OSD of first kind Down states in node, wherein,
M2/(N0-N1+N2)≥h2。
Further, in the one of embodiment of the present invention, when monitor determines the free memory of target storage node
Capacity and the ratio of OSD quantity are more than or equal to the first predetermined threshold value, and when being less than the second predetermined threshold value, refusal is directed to memory node
OSD increase operation.
In this embodiment, monitor determines that the free memory capacity of target storage node and the ratio of OSD quantity are more than
Equal to the first predetermined threshold value, and the 3rd type report that can be sent less than the second predetermined threshold value including receiving target storage node
Accuse, or, detect the free memory capacity of target storage node and the ratio of OSD quantity is more than or equal to the first predetermined threshold value, and
Less than the second predetermined threshold value.
Wherein, the 3rd type of report is that target storage node determines itself free memory capacity and the ratio of OSD quantity
More than or equal to the first predetermined threshold value, and it is less than what is sent during the second predetermined threshold value, and then triggers monitor refusal for storage section
The OSD increase operations of point.
Wherein, target storage node sends specific implementation and the target storage node of the 3rd type of report to prison to monitor
Realizing for visual organ transmission first kind report is similar, and the embodiment of the present invention will not be described here.
It should be noted that when the free memory capacity of target storage node and the ratio of OSD quantity are by pre- less than first
If changes of threshold is more than or equal to the first predetermined threshold value, and when being less than the second predetermined threshold value, illustrates that now memory node can be tieed up
OSD normal work is held, but has been unsatisfactory for the optimal or preferable distribution between OSD and free memory capacity, is now supervised
Visual organ can keep OSD increase operation of the refusal for target storage node, without carrying out other specially treateds.
Further, in the one of embodiment of the present invention, when monitor determines the free memory of target storage node
Capacity and the ratio of OSD quantity are less than the first predetermined threshold value, or, more than or equal to the first predetermined threshold value and are less than the second predetermined threshold value
When, monitor can generate system health daily record, and the system health daily record is used to record target storage node generation memory failure,
So as to, keeper can know that the situation of memory failure occurs for target storage node according to the system health daily record, and according to need
Seek carry out respective handling.
It should be noted that in embodiments of the present invention, after being changed due to the OSD states of memory node, Ceph collection
The Leader (leader) of group needs to update cluster Map (mapping), therefore, if not Leader monitor is by the middle part of memory node
Point OSD is arranged to after first kind Down states, it is also necessary to Leader nodes are notified, by Leader node updates cluster Map,
Therefore, in order to improve cluster Map renewal efficiency, above-mentioned processing can be performed by the Leader of Ceph clusters and operated.
In order that those skilled in the art more fully understand that the embodiment of the present invention provides technical scheme, with reference to specific reality
Example illustrates to technical scheme provided in an embodiment of the present invention.
In this embodiment it is assumed that the first predetermined threshold value (h1) is 1, the second predetermined threshold value (h2) is 1.5;Assuming that target is deposited
The initial free memory capacity of storage node is 64G (assuming that being provided by 16 4G internal memories), and OSD quantity is 32.The then embodiment
Middle fault handling method flow is as follows:
1st, when 5 root memories break down in target storage node, the free memory capacity and OSD numbers of target storage node
The ratio (being depicted below as h3) of amount is h3=(64-4*5)/32=1.375, i.e. h1 < h3 < h2, now target storage node
It can be reported by OSD finger daemons to monitor, i.e., send the 3rd type of report to monitor;
When monitor receives three type of report of target storage node transmission, determine that target storage node is in h1 <
H3 < h2 state, now, monitor can perform following processing:
A, system health daily record is generated, memory failure occurs for record target storage node;
B, OSD increase operation of the refusal for target storage node;
2nd, when other 4 root memory of target storage node breaks down (i.e. totally 9 root memories break down), target storage section
H3=(64-4*9)/32=0.875, i.e. the h3 < h1 of point, now, target storage node can be by OSD finger daemons to prison
Visual organ reports, i.e., sends first kind report to monitor;
When monitor receives the first kind report of target storage node transmission, determine that target storage node is in h3 <
H1 state, now, monitor can perform following processing:
A, system health daily record is generated, serious memory failure occurs for record target storage node;
B, OSD increase operation of the refusal for target storage node;
C, work disposal is carried out to target storage node upper part OSD, its realization is as follows:
I, the quantity o1 for the OSD for needing work disposal is calculated, the h3 of the target storage node after being handled with guarantee is more than
H1, i.e. 1≤(64-4*9)/(32-o1), therefore, o1 minimum value is 4;
Random 4 OSD on target storage node are carried out work disposal by ii, monitor, will select 4 OSD
It is arranged to manage Down states (i.e. first kind Down states), and is moved the data on this 4 OSD by data recovery action
Move on other normal OSD of Ceph clusters;
Wherein, monitor, can be periodically after having carried out going work disposal to the 4 of target storage node OSD
Detect the h3 values of target storage node;
3rd, when keeper has found system health daily record and to find target storage node according to the system health daily record, and carry out
During internal memory recovery processing (assuming that the free memory capacity after recovering is M), the h3 of target storage node can increase, when monitor is visited
When measuring target storage node and being in h3 > h2 state (i.e. h3 > 1.5), monitor can perform following processing:
A, the OSD increase operations for target storage node are allowed;
B, the OSD of target storage node upper part or cura generalis Down states is reverted into UP states, its realization is as follows:
I, the quantity o2 for the OSD for needing to revert to UP states is calculated, the h3 of the target storage node after being handled with guarantee is big
In equal to h2, i.e. 1.5≤M/ (28+o2);
The OSD of o2 quantity reverts to UP states in ii, monitor will manage Down states OSD on target storage node,
And carry out releveling processing.
Wherein, the difference that monitor performs according to the change of the free memory capacity of memory node and the ratio of OSD quantity
The schematic diagram of operation can be as shown in Figure 3.
By above description as can be seen that in technical scheme provided in an embodiment of the present invention, by being provided for indicating
First predetermined threshold value of the memory source deficiency of memory node, when it is determined that the free memory of the target storage node in memory node
When capacity and the ratio of OSD quantity are less than the first predetermined threshold value, refusal operates for the OSD increases of target storage node, and will
The part OSD of target storage node is arranged to first kind Down states so that the free memory capacity of target storage node and
The ratio of remaining OSD quantity is more than or equal to the first predetermined threshold value, has evaded and has been likely to occur OSD on memory node after memory failure
Energy bottleneck and the risk of data recovery failure.
Fig. 4 is referred to, is a kind of structural representation of fault treating apparatus provided in an embodiment of the present invention, wherein, the dress
The monitor that can apply in above method embodiment is put, as shown in figure 4, the fault treating apparatus can include:
Acquiring unit 410, for obtaining the free memory capacity of memory node and the letter of object storage device OSD quantity
Breath;
Determining unit 420, for determining in the memory node with the presence or absence of the memory size of target storage node and right
As the ratio of storage device OSD quantity is less than the first predetermined threshold value;
Processing unit 430, for determining the available of target storage node in the memory node when the determining unit 420
When the ratio of memory size and OSD quantity is less than the first predetermined threshold value, OSD increase behaviour of the refusal for the target storage node
Make, and the part OSD of the target storage node is arranged to first kind inoperative Down states, so that the target stores
The free memory capacity of node and the ratio of remaining OSD quantity are more than or equal to first predetermined threshold value;Wherein, it is described remaining
OSD refers to available in addition to the part OSD for being arranged to first kind Down states in the OSD of the target storage node
OSD。
In an alternative embodiment, the acquiring unit 410, the free memory reported specifically for receiving the memory node
The information of capacity and OSD quantity;Or, the active volume of the detection memory node and the information of OSD quantity.
In an alternative embodiment, the processing unit 430, it is additionally operable to when the determining unit 420 is when the determination target
When the free memory capacity of memory node and the ratio of OSD quantity are more than the second predetermined threshold value, it is allowed to stored for the target
The OSD increase operations of node, and free memory capacity after being handled with recovery and the ratio of OSD quantity are more than or equal to described second
The principle of predetermined threshold value, the part in the part OSD of first kind Down states will be arranged in the target storage node
Or whole OSD revert to work UP states;Wherein, first predetermined threshold value is less than second predetermined threshold value.
In an alternative embodiment, the acquiring unit 410, the free memory for being additionally operable to detect the target storage node hold
The ratio of amount and OSD quantity;
The determining unit 420, specifically for when the free memory capacity and OSD numbers that detect the target storage node
When the ratio of amount is more than the second predetermined threshold value, the free memory capacity of the target storage node and the ratio of OSD quantity are determined
More than the second predetermined threshold value.
In an alternative embodiment, the processing unit 430, it is additionally operable to when the memory size for determining the target storage node
When being more than or equal to first predetermined threshold value with the ratio of OSD quantity, and being less than second predetermined threshold value, refusal is for described
The OSD increase operations of target storage node.
Fig. 5 is referred to, is a kind of structural representation of fault treating apparatus provided in an embodiment of the present invention, wherein, the dress
The memory node that can apply in above method embodiment is put, as shown in figure 5, the fault treating apparatus can include:
Acquiring unit 510, for obtaining the information of itself free memory capacity and object storage object equipment OSD quantity;
Transmitting element 520, for reporting the information of itself free memory capacity and OSD quantity to monitor.
The hardware configuration that Fig. 6 is located at same physical host for a kind of monitor and memory node that disclosure example provides shows
It is intended to.But it is to be understood that monitor and memory node can be located at different physical hosts.The present embodiment is with Fig. 6 situation
Illustrate.The physical host may include processor 601, be stored with the machinable medium 602 of machine-executable instruction
And metadata and/or the OSD603 of copy for data storage object.
Processor 601 can communicate with machinable medium 602 and OSD603 via system bus 604.It is also, logical
Cross and read and perform machine-executable instruction corresponding with fault handling logic, processor 601 in machinable medium 602
It can perform above-described fault handling method.
Machinable medium 602 referred to herein can be any electronics, magnetic, optics or other physical stores
Device, can be included or storage information, such as executable instruction, data, etc..For example, machinable medium can be:
RAM (Radom Access Memory, random access memory), volatile memory, nonvolatile memory, flash memory, storage are driven
Dynamic device (such as hard disk drive), solid state hard disc, any kind of storage dish (such as CD, dvd), or similar storage are situated between
Matter, or combinations thereof.OSD603 can include but is not limited to physical disk.
The function of unit and the implementation process of effect specifically refer to and step are corresponded in the above method in said apparatus
Implementation process, it will not be repeated here.
For device embodiment, because it corresponds essentially to embodiment of the method, so related part is real referring to method
Apply the part explanation of example.Device embodiment described above is only schematical, wherein described be used as separating component
The unit of explanation can be or may not be physically separate, can be as the part that unit is shown or can also
It is not physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to reality
Need to select some or all of module therein to realize the purpose of the present invention program.Those of ordinary skill in the art are not paying
In the case of going out creative work, you can to understand and implement.
Those skilled in the art will readily occur to the present invention its after considering specification and putting into practice invention disclosed herein
Its embodiment.The application be intended to the present invention any modification, purposes or adaptations, these modifications, purposes or
Person's adaptations follow the general principle of the present invention and including undocumented common knowledges in the art of the invention
Or conventional techniques.Description and embodiments are considered only as exemplary, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be appreciated that the invention is not limited in the precision architecture for being described above and being shown in the drawings, and
And various modifications and changes can be being carried out without departing from the scope.The scope of the present invention is only limited by appended claim.
Claims (12)
1. a kind of fault handling method, it is characterised in that this method includes:
Obtain the free memory capacity of memory node and the information of object storage device OSD quantity;
When it is determined that the free memory capacity of target storage node and the ratio of OSD quantity are default less than first in the memory node
During threshold value, refusal is operated for the OSD increases of the target storage node, and the part OSD of the target storage node is set
First kind inoperative Down states are set to, so that the free memory capacity of the target storage node and remaining OSD quantity
Ratio is more than or equal to first predetermined threshold value;Wherein, remaining described OSD refers to remove quilt in the OSD of the target storage node
The available OSD being arranged to outside the part OSD of first kind Down states.
2. according to the method for claim 1, it is characterised in that the free memory capacity and OSD for obtaining memory node
The information of quantity, including:
Receive the information of free memory capacity that the memory node reports and OSD quantity;Or,
Detect the active volume of the memory node and the information of OSD quantity.
3. according to the method for claim 1, it is characterised in that the part OSD by the target storage node is set
After first kind Down states, in addition to:
When it is determined that the free memory capacity of the target storage node and the ratio of OSD quantity are more than the second predetermined threshold value, permit
Perhaps the OSD increase operations of the target storage node, and free memory capacity and the ratio of OSD quantity after being handled with recovery are directed to
Value will be arranged to first kind Down states more than or equal to the principle of second predetermined threshold value in the target storage node
Part OSD in part or all of OSD revert to work UP states;Wherein, first predetermined threshold value is less than described second
Predetermined threshold value.
4. according to the method for claim 3, it is characterised in that the free memory for determining the target storage node holds
The ratio of amount and OSD quantity is more than the second predetermined threshold value, including:
Detect the free memory capacity of the target storage node and the ratio of OSD quantity;
When the free memory capacity and the ratio of OSD quantity that detect the target storage node are more than the second predetermined threshold value,
Determine that the free memory capacity of the target storage node and the ratio of OSD quantity are more than the second predetermined threshold value.
5. according to the method for claim 3, it is characterised in that methods described also includes:
When it is determined that the memory size of the target storage node and the ratio of OSD quantity are more than or equal to first predetermined threshold value,
And when being less than second predetermined threshold value, OSD increase operation of the refusal for the target storage node.
6. a kind of fault handling method, it is characterised in that this method includes:
Obtain the information of itself free memory capacity and object storage object equipment OSD quantity;
The information of itself free memory capacity and OSD quantity is reported to monitor.
7. a kind of fault treating apparatus, it is characterised in that the device includes:
Acquiring unit, for obtaining the free memory capacity of memory node and the information of object storage device OSD quantity;
Determining unit, for determining to set with the presence or absence of the memory size of target storage node and object storage in the memory node
The ratio of standby OSD quantity is less than the first predetermined threshold value;
Processing unit, for determined when the determining unit target storage node in the memory node free memory capacity and
When the ratio of OSD quantity is less than the first predetermined threshold value, refusal for the target storage node OSD increase operations, and by institute
The part OSD for stating target storage node is arranged to first kind inoperative Down states so that the target storage node can
It is more than or equal to first predetermined threshold value with the ratio of memory size He remaining OSD quantity;Wherein, remaining described OSD refers to institute
State the available OSD in addition to the part OSD for being arranged to first kind Down states in the OSD of target storage node.
8. device according to claim 7, it is characterised in that
The acquiring unit, the free memory capacity and the information of OSD quantity reported specifically for the reception memory node;
Or, the active volume of the detection memory node and the information of OSD quantity.
9. device according to claim 7, it is characterised in that
The processing unit, be additionally operable to when the determining unit when the free memory capacity for determining the target storage node and
When the ratio of OSD quantity is more than the second predetermined threshold value, it is allowed to increase operation for the OSD of the target storage node, and with extensive
The ratio of free memory capacity and OSD quantity after multiple processing is more than or equal to the principle of second predetermined threshold value, by the mesh
The part or all of OSD being arranged in mark memory node in the part OSD of first kind Down states reverts to work UP shapes
State;Wherein, first predetermined threshold value is less than second predetermined threshold value.
10. device according to claim 9, it is characterised in that
The acquiring unit, it is additionally operable to detect the free memory capacity of the target storage node and the ratio of OSD quantity;
The determining unit, specifically for when the free memory capacity and the ratio of OSD quantity that detect the target storage node
Value is when being more than the second predetermined threshold value, determines the free memory capacity of the target storage node and the ratio of OSD quantity more than the
Two predetermined threshold values.
11. device according to claim 9, it is characterised in that
The processing unit, it is additionally operable to when the memory size and the ratio of OSD quantity that determine the target storage node are more than etc.
In first predetermined threshold value, and when being less than second predetermined threshold value, OSD increase of the refusal for the target storage node
Operation.
12. a kind of fault treating apparatus, it is characterised in that the device includes:
Acquiring unit, for obtaining the information of itself free memory capacity and object storage object equipment OSD quantity;
Transmitting element, for reporting the information of itself free memory capacity and OSD quantity to monitor.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711015125.1A CN107729185B (en) | 2017-10-26 | 2017-10-26 | Fault processing method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711015125.1A CN107729185B (en) | 2017-10-26 | 2017-10-26 | Fault processing method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107729185A true CN107729185A (en) | 2018-02-23 |
CN107729185B CN107729185B (en) | 2020-12-04 |
Family
ID=61213886
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201711015125.1A Active CN107729185B (en) | 2017-10-26 | 2017-10-26 | Fault processing method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107729185B (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109101357A (en) * | 2018-07-20 | 2018-12-28 | 广东浪潮大数据研究有限公司 | A kind of detection method and device of OSD failure |
CN109213637A (en) * | 2018-11-09 | 2019-01-15 | 浪潮电子信息产业股份有限公司 | Data reconstruction method, device and the medium of distributed file system clustered node |
CN109614276A (en) * | 2018-11-28 | 2019-04-12 | 平安科技(深圳)有限公司 | Fault handling method, device, distributed memory system and storage medium |
CN109669822A (en) * | 2018-11-28 | 2019-04-23 | 平安科技(深圳)有限公司 | The creation method and computer readable storage medium of electronic device, spare memory pool |
CN109710456A (en) * | 2018-12-10 | 2019-05-03 | 新华三技术有限公司 | A kind of data reconstruction method and device |
CN115543862A (en) * | 2022-09-27 | 2022-12-30 | 超聚变数字技术有限公司 | Memory management method and related device |
CN118035924A (en) * | 2024-04-11 | 2024-05-14 | 国网山东省电力公司电力科学研究院 | Power equipment fault data processing method and system based on power big data |
CN118035924B (en) * | 2024-04-11 | 2024-06-21 | 国网山东省电力公司电力科学研究院 | Power equipment fault data processing method and system based on power big data |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102693177A (en) * | 2011-03-23 | 2012-09-26 | ***通信集团公司 | Fault diagnosing and processing methods of virtual machine as well as device and system thereof |
US20150149423A1 (en) * | 2013-10-18 | 2015-05-28 | Hitachi Data Systems Engineering UK Limited | Data redundancy in a cluster system |
US20150254003A1 (en) * | 2014-03-10 | 2015-09-10 | Futurewei Technologies, Inc. | Rdma-ssd dual-port unified memory and network controller |
CN105119737A (en) * | 2015-07-16 | 2015-12-02 | 浪潮软件股份有限公司 | Method for monitoring Ceph cluster through Zabbix |
CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
CN106302717A (en) * | 2016-08-12 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | The method for optimizing resources of a kind of CEPH system and device |
CN107045469A (en) * | 2017-03-28 | 2017-08-15 | 北京精强远科技有限公司 | A kind of intelligent sound warning system and method |
-
2017
- 2017-10-26 CN CN201711015125.1A patent/CN107729185B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102693177A (en) * | 2011-03-23 | 2012-09-26 | ***通信集团公司 | Fault diagnosing and processing methods of virtual machine as well as device and system thereof |
US20150149423A1 (en) * | 2013-10-18 | 2015-05-28 | Hitachi Data Systems Engineering UK Limited | Data redundancy in a cluster system |
US20150254003A1 (en) * | 2014-03-10 | 2015-09-10 | Futurewei Technologies, Inc. | Rdma-ssd dual-port unified memory and network controller |
CN105119737A (en) * | 2015-07-16 | 2015-12-02 | 浪潮软件股份有限公司 | Method for monitoring Ceph cluster through Zabbix |
CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
CN106302717A (en) * | 2016-08-12 | 2017-01-04 | 浪潮(北京)电子信息产业有限公司 | The method for optimizing resources of a kind of CEPH system and device |
CN107045469A (en) * | 2017-03-28 | 2017-08-15 | 北京精强远科技有限公司 | A kind of intelligent sound warning system and method |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109101357A (en) * | 2018-07-20 | 2018-12-28 | 广东浪潮大数据研究有限公司 | A kind of detection method and device of OSD failure |
CN109213637B (en) * | 2018-11-09 | 2022-03-04 | 浪潮电子信息产业股份有限公司 | Data recovery method, device and medium for cluster nodes of distributed file system |
CN109213637A (en) * | 2018-11-09 | 2019-01-15 | 浪潮电子信息产业股份有限公司 | Data reconstruction method, device and the medium of distributed file system clustered node |
CN109614276A (en) * | 2018-11-28 | 2019-04-12 | 平安科技(深圳)有限公司 | Fault handling method, device, distributed memory system and storage medium |
CN109669822A (en) * | 2018-11-28 | 2019-04-23 | 平安科技(深圳)有限公司 | The creation method and computer readable storage medium of electronic device, spare memory pool |
CN109669822B (en) * | 2018-11-28 | 2023-06-06 | 平安科技(深圳)有限公司 | Electronic device, method for creating backup storage pool, and computer-readable storage medium |
WO2020107829A1 (en) * | 2018-11-28 | 2020-06-04 | 平安科技(深圳)有限公司 | Fault processing method, apparatus, distributed storage system, and storage medium |
CN109614276B (en) * | 2018-11-28 | 2021-09-21 | 平安科技(深圳)有限公司 | Fault processing method and device, distributed storage system and storage medium |
CN109710456B (en) * | 2018-12-10 | 2021-03-23 | 新华三技术有限公司 | Data recovery method and device |
CN109710456A (en) * | 2018-12-10 | 2019-05-03 | 新华三技术有限公司 | A kind of data reconstruction method and device |
CN115543862A (en) * | 2022-09-27 | 2022-12-30 | 超聚变数字技术有限公司 | Memory management method and related device |
CN115543862B (en) * | 2022-09-27 | 2023-09-01 | 超聚变数字技术有限公司 | Memory management method and related device |
CN118035924A (en) * | 2024-04-11 | 2024-05-14 | 国网山东省电力公司电力科学研究院 | Power equipment fault data processing method and system based on power big data |
CN118035924B (en) * | 2024-04-11 | 2024-06-21 | 国网山东省电力公司电力科学研究院 | Power equipment fault data processing method and system based on power big data |
Also Published As
Publication number | Publication date |
---|---|
CN107729185B (en) | 2020-12-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107729185A (en) | A kind of fault handling method and device | |
CN109407976B (en) | Distributed storage method and distributed storage device | |
CN106170971A (en) | Arbitration process method, arbitration storage device and system after a kind of cluster Schizencephaly | |
EP2606607B1 (en) | Determining equivalent subsets of agents to gather information for a fabric | |
CN107817950A (en) | A kind of data processing method and device | |
CN106911728A (en) | The choosing method and device of host node in distributed system | |
CN108984107A (en) | Improve the availability of storage system | |
CN109274544A (en) | A kind of fault detection method and device of distributed memory system | |
US20060117101A1 (en) | Node discovery and communications in a network | |
JP2015530639A5 (en) | ||
CN104572344B (en) | A kind of method and system of cloudy data backup | |
US10999131B2 (en) | Method and system for detecting abnormalities in network element operation | |
CN106933659A (en) | The method and apparatus of managing process | |
CN109710456B (en) | Data recovery method and device | |
CN101631048A (en) | Method, device and system for monitoring managed object | |
CN106330531A (en) | Node fault recording and processing method and device | |
CN107065616A (en) | Inter-linked controlling method and device, computer-readable recording medium | |
US7840725B2 (en) | Capture of data in a computer network | |
CN102981939B (en) | Disk monitoring method | |
CN106911519A (en) | A kind of data acquisition monitoring method and device | |
KR20120030938A (en) | Method of data replication in a distributed data storage system and corresponding device | |
CN110532096B (en) | System and method for multi-node grouping parallel deployment | |
US8612524B2 (en) | Cessation of sending network status messages to a server | |
CN107547301A (en) | A kind of master/slave device switching method and device | |
JP2010160660A (en) | Network interface, computer system, operation method therefor, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |