CN113312218A - Method and device for detecting magnetic disk - Google Patents
Method and device for detecting magnetic disk Download PDFInfo
- Publication number
- CN113312218A CN113312218A CN202110350771.3A CN202110350771A CN113312218A CN 113312218 A CN113312218 A CN 113312218A CN 202110350771 A CN202110350771 A CN 202110350771A CN 113312218 A CN113312218 A CN 113312218A
- Authority
- CN
- China
- Prior art keywords
- disk
- sequence
- delay
- model
- upper bound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 239000011159 matrix material Substances 0.000 claims description 32
- 238000012360 testing method Methods 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000007689 inspection Methods 0.000 claims description 4
- 230000003111 delayed effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 24
- 238000012545 processing Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013485 heteroscedasticity test Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 235000021419 vinegar Nutrition 0.000 description 1
- 239000000052 vinegar Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2205—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
- G06F11/2221—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested to test input/output devices or peripheral units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a method and a device for detecting a disk. Wherein, the method comprises the following steps: acquiring the disk model of a disk to be detected; acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence; and determining whether the disk to be detected is the target disk or not according to the comparison result of the upper bound sequence and the delay sequence. The invention solves the technical problem that the slow disc cannot be accurately detected due to inaccurate judgment conditions in the detection of the slow disc in the prior art.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a method and a device for detecting a disk.
Background
With the rapid development of the cloud computing industry, the stability of the storage system is not only more and more important, but also becomes the core competitiveness of the success of cloud computing. The distributed architecture carries out a large amount of fault-tolerant processing on various software and hardware errors, and high availability is realized. Unlike a bad disc, a slow disc can still provide service, but the service time is much slower than a normal disc. When one copy of data is stored in a distributed mode, a plurality of copies are written, the whole writing is slowed due to the slow speed of one copy, if the queuing phenomenon occurs on a disk, the subsequent requests are slowed, and the influence of the slow disk is amplified. Although the related art adopts strategies such as backing read/write and the like to eliminate long tails, the threshold value of the backup read is not easy to set, and the delay of the average request is slowed down although the threshold value is not triggered at many times.
Cloud storage is required to provide high availability and higher latency guarantee. With Non-Volatile Memory Express (NVMe) becoming the mainstream storage medium, microsecond latency makes the slow disk more and more influential on the Service-Level agreement (SLA). Therefore, accurate detection and isolation of the slow disk is an essential part of the work for improving the stability of the storage system.
Aiming at the problem that the slow disc cannot be accurately detected due to inaccurate judgment conditions in the detection of the slow disc in the prior art, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a method and a device for detecting a disk, which at least solve the technical problem that the slow disk cannot be accurately detected due to inaccurate judgment conditions in the detection of the slow disk in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a disk detection method, including: acquiring the disk model of a disk to be detected; acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence; and determining whether the disk to be detected is the target disk or not according to the comparison result of the upper bound sequence and the delay sequence.
Optionally, before obtaining the disk model of the disk to be tested, the method further includes: and performing off-line modeling according to the models and the historical data of various disks to obtain a regression equation of the corresponding models.
Optionally, performing offline modeling according to the model and the historical data of each type of disk, and obtaining a regression equation of the corresponding model includes: acquiring historical data corresponding to models in a preset historical period, wherein the historical data comprises: delay data, bandwidth data and read-write times per second; generating a sample matrix according to the delay data, the bandwidth data and the read-write times per second; obtaining a training set and a test set according to the sample matrix; calculating the correlation between the delay data, the bandwidth data and the read-write times per second according to the training set to obtain a correlation coefficient matrix; and calculating a regression equation corresponding to the model according to the correlation coefficient matrix.
Further, optionally, calculating the regression equation corresponding to the model according to the correlation coefficient matrix includes: obtaining a correlation coefficient between delay data, bandwidth data and the read-write times per second according to the correlation coefficient matrix; selecting parameters meeting target linear correlation according to the correlation coefficient between the delay data, the bandwidth data and the read-write times per second; and obtaining a regression equation according to the linear relation between the parameters and the delay data.
Optionally, the method further includes: calculating a regression result of the training set according to a regression equation, and the scatter distribution of independent variables and residual errors; calculating a regression result of the test set according to a regression equation, and the scatter distribution of independent variables and residual errors; and correcting the regression equation according to the regression result of the training set, the scatter point distribution of the independent variable and the residual error, the regression result of the test set and the scatter point distribution of the independent variable and the residual error to obtain the corrected regression equation.
Further, optionally, calculating according to the disk pressure time sequence, and obtaining an upper bound sequence of a confidence interval corresponding to the disk pressure time sequence includes: taking the magnetic disk pressure time sequence corresponding to the model as an input parameter, inputting a regression equation corresponding to the model, and calculating to obtain a confidence interval; an upper bound sequence of confidence intervals is obtained.
Optionally, determining whether the disk to be tested is the target disk according to the comparison result between the upper bound sequence and the delay sequence includes: comparing the delay sequence with the upper bound sequence; and if the delay sequence is continuously higher than the upper bound sequence, judging that the disk to be tested is a target disk, wherein the target disk is a slow disk.
According to another aspect of the embodiments of the present invention, there is also provided a disk detection apparatus, including: the acquisition module is used for acquiring the disk model of the disk to be detected; the sequence acquisition module is used for acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk model within preset detection time; the calculation module is used for calculating according to the disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the disk pressure time sequence; and the detection module is used for determining whether the disk to be detected is the target disk or not according to the comparison result of the upper bound sequence and the delay sequence.
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, where the non-volatile storage medium includes a stored program, and where the apparatus in which the non-volatile storage medium is controlled to execute the above method when the program runs.
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, where the program executes the method described above.
In the embodiment of the invention, the disk model of the disk to be tested is obtained; acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence; according to the comparison result of the upper bound sequence and the delay sequence, whether the disk to be detected is the target disk or not is determined, the purposes of timely finding and isolating the slow disk and improving the performance of the cloud storage system are achieved, the technical effects of accurately identifying the slow disk, and being low in missing report rate and false report rate are achieved, and the technical problem that the slow disk cannot be accurately detected due to the fact that the detection of the slow disk in the prior art is inaccurate in judgment conditions is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a block diagram of a hardware structure of a computer terminal of a disk detection method according to an embodiment of the present invention;
FIG. 2a is a flowchart of a method for detecting a disk according to a first embodiment of the present invention;
FIG. 2b is a schematic diagram of the delay and pressure timing sequence of disk A in the method for detecting a magnetic disk according to the first embodiment of the present invention;
FIGS. 2c and 2d are schematic diagrams of two-dimensional scatter of latency and IOPS and latency and bandwidth in a disk detection method according to a first embodiment of the present invention;
FIG. 2e is a schematic diagram of confidence intervals in a disk detection method according to a first embodiment of the present invention;
FIG. 2f is a schematic diagram of a prediction result of a regression equation on a test set in the disk inspection method according to the first embodiment of the present invention;
FIG. 2g is a diagram illustrating a prediction result of a regression equation on a test set in a disk inspection method according to an embodiment of the present invention;
FIG. 3a is a timing diagram of normalized disk latency and pressure in a disk detection method according to a first embodiment of the present invention;
FIG. 3b is a scatter diagram of disk latency and IOPS in the disk detection method according to the first embodiment of the present invention;
FIG. 3c is a scatter diagram of disk latency and bandwidth in a disk detection method according to a first embodiment of the present invention;
FIG. 3d is a diagram illustrating the OLS regression prediction value and the 99.9% confidence interval in the disk detection method according to the first embodiment of the present invention;
fig. 3e is an OLS residual scatter diagram in the disk detection method according to the first embodiment of the present invention;
FIG. 3f is a schematic diagram of a regression result of the WLS on the training set in the disk detection method according to the first embodiment of the present invention;
FIG. 3g is a schematic diagram of a regression result of WLS on a prediction set in the detection method of a magnetic disk according to the first embodiment of the present invention;
fig. 4 is a schematic diagram of a magnetic disk detection apparatus according to a second embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical terms related to the present application are:
slow disc: the delay is significantly higher than for a normal case disc.
Example 1
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for disk inspection, including the steps illustrated in the flowchart of the figure as being executable by a computer system such as a set of computer-executable instructions, and where a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that illustrated or described herein.
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking the example of running on a computer terminal, fig. 1 is a hardware structure block diagram of a computer terminal of a disk detection method according to an embodiment of the present invention. As shown in fig. 1, the computer terminal 10 may include one or more (only one shown) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the detection method for a magnetic disk in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the detection method for a magnetic disk of the application program. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
Under the above operating environment, the present application provides a method for detecting a magnetic disk as shown in fig. 2 a. Fig. 2a is a flowchart of a disk detection method according to an embodiment of the present invention. The detection method of the magnetic disk related to the embodiment of the application is as follows:
step S202, acquiring the disk model of the disk to be tested;
in step S202, the method for detecting a disk provided in this embodiment of the present application is different from a disk detection scheme that is made according to industry experience in the related art, and the present application obtains a disk type of the disk through a disk signal, and determines the detection scheme according to a correlation between the disk type and a processing pressure, where the pressure includes a bandwidth and an IOPS (Input/Output Operations Per Second, the number of times of performing read/write Operations Per Second); the method comprises the following specific steps:
optionally, before the obtaining of the disk model of the disk to be tested in step S202, the method for detecting a disk according to the embodiment of the present application further includes: and performing off-line modeling according to the models and the historical data of various disks to obtain a regression equation of the corresponding models.
Optionally, performing offline modeling according to the model and the historical data of each type of disk, and obtaining a regression equation of the corresponding model includes: acquiring historical data corresponding to models in a preset historical period, wherein the historical data comprises: delay data, bandwidth data and read-write times per second; generating a sample matrix according to the delay data, the bandwidth data and the read-write times per second; obtaining a training set and a test set according to the sample matrix; calculating the correlation between the delay data, the bandwidth data and the read-write times per second according to the training set to obtain a correlation coefficient matrix; and calculating a regression equation corresponding to the model according to the correlation coefficient matrix.
Further, optionally, calculating the regression equation corresponding to the model according to the correlation coefficient matrix includes: obtaining a correlation coefficient between delay data, bandwidth data and the read-write times per second according to the correlation coefficient matrix; selecting parameters meeting target linear correlation according to the correlation coefficient between the delay data, the bandwidth data and the read-write times per second; and obtaining a regression equation according to the linear relation between the parameters and the delay data.
Optionally, the method for detecting a disk according to the embodiment of the present application further includes: calculating a regression result of the training set according to a regression equation, and the scatter distribution of independent variables and residual errors; calculating a regression result of the test set according to a regression equation, and the scatter distribution of independent variables and residual errors; and correcting the regression equation according to the regression result of the training set, the scatter point distribution of the independent variable and the residual error, the regression result of the test set and the scatter point distribution of the independent variable and the residual error to obtain the corrected regression equation.
Specifically, the following examples are taken as examples in the embodiments of the present application to explain:
selecting model A, B, and respectively establishing regression models for the model A, B;
taking a disk of type A as an example, obtaining delay, bandwidth and IOPS data of a cluster C from 2020-05-1800: 00:00 to 2020-05-2412: 00:00 to obtain a sample matrix, and randomly and equally dividing the sample matrix into a training set and a test set, wherein each sample point is 19080. Fig. 2b is a schematic diagram of a disc delay and pressure timing sequence in a disc detection method according to a first embodiment of the invention. As shown in fig. 2b, the black solid curve is the delay, the dashed curve is the bandwidth, and the discontinuous two-dot curve is IOPS. It can be seen that there is a positive correlation of delay with bandwidth, IOPS.
And calculating a correlation coefficient matrix, and further obtaining a regression equation according to the positive correlation of the delay, the IOPS and the bandwidth.
Wherein, the matrix of the correlation coefficient is shown in table 1,
TABLE 1
Further, a two-dimensional scatter of the delay versus the IOPS, bandwidth is plotted, and it can be seen that there is a positive correlation between the delay and the pressure. As shown in fig. 2c and 2d, fig. 2c and 2d are schematic diagrams of two-dimensional scatter of latency and IOPS and latency and bandwidth in the disk detection method according to the first embodiment of the present invention. Where the abscissa of FIG. 2c is IOPS and the ordinate is the delay; the abscissa of fig. 2d is the bandwidth and the ordinate is the delay.
As shown in table 1, the correlation coefficient between the bandwidth and the IOPS is 0.885, and there is a strong linear correlation. Therefore, bandwidth is chosen as an argument to predict delay. Looking at fig. 2d, the effect of bandwidth on delay is linearly increasing, using simple linear Regression (OLS). And recording the delay as y and the bandwidth as x to obtain a regression equation:
y＝18.9389+1.54×10-7x
fig. 2e is a schematic diagram of Confidence intervals in the method for detecting a magnetic disk according to the first embodiment of the present invention, and fig. 2f is a schematic diagram of a prediction result of a regression equation on a test set in the method for detecting a magnetic disk according to the first embodiment of the present invention, where, as shown in fig. 2e, the regression prediction value and a Confidence Interval (Confidence Interval) of 99.9% thereof are shown. The dotted line is the upper bound of the confidence interval, and the interrupted double-dot dashed line is the lower bound of the confidence interval. The probability outside the confidence interval is less than 0.001. The black solid curve is the predicted value. Fig. 2f is a scatter diagram of independent variables and residual errors in the disk detection method according to the first embodiment of the present invention, and it is observed that there is no heteroscedasticity in random disturbance. Fig. 2g is a schematic diagram of a prediction result of a regression equation on a test set in the disk detection method according to the first embodiment of the present invention. Mean square error 4.942. Where the abscissa of fig. 2e is the bandwidth and the ordinate is the delay.
Taking the disk of type B as an example, the delay, bandwidth and IOPS data of the cluster D from 2020-05-1700: 00:00 to 2020-05-2312: 00:00 are obtained to form a sample matrix, and the sample matrix is randomly divided into a training set and a testing set, and 19019 samples are respectively obtained. Fig. 3a is a timing diagram after normalization of disk latency and pressure in a disk detection method according to an embodiment of the invention. The black solid curve is the delay, the dashed curve is the bandwidth, and the discontinuous double-dot vinegar curve is the IOPS. It can be seen that there is some dependence of disk latency on bandwidth, IOPS.
A matrix of correlation coefficients is calculated as in table 2.
TABLE 2B disk delay pressure correlation coefficient matrix
Further, two-dimensional scatter plots 3b, 3c of delay versus bandwidth, IOPS are drawn. It can be seen that there is a positive correlation between delay and bandwidth, IOPS.
As shown in Table 2, the bandwidth and IOPS are very highLinear correlation, correlation coefficient 0.976. If a regression model is built using bandwidth and IOPS as arguments, the equation will have severe multicollinearity, and therefore, only one index is used as an argument. Looking at fig. 3b and 3c, although both have heteroscedastic, the heteroscedasticity of the bandwidth is smaller, and therefore the bandwidth is chosen as the argument to predict the delay. Looking at fig. 3c, as the delay increases, the bandwidth increases non-linearly, so polynomial regression is chosen instead of OLS. Considering that the delay is of the order of 107The problem of too large power value can occur by directly using polynomial regression, and the calculation precision is influenced. Thus dealing with the delay. Let x be delay/107Obtaining a regression equation by using a stepwise regression method:
y＝111.4927x-55.3443x2+14.0081x3-1.6650x4+0.0797x5
FIG. 3d shows the OLS regression prediction with 99.9% confidence intervals. The dotted line is the upper bound of the confidence interval, and the interrupted double-dot dotted line is the lower bound of the confidence interval. Black is the predicted value. Fig. 3e is an OLS regression residual scatter diagram, it is easy to observe that there is variance in random disturbance, and as the independent variable increases, the variance increases. Therefore, instead, a Weighted Least Squares (WLS) fit is used.
Obtaining a WLS regression equation,
y＝126.2796x-67.5809x2+17.6303x3-2.1198x4+0.1003x5
the fitted curve is plotted with a 99.9% confidence interval, as in FIG. 3 f. The probability of falling outside the upper and lower bounds is less than 0.1%. The prediction is performed on a test set, fig. 3g, mean square error 179.598.
Step S204, acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time;
in the above step S204 of the present application, in a given detection window of 10 minutes (i.e., a preset detection time in the embodiment of the present application), a disk pressure timing sequence (i.e., a disk pressure timing sequence in the embodiment of the present application) and a delay timing sequence are obtained in each detection period, and are respectively referred to as a disk pressure timing sequence a and a delay timing sequence B.
Step S206, calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence;
further, optionally, calculating according to the disk pressure time sequence, and obtaining an upper bound sequence of a confidence interval corresponding to the disk pressure time sequence includes: taking the magnetic disk pressure time sequence corresponding to the model as an input parameter, inputting a regression equation corresponding to the model, and calculating to obtain a confidence interval; an upper bound sequence of confidence intervals is obtained.
Inputting a corresponding offline model to the obtained type of the disk based on the offline model obtained before the step S202 to obtain a regression equation, and inputting the regression equation to obtain a confidence interval by taking the disk pressure time sequence A in the step S204 as an input parameter; and an upper bound sequence C of confidence intervals is obtained.
And step S208, determining whether the disk to be tested is the target disk according to the comparison result of the upper bound sequence and the delay sequence.
Optionally, determining whether the disk to be tested is the target disk according to the comparison result between the upper bound sequence and the delay sequence includes: comparing the delay sequence with the upper bound sequence; and if the delay sequence is continuously higher than the upper bound sequence, judging that the disk to be tested is a target disk, wherein the target disk is a slow disk.
Specifically, if the delay sequences are all higher than the upper bound sequence of the confidence interval, the disk is a slow disk, and if the delay sequences are slow disks, the disk is a target disk obtained through final detection.
The magnetic disk detection method provided by the embodiment of the application considers the influences of pressure and disk type; and giving a real-time delay prediction value and a confidence interval. The parameters are obtained by a regression method, manual setting is not needed, and sufficient theoretical support is provided. The rate of missing report and the rate of false report are low. In addition, the detection method of the magnetic disk accurately identifies the slow disk based on the embodiment of the application, and has low missing report rate and false report rate; the user is provided with a predicted value and a tolerable confidence interval reference, the interpretability is strong, and the user can easily understand why the disk is judged to be a slow disk; timely discovering and isolating the slow disk, improving the performance of the cloud storage system, and ensuring the Service Level Agreement Level (SLA)
In the embodiment of the invention, the disk model of the disk to be tested is obtained; acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence; according to the comparison result of the upper bound sequence and the delay sequence, whether the disk to be detected is the target disk or not is determined, the purposes of timely finding and isolating the slow disk and improving the performance of the cloud storage system are achieved, the technical effects of accurately identifying the slow disk, and being low in missing report rate and false report rate are achieved, and the technical problem that the slow disk cannot be accurately detected due to the fact that the detection of the slow disk in the prior art is inaccurate in judgment conditions is solved.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the disk detection method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
According to an embodiment of the present invention, there is further provided an apparatus for implementing the method for detecting a magnetic disk, where fig. 4 is a schematic diagram of an apparatus for detecting a magnetic disk according to a second embodiment of the present invention, and as shown in fig. 4, the apparatus for detecting a magnetic disk according to the embodiment of the present invention includes: an obtaining module 42, configured to obtain a disk model of a disk to be tested; the sequence acquisition module 44 is configured to acquire a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; the calculation module 46 is configured to perform calculation according to the disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the disk pressure time sequence; and the detection module 48 is configured to determine whether the disk to be detected is the target disk according to the comparison result between the upper bound sequence and the delay sequence.
Example 3
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, where the non-volatile storage medium includes a stored program, and where the apparatus in which the non-volatile storage medium is controlled to execute the above method when the program runs.
Example 4
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, where the program executes the method described above.
Example 5
The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the disk detection method provided in the first embodiment.
Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring the disk model of a disk to be detected; acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk type within a preset detection time; calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence; and determining whether the disk to be detected is the target disk or not according to the comparison result of the upper bound sequence and the delay sequence.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: before obtaining the disk model of the disk to be tested, off-line modeling is carried out according to the model and historical data of various disks, and a regression equation of the corresponding model is obtained.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: performing off-line modeling according to the models and the historical data of various disks to obtain a regression equation of the corresponding model, wherein the regression equation comprises the following steps: acquiring historical data corresponding to models in a preset historical period, wherein the historical data comprises: delay data, bandwidth data and read-write times per second; generating a sample matrix according to the delay data, the bandwidth data and the read-write times per second; obtaining a training set and a test set according to the sample matrix; calculating the correlation between the delay data, the bandwidth data and the read-write times per second according to the training set to obtain a correlation coefficient matrix; and calculating a regression equation corresponding to the model according to the correlation coefficient matrix.
Further, optionally, in the present embodiment, the storage medium is configured to store program code for performing the following steps: calculating a regression equation corresponding to the model according to the correlation coefficient matrix comprises the following steps: obtaining a correlation coefficient between delay data, bandwidth data and the read-write times per second according to the correlation coefficient matrix; selecting parameters meeting target linear correlation according to the correlation coefficient between the delay data, the bandwidth data and the read-write times per second; and obtaining a regression equation according to the linear relation between the parameters and the delay data.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: calculating a regression result of the training set according to a regression equation, and the scatter distribution of independent variables and residual errors; calculating a regression result of the test set according to a regression equation, and the scatter distribution of independent variables and residual errors; and correcting the regression equation according to the regression result of the training set, the scatter point distribution of the independent variable and the residual error, the regression result of the test set and the scatter point distribution of the independent variable and the residual error to obtain the corrected regression equation.
Further, optionally, in the present embodiment, the storage medium is configured to store program code for performing the following steps: calculating according to the disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the disk pressure time sequence, wherein the upper bound sequence comprises the following steps: taking the magnetic disk pressure time sequence corresponding to the model as an input parameter, inputting a regression equation corresponding to the model, and calculating to obtain a confidence interval; an upper bound sequence of confidence intervals is obtained.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining whether the disk to be tested is the target disk according to the comparison result of the upper bound sequence and the delay sequence comprises the following steps: comparing the delay sequence with the upper bound sequence; and if the delay sequence is continuously higher than the upper bound sequence, judging that the disk to be tested is a target disk, wherein the target disk is a slow disk.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for detecting a disk, comprising:
acquiring the disk model of a disk to be detected;
within preset detection time, acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk model;
calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence;
and determining whether the disk to be tested is a target disk or not according to the comparison result of the upper bound sequence and the delay sequence.
2. The method of claim 1, wherein before obtaining the disk model of the disk under test, the method further comprises:
and performing off-line modeling according to the models and the historical data of various disks to obtain a regression equation corresponding to the models.
3. The method of claim 2, wherein performing offline modeling according to the model and historical data of each type of disk, and obtaining a regression equation corresponding to the model comprises:
acquiring historical data corresponding to the model in a preset historical period, wherein the historical data comprises: delay data, bandwidth data and read-write times per second;
generating a sample matrix according to the delay data, the bandwidth data and the read-write times per second;
obtaining a training set and a testing set according to the sample matrix;
calculating the delay data, the correlation between the bandwidth data and the read-write times per second according to the training set to obtain a correlation coefficient matrix;
and calculating a regression equation corresponding to the model according to the correlation coefficient matrix.
4. The method of claim 3, wherein the calculating the regression equation corresponding to the model according to the correlation coefficient matrix comprises:
obtaining the delay data according to the correlation coefficient matrix, and the correlation coefficient between the bandwidth data and the read-write times per second;
selecting parameters meeting target linear correlation according to the delay data, the bandwidth data and the correlation coefficient between the read-write times per second;
and obtaining the regression equation according to the linear relation between the parameters and the delay data.
5. The method of claim 3, wherein the method further comprises:
calculating a regression result of the training set according to the regression equation, and the scatter distribution of independent variables and residual errors;
calculating the regression result of the test set and the scatter point distribution of independent variables and residual errors according to the regression equation;
and correcting the regression equation according to the regression result of the training set, the scatter point distribution of the independent variable and the residual error and the regression result of the test set, and the scatter point distribution of the independent variable and the residual error to obtain the corrected regression equation.
6. The method of claim 5, wherein the calculating according to the disk pressure time sequence to obtain an upper bound sequence of confidence intervals corresponding to the disk pressure time sequence comprises:
inputting a regression equation corresponding to the model by taking the magnetic disk pressure time sequence corresponding to the model as an input parameter, and calculating to obtain the confidence interval;
and acquiring an upper bound sequence of the confidence interval.
7. The method of claim 6, wherein the determining whether the disk to be tested is a target disk according to the comparison result of the upper bound sequence and the delayed sequence comprises:
comparing the delay sequence with the upper bound sequence;
and if the delay sequence is continuously higher than the upper bound sequence, judging that the disk to be tested is a target disk, wherein the target disk is a slow disk.
8. A disk inspection apparatus comprising:
the acquisition module is used for acquiring the disk model of the disk to be detected;
the sequence acquisition module is used for acquiring a corresponding disk pressure time sequence and a corresponding delay time sequence according to the disk model within preset detection time;
the calculation module is used for calculating according to the magnetic disk pressure time sequence to obtain an upper bound sequence of a confidence interval corresponding to the magnetic disk pressure time sequence;
and the detection module is used for determining whether the disk to be detected is a target disk or not according to the comparison result of the upper bound sequence and the delay sequence.
9. A non-volatile storage medium, wherein the non-volatile storage medium comprises a stored program, and wherein the apparatus on which the non-volatile storage medium is located is controlled to perform the method of claim 1 when the program is run.
10. A processor, wherein the processor is configured to execute a program, wherein the program when executed performs the method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110350771.3A CN113312218A (en) | 2021-03-31 | 2021-03-31 | Method and device for detecting magnetic disk |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110350771.3A CN113312218A (en) | 2021-03-31 | 2021-03-31 | Method and device for detecting magnetic disk |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113312218A true CN113312218A (en) | 2021-08-27 |
Family
ID=77371958
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110350771.3A Pending CN113312218A (en) | 2021-03-31 | 2021-03-31 | Method and device for detecting magnetic disk |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312218A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188346A (en) * | 2013-03-05 | 2013-07-03 | 北京航空航天大学 | Distributed decision making supporting massive high-concurrency access I/O (Input/output) server load balancing system |
| CN103488544A (en) * | 2013-09-26 | 2014-01-01 | 华为技术有限公司 | Processing method and device for detecting slow disk |
| US20150269006A1 (en) * | 2014-03-18 | 2015-09-24 | International Business Machines Corporation | Bottleneck detection for performance management |
| WO2017012392A1 (en) * | 2015-07-17 | 2017-01-26 | 中兴通讯股份有限公司 | Disk check method and apparatus |
| US20180150239A1 (en) * | 2015-07-31 | 2018-05-31 | Huawei Technologies Co., Ltd. | Disk detection method and apparatus |
| US20180157438A1 (en) * | 2015-07-31 | 2018-06-07 | Huawei Technologies Co., Ltd. | Slow-disk detection method and apparatus |
| CN109542695A (en) * | 2017-09-21 | 2019-03-29 | 华为技术有限公司 | The method and apparatus for determining logic storage unit performance |
| CN110688063A (en) * | 2019-09-05 | 2020-01-14 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for screening Raid slow disc |
| US20200327020A1 (en) * | 2019-04-10 | 2020-10-15 | EMC IP Holding Company LLC | Predicting and handling of slow disk |
| CN111933207A (en) * | 2020-08-26 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Slow disk identification method and device, electronic equipment and storage equipment |
| CN112530505A (en) * | 2020-12-29 | 2021-03-19 | 苏州元核云技术有限公司 | Hard disk delay detection method and device and computer readable storage medium |
-
2021
- 2021-03-31 CN CN202110350771.3A patent/CN113312218A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188346A (en) * | 2013-03-05 | 2013-07-03 | 北京航空航天大学 | Distributed decision making supporting massive high-concurrency access I/O (Input/output) server load balancing system |
| CN103488544A (en) * | 2013-09-26 | 2014-01-01 | 华为技术有限公司 | Processing method and device for detecting slow disk |
| US20150269006A1 (en) * | 2014-03-18 | 2015-09-24 | International Business Machines Corporation | Bottleneck detection for performance management |
| WO2017012392A1 (en) * | 2015-07-17 | 2017-01-26 | 中兴通讯股份有限公司 | Disk check method and apparatus |
| US20180150239A1 (en) * | 2015-07-31 | 2018-05-31 | Huawei Technologies Co., Ltd. | Disk detection method and apparatus |
| US20180157438A1 (en) * | 2015-07-31 | 2018-06-07 | Huawei Technologies Co., Ltd. | Slow-disk detection method and apparatus |
| CN109542695A (en) * | 2017-09-21 | 2019-03-29 | 华为技术有限公司 | The method and apparatus for determining logic storage unit performance |
| US20200327020A1 (en) * | 2019-04-10 | 2020-10-15 | EMC IP Holding Company LLC | Predicting and handling of slow disk |
| CN110688063A (en) * | 2019-09-05 | 2020-01-14 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for screening Raid slow disc |
| CN111933207A (en) * | 2020-08-26 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Slow disk identification method and device, electronic equipment and storage equipment |
| CN112530505A (en) * | 2020-12-29 | 2021-03-19 | 苏州元核云技术有限公司 | Hard disk delay detection method and device and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107480015B (en) | Load testing method, device and system, storage medium and pressure testing server | |
| CN106354590B (en) | Disk detection method and device | |
| CN107819631B (en) | Equipment anomaly detection method, device and equipment | |
| CN109561052B (en) | Method and device for detecting abnormal flow of website | |
| CN110471821B (en) | Abnormality change detection method, server, and computer-readable storage medium | |
| CN108366012B (en) | Social relationship establishing method and device and electronic equipment | |
| CN113485931A (en) | Test method, test device, electronic equipment and computer readable storage medium | |
| CN110781605A (en) | Advertisement putting model testing method and device, computer equipment and storage medium | |
| CN111176918A (en) | Server testing method, system, equipment and computer readable storage medium | |
| US8930773B2 (en) | Determining root cause | |
| US20160094392A1 (en) | Evaluating Configuration Changes Based on Aggregate Activity Level | |
| CN111562884A (en) | Data storage method and device and electronic equipment | |
| CN111858704A (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN110602207A (en) | Method, device, server and storage medium for predicting push information based on off-network | |
| CN106294457B (en) | Network information pushing method and device | |
| CN113312218A (en) | Method and device for detecting magnetic disk | |
| CN109039695B (en) | Service fault processing method, device and equipment | |
| CN108021464B (en) | Bottom-pocketing processing method and device for application response data | |
| CN112822482B (en) | Method and equipment for determining evaluation score of audio and video call | |
| CN112671590B (en) | Data transmission method and device, electronic equipment and computer storage medium | |
| CN106776264B (en) | Application program code testing method and device | |
| CN113676377A (en) | Online user number evaluation method, device, equipment and medium based on big data | |
| CN111143644B (en) | Identification method and device of Internet of things equipment | |
| CN113079063A (en) | Offline judgment method, system and device for charging device and computer storage medium | |
| CN110019068B (en) | Log text processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40069593 Country of ref document: HK |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20240313 Address after: # 03-06, Lai Zan Da Building 1, 51 Belarusian Road, Singapore Applicant after: Alibaba Innovation Co. Country or region after: Singapore Address before: Room 01, 45th Floor, AXA Building, 8 Shanton Road, Singapore Applicant before: Alibaba Singapore Holdings Ltd. Country or region before: Singapore |
|
| TA01 | Transfer of patent application right |