CN117725543A - Multi-element time sequence anomaly prediction method, electronic equipment and storage medium - Google Patents
Multi-element time sequence anomaly prediction method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117725543A CN117725543A CN202410179705.8A CN202410179705A CN117725543A CN 117725543 A CN117725543 A CN 117725543A CN 202410179705 A CN202410179705 A CN 202410179705A CN 117725543 A CN117725543 A CN 117725543A
- Authority
- CN
- China
- Prior art keywords
- result
- time
- abnormal
- module
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000002159 abnormal effect Effects 0.000 claims abstract description 40
- 230000005856 abnormality Effects 0.000 claims abstract description 13
- 239000011159 matrix material Substances 0.000 claims description 39
- 238000012544 monitoring process Methods 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 27
- 230000007246 mechanism Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 14
- 230000004913 activation Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 101000995924 Homo sapiens Nucleosome assembly protein 1-like 5 Proteins 0.000 claims description 4
- 102100034530 Nucleosome assembly protein 1-like 5 Human genes 0.000 claims description 4
- 230000001364 causal effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000004590 computer program Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the field of computer technology application, in particular to a multivariate time sequence anomaly prediction method, electronic equipment and a storage medium, comprising the following steps: inputting the multi-element time sequence X of the monitored server which needs to be predicted currently into a data embedding module of a multi-element time sequence abnormal prediction model to obtain a corresponding data embedding result; inputting the data embedding result into an encoder of a multi-element time sequence abnormal prediction model to obtain a corresponding encoding result; inputting the coding result into a decoder of a multi-element time sequence abnormal prediction model to obtain a corresponding decoding result; and determining whether the monitored server is abnormal in w time stamps after the prediction time and an index of the abnormality based on at least X and a decoding result. The invention can accurately predict whether the monitored server generates abnormality or not and the specific abnormal indexes in w time stamps after the prediction time.
Description
Technical Field
The present invention relates to the field of computer technology, and in particular, to a method for predicting multiple time series anomalies, an electronic device, and a storage medium.
Background
With the rapid development of computing technology, complex systems (e.g., social networks, cloud computing, water quality detection, etc.) have become more complex and sensitive. To ensure the reliability of these systems, various performance counters and/or sensors are widely used to closely monitor the status of the running objects (e.g., servers, services). These monitoring data are collected at equal time intervals and form a Multiple Time Series (MTS); each monitoring indicator forms a univariate time series. If some error or failure occurs in the system, such as network overload, application error, hardware failure, the monitoring data will be abnormal (e.g., surge, steep rise or fall). System failures can lead to service outages, data loss, and significant economic loss.
The current focus is mainly on identifying abnormal behavior (i.e., abnormal detection) in the MTS, which helps the operator find and recover from a fault after it occurs. However, when an anomaly is detected, a fault has occurred and the reliability of the system has been reduced. In contrast, predicting an anomaly before the anomaly actually occurs may inform the operator to take action in advance. In addition, the operator may be aided in improving the efficiency of solving the underlying problem by locating a set of most abnormal indicators to account for the predicted anomalies. Thus, with adequate predictive performance, MTS anomaly prediction and interpretation can significantly improve the reliability of complex systems.
Disclosure of Invention
Aiming at the technical problems, the invention adopts the following technical scheme:
the embodiment of the invention provides a multi-element time sequence anomaly prediction method, which comprises the following steps:
s100, inputting a multi-element time sequence X= (Xt-w+1, xt-w+2, … …, xi, … … and Xt) of a monitored server to be predicted into a data embedding module of a multi-element time sequence abnormal prediction model so as to perform position embedding and space embedding on data in X, and obtaining a corresponding data embedding result; wherein Xi is a multi-element time sequence corresponding to the ith timestamp before the prediction time t, xi= { Xi1, xi2, … …, xis, … …, xin }, i has a value of t-w+1 to t, t is the prediction time, and w is the prediction window size; xis is the value corresponding to the s-th monitoring index in Xi, the value of s is 1 to n, and n is the number of the monitoring indexes of the monitored server.
S200, inputting the data embedding result into an encoder of a multi-element time sequence abnormal prediction model to obtain a corresponding encoding result; the encoder comprises first to third encoding modules which are connected in sequence and have the same structure, wherein each encoding module at least comprises a multi-head attention mechanism module, a horizontal drawing attention module and a multi-scale feedforward network module which are connected in sequence; the first coding module is also connected with the data embedding module.
S300, inputting the coding result into a decoder of a multi-element time sequence abnormal prediction model to obtain a corresponding decoding result; the decoder at least comprises a first linear layer, a dimension relation learning module and a second linear layer which are sequentially connected, wherein the first linear layer is connected with the third coding module; the decoding result comprises a first result obtained by the first linear layer, an inter-dimension relation dependency matrix obtained by the inter-dimension relation learning module and a second result obtained by the second linear layer.
S400, determining whether the monitored server is abnormal or not and an index of the abnormality in w time stamps after the prediction time t at least based on the X, the inter-dimension relation dependency matrix and the second result.
The multivariate time sequence anomaly prediction model is a model which is obtained based on sample data training of a monitored server.
The invention has at least the following beneficial effects:
according to the method, the time dependence and the inter-dimensional dependence in the multi-element time sequence are jointly learned by adopting the transducer and the graph annotation force network framework, so that the prediction result can be more accurate.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a multivariate time series anomaly prediction method according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
For the multivariate time series MTS, the inventors of the present invention noted that three factors lead to more complex predictions and interpretations of MTS anomalies:
(1) There is a lack of sufficient samples of anomaly markers. In long-term stable operation of complex systems, there are relatively few failure events, and abnormal samples in the monitoring stream are typically covered by large amounts of regulatory data. It is impractical and expensive to manually identify and mark anomalies.
(2) Complex patterns of MTS in modern systems. With the increasing number of monitoring metrics, the multi-component time series has been transformed into a high-dimensional time series with complex patterns. This requires that the predictive model not only identify the abnormal changes of each monitored indicator over the long term, but also mine the relationships between these indicators.
(3) Subtle changes before abnormal behavior. The fault (or anomaly) does not occur suddenly, but gradually affects the monitoring indicator, such as a memory leak fault. In order to accurately identify the warning signal of an impending anomaly in a timely manner, it is necessary to capture subtle changes in the MTS that are not readily detectable.
In view of the above problems, the present invention provides a multi-component time series anomaly prediction scheme, which aims to predict whether anomalies will occur in the monitored server in the upcoming time, and how to find a set of most relevant indexes to explain the upcoming anomalies. The scheme learns both time and dimensional dependencies in MTS through a transducer and a graph-annotation network and adopts a multitasking resistance training strategy to amplify differences between normal and abnormal.
Further, as shown in fig. 1, the method for predicting multi-element time series anomalies provided by the embodiment of the invention may include the following steps:
s100, the multivariate time sequence X= (X) of the monitored server needing prediction currently t-w+1 ,X t-w+2 ,……,X i , ……,X t ) Inputting the data into a data embedding module of the multivariate time sequence abnormal prediction model to perform position embedding and space embedding on the data in X to obtain a corresponding data embedding result; wherein X is i To predict the multiple time series corresponding to the ith timestamp before time t, X i ={x i1 ，x i2 ，……，x is ，……，x in The value of i is t-w+1 to t, t is the prediction time, and w is the prediction window size; x is x is Is X i The value of s is 1 to n, and n is the number of the monitoring indexes of the monitored server.
In the embodiment of the invention, the monitoring index of the monitored server can be a parameter representing the performance of the monitored server, the condition occurring inside the monitored server can be better known through the monitoring index, and all indexes comprehensively represent the state of the monitored server. The monitoring index of the monitored server may be determined based on actual situations, for example, the monitoring index of the monitored server may include broadband usage, CPU usage, network IO, disk usage, writing amount of the hard disk, and the like, and the present invention is not particularly limited. In the embodiment of the invention, the monitoring index of the monitored server can be obtained based on the existing mode, for example, the monitoring index is obtained by sampling the monitoring index by a sampler built in the monitored server.
In the embodiment of the present invention, the intervals before two adjacent time stamps are the same, and the specific interval may be set based on actual needs, for example, may be 5 minutes or 1 hour.
Multidimensional time series have two dependencies: time dependence, representing the correlation of data in each index along with time; spatial dependence characterizes the correlation between different index data at the same time. An unexpected point or subsequence that violates these dependencies is often considered an exception.
The invention predicts MTS abnormality based on sliding window. For each timestamp t, the monitoring data collected in its first w timestamps is constructed as one sample, i.e. x= (X) t-w+1 ,X t-w+2 ,……,X i , ……,X t ）。
Further, in the embodiment of the present invention, in order to be able to learn the time dependency and the inter-index dependency in the monitored data, the present invention embeds two kinds of information: (a) time embedding along time; (b) monitoring data space embedding between metrics. In addition, to reduce the effects of outlier data that may be mixed into the training data, the present invention introduces an additional causal convolution to smooth the raw input data. Specifically, the data embedding result is obtained based on the embedding layer, and specifically the following conditions can be satisfied:
Embedding（X）=Cov（PE（X）+SE（X）+Cacov（X））。
wherein, the data Embedding result with the Embedding (X) being X is PE (X) represents time or sequence information of adding each monitoring index in the time dimension of X, SE (X) represents information between adding indexes in each time stamp, and SE (X) =w se ×CosSim（X），W se For a learnable weight matrix of size n×w, cosSim (X) is a matrix composed of cosine similarities between different monitoring indexes in X, and each element in CosSim (X) isCorresponding to cosine similarity between two monitoring indexes. Cacov (X) denotes performing causal convolution operations on X for smoothing input data, thereby improving the ability of the graph transformer to capture normal modes in MTS. Cov () represents a one-dimensional convolution operation for converting embedded data into a high-dimensional space (i.eWei (dimension)>>n), which enables the graph transformer to learn the local patterns useful for capturing the underlying structure of the MTS data.
S200, inputting the data embedding result into an encoder of a multi-element time sequence abnormal prediction model to obtain a corresponding encoding result; the encoder comprises first to third encoding modules which are sequentially connected and have the same structure, wherein each encoding module at least comprises three sub-layers which are sequentially connected, and the sub-layers are a multi-head attention mechanism module, a horizontal drawing attention module and a multi-scale feedforward network module respectively; the first coding module is also connected with the data embedding module, namely, the input of the first coding module is the output of the data embedding module, the output of the first coding module is used as the input of the second coding module, and the output of the second coding module is used as the input of the third coding module.
In the embodiment of the invention, residual connection and layer standardization are applied to each sub-layer of each coding module. To facilitate residual connection, the number of dimensions of each sub-layer is set toThe same number of dimensions as the embedded layer.
In embodiments of the present invention, the multi-headed attention mechanism module may be an 8-headed multi-headed attention mechanism module to capture the richer time dependence from multiple angles. It is known to those skilled in the art that the specific operation of the multi-head attention mechanism module may be prior art, i.e. dividing the input data into multiple heads, performing self-attention calculation on each head, and finally splicing the results of the multiple heads to obtain the final output. The expression is as follows:
MultiHeadAtt（Q，K，V）=Contact（H 1 ，H 2 ，……，H 8 ）。
wherein MultiHeadAtt (Q, K, V) is the output of the multi-head attention mechanism module, H p For the output of the p-th sub-head attention mechanism module, p takes on a value of 1 to 8,H p =softmax(（Q p K T p /d 1/2 ）V p ) Q, K, V are different linear transformations of the input of the multi-headed attention mechanism module, specifically a query matrix, a key matrix, and a value matrix, which are convolved with different convolution kernels. Q (Q) p 、K p And V p For different linear transformations of the input of the p-th sub-head attention mechanism module, contat () represents a concatenation operation. Softmax () is an activation function, representing the normalization operation.
In the embodiment of the invention, the attention is paid to d 1/2 The term scales the weights to reduce the variance of the weights, facilitating stable training.
In order to solve the problem of excessive simplification weight propagation caused by dot products in the attention mechanism, the invention adds a new propagation mechanism to learn time dependence. Specifically, the graph attention is performed on the input time series according to a horizontal relation matrix, and the output result HGAT (Fh) of the horizontal attention module may satisfy the following condition:
HGAT（Fh）=GAT（V，Softmax（QK T ））；
wherein Q, K and V are respectively a query matrix, a key matrix and a value matrix obtained by convolving the input Fh of the horizontal attention module with different convolution kernels, and K T A transposed matrix of K; softmax () is the activation function, softmax (QK T ) Representation pair QK T And carrying out normalization operation to form a discrete distribution in each row. GAT () is a mechanism for executing graph attention, GAT (V, softmax (QK) T ) Specifically according to Softmax (QK) T ) The graph attention mechanism is performed on V to obtain the output of the sub-layer, which includes a dimension conversion process by multiple heads, i.e., d to d/2, to d.
In the embodiment of the invention, in order to better extract the characteristics from the time sequence, the feedforward neural network is optimized so that the characteristics can be extracted on multiple scales. The invention uses 3 convolutions with different kernel sizes and activation function results, and the output result MFFN (Fm) of the multi-scale feedforward feedback meets the following conditions:
MFFN（Fm）=Contat（S 1 ，S 2 ，S 3 ）×W o wherein the j-th extraction result S j =sigmoid（Conv j （Fm））+tanh（Conv j (Fm)), j has a value of 1 to 3. Wherein Contat () represents a splicing operation, W o For the projection parameters of the full connection layer Linear, sigmoid () and tanh () are activation functions, the activation distribution of different sigmoids is different, and the activation distribution of different tanh is different. Conv j (Fm) means convolving the input Fm of the multi-scale feed-forward feedback with a j-th convolution kernel.
S300, inputting the coding result into a decoder of a multi-element time sequence abnormal prediction model to obtain a corresponding decoding result; the decoder at least comprises a first linear layer, a dimension relation learning module and a second linear layer which are sequentially connected, wherein the first linear layer is connected with the third coding module; the decoding result comprises a first result obtained by the first linear layer, an inter-dimension relation dependency matrix obtained by the inter-dimension relation learning module and a second result obtained by the second linear layer.
In the embodiment of the invention, the relationship between dimensions is represented by the relationship learning module between dimensions by adopting a directed graph, and the output result DRLM (Fv) meets the following conditions:
DRLM（Fv）= Softmax(CosSim（TCN(Q)）)；
the Softmax (CosSim (TCN (Q))) represents a dependency relation matrix among dimensions, and captures the dependency learned among different indexes, wherein the size is n×n. Softmax () is an activation function, TCN () is an execution time convolution network, and Q is a query matrix obtained by convolving an input Fv of the inter-dimensional relationship learning module with different convolution kernels.
It is known to those skilled in the art that the structure of the feedforward neural network may be an existing network structure.
S400, determining whether the monitored server is abnormal or not and an index of the abnormality in w time stamps after the prediction time t at least based on the X, the inter-dimension relation dependency matrix and the second result. Further, S400 may specifically include:
s401, obtaining an abnormality determination value S (X) = |X-X2 corresponding to X 0 ∣∣·（KL（M，M 0 ) A) is provided; wherein X2 0 As a result of the second outcome of the process, |||| means || an absolute value function of the absolute value, i.e., |X-X2 0 || represents X and X2 0 Mean square error between. Represents the dot product, KL () represents the bi-directional Kullback-Leibler divergence function, M is a matrix of cosine similarities between different monitor indices in X, M 0 Is a dependency matrix among dimensions.
In the embodiment of the invention, only X2 is used in the calculation formula of the abnormal judgment value of X 0 Is not necessarily completely close to the input. Meanwhile, errors of the inter-dimension dependency relationship are included in the score, so that early change signals caused by context abnormality and the inter-dimension dependency relationship abnormality can be effectively captured.
S402, if S (X) > S0, determining that the monitored server is abnormal in w time stamps after the prediction time t, and executing S403; otherwise, determining that the monitored server cannot be abnormal in w time stamps after the predicted time t; s0 is a preset abnormal judgment value threshold value.
S403, obtaining the abnormal judgment value IS of the S-th monitoring index s =k1×∣∣Y s -Y 0 s ∣∣+k2×KL（M，M 0 ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein Y is s Is the index vector corresponding to the s-th monitoring index in X, Y s =（x （t-w+1）s ，……，x is ，……，x ts ），Y 0 s Is the sum Y of the second result s A corresponding result; k1 and k2 are a first preset weight and a second preset weight, respectively, and may be empirical values, in one exemplary embodiment k1=0.001, k2=1.
S404, IS 1 To IS n And sequencing according to the sequence from big to small, and taking monitoring indexes corresponding to the first K sequenced abnormal judgment values as indexes for generating the abnormality.
In the embodiment of the present invention, K may be set based on actual needs, for example, k=3.
In the embodiment of the invention, the multivariate time series anomaly prediction model is a model trained based on sample data of a monitored server, and can comprise an embedded layer, an encoder and a decoder. The specific structure of the embedded layer, encoder and decoder can be seen from the foregoing description. The structure of the multivariate time series anomaly prediction model facilitates learning potential dependencies from deep multiscale features. Because of its structural features (lack of recursion), the transducer ignores the position and structural information of the original time series when applied directly to the time series. This may reduce its ability to capture sequence information and dependencies in the MTS data. Thus, a data embedding module is added to alleviate this problem.
The training process of the multivariate time series anomaly prediction model can be specifically obtained based on the following steps:
s1, acquiring a sample set; the sample data in the sample set may be a historical multi-dimensional time series of the monitored object.
S2, inputting the sample data of the current batch into the current multivariate time sequence anomaly prediction model for training to obtain a corresponding prediction result. The length of the sample data for each batch is w.
Those skilled in the art will appreciate that the specific implementation of S2 may refer to the specific implementations of S100 to S300 described above. The prediction result may be a decoding result.
S3, acquiring a current loss function value of the current multi-element time sequence abnormal prediction model based on the prediction result of the current batch and the corresponding real result, judging whether the current loss function value accords with a preset model training ending condition, if so, executing the step S5, otherwise, executing the step S4.
In the embodiment of the invention, in order to be more comprehensiveThe anomaly is effectively predicted, the reconstruction error is not directly used, and the two-stage multitasking countermeasure loss is adopted as an optimization target. In the first stage, X1 is actuated 0 、X2 0 Approximating the original input X and driving the relationship matrix close to the original matrix. In the second stage, the final output is optimized away from the input and the difference of the relationship matrix is enlarged, so that the relationship matrix is free from the limitation of the original matrix and the difference between the normal sample and the abnormal sample is enlarged. That is, the multivariate time series anomaly prediction model uses a two-stage loss function during training, wherein the first stage loss function L1= |X-X1 0 ∣∣+α×（KL（M，M 0 ）+β×∣∣X-X2 0 ||, second stage loss function l2= |x-X1 0 ∣∣-α/2×（KL（M，M 0 ）-β/2×∣∣X-X2 0 And |. Wherein X1 0 As a first result, X2 0 As a result of the second outcome of the process, |||| means || an absolute value function of the absolute value, represents the dot product, KL () represents the bi-directional Kullback-Leibler divergence function,the degree of matching of the dependency between the dimensions learned by the loss metric model with the initial relationship. M is a matrix formed by cosine similarity between different monitoring indexes in X, M 0 And alpha and beta are super parameters for adjusting weights of different losses respectively for the dependency relation matrix among the dimensions. When->，/><When 0, the optimization aim is to enlarge the difference between the normal and the abnormal samples, so that the difference between the normal samples and the abnormal samples is 10% -30%.
In the embodiment of the present invention, the preset model training ending condition may be set based on actual needs, for example, L1 or L2 may not change any more in a set time period, for example, in a continuous 3-wheel training process, and the training may be ended.
S4, updating parameters of a current time sequence abnormality prediction model based on the current loss function value, and taking sample data of a next batch as sample data of the current batch to execute S2.
And S5, taking the current multivariate time sequence anomaly prediction model as a target multivariate time sequence anomaly prediction model.
In one embodiment of the present invention, S0 may be an empirical value.
In another embodiment of the invention, S0 may be derived based on a test set. Specifically, first, the test set is processed according to the steps S100 to S400, so as to obtain corresponding abnormal judgment values at all the prediction moments. Then, the minimum value of 5% of the data sorted from high to low is set as S0.
In summary, the multivariate time series anomaly prediction model used in the invention utilizes the variant of the transducer, the horizontal drawing attention module and the relationship learning module between dimensions to model the time dependency relationship and the relationship between dimensions simultaneously so as to conduct anomaly prediction, adopts a multitask countermeasure training strategy to expand the difference between normal time points and abnormal time points, can improve the performance of the model, and further improves the performance in the aspects of anomaly prediction, anomaly detection and anomaly interpretation.
According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.
In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the above embodiments.
In an exemplary embodiment, the readable storage medium may be a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method according to the above embodiment.
In an exemplary embodiment, the computer program product comprises a computer program which, when executed by a processor, implements the method according to the above embodiments.
Electronic devices are intended to represent various forms of user terminals, various forms of digital computers, such as desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
In one exemplary embodiment, the electronic device may include a computing unit that may perform various suitable actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) or a computer program loaded from a storage unit into a Random Access Memory (RAM). In the RAM, various programs and data required for the operation of the device may also be stored. The computing unit and the RAM are connected to each other by a bus. An input/output (I/O) interface is also connected to the bus.
Further, a plurality of components in the electronic device are connected to the I/O interface, including: an input unit such as a keyboard, a mouse, etc.; an output unit such as various types of displays, speakers, and the like; a storage unit such as a magnetic disk, an optical disk, or the like; and communication units such as network cards, modems, wireless communication transceivers, and the like. The communication unit allows the device to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing units include, but are not limited to, central Processing Units (CPUs), graphics Processing Units (GPUs), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processors, controllers, microcontrollers, and the like. The computing unit performs the various methods and processes described above, such as the multivariate time series anomaly prediction method. For example, in some embodiments, the multivariate time series anomaly prediction method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device via the ROM and/or the communication unit. When the computer program is loaded into RAM and executed by the computing unit, one or more steps of the multivariate time series anomaly prediction method described above may be performed. Alternatively, in other embodiments, the computing unit may be configured to perform the multivariate time series exception prediction method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present invention may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution disclosed in the present invention can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (9)
1. A method for multivariate time series anomaly prediction, the method comprising the steps of:
s100, the multivariate time sequence X= (X) of the monitored server needing prediction currently t-w+1 ,X t-w+2 ,……,X i , ……,X t ) Inputting the data into a data embedding module of the multivariate time sequence abnormal prediction model to perform position embedding and space embedding on the data in X to obtain a corresponding data embedding result; wherein X is i For the multiple corresponding to the ith timestamp before the predicted time tTime series, X i ={x i1 ，x i2 ，……，x is ，……，x in The value of i is t-w+1 to t, t is the prediction time, and w is the prediction window size; x is x is Is X i The value of s is 1 to n, and n is the number of the monitoring indexes of the monitored server;
s200, inputting the data embedding result into an encoder of a multi-element time sequence abnormal prediction model to obtain a corresponding encoding result; the encoder comprises first to third encoding modules which are connected in sequence and have the same structure, wherein each encoding module at least comprises a multi-head attention mechanism module, a horizontal drawing attention module and a multi-scale feedforward network module which are connected in sequence; the first coding module is also connected with the data embedding module;
s300, inputting the coding result into a decoder of a multi-element time sequence abnormal prediction model to obtain a corresponding decoding result; the decoder at least comprises a first linear layer, a dimension relation learning module and a second linear layer which are sequentially connected, wherein the first linear layer is connected with the third coding module; the decoding result comprises a first result obtained by a first linear layer, an inter-dimension relation dependency matrix obtained by the inter-dimension relation learning module and a second result obtained by a second linear layer;
s400, determining whether the monitored server is abnormal or not and an index of the abnormality in w time stamps after the prediction time t at least based on the X, the relation dependency matrix among the dimensions and the second result;
the multivariate time sequence anomaly prediction model is a model which is obtained based on sample data training of a monitored server.
2. The method of claim 1, wherein the data embedding result satisfies the following condition:
Embedding（X）=Cov（PE（X）+SE（X）+Cacov（X））；
embedding X data with the result of Embedding and PE (X) representing adding each in the time dimension of XMonitoring time or sequence information of the indicators, SE (X) representing information between adding indicators at each time stamp, SE (X) =w se ×CosSim（X），W se The weight matrix with the size of n multiplied by w is a matrix formed by cosine similarity among different monitoring indexes in the X, wherein CosSim (X) is a matrix formed by cosine similarity among different monitoring indexes in the X; cacov (X) denotes performing a causal convolution operation on X, and Cov () denotes a one-dimensional convolution operation.
3. The method according to claim 1, wherein the output result HGAT (Fh) of the horizontal attention module satisfies the following condition:
HGAT（Fh）=GAT（V，Softmax（QK T ））；
wherein Q, K and V are respectively a query matrix, a key matrix and a value matrix obtained by convolving the input Fh of the horizontal attention module with different convolution kernels, and K T A transposed matrix of K; softmax () is the activation function and GAT () is the execution graph attention mechanism.
4. The method according to claim 1, characterized in that the output result MFFN (Fm) of the multi-scale feed-forward feedback satisfies the following condition:
MFFN（Fm）=Contat（S 1 ，S 2 ，S 3 ）×W o wherein the j-th extraction result S j =sigmoid（Conv j （Fm））+tanh（Conv j (Fm)) and j has a value of 1 to 3;
wherein Contat () represents a splicing operation, W o For projection parameters, sigmoid () and tanh () are activation functions, conv j (Fm) means convolving the input Fm of the multi-scale feed-forward feedback with a j-th convolution kernel.
5. The method of claim 1, wherein the output of the inter-dimensional relationship learning module is DRLM (Fv) satisfying the following condition:
DRLM（Fv）=Softmax(CosSim（TCN(Q)）)；
softmax (CosSim (TCN (Q))) represents the inter-dimension dependency matrix, softmax () is the activation function, cosSim () is the cosine similarity calculated between different rows, TCN () is the execution time convolution network, and Q is the query matrix obtained by convolving the input Fv of the inter-dimension relationship learning module with different convolution kernels.
6. The method of claim 5, wherein S400 specifically comprises:
s401, obtaining an abnormality determination value S (X) = |X-X2 corresponding to X 0 ∣∣·（KL（M，M 0 ) A) is provided; wherein X2 0 As a result of the second outcome of the process, |||| denotes an absolute value function, represents the dot product, KL () represents the bi-directional Kullback-Leibler divergence function, M is a matrix formed by cosine similarity between different monitoring indexes in X, M 0 Is a dependency relation matrix among dimensions;
s402, if S (X) > S0, determining that the monitored server is abnormal in w time stamps after the prediction time t, and executing S403; otherwise, determining that the monitored server cannot be abnormal in w time stamps after the predicted time t; s0 is a preset abnormal judgment value threshold value;
s403, obtaining the abnormal judgment value IS of the S-th monitoring index s =k1×∣∣Y s -Y 0 s ∣∣+k2×KL（M，M 0 ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein Y is s Is the index vector corresponding to the s-th monitoring index in X, Y s =（x （t-w+1）s ，……，x is ，……，x ts ），Y 0 s Is the sum Y of the second result s A corresponding result; k1 and k2 are a first preset weight and a second preset weight respectively;
s404, IS 1 To IS n And sequencing according to the sequence from big to small, and taking monitoring indexes corresponding to the first K sequenced abnormal judgment values as indexes for generating the abnormality.
7. The method of claim 1, wherein the multivariate time series anomaly prediction model employs a two-stage loss function during training, wherein the first orderSegment loss function l1= |x-X1 0 ∣∣+α×KL（M，M 0 ）+β×∣∣X-X2 0 ||, second stage loss function l2= |x-X1 0 ∣∣-α/2×KL（M，M 0 ）-β/2×∣∣X-X2 0 ∣∣；
Wherein X1 0 As a first result, X2 0 As a result of the second outcome of the process, |||| denotes an absolute value function, represents the dot product, KL () represents the bi-directional Kullback-Leibler divergence function, M is a matrix formed by cosine similarity between different monitoring indexes in X, M 0 The alpha and the beta are super parameters respectively, which are dependency relation matrixes among dimensions.
8. An electronic device comprising a processor and a memory;
the processor is adapted to perform the steps of the method according to any of claims 1 to 7 by invoking a program or instruction stored in the memory.
9. A non-transitory computer-readable storage medium storing a program or instructions that cause a computer to perform the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410179705.8A CN117725543B (en) | 2024-02-18 | 2024-02-18 | Multi-element time sequence anomaly prediction method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410179705.8A CN117725543B (en) | 2024-02-18 | 2024-02-18 | Multi-element time sequence anomaly prediction method, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117725543A true CN117725543A (en) | 2024-03-19 |
| CN117725543B CN117725543B (en) | 2024-05-03 |
Family
ID=90207423
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410179705.8A Active CN117725543B (en) | 2024-02-18 | 2024-02-18 | Multi-element time sequence anomaly prediction method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117725543B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825289A (en) * | 2015-12-17 | 2016-08-03 | 国网江苏省电力公司经济技术研究院 | Prediction method for wind power time sequence |
| US20190228312A1 (en) * | 2018-01-25 | 2019-07-25 | SparkCognition, Inc. | Unsupervised model building for clustering and anomaly detection |
| US20190294980A1 (en) * | 2016-05-26 | 2019-09-26 | Ogma Intelligent Systems Corp. | Processing time-varying data streams using sparse predictive hierarchies |
| CN110502627A (en) * | 2019-08-28 | 2019-11-26 | 上海海事大学 | A kind of answer generation method based on multilayer Transformer polymerization encoder |
| CN114519469A (en) * | 2022-02-22 | 2022-05-20 | 重庆大学 | Construction method of multivariate long sequence time sequence prediction model based on Transformer framework |
| CN114626479A (en) * | 2022-03-23 | 2022-06-14 | 南京大学 | Time sequence anomaly detection method and device based on variational self-coding time sequence decomposition |
| CN114706293A (en) * | 2022-03-25 | 2022-07-05 | 上海大学 | Coal slime water intelligent dosing control method based on GRU-Attention and self-adaptive fuzzy PID |
| CN115758173A (en) * | 2022-11-16 | 2023-03-07 | 西华大学 | Cloud platform system anomaly detection method and device based on parallel graph attention network |
| CN116522265A (en) * | 2023-04-25 | 2023-08-01 | 国网上海市电力公司 | Industrial Internet time sequence data anomaly detection method and device |
| CN116796275A (en) * | 2023-06-27 | 2023-09-22 | 重庆科技学院 | Multi-mode time sequence anomaly detection method for industrial equipment |
| CN117290818A (en) * | 2023-11-22 | 2023-12-26 | 中国民航大学 | Multi-dimensional time sequence prediction method, electronic equipment and storage medium |
| WO2024012735A1 (en) * | 2022-07-13 | 2024-01-18 | Continental Automotive Technologies GmbH | Training of a machine learning model for predictive maintenance tasks |
| CN117422695A (en) * | 2023-11-08 | 2024-01-19 | 上海朗驰佰特智能技术有限公司 | CR-deep-based anomaly detection method |
-
2024
- 2024-02-18 CN CN202410179705.8A patent/CN117725543B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825289A (en) * | 2015-12-17 | 2016-08-03 | 国网江苏省电力公司经济技术研究院 | Prediction method for wind power time sequence |
| US20190294980A1 (en) * | 2016-05-26 | 2019-09-26 | Ogma Intelligent Systems Corp. | Processing time-varying data streams using sparse predictive hierarchies |
| US20190228312A1 (en) * | 2018-01-25 | 2019-07-25 | SparkCognition, Inc. | Unsupervised model building for clustering and anomaly detection |
| CN110502627A (en) * | 2019-08-28 | 2019-11-26 | 上海海事大学 | A kind of answer generation method based on multilayer Transformer polymerization encoder |
| CN114519469A (en) * | 2022-02-22 | 2022-05-20 | 重庆大学 | Construction method of multivariate long sequence time sequence prediction model based on Transformer framework |
| CN114626479A (en) * | 2022-03-23 | 2022-06-14 | 南京大学 | Time sequence anomaly detection method and device based on variational self-coding time sequence decomposition |
| CN114706293A (en) * | 2022-03-25 | 2022-07-05 | 上海大学 | Coal slime water intelligent dosing control method based on GRU-Attention and self-adaptive fuzzy PID |
| WO2024012735A1 (en) * | 2022-07-13 | 2024-01-18 | Continental Automotive Technologies GmbH | Training of a machine learning model for predictive maintenance tasks |
| CN115758173A (en) * | 2022-11-16 | 2023-03-07 | 西华大学 | Cloud platform system anomaly detection method and device based on parallel graph attention network |
| CN116522265A (en) * | 2023-04-25 | 2023-08-01 | 国网上海市电力公司 | Industrial Internet time sequence data anomaly detection method and device |
| CN116796275A (en) * | 2023-06-27 | 2023-09-22 | 重庆科技学院 | Multi-mode time sequence anomaly detection method for industrial equipment |
| CN117422695A (en) * | 2023-11-08 | 2024-01-19 | 上海朗驰佰特智能技术有限公司 | CR-deep-based anomaly detection method |
| CN117290818A (en) * | 2023-11-22 | 2023-12-26 | 中国民航大学 | Multi-dimensional time sequence prediction method, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| ZHENGHAO ZHANG 等: "SR-HGAT: Symmetric Relations Based Heterogeneous Graph Attention Network", 《DIGITAL OBJECT IDENTIFIER》, 22 September 2022 (2022-09-22), pages 165631 - 165655 * |
| 李文;邓升;段妍;杜守国;: "时间序列预测与深度学习:文献综述与应用实例", 计算机应用与软件, no. 10, 12 October 2020 (2020-10-12) * |
| 杨英;唐平;: "VAE＿LSTM算法在时间序列预测模型中的研究", 湖南科技大学学报(自然科学版), no. 03, 20 September 2020 (2020-09-20) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117725543B (en) | 2024-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021204010A1 (en) | Time series anomaly detection method and apparatus, and computer device and storage medium | |
| EP3139313B1 (en) | Anomaly detection system and method | |
| CN111459700A (en) | Method and apparatus for diagnosing device failure, diagnostic device, and storage medium | |
| CN113852603B (en) | Abnormality detection method and device for network traffic, electronic equipment and readable medium | |
| CN108460397B (en) | Method and device for analyzing equipment fault type, storage medium and electronic equipment | |
| CN109472097B (en) | Fault diagnosis method for online monitoring equipment of power transmission line | |
| CN116340796B (en) | Time sequence data analysis method, device, equipment and storage medium | |
| CN103197663A (en) | Method and system of failure prediction | |
| Shajihan et al. | CNN based data anomaly detection using multi-channel imagery for structural health monitoring | |
| CN116821646A (en) | Data processing chain construction method, data reduction method, device, equipment and medium | |
| CN115769235A (en) | Method and system for providing an alert related to the accuracy of a training function | |
| CN115375039A (en) | Industrial equipment fault prediction method and device, electronic equipment and storage medium | |
| CN116562120A (en) | RVE-based turbine engine system health condition assessment method and RVE-based turbine engine system health condition assessment device | |
| CN115081673A (en) | Oil and gas pipeline abnormity prediction method and device, electronic equipment and medium | |
| CN114881167A (en) | Abnormality detection method, abnormality detection device, electronic apparatus, and medium | |
| CN112882898B (en) | Anomaly detection method, system, device and medium based on big data log analysis | |
| CN117094451B (en) | Power consumption prediction method, device and terminal | |
| CN117290818A (en) | Multi-dimensional time sequence prediction method, electronic equipment and storage medium | |
| CN117195114A (en) | Chemical production line identification method and system | |
| CN117725543B (en) | Multi-element time sequence anomaly prediction method, electronic equipment and storage medium | |
| CN117234844A (en) | Cloud server abnormality management method and device, computer equipment and storage medium | |
| CN116258167A (en) | Data detection method, device, electronic equipment and medium | |
| CN114923715A (en) | Abnormal state detection method and system for homogenization and tackifying reaction kettle device | |
| CN111913463B (en) | State monitoring method for chemical volume control system of nuclear power plant | |
| Hu et al. | A multi-feature-based fault diagnosis method based on the weighted timeliness broad learning system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |