CN116821619A - Time sequence anomaly detection method based on multi-element time sequence relation learning - Google Patents
Time sequence anomaly detection method based on multi-element time sequence relation learning Download PDFInfo
- Publication number
- CN116821619A CN116821619A CN202310761069.5A CN202310761069A CN116821619A CN 116821619 A CN116821619 A CN 116821619A CN 202310761069 A CN202310761069 A CN 202310761069A CN 116821619 A CN116821619 A CN 116821619A
- Authority
- CN
- China
- Prior art keywords
- time sequence
- time
- learning
- encoder
- correlation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 20
- 230000006870 function Effects 0.000 claims abstract description 22
- 230000005856 abnormality Effects 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims abstract description 18
- 238000012544 monitoring process Methods 0.000 claims abstract description 9
- 238000005457 optimization Methods 0.000 claims abstract description 6
- 238000009826 distribution Methods 0.000 claims description 21
- 238000000034 method Methods 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 17
- 125000004122 cyclic group Chemical group 0.000 claims description 10
- 238000012952 Resampling Methods 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 description 6
- 238000010801 machine learning Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention discloses a time sequence anomaly detection method based on multi-element time sequence relation learning, which comprises the following steps of S1: based on T train Normal data and variations within a time period are derived from an encoder framework, and a correlation relationship between model learning multivariate time sequences is established, comprising the following sub-steps: step A1: using an encoder to characterize a multi-element time sequence correlation; step A2: sampling in the hidden space based on the learned parameters; step A3: realizing time sequence reconstruction based on the variation self-encoder decoder; step S2: establishing a model objective function and an optimization scheme; step S3: detecting and diagnosing the real-time abnormality of the monitoring system; the invention aims to provide a time sequence anomaly detection method based on multi-element time sequence relation learning, which realizes the effective learning of multi-element time sequence correlation relationThe system abnormality of the complex running system is detected and diagnosed, the credibility and the problem of the model are improved, and meanwhile, good interpretability is provided for abnormality monitoring.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a time sequence anomaly detection method based on multi-element time sequence relation learning.
Background
In industrial production, multivariate time series data is one of the most basic data forms collected by complex systems. With the continuous development of the system scale, the scale of the multi-element time sequence is continuously enlarged, and the difficulty of abnormality detection and diagnosis of the complex system is also continuously increased. In the machine learning field, an efficient anomaly detection model needs to be built, potential anomalies in the system are judged in real time, and warning is given. Therefore, how to efficiently perform real-time abnormality diagnosis is an important research problem.
Around this research problem, most of the existing methods are concerned with the reconstruction and prediction of multivariate time sequences. For the data collected in the normal state of the system, a machine learning model is used for reconstructing the current sequence or effectively predicting the future system state, and model training is realized based on the target. In addition, for the system to be monitored, the model which is trained in the previous step is used, if the reconstruction effect or the prediction error of the model is higher than a certain set threshold value, the current system state is considered to be significantly deviated from the normal system state used during training, and a potential abnormal warning is given to warn a system user of possible system risks. However, in a complex system, the multiple time sequences from different sensors have natural dependency relationships, and the change of the correlation relationship between the sensors is also an effective predictor of system anomalies, so a new anomaly detection method is needed to analyze the system dependency relationships through collected data and detect the system anomalies accordingly.
Disclosure of Invention
The invention aims to provide a time sequence anomaly detection method based on multi-element time sequence relation learning, which realizes detection and diagnosis of system anomalies of a complex running system through effective learning of multi-element time sequence relation, improves the credibility and the questionability of a model, provides better interpretability for anomaly monitoring, and has better guiding significance in practical application.
To achieve the purpose, the invention adopts the following technical scheme: a time sequence anomaly detection method based on multi-element time sequence relation learning comprises the following steps:
step S1: based on T train Normal data and variations within a time period are derived from an encoder framework, and a correlation relationship between model learning multivariate time sequences is established, comprising the following sub-steps:
step A1: using an encoder to characterize a multi-element time sequence correlation;
step A2: sampling in the hidden space based on the learned parameters;
step A3: realizing time sequence reconstruction based on the variation self-encoder decoder;
step S2: establishing a model objective function and an optimization scheme;
step S3: and detecting and diagnosing the abnormality of the monitoring system in real time.
Preferably, in said step A1, in particular comprised in the encoder, a linear layer W is used l Learning input window data S t Is embedded with features H t ＝S t W l And calculating the dependency relationship among multiple time sequences based on a multi-head self-attention mechanism, and substituting a polynomial under the single-head condition:
wherein the method comprises the steps ofIn the multi-head attention mechanism, t is the index of the current time window, i is the first sensor, j is the second sensor, and N is the total number of sensors.
Preferably, in the step A2, the method specifically includes a framework based on a variational self-encoder, randomly sampling hidden space variables according to the polynomial distribution in the step A1, and based on the learned probability vectorSampling is carried out by adopting Gumbel-Softmax resampling technique, and the variable formula is as follows:
where g is a random variable sampled from the Gumbel (0, 1) distribution, gumbel (0, 1) is a Gumbel distribution with parameters 0,1, the temperature variable τ is a resampling superparameter, and Softmax () is a Softmax normalized exponential function.
Preferably, in the step A3, in the variable self-encoder, the DCGRU module is used to implement cyclic reconstruction of the sequence based on the correlation relationship between the sequences sampled in the step A2, and the specific calculation mode is as follows:
wherein W is Q ，b Q (Q=R, C, U) is a model-learnable parameter, ||is a stacking side-by-side operation of the matrix,the hidden state of the t 'time point in the cyclic reconstruction, and Xt' is the input of the current time point; r is R t′ Is a cyclic neural network parameter; a is a graph modeling form of the multi-element time sequence correlation obtained after sampling in the step A2, and the ∈A is graph convolution operation, and a specific calculation mode is defined as follows:
wherein D is O As an output matrix of the adjacency matrix A, D I For the incoupling matrix adjacent to matrix a, k is the diffuseness coefficient of the graph convolution,and Y is a signal to be convolved, and is a learnable parameter.
Preferably, in the step S2, the optimization is performed by using a final objective function of the model, where the formula of the objective function is:
wherein Z is t Is a hidden variable; s is S t Is the actual sampling variable; q φ Posterior distribution probability density functions; p is p ψ Is a priori probability density function;reconstructing an error for a cycle of the sequence; KL (q) φ (Z t |S t )||p ψ (Z t ) And the I is KL divergence between the multivariate time sequence correlation relationship learned by the model and the prior distribution.
Preferably, in said step S3, it is specifically included in the utilization of T train And the normal data in the time period realizes the description of the state of the normal working system, and the normal dependence relationship of each sensor in normal working is obtained by integrating the correlation relationship among multiple time sequences under all data.
The technical scheme of the invention has the beneficial effects that: using a graph structure to describe the relation of different time dimensions in a multi-element time sequence system, wherein different nodes represent different sequence dimensions, edges represent the correlation between sequences, and the correlation between sequences of each dimension is modeled and learned on normal data when the system works normally; based on the reconstruction and variation self-encoder, the unsupervised learning of the graph structure is realized, and a dynamic anomaly detection model of the industrial system under the driving of data is established.
The invention realizes the detection and diagnosis of the system abnormality of the complex operation system through the effective study of the multi-element time sequence correlation, improves the credibility and the problem of the model, provides better interpretability for the abnormality monitoring, has better guiding significance in the practical application, and can be widely applied to the rapid and accurate abnormality detection of the multi-element time sequence data which are rapidly acquired in the complex system in the actual production.
Drawings
FIG. 1 is a schematic diagram of a time series anomaly detection framework based on multivariate timing relationship learning in accordance with one embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further described below by the specific embodiments with reference to the accompanying drawings.
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
Referring to fig. 1, it is assumed that in a complex system with N sensors, data acquired over T time is represented by a matrix X with N rows and T columns, and the time point at which data is acquired and the sensor number are represented by a superscript and a subscript, respectively, in the present invention. In the present invention, it is assumed that the target system is the previous T train Is normally operated in time, and our goal is to judge T train Whether or not there is an abnormality at each time point after the time. Simultaneously, a sliding time window with the width w is used for sliding the data set for establishing the input model, and the data sample acquired after sliding at the moment t is recorded as S t Thereby creating a machine learning model denoted as f, and outputting a determination f regarding abnormality for the acquired sliding sample using the model (S t )。
A time sequence anomaly detection method based on multi-element time sequence relation learning comprises the following steps:
step S1: based on T train Normal data and variations within a time period are derived from an encoder framework, and a correlation relationship between model learning multivariate time sequences is established, comprising the following sub-steps:
step A1: using an encoder to characterize a multi-element time sequence correlation;
step A2: sampling in the hidden space based on the learned parameters;
step A3: realizing time sequence reconstruction based on the variation self-encoder decoder;
step S2: establishing a model objective function and an optimization scheme;
step S3: and detecting and diagnosing the abnormality of the monitoring system in real time.
The learned model is deployed in a system to be monitored, correlation coefficients of the multivariate time sequences learned in the step A1 and the step A2 are calculated in real time, correlation relations among the sensors obtained by real-time monitoring are compared with correlation relations under normal operation obtained in the previous step, and differences between the correlation coefficients and the correlation coefficients are calculated. And when the difference between the two is higher than a certain set threshold value, alarming for system abnormality is carried out, so that real-time abnormality detection is realized. Meanwhile, the correlation relationship with the largest difference between the two is obtained by comparing the difference between the two, and the correlation relationship is reported as an abnormal point possibly existing in a real-time system, so that abnormal diagnosis is realized, and the interpretability of abnormal detection of the model is improved.
The invention realizes the detection and diagnosis of the system abnormality of the complex operation system through the effective study of the multi-element time sequence correlation, improves the credibility and the problem of the model, provides better interpretability for the abnormality monitoring and has better guiding significance in the practical application.
Preferably, in said step A1, in particular comprised in the encoder, a linear layer W is used l Learning input window data S t Is embedded with features H t ＝S t W l And calculating the dependency relationship among multiple time sequences based on a multi-head self-attention mechanism, and substituting a polynomial under the single-head condition:
wherein the method comprises the steps ofIn the multi-head attention mechanism, t is the index of the current time window, i is the first sensor, j is the second sensor, and N is the total number of sensors.
In the multi-head attention mechanism, the final parameters learned by the model are a series of vectorsIn this process, the correlation between the first sensor i and the second sensor j can be assumed without loss of generality, and the self-parameters are sampled from the parameters of theta ij t Is a polynomial distribution of (a). By using the multi-head attention mechanism of the h head, we can characterize the association in h-1 that may exist in the system, and the other attention head considers that the two sensors are irrelevant. Wherein, constructing a vector based on multiple head attentions +.>And when the vector element sum is 1 through Softmax operation, the basic condition of polynomial distribution is satisfied.
Specifically, in the step A2, the method specifically includes a framework based on a variational self-encoder, randomly sampling hidden space variables according to the polynomial distribution in the step A1, and based on the learned probability vectorSampling is carried out by adopting Gumbel-Softmax resampling technique, and the variable formula is as follows:
where g is a random variable sampled from the Gumbel (0, 1) distribution, gumbel (0, 1) is a Gumbel distribution with parameters 0,1, the temperature variable τ is a resampling superparameter, and Softmax () is a Softmax normalized exponential function.
In the hidden space, in the framework of a variational self-encoder (VAE), we need to randomly sample the hidden space variables, and the sampling is implemented by using gummel-Softmax resampling technique according to the polynomial distribution learned in the previous substep A1, since the polynomial distribution belongs to discrete random distribution, in order to avoid the problem that the model is not differentiable and is not gradiently returned due to discontinuity occurring therein.
Preferably, in the step A3, in the variable self-encoder, the DCGRU module is used to implement cyclic reconstruction of the sequence based on the correlation relationship between the sequences sampled in the step A2, and the specific calculation mode is as follows:
wherein W is Q ，b Q (Q=R, C, U) is a model-learnable parameter, ||is a stacking side-by-side operation of the matrix,is the hidden state, X, at the t' th time point in the cyclic reconstruction t′ Input for the current point in time; r is R t′ Is a cyclic neural network parameter; a is a graph modeling form of a multi-element time sequence correlation obtained after sampling in the step A2, A is a graph convolution operation, and a specific calculation mode is defined as follows:
wherein D is O As an output matrix of the adjacency matrix A, D I For the incoupling matrix adjacent to matrix a, k is the diffuseness coefficient of the graph convolution,and Y is a signal to be convolved, and is a learnable parameter.
The graph modeling form of a is characterized by using a graph adjacency matrix, and in reconstruction decoding, the decoding hidden state obtained at the last time point is used for predicting the system state at the next time point.
Specifically, in the step S2, optimization is performed by using a final objective function of the model, where a formula of the objective function is:
wherein Z is t Is a hidden variable; s is(s) t Is the actual sampling variable; q φ Posterior distribution probability density functions; p is p ψ Is a priori probability density function;reconstructing an error for a cycle of the sequence; KL [ q ] φ (Z t |S t )||p ψ (Z t )]KL divergence between the multivariate time series correlation relationship learned for the model and the prior distribution.
Qualitatively, the better the cyclic reconstruction effect of the model, the closer the learned multi-element time sequence correlation relation is to the prior distribution, namely the larger the formula value of the objective function. Thus, the objective function described above will be optimized within the framework of deep learning using a gradient-lifting algorithm to learn an optimal model.
Preferably, in said step S3, it is specifically included in the utilization of T train And the normal data in the time period realizes the description of the state of the normal working system, and the normal dependence relationship of each sensor in normal working is obtained by integrating the correlation relationship among multiple time sequences under all data.
In the description herein, reference to the term "embodiment," "example," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The technical principle of the present invention is described above in connection with the specific embodiments. The description is made for the purpose of illustrating the general principles of the invention and should not be taken in any way as limiting the scope of the invention. Other embodiments of the invention will be apparent to those skilled in the art from consideration of this specification without undue burden.
Claims (6)
1. The time sequence anomaly detection method based on multi-element time sequence relation learning is characterized by comprising the following steps of:
step S1: based on T train Normal data and variations within a time period are derived from an encoder framework, and a correlation relationship between model learning multivariate time sequences is established, comprising the following sub-steps:
step A1: using an encoder to characterize a multi-element time sequence correlation;
step A2: sampling in the hidden space based on the learned parameters;
step A3: realizing time sequence reconstruction based on the variation self-encoder decoder;
step S2: establishing a model objective function and an optimization scheme;
step S3: and detecting and diagnosing the abnormality of the monitoring system in real time.
2. The method for detecting a time series anomaly based on multivariate time series relationship learning of claim 1, wherein in said step A1, specifically included in an encoder, a linear layer W is used l Learning input window data S t Is embedded with features H t ＝S t W l And calculating the dependency relationship among multiple time sequences based on a multi-head self-attention mechanism, and substituting a polynomial under the single-head condition:
wherein the method comprises the steps ofIs a multi-head attention mechanismWherein t is the index of the current time window, i is the first sensor, j is the second sensor, and N is the total number of sensors.
3. The method for detecting abnormal time series based on multi-element time series relation learning according to claim 2, wherein in said step A2, the method specifically comprises a framework based on a variational self-encoder, wherein the method comprises randomly sampling hidden space variables according to the polynomial distribution of step A1, and based on the learned probability vectorSampling is carried out by adopting Gumbel-Softmax resampling technique, and the variable formula is as follows:
where g is a random variable sampled from the Gumbel (0, 1) distribution, gumbel (0, 1) is a Gumbel distribution with parameters 0,1, the temperature variable τ is a resampling superparameter, and Softmax () is a Softmax normalized exponential function.
4. The method for detecting abnormal time series based on multi-element time series relation learning according to claim 3, wherein in the step A3, a DCGRU module is used in a variable self-encoder to realize cyclic reconstruction of the sequence based on the correlation between the sequences obtained by sampling in the step A2, and the specific calculation mode is as follows:
wherein W is Q ，b Q (Q=R, C, U) is a model-learnable parameter, ||is a stacking side-by-side operation of the matrix,is the hidden state, X, at the t' th time point in the cyclic reconstruction t′ Input for the current point in time; r is R t′ Is a cyclic neural network parameter; a is a graph modeling form of the multi-element time sequence correlation obtained after sampling in the step A2, and the ∈A is graph convolution operation, and a specific calculation mode is defined as follows:
wherein D is O As an output matrix of the adjacency matrix A, D I For the incoupling matrix adjacent to matrix a, k is the diffuseness coefficient of the graph convolution,and Y is a signal to be convolved, and is a learnable parameter.
5. The method for detecting abnormal time series based on multivariate time series relation learning according to claim 1, wherein in the step S2, the method specifically comprises optimizing a final objective function by using a model, wherein a formula of the objective function is as follows:
wherein Z is t Is a hidden variable; s is S t Is the actual sampling variable; q φ Posterior distribution probability density functions; p is p ψ Is a priori probability density function;reconstructing an error for a cycle of the sequence; KL [ q ] φ (Z t |S t )||p ψ (Z t )]KL divergence between the multivariate time series correlation relationship learned for the model and the prior distribution.
6. The method for detecting a time series abnormality based on multivariate time series relationship learning of claim 1, wherein in said step S3, specifically comprising the use of T train And the normal data in the time period realizes the description of the state of the normal working system, and the normal dependence relationship of each sensor in normal working is obtained by integrating the correlation relationship among multiple time sequences under all data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310761069.5A CN116821619A (en) | 2023-06-25 | 2023-06-25 | Time sequence anomaly detection method based on multi-element time sequence relation learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310761069.5A CN116821619A (en) | 2023-06-25 | 2023-06-25 | Time sequence anomaly detection method based on multi-element time sequence relation learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116821619A true CN116821619A (en) | 2023-09-29 |
Family
ID=88142352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310761069.5A Pending CN116821619A (en) | 2023-06-25 | 2023-06-25 | Time sequence anomaly detection method based on multi-element time sequence relation learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116821619A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117370919A (en) * | 2023-12-08 | 2024-01-09 | 吉林省拓达环保设备工程有限公司 | Remote monitoring system for sewage treatment equipment |
-
2023
- 2023-06-25 CN CN202310761069.5A patent/CN116821619A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117370919A (en) * | 2023-12-08 | 2024-01-09 | 吉林省拓达环保设备工程有限公司 | Remote monitoring system for sewage treatment equipment |
| CN117370919B (en) * | 2023-12-08 | 2024-03-01 | 吉林省拓达环保设备工程有限公司 | Remote monitoring system for sewage treatment equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111967486A (en) | Complex equipment fault diagnosis method based on multi-sensor fusion | |
| CN112926273A (en) | Method for predicting residual life of multivariate degradation equipment | |
| KR20140043439A (en) | System of sequential kernel regression modeling for forecasting and prognostics | |
| CN116821619A (en) | Time sequence anomaly detection method based on multi-element time sequence relation learning | |
| Gugulothu et al. | Sparse neural networks for anomaly detection in high-dimensional time series | |
| CN115484102A (en) | Industrial control system-oriented anomaly detection system and method | |
| CN115809405A (en) | Fan main shaft gear box temperature anomaly detection method based on multi-feature fusion | |
| CN116595876A (en) | Multi-sensor data fusion motor temperature situation prediction method based on GCN-LSTM | |
| CN114841076A (en) | Power battery production process fluctuation abnormity detection method based on space-time diagram model | |
| CN114282443A (en) | Residual service life prediction method based on MLP-LSTM supervised joint model | |
| CN117175588B (en) | Space-time correlation-based electricity load prediction method and device | |
| CN114358435A (en) | Pollution source-water quality prediction model weight influence calculation method of two-stage space-time attention mechanism | |
| CN114500004A (en) | Anomaly detection method based on conditional diffusion probability generation model | |
| CN116663613A (en) | Multi-element time sequence anomaly detection method for intelligent Internet of things system | |
| CN115840917A (en) | Motor temperature rise prediction method | |
| CN113779879B (en) | Middle-long-term electricity utilization abnormality detection method based on LSTM-seq2seq-attention model | |
| JP2005242803A (en) | Performance estimator, performance estimating method, and performance estimating program of machine | |
| CN116821828A (en) | Multi-dimensional time sequence prediction method based on industrial data | |
| CN116340872A (en) | Method for determining abnormality based on combination of reconstruction and prediction | |
| Song et al. | A novel framework for machine remaining useful life prediction based on time series analysis | |
| CN114548701A (en) | Process early warning method and system for analyzing and estimating coupling structure of full-scale measuring point | |
| CN114841196A (en) | Mechanical equipment intelligent fault detection method and system based on supervised learning | |
| Zhang et al. | Remaining useful life predictions for turbofan engine using semi-supervised DBN-LSTM joint training model | |
| KR20220028727A (en) | Method and Apparatus for Real Time Fault Detection Using Time series data According to Degradation | |
| Ameli et al. | Explainable Unsupervised Multi-Sensor Industrial Anomaly Detection and Categorization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |