CN116204770B - Training method and device for detecting abnormality of bridge health monitoring data - Google Patents
Training method and device for detecting abnormality of bridge health monitoring data Download PDFInfo
- Publication number
- CN116204770B CN116204770B CN202211599237.7A CN202211599237A CN116204770B CN 116204770 B CN116204770 B CN 116204770B CN 202211599237 A CN202211599237 A CN 202211599237A CN 116204770 B CN116204770 B CN 116204770B
- Authority
- CN
- China
- Prior art keywords
- monitoring data
- trained
- sample
- network
- teacher
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 186
- 230000036541 health Effects 0.000 title claims abstract description 119
- 238000012549 training Methods 0.000 title claims abstract description 108
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000005856 abnormality Effects 0.000 title claims abstract description 29
- 230000002159 abnormal effect Effects 0.000 claims abstract description 43
- 238000013140 knowledge distillation Methods 0.000 claims abstract description 22
- 239000012634 fragment Substances 0.000 claims description 39
- 238000001514 detection method Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004821 distillation Methods 0.000 claims description 10
- 238000011478 gradient descent method Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 5
- 230000002547 anomalous effect Effects 0.000 claims 1
- 238000003062 neural network model Methods 0.000 abstract description 5
- 239000010410 layer Substances 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000013526 transfer learning Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention provides a training method and a training device for detecting abnormality of bridge health monitoring data, wherein the method comprises the following steps: acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, and training until convergence is completed to complete the training of the teacher network; acquiring a bridge health monitoring data sample set and a student network to be trained, respectively inputting a plurality of trained teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence is achieved to complete the training of the student network; and carrying out abnormal prediction on the bridge health monitoring data to be detected by using the trained student network. The method solves the problem that the existing neural network model cannot detect the abnormality of the target bridge health monitoring data because the bridge monitoring abnormal data set in the prior art has no label and the normal sample is insufficient.
Description
Technical Field
The invention belongs to the technical field of bridge monitoring, and particularly relates to a training method and device for detecting abnormality of bridge health monitoring data.
Background
At present, methods for detecting anomalies of various types of bridge monitoring data are generally divided into two main types, namely a traditional model-driven time sequence anomaly detection method, including a statistical-based method, a reconstruction-based method and a clustering-class classification-based method; the second is a data-driven time sequence anomaly detection method based on Deep learning, which mainly comprises Deep classification, deep one-class, autoencoder and other models.
The traditional time sequence anomaly detection method mostly depends on expert knowledge and a manually designed feature extractor, and can not utilize a mass sensor to monitor data mining useful information, so that the method is inferior to the time sequence anomaly detection method based on deep learning in the aspects of model expandability, anomaly judgment accuracy, stability and the like.
At present, the deep learning technology is widely applied and verified in various fields, and the application of the deep learning technology to the abnormal detection of bridge health monitoring data faces two main problems: firstly, on the data layer, training of a depth model depends on a large amount of labeled data, but the existing bridge monitoring abnormal data set with completed labeling is lacking, for bridge monitoring, abnormal data examples are very rare and only occupy a small part of normal examples, and training of a depth model for bridge abnormality detection is very difficult due to the situations; secondly, on the model level, the existing depth model is often provided with a plurality of parameters, is complex in structure and low in running speed, and cannot meet the real-time requirement of bridge abnormality detection in practical application.
Disclosure of Invention
The invention provides a training method and device for detecting bridge health monitoring data anomalies, and aims to solve the problem that an existing neural network model cannot detect target bridge health monitoring data anomalies due to the lack of marked bridge monitoring anomaly data sets in the prior art.
The invention provides a training method for detecting abnormality of bridge health monitoring data, which comprises the following steps:
acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
acquiring a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into the trained multiple teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the trained multiple teacher networks until convergence to obtain the trained student network;
the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
According to the training method for detecting the abnormality of the bridge health monitoring data provided by the invention, each teacher network comprises a first encoder and a first decoder;
correspondingly, inputting the time sequence monitoring data sample set into each corresponding teacher network to be trained according to the category to train until convergence, and specifically comprising the following steps:
dividing the time series monitoring data sample set into a first sample segment sequence according to time series;
inputting the first sequence of sample segments into the first encoder;
the first encoder extracts a first concealment state for the first sequence of sample segments;
the first decoder outputs a first reconstructed sample sequence according to the first concealment state;
bringing the first reconstructed sample sequence and the first sample fragment sequence into a first loss function, and completing training when the first loss function converges;
repeating the steps until the plurality of teacher networks to be trained complete training.
According to the training method for detecting the abnormality of the bridge health monitoring data provided by the invention, after the teacher network to be trained finishes training, the training method further comprises the step of carrying out abnormality prediction on the monitoring data in the respective fields by utilizing the teacher network, and specifically comprises the following steps:
Acquiring a monitoring data segment to be detected;
inputting the monitoring data segment to be detected into the corresponding teacher network to obtain a first prediction result;
calculating a first anomaly score value based on the monitored data segment to be detected and the first prediction result;
and judging the first anomaly score value according to a preset first threshold value and outputting an anomaly prediction result corresponding to the field monitoring data.
According to the training method for detecting the abnormality of the bridge health monitoring data provided by the invention, the bridge health monitoring data sample set is respectively input into the trained multiple teacher networks and the student network to be trained, and knowledge distillation training is carried out on the student network to be trained by utilizing the trained multiple teacher networks until convergence, and the training method specifically comprises the following steps:
dividing the bridge health monitoring data sample set into a second sample fragment sequence according to time sequence;
inputting the second sample fragment sequence into the student network to be trained to obtain a second reconstructed sample sequence output by the student network;
inputting the second sample fragment sequence into the trained multiple teacher networks to obtain a third reconstructed sample sequence output by each teacher network;
And calculating distillation loss according to the second sample fragment sequence, the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and guiding the student network training by adopting a counter-propagation gradient descent method.
According to the training method for bridge health monitoring data anomaly detection provided by the invention, distillation loss is calculated according to the second sample fragment sequence, the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and a counter-propagation gradient descent method is adopted to guide student network training, which comprises the following steps:
calculating a first reconstructed sample error value for the student network based on the second sample fragment sequence and the second reconstructed sample sequence;
calculating a reconstructed sample error value of each teacher network based on the second sample fragment sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a second reconstructed sample error value;
calculating a reconstructed sample error value between each teacher network and each student network based on the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a third reconstructed sample error value;
And adding the first reconstructed sample error value, the second reconstructed sample error value and the third reconstructed sample error value to obtain an overall error value, and iteratively optimizing the network parameters of the student network until convergence by taking the overall error value as distillation loss to obtain the student network with training completed.
According to the training method for detecting the abnormality of the bridge health monitoring data, the student network comprises a second encoder and a second decoder;
correspondingly, the second sample fragment sequence is input into the student network to be trained to obtain a second reconstructed sample sequence output by the student network, and the method specifically comprises the following steps:
inputting the second sequence of sample fragments into the second encoder;
the second encoder extracts a second concealment state for the second sequence of sample segments;
the second decoder outputs the second reconstructed sample sequence according to the second concealment state.
According to the training method for detecting the abnormality of the bridge health monitoring data provided by the invention, the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal, and the training method specifically comprises the following steps:
Acquiring bridge health monitoring data segments to be detected;
inputting the bridge health monitoring data segment to be detected into the student network to obtain a second prediction result;
calculating a second abnormal score value based on the bridge health monitoring data segment to be detected and the second prediction result;
and judging the second abnormal score value according to a preset second threshold value and outputting an abnormal prediction result of the bridge health monitoring data.
The invention also provides a training device for detecting the abnormality of the bridge health monitoring data, which comprises:
the teacher network training module is used for acquiring time sequence monitoring data sample sets of a plurality of categories in different fields and a plurality of teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring a plurality of trained teacher networks;
the student network training module is used for acquiring a bridge health monitoring data sample set and a student network to be trained, inputting the bridge health monitoring data sample set into the trained multiple teacher networks and the student network to be trained respectively, and performing knowledge distillation training on the student network to be trained by utilizing the trained multiple teacher networks until convergence to obtain the trained student network;
The trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the training method for detecting the abnormality of the bridge health monitoring data is realized by the processor when the processor executes the program.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a training method for bridge health monitoring data anomaly detection as described in any one of the above.
On one hand, for the problem that the existing neural network model cannot perform abnormal detection on target bridge health monitoring data due to the fact that no label exists on a data layer and a positive sample is insufficient, the invention introduces the idea of transfer learning, trains a plurality of teacher networks on the basis of acquiring a disclosed time sequence monitoring data sample set in other fields, acquires abundant abnormal modes, and transfers learned knowledge to an abnormal detection task of the target bridge health monitoring data; on the other hand, for a time sequence monitoring data set, the model level has the problem of operation real-time, the invention adopts the idea of knowledge distillation, and trains a target student network for bridge health monitoring data anomaly detection through a plurality of teacher networks, wherein the teacher network has a complex structure and more parameters, but can better learn a complex time sequence anomaly change mode, and the student network has a simple structure, fewer parameters and high operation speed, and can fully learn the time sequence anomaly feature mining capability of the teacher network so as to meet the real-time requirement of performance and operation speed in actual deployment. Therefore, the method and the system can fully mine the abnormal mode and the time sequence abnormal change mode in the mass monitoring data, accurately judge the occurrence of abnormal conditions, have instantaneity and can be well applied to daily maintenance management of bridges.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a training method for detecting anomalies in bridge health monitoring data according to the present invention;
fig. 2 is a schematic diagram of training a teacher network in the training method for detecting abnormality of bridge health monitoring data according to the present invention;
FIG. 3 is a schematic diagram of knowledge distillation training of a student network to be trained by using a plurality of trained teacher networks in the training method for detecting abnormality of bridge health monitoring data provided by the invention;
fig. 4a is a schematic diagram of a network structure of a teacher network encoder and decoder in a training method for detecting anomalies in bridge health monitoring data according to the present invention;
fig. 4b is a schematic diagram of a network structure of a student network encoder and decoder in a training method for detecting anomalies in bridge health monitoring data according to the present invention;
Fig. 5 is a schematic structural diagram of a training device for detecting abnormality of bridge health monitoring data according to the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Reference numerals:
21: a teacher network training module; 22: and the student network training module.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Knowledge distillation (Knowledge distillation, KD), which can transfer knowledge of one or more networks to another homogeneous or heterogeneous network, is widely used in transfer learning and model compression. Knowledge distillation requires training one or more teacher networks first and then training student networks using the output of these teacher networks and the true labels of the data. Knowledge distillation can be used to transform the network from a large teacher network to a small student network, implementing compression of the model and preserving performance close to that of the large network; knowledge of multiple teacher networks may also be transferred to one student network so that the performance of a single network approaches the results of an emsemble.
The core idea of the transfer learning is to maximally utilize the knowledge of the marked domain to assist the knowledge acquisition and learning of the target domain. And (5) finding out the similarity between the source field and the target field and reasonably utilizing the similarity. Such similarity is very common, e.g. the body construction of different people is similar; the riding patterns of bicycles and motorcycles are similar; chess and Chinese chess are similar; the playing modes of shuttlecocks and tennis balls are similar. This similarity can also be understood as invariant. With this similarity, the next step is how to measure and exploit this similarity. The goal of the metrology work is two-fold: firstly, the similarity of two fields is well measured, and not only is the similarity of two fields qualitatively taught to us, but also the similarity degree is given more quantitatively. Secondly, the measurement is taken as a criterion, and the similarity between the two fields is increased by a learning means to be adopted, so that the transfer learning is completed.
Referring to fig. 1, a training method for detecting anomalies in bridge health monitoring data provided in this embodiment includes:
step S1: acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
In this step, a sample set of published time series monitoring data of different categories in different fields is obtainedTraining corresponding Teacher networks Teacher on these data sets 1 ,…,Teacher K As shown in fig. 2. After training is completed, each teacher network is saved for subsequent training of student networks.
Step S2: the method comprises the steps of obtaining a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student network;
in this step, a knowledge distillation method is adopted on a target data set, i.e. a bridge health monitoring data sample set, and a Student network Student is trained by using K teacher networks after training, as shown in fig. 3.
According to the training method for detecting the abnormality of the bridge health monitoring data, the plurality of teacher networks are trained by acquiring the time sequence monitoring data sample sets of the plurality of categories in different fields, and the training of the teacher networks is completed; and respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and a plurality of student networks to be trained by acquiring the bridge health monitoring data sample set and the student networks to be trained, and carrying out knowledge distillation training on the student networks to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student networks. And performing anomaly detection on the bridge health monitoring data to be detected by using the trained student network, namely inputting the bridge health monitoring data to be detected into the student network, performing anomaly prediction on the student network, and outputting 0 (no anomaly on the representative monitoring data) or 1 (anomaly on the representative monitoring data). On one hand, for the problem that the existing neural network model cannot detect the abnormality of the target bridge health monitoring data due to the fact that no label exists on the data layer and the positive sample is insufficient, the idea of migration learning is introduced, a plurality of teacher networks are trained on the basis of acquiring a disclosed time sequence monitoring data sample set in other fields, rich abnormality modes are obtained, and learned knowledge is migrated to a target bridge health monitoring data abnormality detection task; on the other hand, for a time sequence monitoring data set, the model level has the problem of operation real-time, the invention adopts the idea of knowledge distillation, and trains a target student network for bridge health monitoring data anomaly detection through a plurality of teacher networks, wherein the teacher network has a complex structure and more parameters, but can better learn a complex time sequence anomaly change mode, and the student network has a simple structure, fewer parameters and high operation speed, can fully learn the time sequence anomaly feature mining capability of the teacher network, and meets the real-time requirements of performance and operation speed in actual deployment. Therefore, the neural network model obtained by training can fully mine abnormal modes and time sequence abnormal change modes in mass monitoring data, accurately judge abnormal conditions, has instantaneity and can be well applied to daily maintenance management of bridges.
In this embodiment, obtaining a set of published time series monitoring data samples of different categories in different fields includes: the number of teacher networks K in this embodiment is set to 3, and the SMD data set, the PSM data set, and the SWaT data set are server operation monitoring data sets. These datasets were divided in time series into sequences of segments of dimension 24 x 3, where dimension 24 represents the length of the input monitoring time series being 24 and dimension 3 represents the monitoring of 3 indices per instant. For encoders of teacher and student networks, the input dimension is 24×3, the output dimension is 24×64, for decoders, the input dimension is 24×64, the output dimension is 24×3, and the hidden layer dimension of the depth network is set to 64.
Teacher of each Teacher's network K Comprising a first Encoder Encoder K And a first Decoder K ；
Correspondingly, inputting the time sequence monitoring data sample set into each corresponding teacher network to be trained according to the category to train until convergence, and specifically comprising the following steps:
sample set of time series monitoring dataDividing into a first sample segment sequence X in time series t ；
Sequence of first sample segments X t Input to a first Encoder Encoder K ；
First Encoder Encoder K Extracting a first sample fragment sequence X t Is the first hidden state Z of (1) t ；
First Decoder K Outputting a first reconstructed sample sequence according to a first concealment state
First reconstructing the sample sequenceWith the first sample fragment sequence->Bringing a first loss function, and completing training when the first loss function converges;
repeating the steps until a plurality of Teacher networks Teacher to be trained 1 ，...，Teacher K And (5) training is completed.
In this embodiment, after training is completed by the teacher network to be trained, the method further includes performing anomaly prediction on the monitoring data in the respective fields by using the teacher network, and specifically includes:
acquiring a monitoring data segment X to be detected k ；
The monitoring data segment X to be detected k Inputting corresponding Teacher network Teacher K To obtain a first prediction result Teacher K (X k )；
Based on the monitored data segment X to be detected k And a first predictor Teacher K (X k ) Calculating a first anomaly score value
According to a preset first threshold sigma, the first anomaly score valueAnd judging and outputting an abnormal prediction result of the corresponding field monitoring data.
Specifically, because the bridge health monitoring data set does not have any labeling information, an unsupervised training mode is needed. The present embodiment designs a depth anomaly detection network based on a self encoder (Autoencoder). A self-encoder network for reconstructing normal instances is first trained, and then the trained network is used in combination with a set first threshold value to detect whether an abnormality exists in a certain monitored data segment. As shown in fig. 2, a set of data samples is monitored for a time series On top of that, divided into a first sample segment sequence X in time sequence t Teacher's network for Teacher K Encoder Encoder K Receiving a first sample fragment sequence X t As input, the intermediate first hidden state Z is derived by equation (1) t Then Decoder K To put the first hidden state Z t As input, the first reconstructed sample sequence is output by equation (2)>
Z t ＝Encoder K (X t )； (1)
Equation (3) characterizes the first loss functionLoss value of (i.e. first sample fragment sequence X) t And the first reconstructed sample sequence->Is the norm L2 distance of (2):
based on a first loss functionAnd (3) carrying out iterative optimization updating on the network parameters of the teacher network by adopting an ADAM algorithm so as to complete training.
Teacher network Teacher after training K The method is used for predicting the abnormality of the monitoring data in the corresponding field, and comprises the following steps:
the monitoring data segment X to be detected k Teacher network Teacher for inputting corresponding field K To obtain a Teacher network Teacher K The output first prediction result Teacher K (X k ) Calculating a first anomaly score value of the monitored data segment X to be detected through a formula (4)
Judging the first abnormal score value through a formula (5) to obtain an abnormal prediction result y1 of the corresponding field monitoring data, and adopting a binary classification method to judge the first abnormal score value Making categorical predictions, i.e. judging ifThen it indicates that the corresponding domain monitoring data is abnormal (y1=0), otherwise it indicates that the corresponding domain monitoring data is abnormal (y1=1), where σ is a first threshold value defined by user.
In this embodiment, the input bridge health data sample set is also divided into segment sequences with dimensions of 24×3 according to time sequence, wherein dimension 24 represents the length of the input monitoring time sequence being 24, and dimension 3 represents 3 indexes monitored at each moment, namely, ambient temperature, girder displacement and component fatigue. Referring to fig. 3, a bridge health monitoring data sample set is respectively input into a plurality of trained teacher networks and a student network to be trained, and knowledge distillation training is performed on the student network to be trained by using the plurality of trained teacher networks until convergence, which specifically includes:
sample set of bridge health monitoring dataTime-series division into a second sequence of sample fragments X b ；
Sequence of second sample fragments X S Inputting the Student network Student to be trained to obtain a second reconstructed sample sequence output by the Student network Student
Sequence of second sample fragments X S Inputting trained multiple Teacher networks Teacher 1 ，...，Teacher K Obtaining each ofTeacher of personal Teacher network K Output third reconstructed sample sequence
According to the second sample fragment sequence X S A second reconstructed sample sequenceAnd a third reconstructed sample sequence per teacher network output +.>The distillation loss is calculated, and a counter-propagation gradient descent method is adopted to guide the student to train on the network.
In this embodiment, according to the second sample fragment sequence X S A second reconstructed sample sequenceAnd a third reconstructed sample sequence per teacher network output +.>Calculating distillation loss, and guiding student network training by adopting a counter-propagation gradient descent method, wherein the method specifically comprises the following steps of:
based on the second sample fragment sequence X S And a second reconstructed sample sequenceCalculating a first reconstructed sample error value +.>See formula (6):
based on the second sample fragment sequence X S And a third reconstructed sample sequence output by each teacher networkCalculating the reconstructed sample error value of each teacher network, and accumulating to form a second reconstructed sample error value +.>See formula (7):
based on the second reconstructed sample sequenceAnd a third reconstructed sample sequence per teacher network output +.>Calculating the error value of the reconstructed sample between each teacher network and each student network, and accumulating to form a third reconstructed sample error value +.>See formula (8):
error value of first reconstructed sample Second reconstructed sample error value +.>And a third reconstructed sample error value +.>Adding to obtain an overall error value->See formula (9):
the total error value is reduced by adopting a counter-propagation gradient descent method through a formula (10)And carrying out iterative optimization on the network parameters of the student network as distillation loss until convergence to obtain the training-completed student network.
Wherein θ is a network parameter of the student network.
Specifically, the backward propagation gradient descent method adopts an ADAM algorithm to update and optimize the network parameters of the Student network Student.
In this embodiment, the Student network Student includes a second Encoder and a second Decoder;
correspondingly, the second sample fragment sequence X S Inputting the Student network Student to be trained to obtain a second reconstructed sample sequence output by the Student network StudentThe method specifically comprises the following steps:
sequence of second sample fragments X s Inputting a second Encoder;
the second Encoder extracts a second sequence of sample fragments X S Is the second hidden state Z of (2) b ；
The second Decoder decodes the first signal according to the second hidden state Z b Outputting a second reconstructed sample sequence
In this embodiment, the trained student network is configured to predict bridge health monitoring data to be detected, so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal, and specifically includes:
Obtaining bridge health monitoring data segment X to be detected q ；
Bridge health monitoring data segment X to be detected q Inputting Student network Student to obtain second prediction result Student (X q )；
Bridge health monitoring data segment X based on to-be-detected q And a second predictor Student (X q ) Calculating a second outlier scoreSee formula (11);
a second abnormality score value according to a preset second threshold value betaAnd judging and outputting an abnormal prediction result y2 of the bridge health monitoring data, wherein the abnormal prediction result y2 is shown in a formula (12).
For the second outlier value by equation (12)Judging and obtaining an abnormal prediction result y2 of the bridge health monitoring data segment, and inheriting a binary classification method of the teacher network to carry out second abnormal score value +.>Making a classification prediction, i.e. determining if +.>And if the bridge health monitoring data segment is abnormal (y2=0), otherwise, the bridge health monitoring data segment is abnormal (y2=1), wherein beta is a self-defined second threshold value.
In this embodiment, as shown in FIG. 4a, the Teacher network Teacher K The encoder and decoder of (a) adopt a transform structure, wherein the position coding adopts the existing relative position coding method, the number of heads of a multi-head attention mechanism is set to be 5, and the number of heads of an add is set to be 5 &Norms represent residual and normalized operations, and feed forward networks employ fully connected neural networks with two hidden layers. Compared with a teacher network, the Student network Student structure is simpler, so that the real-time requirement of abnormality detection on bridge monitoring data is guaranteed by using the trained Student network, and the Student network only comprises a feed-forward full-connection neural network with two layers, as shown in fig. 4 b. The input coding layers of the teacher network and the student network are all single-layer fully-connected neural networks.
Example two
Referring to fig. 5, this embodiment provides a training device for detecting abnormality of bridge health monitoring data, including:
the teacher network training module 21 is configured to obtain a plurality of time series monitoring data sample sets of categories in different fields and a plurality of teacher networks to be trained, input the time series monitoring data sample sets into each corresponding teacher network to be trained according to the categories, train until convergence, and obtain a plurality of trained teacher networks;
further, the teacher network training module 21 includes: the first acquisition unit is used for acquiring time sequence monitoring data sample sets of a plurality of categories in different fields; the second acquisition unit is used for acquiring a plurality of teacher networks to be trained; the first training unit is used for inputting the time sequence monitoring data sample set into each corresponding teacher network to be trained according to the category to train until convergence, and obtaining a plurality of trained teacher networks.
The student network training module 22 is configured to obtain a bridge health monitoring data sample set and a student network to be trained, respectively input the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, and perform knowledge distillation training on the student network to be trained by using the plurality of trained teacher networks until convergence to obtain the trained student network;
further, the student network training module 22 includes: the third acquisition unit is used for acquiring a bridge health monitoring data sample set; a fourth acquisition unit for acquiring a student network to be trained; and the second training unit is used for respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and a student network to be trained, and carrying out knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student network.
The trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
Further, the device also comprises a monitoring data detection module, which is used for acquiring bridge health monitoring data to be detected, inputting the bridge health monitoring data to be detected into a trained student network, and obtaining a prediction result of whether the bridge health monitoring data to be detected is abnormal.
The implementation process of the functions and actions of each module in the above device is specifically detailed in the implementation process of the corresponding steps in the above method, so relevant parts only need to be referred to in the description of the method embodiments, and are not repeated here. The system embodiments described above are merely illustrative, and some or all of the modules may be selected according to actual needs to achieve the objectives of the inventive arrangements.
Example III
As shown in fig. 6, the present embodiment provides an electronic apparatus including: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. Processor 310 may invoke logic instructions in memory 330, processor 310 executing a training method for bridge health monitoring data anomaly detection, the method comprising:
acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
The method comprises the steps of obtaining a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student network;
the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions to cause a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, where the computer program, when executed by a processor, can perform the training method for detecting anomalies in bridge health monitoring data according to the above method embodiment, and the method includes:
acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
the method comprises the steps of obtaining a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student network;
the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
Example IV
The present embodiment provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a training method for bridge health monitoring data anomaly detection as described in the above method embodiment, the method comprising:
acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
the method comprises the steps of obtaining a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, and performing knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence to obtain the trained student network;
the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. The training method for detecting the abnormality of the bridge health monitoring data is characterized by comprising the following steps of:
acquiring time sequence monitoring data sample sets of multiple categories in different fields and multiple teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring multiple trained teacher networks;
the method comprises the steps of obtaining a bridge health monitoring data sample set and a student network to be trained, respectively inputting the bridge health monitoring data sample set into a plurality of trained teacher networks and the student network to be trained, carrying out knowledge distillation training on the student network to be trained by utilizing the plurality of trained teacher networks until convergence, and obtaining the trained student network, wherein the method specifically comprises the following steps of:
dividing the bridge health monitoring data sample set into a second sample fragment sequence according to time sequence;
inputting the second sample fragment sequence into the student network to be trained to obtain a second reconstructed sample sequence output by the student network;
inputting the second sample fragment sequence into the trained multiple teacher networks to obtain a third reconstructed sample sequence output by each teacher network;
Calculating distillation loss according to the second sample fragment sequence, the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and guiding student network training by adopting a counter-propagation gradient descent method, wherein the method specifically comprises the following steps:
calculating a first reconstructed sample error value for the student network based on the second sample fragment sequence and the second reconstructed sample sequence;
calculating a reconstructed sample error value of each teacher network based on the second sample fragment sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a second reconstructed sample error value;
calculating a reconstructed sample error value between each teacher network and each student network based on the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a third reconstructed sample error value;
adding the first reconstruction sample error value, the second reconstruction sample error value and the third reconstruction sample error value to obtain an overall error value, and iteratively optimizing the network parameters of the student network until convergence by taking the overall error value as distillation loss to obtain the student network with training completed;
The trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
2. The training method for bridge health monitoring data anomaly detection of claim 1, wherein each of the teacher networks includes a first encoder and a first decoder;
correspondingly, inputting the time sequence monitoring data sample set into each corresponding teacher network to be trained according to the category to train until convergence, and specifically comprising the following steps:
dividing the time series monitoring data sample set into a first sample segment sequence according to time series;
inputting the first sequence of sample segments into the first encoder;
the first encoder extracts a first concealment state for the first sequence of sample segments;
the first decoder outputs a first reconstructed sample sequence according to the first concealment state;
bringing the first reconstructed sample sequence and the first sample fragment sequence into a first loss function, and completing training when the first loss function converges;
repeating the steps until the plurality of teacher networks to be trained complete training.
3. The training method for detecting anomalies of bridge health monitoring data according to claim 2, further comprising, after the training of the teacher network to be trained is completed, utilizing the teacher network to perform anomaly prediction on monitoring data of respective fields, specifically comprising:
acquiring a monitoring data segment to be detected;
inputting the monitoring data segment to be detected into the corresponding teacher network to obtain a first prediction result;
calculating a first anomaly score value based on the monitored data segment to be detected and the first prediction result;
and judging the first anomaly score value according to a preset first threshold value and outputting an anomaly prediction result corresponding to the field monitoring data.
4. The training method for bridge health monitoring data anomaly detection of claim 1, wherein the student network comprises a second encoder and a second decoder;
correspondingly, the second sample fragment sequence is input into the student network to be trained to obtain a second reconstructed sample sequence output by the student network, and the method specifically comprises the following steps:
inputting the second sequence of sample fragments into the second encoder;
The second encoder extracts a second concealment state for the second sequence of sample segments;
the second decoder outputs the second reconstructed sample sequence according to the second concealment state.
5. The training method for detecting anomalies of bridge health monitoring data according to claim 1, wherein the trained student network is configured to predict bridge health monitoring data to be detected to obtain a prediction result of whether the bridge health monitoring data to be detected is anomalous, and specifically comprises:
acquiring bridge health monitoring data segments to be detected;
inputting the bridge health monitoring data segment to be detected into the student network to obtain a second prediction result;
calculating a second abnormal score value based on the bridge health monitoring data segment to be detected and the second prediction result;
and judging the second abnormal score value according to a preset second threshold value and outputting an abnormal prediction result of the bridge health monitoring data.
6. A trainer for bridge health monitoring data anomaly detection, characterized by comprising:
the teacher network training module is used for acquiring time sequence monitoring data sample sets of a plurality of categories in different fields and a plurality of teacher networks to be trained, inputting the time sequence monitoring data sample sets into each corresponding teacher network to be trained according to the categories, training until convergence, and acquiring a plurality of trained teacher networks;
The student network training module is used for acquiring a bridge health monitoring data sample set and a student network to be trained, inputting the bridge health monitoring data sample set into the trained multiple teacher networks and the student network to be trained respectively, and performing knowledge distillation training on the student network to be trained by utilizing the trained multiple teacher networks until convergence to obtain the trained student network, wherein the training module is specifically used for:
dividing the bridge health monitoring data sample set into a second sample fragment sequence according to time sequence;
inputting the second sample fragment sequence into the student network to be trained to obtain a second reconstructed sample sequence output by the student network;
inputting the second sample fragment sequence into the trained multiple teacher networks to obtain a third reconstructed sample sequence output by each teacher network;
calculating distillation loss according to the second sample fragment sequence, the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and guiding student network training by adopting a counter-propagation gradient descent method, wherein the method is particularly used for:
calculating a first reconstructed sample error value for the student network based on the second sample fragment sequence and the second reconstructed sample sequence;
Calculating a reconstructed sample error value of each teacher network based on the second sample fragment sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a second reconstructed sample error value;
calculating a reconstructed sample error value between each teacher network and each student network based on the second reconstructed sample sequence and the third reconstructed sample sequence output by each teacher network, and accumulating to form a third reconstructed sample error value;
adding the first reconstruction sample error value, the second reconstruction sample error value and the third reconstruction sample error value to obtain an overall error value, and iteratively optimizing the network parameters of the student network until convergence by taking the overall error value as distillation loss to obtain the student network with training completed;
the trained student network is used for predicting the bridge health monitoring data to be detected so as to obtain a prediction result of whether the bridge health monitoring data to be detected is abnormal or not.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the training method for bridge health monitoring data anomaly detection of any one of claims 1-5 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the training method for bridge health monitoring data anomaly detection of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211599237.7A CN116204770B (en) | 2022-12-12 | 2022-12-12 | Training method and device for detecting abnormality of bridge health monitoring data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211599237.7A CN116204770B (en) | 2022-12-12 | 2022-12-12 | Training method and device for detecting abnormality of bridge health monitoring data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116204770A CN116204770A (en) | 2023-06-02 |
| CN116204770B true CN116204770B (en) | 2023-10-13 |
Family
ID=86518153
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211599237.7A Active CN116204770B (en) | 2022-12-12 | 2022-12-12 | Training method and device for detecting abnormality of bridge health monitoring data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116204770B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117076928A (en) * | 2023-08-25 | 2023-11-17 | 中交路桥科技有限公司 | Bridge health state monitoring method, device and system and electronic equipment |
| CN117216701B (en) * | 2023-09-13 | 2024-03-15 | 华夏安信物联网技术有限公司 | Intelligent bridge monitoring and early warning method and system |
| CN117708643B (en) * | 2023-11-07 | 2024-07-09 | 中交公路长大桥建设国家工程研究中心有限公司 | Bridge monitoring abnormal data identification method and system based on fusion sequence characteristics |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019240964A1 (en) * | 2018-06-12 | 2019-12-19 | Siemens Aktiengesellschaft | Teacher and student based deep neural network training |
| CN113610126A (en) * | 2021-07-23 | 2021-11-05 | 武汉工程大学 | Label-free knowledge distillation method based on multi-target detection model and storage medium |
| CN113610173A (en) * | 2021-08-13 | 2021-11-05 | 天津大学 | Knowledge distillation-based multi-span domain few-sample classification method |
| CN114239718A (en) * | 2021-12-15 | 2022-03-25 | 杭州电子科技大学 | High-precision long-term time sequence prediction method based on multivariate time sequence data analysis |
| CN114462617A (en) * | 2021-12-21 | 2022-05-10 | 北京理工大学 | Data anomaly detection method based on distillation mechanism |
| CN114881861A (en) * | 2022-05-25 | 2022-08-09 | 厦门大学 | Unbalanced image over-resolution method based on double-sampling texture perception distillation learning |
| CN114880659A (en) * | 2022-04-29 | 2022-08-09 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, equipment and medium for time series data abnormity detection |
| CN114996124A (en) * | 2022-05-09 | 2022-09-02 | 北京云集智造科技有限公司 | Multi-dimensional time series anomaly detection method based on characteristic distillation |
| CN115019183A (en) * | 2022-07-28 | 2022-09-06 | 北京卫星信息工程研究所 | Remote sensing image model migration method based on knowledge distillation and image reconstruction |
| WO2022212852A1 (en) * | 2021-04-02 | 2022-10-06 | Liveperson, Inc. | Domain adaptation of ai nlp encoders with knowledge distillation |
| US11487944B1 (en) * | 2019-12-09 | 2022-11-01 | Asapp, Inc. | System, method, and computer program for obtaining a unified named entity recognition model with the collective predictive capabilities of teacher models with different tag sets using marginal distillation |
| CN115375986A (en) * | 2022-07-08 | 2022-11-22 | 际络科技(上海)有限公司 | Model distillation method and device |
| CN115461778A (en) * | 2020-05-05 | 2022-12-09 | 因美纳有限公司 | Equalization-based image processing and spatial crosstalk attenuator |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9922307B2 (en) * | 2014-03-31 | 2018-03-20 | Elwha Llc | Quantified-self machines, circuits and interfaces reflexively related to food |
| SI3349776T1 (en) * | 2015-09-17 | 2021-04-30 | Histide Ag | Pharmaceutical association of growth factor receptor agonist and adhesion protein inhibitor for converting a neoplastic cell into a non-neoplastic cell and uses thereof |
| KR20200037816A (en) * | 2017-08-02 | 2020-04-09 | 스트롱 포스 아이오티 포트폴리오 2016, 엘엘씨 | Methods and systems for detection in an industrial Internet of Things data collection environment with large data sets |
| WO2020123710A1 (en) * | 2018-12-13 | 2020-06-18 | Ascava, Inc. | Efficient retrieval of data that has been losslessly reduced using a prime data sieve |
| US11347965B2 (en) * | 2019-03-21 | 2022-05-31 | Illumina, Inc. | Training data generation for artificial intelligence-based sequencing |
| US20220187847A1 (en) * | 2019-11-05 | 2022-06-16 | Strong Force Vcn Portfolio 2019, Llc | Robot Fleet Management for Value Chain Networks |
-
2022
- 2022-12-12 CN CN202211599237.7A patent/CN116204770B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019240964A1 (en) * | 2018-06-12 | 2019-12-19 | Siemens Aktiengesellschaft | Teacher and student based deep neural network training |
| US11487944B1 (en) * | 2019-12-09 | 2022-11-01 | Asapp, Inc. | System, method, and computer program for obtaining a unified named entity recognition model with the collective predictive capabilities of teacher models with different tag sets using marginal distillation |
| CN115461778A (en) * | 2020-05-05 | 2022-12-09 | 因美纳有限公司 | Equalization-based image processing and spatial crosstalk attenuator |
| WO2022212852A1 (en) * | 2021-04-02 | 2022-10-06 | Liveperson, Inc. | Domain adaptation of ai nlp encoders with knowledge distillation |
| CN113610126A (en) * | 2021-07-23 | 2021-11-05 | 武汉工程大学 | Label-free knowledge distillation method based on multi-target detection model and storage medium |
| CN113610173A (en) * | 2021-08-13 | 2021-11-05 | 天津大学 | Knowledge distillation-based multi-span domain few-sample classification method |
| CN114239718A (en) * | 2021-12-15 | 2022-03-25 | 杭州电子科技大学 | High-precision long-term time sequence prediction method based on multivariate time sequence data analysis |
| CN114462617A (en) * | 2021-12-21 | 2022-05-10 | 北京理工大学 | Data anomaly detection method based on distillation mechanism |
| CN114880659A (en) * | 2022-04-29 | 2022-08-09 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, equipment and medium for time series data abnormity detection |
| CN114996124A (en) * | 2022-05-09 | 2022-09-02 | 北京云集智造科技有限公司 | Multi-dimensional time series anomaly detection method based on characteristic distillation |
| CN114881861A (en) * | 2022-05-25 | 2022-08-09 | 厦门大学 | Unbalanced image over-resolution method based on double-sampling texture perception distillation learning |
| CN115375986A (en) * | 2022-07-08 | 2022-11-22 | 际络科技(上海)有限公司 | Model distillation method and device |
| CN115019183A (en) * | 2022-07-28 | 2022-09-06 | 北京卫星信息工程研究所 | Remote sensing image model migration method based on knowledge distillation and image reconstruction |
Non-Patent Citations (4)
| Title |
|---|
| Enabling technologies for low-latency service migration in 5G transport networks;Jun Li;《Journal of Optical Communications and Networking 》;全文 * |
| 一种用于卷积神经网络压缩的混合剪枝方法;靳丽蕾;杨文柱;王思乐;崔振超;陈向阳;陈丽萍;;小型微型计算机***(第12期);全文 * |
| 基于深度学习的计算机辅助肺癌诊断方法;张鹏;徐欣楠;王洪伟;冯元力;冯浩哲;张建伟;闫守琨;侯宇轩;宋怡文;李佳翔;刘新国;;计算机辅助设计与图形学学报(第01期);全文 * |
| 平行堆栈式自编码器及其在过程建模中的应用;李荣雨;徐宏宇;;电子测量与仪器学报(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116204770A (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116204770B (en) | Training method and device for detecting abnormality of bridge health monitoring data | |
| CN112784965B (en) | Large-scale multi-element time series data anomaly detection method oriented to cloud environment | |
| CN111914873B (en) | Two-stage cloud server unsupervised anomaly prediction method | |
| CN109034368A (en) | A kind of complex device Multiple Fault Diagnosis Method based on DNN | |
| CN111881722B (en) | Cross-age face recognition method, system, device and storage medium | |
| CN111597340A (en) | Text classification method and device and readable storage medium | |
| CN112749562A (en) | Named entity identification method, device, storage medium and electronic equipment | |
| CN110956309A (en) | Flow activity prediction method based on CRF and LSTM | |
| CN114756687A (en) | Self-learning entity relationship combined extraction-based steel production line equipment diagnosis method | |
| CN111382798A (en) | Sample picture label correction method, device, equipment and storage medium | |
| KR102540027B1 (en) | Power control system and method characterized in that possible to predict power demand and detect abnormalities | |
| CN115169430A (en) | Cloud network end resource multidimensional time sequence anomaly detection method based on multi-scale decoding | |
| CN116205482A (en) | Important personnel risk level assessment method and related equipment | |
| CN115292568B (en) | Civil news event extraction method based on joint model | |
| CN113076235B (en) | Time sequence abnormity detection method based on state fusion | |
| CN115994630A (en) | Multi-scale self-attention-based equipment residual service life prediction method and system | |
| CN115328782A (en) | Semi-supervised software defect prediction method based on graph representation learning and knowledge distillation | |
| CN115935372A (en) | Vulnerability detection method based on graph embedding and bidirectional gated graph neural network | |
| CN114048546B (en) | Method for predicting residual service life of aeroengine based on graph convolution network and unsupervised domain self-adaption | |
| CN115455194A (en) | Knowledge extraction and analysis method and device for railway faults | |
| CN114416479A (en) | Log sequence anomaly detection method based on out-of-stream regularization | |
| CN113553245B (en) | Log anomaly detection method combining bidirectional slice GRU and gate control attention mechanism | |
| CN112579777B (en) | Semi-supervised classification method for unlabeled text | |
| CN112381213A (en) | Industrial equipment residual life prediction method based on bidirectional long-term and short-term memory network | |
| CN116578336A (en) | Software clone detection method based on plagiarism-detector countermeasure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |