CN114549589A - Rotating body vibration displacement measurement method and system based on lightweight neural network - Google Patents
Rotating body vibration displacement measurement method and system based on lightweight neural network Download PDFInfo
- Publication number
- CN114549589A CN114549589A CN202210193439.5A CN202210193439A CN114549589A CN 114549589 A CN114549589 A CN 114549589A CN 202210193439 A CN202210193439 A CN 202210193439A CN 114549589 A CN114549589 A CN 114549589A
- Authority
- CN
- China
- Prior art keywords
- neural network
- rotator
- data
- lightweight
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006073 displacement reaction Methods 0.000 title claims abstract description 94
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 22
- 238000000691 measurement method Methods 0.000 title description 10
- 238000012549 training Methods 0.000 claims abstract description 79
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 53
- 238000012360 testing method Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 27
- 238000012216 screening Methods 0.000 claims abstract description 20
- 230000008014 freezing Effects 0.000 claims abstract description 17
- 238000007710 freezing Methods 0.000 claims abstract description 17
- 238000002372 labelling Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000010606 normalization Methods 0.000 claims abstract description 12
- 238000010586 diagram Methods 0.000 claims description 18
- 238000005259 measurement Methods 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000005070 sampling Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 208000000044 Amnesia Diseases 0.000 description 3
- 208000026139 Memory disease Diseases 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000006984 memory degeneration Effects 0.000 description 3
- 208000023060 memory loss Diseases 0.000 description 3
- 230000004044 response Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for measuring vibration displacement of a rotating body based on a lightweight neural network, wherein the method comprises the following steps: collecting rotator image data and eddy current data; labeling the rotator image data to obtain a training data set and a test data set; building a lightweight convolutional neural network model; training the model by using a training data set to obtain a series of weight files to be selected; testing a weight file to be selected by using the test data set, comparing eddy current data, and screening to obtain an optimal weight parameter; loading the optimal weight parameters into a lightweight convolutional neural network model to obtain a freezing model; inputting video data to be detected into a freezing model for detection to obtain a multi-frame target detection result; correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data; and carrying out normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve. The invention can be used for measuring the vibration displacement of the rotating body in video.
Description
Technical Field
The invention relates to a rotator vibration displacement measurement method and system based on a lightweight neural network, and belongs to the field of artificial intelligence vibration detection and computer vision tracking. .
Background
The measurement of the vibration displacement field of precise parts such as a rotating body and the like is taken as a necessary means in the early stage of health monitoring, and the service life, the fault type and the dynamic balance characteristic of mechanical structures such as a shaft, a bearing, a rotor and the like can be effectively diagnosed. At present, displacement measurement of a structural body mainly comprises contact measurement and non-contact measurement, but the contact displacement measurement mode of an acceleration sensor cannot implement body installation because of the limited state characteristic of rotary motion.
As a remote, non-contact and non-destructive displacement measurement method, vision measurement is gradually accepted by experts and scholars in the recent years. However, in the conventional visual vibration measurement method, the method is limited by inherent sampling constraints of a common image sensor, and efficient vibration target identification and relevance tracking cannot be realized on a rotating body target in a low-resolution video on the premise of a high sampling rate.
Disclosure of Invention
The invention provides a rotator vibration displacement measuring method and system based on a lightweight neural network, which are used for realizing anchor-free frame type obtaining of rotator vibration displacement by constructing a lightweight convolutional neural network model.
The technical scheme of the invention is as follows: a rotating body vibration displacement measurement method based on a lightweight neural network comprises the following steps:
step 3, building a lightweight convolutional neural network model;
step 4, training the model by using the training data set to obtain a series of weight files to be selected;
step 5, testing the weight file to be selected by using the test data set, comparing eddy current data, and screening to obtain an optimal weight parameter;
step 6, loading the optimal weight parameters into a lightweight convolutional neural network model to obtain a freezing model;
step 7, inputting the video data to be detected into a freezing model for detection to obtain a multi-frame target detection result;
step 8, correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data;
and 9, performing normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
The step 1 comprises the following steps: synchronously acquiring image signals and voltage signals of the rotating body at different rotating speeds through a high-speed industrial camera and an eddy current sensor; the image signal collected by the high-speed industrial camera is used as rotator image data, and the voltage signal collected by the eddy current sensor is used as eddy current data.
The step 2 includes:
step 2.1, manually marking the rotator image data obtained in the step 1 by using a marking tool, wherein marking frames are uniform in size during marking;
and 2.2, dividing the marked data set into a training data set and a testing data set.
The step 3 comprises the following steps: the DLA-34 backbone network in the CenterNet detection algorithm is replaced by an improved MobileNet V2 lightweight backbone network, three parallel heads used for estimating thermodynamic diagrams, object center offset and boundary box size learn the characteristics of each magnetic head through a 3 x 3 convolution layer and a 1 x 1 convolution layer and then are added to the backbone network, and finally the coordinate information containing the target center point of the rotator is mapped.
The improved MobileNet V2 lightweight backbone network specifically comprises: using MobileNetV2 as a basic framework, first extending a bottleneck layer in which the standard convolution is replaced with a deep separable convolution for filtering; the last three convolutional pooling layers in the MobileNetV2 network were then changed to three deconvolution layers.
The step 4 comprises the following steps:
step 4.1, configuring training hyper-parameters in a train.py file of the lightweight convolutional neural network model before formal training, wherein the hyper-parameters of the configuration file mainly comprise learning rate, number of extracted pictures, iteration times and weight attenuation coefficients, and the rest hyper-parameters are default values;
step 4.2, training the lightweight convolutional neural network model; the trained subjects include: the coordinates of the center point of each marking frame and a rectangular frame; the expression form of the training result is a weight file obtained after each training iteration;
and 4.3, calling a train file in the lightweight convolutional neural network model to start training, and screening all obtained weight files through performance evaluation indexes carried by the network model after the set training times are reached to obtain a plurality of weight files to be selected.
The step 5 comprises the following steps:
step 5.1, inputting the test data sets and the weight files to be selected one by one into a lightweight convolutional neural network model;
step 5.2, generating a boundary box of the target by the lightweight convolutional neural network and extracting the central point coordinate of the boundary box of the target as a multi-frame target detection result;
step 5.3, the tracking branch correlates the multi-frame target detection results;
step 5.4, using the position information of the target in the first frame of the video as a reference frame for calculating the offset, and regressing the vibration displacement offset of the target center point in the pixel coordinate system in all the subsequent frames to obtain rotator displacement data;
and 5.5, carrying out normalization processing on the rotator displacement data and the synchronously acquired eddy current data to obtain a time domain graph of the rotator displacement data, carrying out fast Fourier transform on the time domain graph to obtain a frequency domain graph, respectively comparing the time domain graph and the frequency domain graph, calculating the variance of the two displacement data, and if the two data are fitted, determining the weight file to be selected as the optimal weight parameter.
The fit is: the variance of the displacement data of the two is smaller than a threshold value, wherein the threshold value is 0.05; if there are multiple qualifying conditions, the variance is selected to be the smallest.
The step 5.3 comprises the following steps:
step 5.3.1, taking a multi-frame target detection result output by the lightweight convolutional neural network as an input quantity of the tracking branch;
step 5.3.2, preliminarily screening the bounding box generated by the lightweight convolutional neural network by using confidence coefficient;
step 5.3.3, generating a track predictor by utilizing Kalman filtering, and performing cascade matching on a track generated by the predictor and a multi-frame target detection result by Mahalanobis distance matching, cosine distance matching and cross-over matching;
and 5.3.4, updating the parameters of the predictor, and performing frame-by-frame circulation to achieve the purpose of associating multi-frame target detection results.
A rotating body vibration displacement measurement system based on a lightweight neural network comprises:
the collecting module is used for collecting rotator image data and eddy current data;
the first obtaining module is used for labeling the rotator image data to obtain a training data set and a test data set;
the model building module is used for building a lightweight convolutional neural network model;
the second obtaining module is used for training the model by using the training data set to obtain a series of weight files to be selected;
the screening module is used for testing the weight file to be selected by utilizing the test data set, comparing the eddy current data and screening to obtain an optimal weight parameter;
the third obtaining module is used for loading the optimal weight parameters into the lightweight convolutional neural network model to obtain a freezing model;
the fourth obtaining module is used for inputting the video data to be detected into the freezing model for detection to obtain a multi-frame target detection result;
the fifth obtaining module is used for correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data;
and the sixth obtaining module is used for carrying out normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
The invention has the beneficial effects that: the method takes a high-speed industrial camera as a collecting medium, takes a rotor and other rotating bodies in a high-speed video as vibration displacement measurement objects, introduces a depth convolution neural network into the field of visual vibration measurement, balances target detection precision and displacement tracking speed on the basis, and utilizes a designed lightweight convolution neural network model to measure the vibration displacement of the rotating bodies in the video. The method uses an anchor-frame-free detection algorithm to strengthen the shape structure characteristics and the position characteristics of the obtained target, performs light weight processing on the backbone network on the premise that the precision meets the expected requirement, performs convolution filtering by using deep separable convolution and point-by-point convolution instead of conventional convolution, can solve the problems of the calculation efficiency and the parameter quantity of the convolution network under the condition of similar loss precision of the whole model, and can meet the industrial landing requirement; furthermore, the phenomenon of short-time memory loss existing in the detection algorithm is avoided through the tracking branch, and the displacement relevance of the rotating body between frames is strengthened, so that the displacement offset of the vibrating body is measured more accurately; furthermore, when the optimal weight parameters are screened, the eddy current displacement signals which are synchronously acquired are used as standard displacement offsets, the visual displacement measurement results under different rotating speeds are compared, and a vibration displacement curve which is regressed by the lightweight convolution neural network and a displacement signal which is obtained by the eddy current sensor have higher fitting degree through a time domain graph and a frequency domain graph, so that the reliability of the method in the measurement of the vibration displacement of the rotating body is further verified.
Drawings
FIG. 1 is a block diagram of the present invention;
FIG. 2 is a diagram illustrating the effect of a YOLO-mark tool on the marking of a rotating body;
FIG. 3 is a diagram of a lightweight convolutional neural network framework;
FIG. 4 is a filter diagram of a depth separable convolution and a conventional convolution;
FIG. 5 is a Gaussian thermodynamic diagram of a rotor image;
FIG. 6 is a flow chart of model training and generating training weights;
FIG. 7 is a flow chart of screening optimal weight parameters by comparing eddy current data;
FIG. 8 is a schematic diagram of vibration displacement offset extraction;
FIG. 9 is a time domain plot comparison of the present invention and eddy current measurement methods in a unified coordinate system (abscissa-time, ordinate-amplitude, solid line represents eddy current measurement method, dashed line represents the measurement method of the present invention);
FIG. 10 is a plot of the frequency domain of the present invention versus eddy current measurement in a unified coordinate system (abscissa-frequency, ordinate-amplitude, solid line for eddy current measurement, dashed line for measurement according to the present invention);
FIG. 11 is a trace branch flow diagram.
Detailed Description
The invention will be further described with reference to the following figures and examples, but the scope of the invention is not limited thereto.
Example 1: as shown in fig. 1 to 11, a method for measuring vibration displacement of a rotating body based on a lightweight neural network includes: step 1, collecting rotator image data and eddy current data; step 2, labeling image data of the rotating body to obtain a training data set and a test data set; step 3, building a lightweight convolutional neural network model; step 4, training the model by using the training data set to obtain a series of weight files to be selected; step 5, testing the weight file to be selected by using the test data set, comparing eddy current data, and screening to obtain an optimal weight parameter; step 6, loading the optimal weight parameters into a lightweight convolutional neural network model to obtain a freezing model; step 7, inputting the video data to be detected into a freezing model for detection to obtain multi-frame target detection results; step 8, correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data; and 9, performing normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
Optionally, the step 1 includes: synchronously acquiring image signals and voltage signals of the rotating body at different rotating speeds through a high-speed industrial camera and an eddy current sensor; the image signal collected by the high-speed industrial camera is used as rotator image data, and the voltage signal collected by the eddy current sensor is used as eddy current data.
Optionally, the step 2 includes:
step 2.1, manually marking the image data obtained in the step 1 by using a marking tool, wherein marking frames are uniform in size during marking so as to reduce manual marking errors to the maximum extent; wherein, the marking tool adopts YOLO-mark;
and 2.2, dividing the labeled data set into a training data set and a testing data set according to 80% and 20% of the number of the labeled data sets.
Optionally, the step 3 includes: replacing a DLA-34 backbone network in a CenterNet detection algorithm with an improved MobileNet V2 lightweight backbone network, learning the characteristics of each magnetic head by three parallel heads for estimating thermodynamic diagrams, object center offset and boundary box size through a 3 x 3 convolution layer and a 1 x 1 convolution layer, and then attaching the three parallel heads to the backbone network, and finally mapping out coordinate information containing a target center point of a rotator;
optionally, the improved MobileNetV2 lightweight backbone network specifically includes: by taking the MobileNetV2 as a basic framework, a bottleneck layer in which the standard convolution is replaced by a deep separable convolution for filtering operation is firstly adopted, so that the calculation efficiency can be improved by reducing the number of parameters under the condition of similar loss precision; and secondly, the last three convolution pooling layers in the MobileNet V2 network are converted into three deconvolution layers, and an output characteristic diagram with higher resolution can be obtained by utilizing the up-sampling effect of deconvolution, so that the target loss rate is favorably reduced, and the target correlation between frames is improved.
Optionally, the step 4 includes:
step 4.1, configuring training hyper-parameters in a train.py file of the lightweight convolutional neural network model before formal training, wherein the hyper-parameters of the configuration file mainly comprise a learning rate, the number of extracted pictures, iteration times and weight attenuation coefficients, and the rest hyper-parameters are default values;
step 4.2, training the lightweight convolutional neural network model; the trained subjects include: the coordinates of the center point of each marking frame and a rectangular frame; the expression form of the training result is a weight file obtained after each training iteration;
and 4.3, calling a train file in the lightweight convolutional neural network model to start training, and screening all obtained weight files through performance evaluation indexes carried by the network model after the set training times are reached to obtain a plurality of weight files to be selected.
Optionally, the specific process of the step 4.3 training is as follows:
step 4.3.1, calling a train.
4.3.2, randomly extracting a batch _ size image in a training set as a current training sample; wherein, batch _ size represents the number of pictures extracted from the training set at a single time;
4.3.3, sequentially putting each image in the training sample in the step 4.3.2 into a network model for updating the weight parameters; the updating of the weight parameter specifically includes: randomly initializing weight parameters or loading pre-training weight parameters, performing forward propagation calculation of the convolutional neural network to obtain a group of intermediate parameters, and performing backward propagation to update the weight parameters by using the intermediate parameters; the new weight parameter will replace the old weight parameter previously used to compute the forward propagation;
step 4.3.4, recording the process of one forward and backward propagation of all the images of the training data set in step 4.3.2 as a training of the network model, and storing a weight file in each training; repeating the step 4.3.2 to the step 4.3.3 until the training times of the network model reach the set iteration times;
and 4.3.5, after the set training times are reached, screening all the obtained weight files through the performance evaluation indexes carried by the network model to obtain a plurality of weight files to be selected.
Optionally, the step 5 includes:
step 5.1, inputting the rotator test data set and the weight files to be selected one by one into a lightweight convolutional neural network model;
step 5.2, generating a boundary box of the target by the lightweight convolutional neural network and extracting the central point coordinate of the boundary box of the target as a multi-frame target detection result;
step 5.3, the tracking branch correlates the multi-frame target detection results;
step 5.4, using the position information of the target in the first frame of the video as a reference frame for calculating the offset, and regressing the vibration displacement offset of the target center point in the pixel coordinate system in all the subsequent frames to obtain rotator displacement data;
and 5.5, carrying out normalization processing on the rotator displacement data and the synchronously acquired eddy current data to obtain a time domain graph of the rotator displacement data, carrying out fast Fourier transform on the time domain graph to obtain a frequency domain graph, respectively comparing the time domain graph and the frequency domain graph, calculating the variance of the two displacement data, and if the two data are fitted, determining the weight file to be selected as the optimal weight parameter.
Optionally, the fitting is: the variance of the displacement data of the two is smaller than a threshold value, which represents fitting, wherein the threshold value is 0.05; if there are multiple fit conditions, the variance is selected to be the smallest.
Optionally, the step 5.3 includes:
step 5.3.1, taking a multi-frame target detection result output by the lightweight convolutional neural network as an input quantity of the tracking branch;
step 5.3.2, preliminarily screening the bounding box generated by the lightweight convolutional neural network by using confidence coefficient;
step 5.3.3, generating a track predictor by utilizing Kalman filtering, and performing cascade matching on a track generated by the predictor and a multi-frame target detection result by Mahalanobis distance matching, cosine distance matching and cross-over matching;
and 5.3.4, updating the parameters of the predictor, and performing frame-by-frame circulation to achieve the purpose of associating multi-frame target detection results.
The tracking branch can avoid the phenomenon of short-time memory loss existing in the lightweight convolutional neural network, the lightweight convolutional neural network aims to predict the position and the type of a target object and neglects the space-time correlation between adjacent frames, and the tracking branch can strengthen the displacement correlation of a rotator between the frames and measure the displacement offset of the vibrator more accurately.
Example 2: an alternative embodiment of the present invention is described in detail below, as shown in fig. 1-11. A rotating body vibration displacement measurement method based on a lightweight neural network comprises the following steps: step 1, synchronously acquiring rotator image data and eddy current data; step 2, labeling the collected rotator image data to obtain a training data set and a test data set; step 3, building a lightweight convolutional neural network model; step 4, training the model by using a training data set to obtain a series of weight files to be selected; step 5, testing the weight file to be selected by using the test data set, comparing eddy current data, and screening to obtain an optimal weight parameter; step 6, loading the optimal weight parameters into a lightweight convolutional neural network model, and obtaining a freezing model by utilizing PyTorch; step 7, inputting the video data to be detected into a freezing model for detection to obtain multi-frame target detection results; step 8, correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data; and 9, performing normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
And (3) acquiring the rotator image data and the eddy current data in the step (1) on a high-speed rotor vibration test bed (Nanjing Dongda Z-03). The method comprises the steps of taking two sensors with different acquisition principles, namely a high-speed industrial camera (Qianyou 5F01, the set resolution is 512 multiplied by 512, the frame rate is 2000fps) and an eddy current sensor (Shanghai Euro-duo-9000, the set signal sampling rate is 2000Hz), as meters for rotating body displacement acquisition, and synchronously acquiring image data and voltage displacement signals of a rotor, wherein the image data consists of 10000 sequence frames of images at three rotating speeds (6r/s, 13r/s and 35r/s) and 1000 images at the rotating speed of 6r/s which are acquired independently.
The specific content of the step 2 comprises:
and 2.1, manually labeling the image data obtained in the step 1 by using a labeling tool YOLO-mark, wherein the labeling frames are uniform in size during labeling so as to meet the target size characteristic of the rotor target in the image sequence, and therefore errors caused by human factors are reduced. The effect of the YOLO-mark marking is shown in FIG. 2;
and 2.2, dividing the labeled data sets into training data sets and testing data sets according to 80% and 20% of the number of the labeled data sets.
The lightweight convolutional neural network model structure constructed in the step 3 is shown in fig. 3: the step 3 comprises the following steps: the DLA-34 backbone network in the CenterNet detection algorithm is replaced by an improved MobileNet V2 lightweight backbone network, three parallel heads used for estimating thermodynamic diagrams, object center offset and boundary box sizes learn the characteristics of each magnetic head through a 3 x 3 convolution layer and a 1 x 1 convolution layer and then are added to the backbone network, and finally the coordinate information containing the target center point of the rotator rotor is mapped.
The CenterNet detection algorithm belongs to an anchor-frame-free detection algorithm, a traditional multi-type target framing mode is abandoned when a target is detected, the target detection problem is converted into a prediction problem of a target central point, namely the target is represented by the central point of the target, and a rectangular frame of the target is obtained by predicting the offset and the width and the height of the target central point; based on the characteristic that vibration displacement tracking of the rotating body is actually the position tracking of the central point of the rotating body, obviously, the model constructed based on the improved CenterNet detection algorithm has the characteristics of not exhaustively exhausting a large number of prediction frames at the position of a suspected target and saving a large amount of inference time, and has absolute advantages in precision, speed and matching degree in the field of target tracking of the rotating body, thereby showing that the model of the invention is more in line with actual requirements.
Wherein the improved MobileNet V2 lightweight backbone network framework is shown in Table 1: input represents the size of the network Input characteristic diagram; operator stands for convolution operation; t represents the expansion factor in the convolution operation; c represents the number of channels for the next convolution operation; n represents the number of convolution operations; s represents the convolution operation step size. With MobileNetV2 as the basic framework, first following the bottleneck layer where the filtering operation is performed with the standard convolution replaced with the depth separable convolution, the filtering principles of the conventional convolution and the depth separable convolution are shown as (a) and (b) in fig. 4, respectively: taking an RGB three-channel image with a size of 5 × 5 as an example, 4 feature maps can be generated after an input image is subjected to 4 conventional convolution kernels with a size of 3 × 3 × 3. Depth separable convolution first makes use of a single convolution kernel of size 3 x 1, which can generate a feature map of 3 single channels. All feature maps are then convolved point by point, i.e. 4 feature maps are generated by 4 convolution kernels of size 1 × 1 × 3, thus ensuring that the generated feature maps are the same size as conventional convolution. By comparison, we can find that the parameter calculation amount (o (n) ═ 3 × 3 × 3+1 × 3 × 4 ═ 29) of the depth separable convolution in the convolution filtering is only 1/3 of the conventional convolution (o (n) ═ 3 × 3 × 3 × 4 ═ 108), and can improve the calculation efficiency by reducing the parameter amount with similar loss of precision.
And secondly, the last three convolution pooling layers are converted into three deconvolution layers, and an output characteristic diagram with higher resolution can be obtained by utilizing the up-sampling effect of deconvolution, so that the target loss rate is reduced, and the target correlation between frames is improved.
TABLE 1 backbone network framework architecture composition
Wherein the thermodynamic diagram parallels the classification information of the objects in the diagram and is responsible for estimating the location of the center of the object, each class resulting in a separate thermodynamic map. Single target (x)k,yk) GT frame bkCenter point of object in frame ckAnd ckDownsampled matched thermal mapCenter point cklCan be respectively expressed as:
wherein, the first and the second end of the pipe are connected with each other,
if a certain coordinate in the thermodynamic diagram contains the center point of the rotor, a key point represented by a Gaussian circle is mapped at the target. If the predicted center point is exactly (c)i,ci) The thermodynamic response at that point is predicted to be 1. Otherwise the thermodynamic response value will exhibit a penalized exponential decay with increasing offset of the predicted position from the center of the predicted object. Taking the experimental object rotor of the present application as an example, the central point position predicted by thermodynamic diagram is shown in fig. 5, where (a) represents the target central point of the gaussian function map; (b) representing a two-dimensional map of the center point; (c) representing a three-dimensional mapping of a center point; the circle represents the center point of the object.
The existence of a sampling span between the input image and the output feature map can cause the mapping terminal to generate a rotor target offset. To mitigate quantization errors due to upsampling and downsampling operations, we define an object center offset parallel header to estimate the continuous offset of each pixel with respect to the object center, and a bounding box size parallel header to infer the height and width of the target box at each position. Size S of the bounding boxkAnd center offset OkRespectively expressed as:
the specific content of the step 4 is as follows:
step 4.1, referring to fig. 6, before formal training, configuring training hyper-parameters in a train.py file of the lightweight convolutional neural network model, wherein the hyper-parameters of the configuration file mainly comprise learning rate, number of extracted pictures, iteration times and weight attenuation coefficient, and the rest hyper-parameters are default values; in this embodiment, the learning rate is 0.0001, the number of extracted pictures is 8, the number of iterations is 20000, the weight attenuation coefficient is 0.0005, and the remaining hyper-parameters are default values. The experimental equipment is desktop GPU NVIDIA GeForce GTX 2080s, the memory is 16g, and the CPU is Intel core i5-10600 KF;
step 4.2, training the lightweight convolutional neural network model; the trained subjects include: coordinates of the center point of each labeling frame
And a rectangular frame
(i.e., the coordinates of the upper left corner and the lower right corner of the rectangular frame); the expression form of the training result is a weight file obtained after each training iteration;
and 4.3, calling a train in a trace file in the lightweight convolutional neural network model to start training, wherein the training process is as follows:
step 4.3.1, calling a train.
4.3.2, randomly extracting a batch _ size image in a training set as a current training sample; wherein, batch _ size represents the number of pictures extracted from the training set at a single time;
4.3.3, sequentially putting each image in the training sample in the step 4.3.2 into a network model for updating the weight parameters; the updating of the weight parameter specifically includes: randomly initializing weight parameters or loading pre-training weight parameters, performing forward propagation calculation of the convolutional neural network to obtain a group of intermediate parameters, and performing backward propagation to update the weight parameters by using the intermediate parameters; the new weight parameter will replace the old weight parameter previously used to compute the forward propagation;
step 4.3.4, recording the process of one forward and backward propagation of all the images of the training data set in step 4.3.2 as a training of the network model, and storing a weight file in each training; repeating the step 4.3.2 to the step 4.3.3 until the training times of the network model reach the set iteration times;
and 4.3.5, after the set training times are reached, screening all the obtained weight files through the performance evaluation indexes carried by the network model to obtain a plurality of weight files to be selected.
The specific content of step 5 is shown in fig. 7:
step 5.1, inputting the rotator test data set and the weight files to be selected one by one into a lightweight convolutional neural network model;
step 5.2, generating a boundary box of the target by the lightweight convolutional neural network and extracting the central point coordinates of the boundary box of the target to obtain a multi-frame target detection result;
step 5.3, the tracking branch correlates the multi-frame detection results;
step 5.4, referring to the attached figure 8, returning the vibration displacement offset of the target center points in all the subsequent frames in the pixel coordinate system by taking the position information of the target in the first frame of the video as a reference frame for calculating the offset to obtain rotator displacement data;
step 5.5, carrying out normalization processing on the rotator displacement data and the synchronously acquired eddy current data to obtain a time domain graph of the rotator displacement data, carrying out fast Fourier transform on the time domain graph to obtain a frequency domain graph, respectively comparing the time domain graph and the frequency domain graph, calculating the variance of the two displacement data, and if the two data are fitted, determining the weight file to be selected at the moment as the optimal weight parameter; the fit is: the variance of the displacement data of the two is smaller than a threshold value, wherein the threshold value is 0.05; if there are multiple qualifying conditions, the variance is selected to be the smallest. As shown in fig. 9 and fig. 10, if the two data are basically fitted, the candidate weight file at this time is the optimal weight parameter;
and 6, loading the optimal weight parameters into a lightweight convolutional neural network model, and generating a freezing model by using PyTorch.
The tracking branch can avoid the phenomenon of short-time memory loss existing in the lightweight convolutional neural network, the lightweight convolutional neural network aims at predicting the position and the type of a target object and neglects the space-time correlation between adjacent frames, and the tracking branch can strengthen the displacement correlation of a rotator between the frames so as to measure the displacement offset of the vibrator more accurately.
In the step 5.3, as shown in fig. 11, the overall process is that a detection result of a t-th frame image output by the lightweight convolutional neural network is used as an input quantity of a tracking branch, firstly, a generated target frame is preliminarily screened by using a confidence coefficient, and the confidence coefficient is used for eliminating a target boundary frame except for setting (generally, a confidence coefficient threshold is set to be 0.6); and secondly, generating a track predictor by using Kalman filtering, performing cascade matching and updating parameters of the predictor, and performing frame-by-frame circulation to achieve the purpose of associating multi-frame detection results.
The Kalman filtering track predictor predicts the state of the central point of the next frame of target according to the current state, and updates the prediction result and the detection result so as to achieve the target tracking purpose, which is exactly corresponding to the regression of the position of the target to be detected by using the central point by an anchor-frame-free detection algorithm.
In the cascade matching process, aiming at the calculation characteristics of the motion information and the appearance information of the rotating body, the similarity between the detection result of the target and the predicted track is compared by using the Mahalanobis distance and the cosine distance, and the intersection ratio of the detection result of the target and the predicted track is taken as the confidence coefficient, so that better matching is realized in an optimization measurement mode;
in the process of updating the parameters of the predictor, for the matched target, continuously keeping matching in the current frame, and updating by using a Kalman filtering tracker; for unmatched detection targets, generating a new track and matching the new track with a new track predictor in the next round; for a track predictor that does not match the detected target, we will delete this predictor after 30 consecutive matching failures.
The rotor has strong universality and interchangeability, and the form is regular and has obvious characteristics, so the light tracking network is particularly suitable for the visual vibration displacement measurement of the rotating body.
Example 3: a rotating body vibration displacement measurement system based on a lightweight neural network comprises: the collecting module is used for collecting rotator image data and eddy current data; the first obtaining module is used for labeling the rotator image data to obtain a training data set and a test data set; the model building module is used for building a lightweight convolutional neural network model; the second obtaining module is used for training the model by using the training data set to obtain a series of weight files to be selected; the screening module is used for testing the weight file to be selected by utilizing the test data set, comparing the eddy current data and screening to obtain an optimal weight parameter; the third obtaining module is used for loading the optimal weight parameters into the lightweight convolutional neural network model to obtain a freezing model; the fourth obtaining module is used for inputting the video data to be detected into the freezing model for detection to obtain a multi-frame target detection result; the fifth acquisition module is used for correlating the multi-frame detection results through the target tracking branch to acquire rotator displacement data; and the sixth obtaining module is used for carrying out normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve. It should be noted that the above modules may be implemented by software or hardware, for example, for the latter, the following may be implemented: the modules can be located in the same processor; and/or the modules are located in different processors in any combination. The terms "first," "second," and the like in the description and in the claims, and in the drawings described above, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.
While the present invention has been described in detail with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, and various changes can be made without departing from the spirit of the present invention within the knowledge of those skilled in the art.
Claims (10)
1. A method for measuring vibration displacement of a rotating body based on a lightweight neural network is characterized by comprising the following steps: the method comprises the following steps:
step 1, collecting rotator image data and eddy current data;
step 2, labeling image data of the rotating body to obtain a training data set and a test data set;
step 3, building a lightweight convolutional neural network model;
step 4, training the model by using the training data set to obtain a series of weight files to be selected;
step 5, testing the weight file to be selected by using the test data set, comparing eddy current data, and screening to obtain an optimal weight parameter;
step 6, loading the optimal weight parameters into a lightweight convolutional neural network model to obtain a freezing model;
step 7, inputting the video data to be detected into a freezing model for detection to obtain a multi-frame target detection result;
step 8, correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data;
and 9, performing normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
2. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 1, wherein: the step 1 comprises the following steps: synchronously acquiring image signals and voltage signals of the rotating body at different rotating speeds through a high-speed industrial camera and an eddy current sensor; the image signal collected by the high-speed industrial camera is used as rotator image data, and the voltage signal collected by the eddy current sensor is used as eddy current data.
3. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 1, wherein: the step 2 includes:
step 2.1, manually marking the rotator image data obtained in the step 1 by using a marking tool, wherein marking frames are uniform in size during marking;
and 2.2, dividing the marked data set into a training data set and a testing data set.
4. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 1, wherein: the step 3 comprises the following steps: the DLA-34 backbone network in the CenterNet detection algorithm is replaced by an improved MobileNet V2 lightweight backbone network, three parallel heads used for estimating thermodynamic diagrams, object center offset and boundary box size learn the characteristics of each magnetic head through a 3 x 3 convolution layer and a 1 x 1 convolution layer and then are added to the backbone network, and finally the coordinate information containing the target center point of the rotator is mapped.
5. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 4, wherein: the improved MobileNet V2 lightweight backbone network specifically comprises: using MobileNetV2 as a basic framework, first extending a bottleneck layer in which the standard convolution is replaced with a deep separable convolution for filtering; the last three convolutional pooling layers in the MobileNetV2 network were then changed to three deconvolution layers.
6. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 1, wherein: the step 4 comprises the following steps:
step 4.1, configuring training hyper-parameters in a train.py file of the lightweight convolutional neural network model before formal training, wherein the hyper-parameters of the configuration file mainly comprise learning rate, number of extracted pictures, iteration times and weight attenuation coefficients, and the rest hyper-parameters are default values;
step 4.2, training the lightweight convolutional neural network model; the trained subjects include: the coordinates of the center point of each marking frame and a rectangular frame; the expression form of the training result is a weight file obtained after each training iteration;
and 4.3, calling a train file in the lightweight convolutional neural network model to start training, and screening all obtained weight files through performance evaluation indexes carried by the network model after the set training times are reached to obtain a plurality of weight files to be selected.
7. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 1, wherein: the step 5 comprises the following steps:
step 5.1, inputting the test data sets and the weight files to be selected one by one into a lightweight convolutional neural network model;
step 5.2, generating a boundary frame of the target by the lightweight convolutional neural network and extracting the coordinates of the central point of the boundary frame of the target to be used as a multi-frame target detection result;
step 5.3, the tracking branch correlates the multi-frame target detection results;
step 5.4, using the position information of the target in the first frame of the video as a reference frame for calculating the offset, and regressing the vibration displacement offset of the target center point in the pixel coordinate system in all the subsequent frames to obtain rotator displacement data;
and 5.5, carrying out normalization processing on the rotator displacement data and the synchronously acquired eddy current data to obtain a time domain graph of the rotator displacement data, carrying out fast Fourier transform on the time domain graph to obtain a frequency domain graph, respectively comparing the time domain graph and the frequency domain graph, calculating the variance of the two displacement data, and if the two data are fitted, determining the weight file to be selected as the optimal weight parameter.
8. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 7, wherein: the fit is: the variance of the displacement data of the two is smaller than a threshold value, wherein the threshold value is 0.05; if there are multiple qualifying conditions, the variance is selected to be the smallest.
9. The method for measuring vibration displacement of a rotating body based on a lightweight neural network according to claim 7, wherein: the step 5.3 comprises the following steps:
step 5.3.1, taking a multi-frame target detection result output by the lightweight convolutional neural network as an input quantity of the tracking branch;
step 5.3.2, preliminarily screening the bounding box generated by the lightweight convolutional neural network by using confidence coefficient;
step 5.3.3, generating a track predictor by utilizing Kalman filtering, and performing cascade matching on a track generated by the predictor and a multi-frame target detection result by Mahalanobis distance matching, cosine distance matching and cross-over matching;
and 5.3.4, updating the parameters of the predictor, and performing frame-by-frame circulation to achieve the purpose of associating multi-frame target detection results.
10. The utility model provides a rotator vibration displacement measurement system based on lightweight neural network which characterized in that: the method comprises the following steps:
the collecting module is used for collecting rotator image data and eddy current data;
the first obtaining module is used for labeling the rotator image data to obtain a training data set and a test data set;
the model building module is used for building a lightweight convolutional neural network model;
the second obtaining module is used for training the model by using the training data set to obtain a series of weight files to be selected;
the screening module is used for testing the weight file to be selected by utilizing the test data set, comparing the eddy current data and screening to obtain an optimal weight parameter;
the third obtaining module is used for loading the optimal weight parameters into the lightweight convolutional neural network model to obtain a freezing model;
the fourth obtaining module is used for inputting the video data to be detected into the freezing model for detection to obtain a multi-frame target detection result;
the fifth obtaining module is used for correlating the multi-frame detection results through the target tracking branch to obtain rotator displacement data;
and the sixth obtaining module is used for carrying out normalization processing on the obtained rotator displacement data to obtain a rotator vibration displacement curve.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210193439.5A CN114549589A (en) | 2022-03-01 | 2022-03-01 | Rotating body vibration displacement measurement method and system based on lightweight neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210193439.5A CN114549589A (en) | 2022-03-01 | 2022-03-01 | Rotating body vibration displacement measurement method and system based on lightweight neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114549589A true CN114549589A (en) | 2022-05-27 |
Family
ID=81661000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210193439.5A Pending CN114549589A (en) | 2022-03-01 | 2022-03-01 | Rotating body vibration displacement measurement method and system based on lightweight neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114549589A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117195440A (en) * | 2023-11-08 | 2023-12-08 | 上海诺倬力机电科技有限公司 | Main shaft system structure optimization method, device, equipment and storage medium |
-
2022
- 2022-03-01 CN CN202210193439.5A patent/CN114549589A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117195440A (en) * | 2023-11-08 | 2023-12-08 | 上海诺倬力机电科技有限公司 | Main shaft system structure optimization method, device, equipment and storage medium |
| CN117195440B (en) * | 2023-11-08 | 2024-01-30 | 上海诺倬力机电科技有限公司 | Main shaft system structure optimization method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110660052B (en) | Hot-rolled strip steel surface defect detection method based on deep learning | |
| CN108732465B (en) | Power distribution network fault positioning method based on wavelet transformation and CNN | |
| CN109145830B (en) | Intelligent water gauge identification method | |
| CN113436169B (en) | Industrial equipment surface crack detection method and system based on semi-supervised semantic segmentation | |
| CN111122162B (en) | Industrial system fault detection method based on Euclidean distance multi-scale fuzzy sample entropy | |
| CN113808174B (en) | Radar small target tracking method based on full convolution network and Kalman filtering | |
| CN114818774A (en) | Intelligent gearbox fault diagnosis method based on multi-channel self-calibration convolutional neural network | |
| CN109708658B (en) | Visual odometer method based on convolutional neural network | |
| CN111161224A (en) | Casting internal defect grading evaluation system and method based on deep learning | |
| CN110399866A (en) | Based on the alternate space junk observation method of CCD camera different exposure time | |
| CN113392881A (en) | Rotary machine fault diagnosis method based on transfer learning | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| CN116229052B (en) | Method for detecting state change of substation equipment based on twin network | |
| CN114677339A (en) | Power transmission line bolt out-of-stock defect detection method introducing attention mechanism | |
| CN114549589A (en) | Rotating body vibration displacement measurement method and system based on lightweight neural network | |
| CN114842308B (en) | Method for establishing target pre-arbitration model based on full feature fusion | |
| CN115049945A (en) | Method and device for extracting lodging area of wheat based on unmanned aerial vehicle image | |
| CN111624606B (en) | Radar image rainfall identification method | |
| CN116843725B (en) | River surface flow velocity measurement method and system based on deep learning optical flow method | |
| CN117312948A (en) | Rotary machine fault diagnosis method based on multi-sensor fusion and ECA-CNN | |
| CN116682045A (en) | Beam pumping unit fault detection method based on intelligent video analysis | |
| CN114565511B (en) | Lightweight image registration method, system and device based on global homography estimation | |
| CN116051808A (en) | YOLOv 5-based lightweight part identification and positioning method | |
| CN115796045A (en) | Glacier recognition model modeling method based on deep learning network and semi-supervised learning | |
| CN114581705A (en) | Fruit ripening detection method and system based on YOLOv4 model and convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |