CN116188846A - Equipment fault detection method and device based on vibration image - Google Patents
Equipment fault detection method and device based on vibration image Download PDFInfo
- Publication number
- CN116188846A CN116188846A CN202211733357.1A CN202211733357A CN116188846A CN 116188846 A CN116188846 A CN 116188846A CN 202211733357 A CN202211733357 A CN 202211733357A CN 116188846 A CN116188846 A CN 116188846A
- Authority
- CN
- China
- Prior art keywords
- image
- vibration
- frequency amplitude
- vibration image
- amplitude histogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M13/00—Testing of machine parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
Abstract
The embodiment of the invention provides a device fault detection method and device based on a vibration image, which are characterized in that the vibration image of a device is acquired through image acquisition, and a frequency amplitude two-dimensional histogram of the vibration frequency and amplitude value of the vibration image is obtained after the characteristics of the vibration image are extracted; according to a preset convolutional neural network model through classification, performing classification calculation on the acquired vibration image and frequency amplitude histogram, obtaining classification results aiming at the vibration image and the frequency amplitude histogram, and finally performing abnormal fault judgment on equipment according to the classification results; the vibration image characteristics of the equipment are analyzed around the improved classified convolution neural network model, the vibration of the equipment is analyzed by a non-contact information acquisition method, and a vibration signal detection means for industrial equipment is expanded.
Description
Technical Field
The invention relates to the technical field of machine vision CV (computer vision system), in particular to an equipment fault detection method and device based on vibration images, electronic equipment and a readable storage medium.
Background
Industrial equipment is used as continuously rich production data, so that the working efficiency of labor production can be greatly improved, and the method has the advantages of stability and reliability; when the equipment is in operation, certain vibration noise is often accompanied by mechanical activity of various functional components in the equipment; by collecting and analyzing the sound wave characteristics in the noise, the running state of the equipment can be effectively analyzed, and the method has important significance for detection and maintenance of the equipment.
In the related art, it is common to arrange vibration sensors to capture vibration signals generated when an industrial apparatus is operated, and then determine the operation state of the apparatus by analyzing and processing the collected vibration signals, and determine whether an abnormal fault exists.
However, in the existing scheme, the sensor world is required to be connected to equipment, signal acquisition is carried out through contact deployment, the whole detection link step is very complicated, and the application method is too limited.
Disclosure of Invention
The embodiment of the invention provides a device fault detection method and device based on a vibration image, electronic equipment and a readable storage medium, which are used for solving the problem that detection means of industrial equipment are too limited in the prior art.
In order to solve the technical problems, the embodiment of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for detecting a device failure based on a vibration image, the method including:
acquiring a vibration image of the equipment;
extracting features of the vibration image to obtain a frequency amplitude histogram of the vibration image;
obtaining classification results aiming at the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model;
And determining whether the equipment has faults according to the classification result.
Optionally, the obtaining a classification result for the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model includes:
fusing the image channel of the vibration image with the image channel of the frequency amplitude histogram to obtain a target fusion image;
and classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result.
Optionally, the vibration image and the frequency amplitude histogram are two-dimensional histograms; the fusing the image channel of the vibration image and the image channel of the frequency amplitude histogram to obtain a target fused image comprises the following steps:
obtaining a vibration image matrix collection comprising at least one vibration image matrix according to the image channel information of the vibration image, wherein a single vibration image matrix comprises all content information of an image channel of one vibration image;
obtaining a frequency amplitude histogram matrix aggregate comprising at least one frequency amplitude histogram matrix according to the image channel information of the frequency amplitude histogram, wherein a single frequency amplitude histogram matrix comprises all content information of the image channel of one frequency amplitude histogram;
Fusing the vibration image matrix set with the image channels in the frequency amplitude histogram matrix set to obtain a fused matrix set; wherein the fusion matrix aggregate includes all content information of the image channels of the vibration image and the image channels of the frequency amplitude histogram;
and generating the target fusion image according to the fusion matrix collection.
Optionally, the classified convolutional neural network model comprises at least one classifier; the classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result comprises the following steps:
evaluating and scoring the target fusion image through a classifier in a classified convolutional neural network model;
obtaining an arithmetic average of all scoring results of the classifier;
the arithmetic average value is taken as a classification result.
Optionally, the determining whether the equipment has a fault according to the classification result includes:
if the arithmetic average value of the target fusion image in the classification result is larger than or equal to a first preset threshold value, determining that the equipment has faults;
and if the arithmetic average value of the target fusion image is smaller than a first preset threshold value, determining that the equipment has no fault.
Optionally, the vibration image includes: a normal vibration image sample and an abnormal vibration image sample;
before extracting features of the vibration image and obtaining a frequency amplitude histogram of the vibration image, the method further includes:
and carrying out sample quantity allocation on the vibration image through a sample sampling function, so that the ratio of the sample quantity of the abnormal vibration image sample to the sample quantity of the normal vibration image sample in the vibration image is equal to a first preset proportion threshold value.
Optionally, after classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result, the method further includes:
and model training is carried out on the classifier of the classified convolutional neural network model through a loss function, and calculation parameters used by the classifier in classifying the target fusion image are optimized.
In a second aspect, an embodiment of the present invention provides an apparatus for detecting a device failure based on a vibration image, the apparatus including:
the image acquisition module is used for acquiring a vibration image of the equipment;
the feature extraction module is used for extracting features of the vibration image and obtaining a frequency amplitude histogram of the vibration image;
The image classification module is used for obtaining classification results aiming at the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model;
and the equipment fault determining module is used for determining whether equipment has faults or not according to the classification result.
Optionally, the image classification module further includes:
the fusion image acquisition sub-module is used for fusing the image channel of the vibration image with the image channel of the frequency amplitude histogram to obtain a target fusion image;
and the classification algorithm execution sub-module is used for classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result.
Optionally, the fused image acquisition sub-module further includes:
a vibration image channel matrix obtaining unit, configured to obtain a vibration image matrix collection including at least one vibration image matrix according to image channel information of the vibration image, where a single vibration image matrix includes all content information of an image channel of the vibration image;
a frequency amplitude image channel matrix obtaining unit, configured to obtain, according to image channel information of the frequency amplitude histogram, a frequency amplitude histogram matrix aggregate including at least one frequency amplitude histogram matrix, a single frequency amplitude histogram matrix including all content information of an image channel of the frequency amplitude histogram;
The matrix fusion unit is used for fusing the vibration image matrix collection with the image channels in the frequency amplitude histogram matrix collection to obtain a fusion matrix collection; wherein the fusion matrix aggregate includes all content information of the image channels of the vibration image and the image channels of the frequency amplitude histogram;
and the fusion image generation unit is used for generating the target fusion image according to the fusion matrix collection.
Optionally, the classification algorithm execution sub-module further includes:
the classification result acquisition unit is used for evaluating and scoring the target fusion image through a classifier in the classification convolutional neural network model;
a classification result averaging unit, configured to obtain an arithmetic average value of all scoring results of the classifier;
and the classification result generating unit is used for taking the arithmetic average value as a classification result.
Optionally, the apparatus further includes:
the sample proportion allocation module is used for allocating the sample number of the vibration image through a sample sampling function, so that the ratio of the sample number of the abnormal vibration image sample to the sample number of the normal vibration image sample in the vibration image is equal to a first preset proportion threshold value.
Optionally, the apparatus further includes:
and the network training execution module is used for carrying out model training on the classifier of the classified convolutional neural network model through a loss function and optimizing calculation parameters used by the classifier when classifying the target fusion image.
In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any of the above.
In a fourth aspect, embodiments of the present invention provide a storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform any one of the methods described above.
In the embodiment of the invention, firstly, a vibration image of equipment is acquired through image acquisition, and after feature extraction is carried out on the vibration image, a frequency amplitude two-dimensional histogram of vibration frequency and amplitude value of the vibration image is obtained; according to a preset convolutional neural network model through classification, performing classification calculation on the acquired vibration image and frequency amplitude histogram, obtaining classification results aiming at the vibration image and the frequency amplitude histogram, and finally performing abnormal fault judgment on equipment according to the classification results; the vibration image characteristics of the equipment are analyzed around the improved classified convolution neural network model, the vibration of the equipment is analyzed by a non-contact information acquisition method, and a vibration signal detection means for industrial equipment is expanded.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 is a schematic step flowchart of a method for detecting equipment failure based on a vibration image according to an embodiment of the present invention;
fig. 2 is a flowchart of a step of a detailed embodiment of a method for detecting a device failure based on a vibration image according to an embodiment of the present invention;
FIG. 3 is a logic relationship diagram for a complete implementation of a method for detecting a device failure based on a vibration image according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a module composition of an apparatus fault detection device based on a vibration image according to an embodiment of the present invention;
FIG. 5 is a functional component relationship diagram of an electronic device according to an embodiment of the present invention;
fig. 6 is a functional component relationship diagram of another electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Referring to fig. 1, a schematic step flowchart of a vibration image-based equipment fault detection method according to an embodiment of the present invention is shown; as shown in fig. 1, the method steps may include:
step 101: a vibration image of the device is acquired.
During the operation of industrial equipment, complex vibration signals are generated due to the relative movement of all parts in the facility; the vibration signal of the equipment contains rich equipment abnormality or fault information, and important feedback indexes of the running state of the equipment play an important role in detecting and monitoring the running state of the equipment and analyzing the abnormality of the equipment by capturing and analyzing the vibration signal.
According to the equipment fault detection method based on the vibration image, firstly, through the third equipment, vibration images generated by industrial equipment in an operation state are continuously collected, and then vibration characteristic information is extracted from the images for analysis. The vibration image is an amplitude and frequency image of vibration of each pixel point on the equipment object.
Specifically, in the embodiment of the invention, a high-precision camera is used for acquiring a vibration image of equipment to be measured. Compared with the traditional method, the method for capturing the vibration signals by installing the contact type vibration sensor for the equipment has more flexible practicability, and meanwhile, different shooting angles can be switched, and the state of the industrial equipment in operation is shot from multiple dimensions so as to acquire the vibration image.
Step 102: and extracting the characteristics of the vibration image to obtain a frequency amplitude histogram of the vibration image.
For the vibration image shot by the high-definition camera, normalization processing is required to be carried out on the content contained in the image before analysis and calculation are carried out, so that feature extraction of the vibration image is introduced, and a frequency amplitude histogram is obtained.
It should be noted that, the image histogram is a very important pixel statistical result in the image processing, and reflects the probability distribution situation of the image pixels. The histogram obtained by feature extraction is different from the original captured vibration image, and the texture information of the image is not characterized any more to display the image content, but statistical information of image pixels. Since the same object has the same gray value in the image whether rotating or translating, the histogram has the advantages of translational invariance, scaling invariance and the like.
The histogram obtained by feature extraction results in a statistical graph with the horizontal axis representing the image gray values and the vertical axis representing the number of gray values or the ratio of gray values.
Specifically, in the embodiment of the present invention, the frequency amplitude histogram obtained by feature extraction is a statistical histogram containing frequency amplitude information of gray values of pixel points in a vibration image.
Step 103: and obtaining classification results aiming at the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model.
After receiving step 102, the vibration image and the frequency amplitude histogram are obtained, the image may be subjected to a feature analysis processing operation.
Analyzing the image by adopting a Mobile Net V3 network model: the Mobile Net series lightweight network is a lightweight deep convolution computing network provided for embedded equipment such as Mobile phones, and the core idea of the network is depth separable convolution, so that the area of an object to be measured can be accurately segmented through object area segmentation. The neural network improves the quality of the representation produced by the network by explicitly modeling the interdependencies between the convolutions of the characteristic channels of the network.
It should be noted that, in the Mobile Net V3 network model used in the embodiment of the present invention, a multi-head selection (multi-head) classification method is used, so that the model network can capture more abundant feature information. Wherein each selection header (head) is an independent classifier with different selection class calculation parameters. And (3) taking the classification result of the two-dimensional image as a specific score generated by a classification algorithm, and then carrying out arithmetic average summation on scoring results of a plurality of classification heads in the model to obtain a final classification result.
Step 104: and determining whether the equipment has faults according to the classification result.
And judging the equipment state of the industrial equipment according to the classification result obtained by the Mobile Net V3 classification neural network, and judging whether the equipment has abnormal faults or not.
In the implementation process, the classification result of each classifier in the general Mobile Net V3 classification neural network model is any decimal fraction between 0 and 1, such as: 0.4, 0.6, 0.8, etc.; the result obtained by arithmetic averaging the score results of all the classification heads is 0.5 score if the average value result is larger than a certain judgment threshold value; the abnormal operation state of the industrial equipment from which the photographed image is derived can be determined.
In summary, according to the device fault detection method based on the vibration image provided by the embodiment of the invention, firstly, the vibration image of the device is acquired through image acquisition, and after the feature extraction is performed on the vibration image, a frequency amplitude two-dimensional histogram about the vibration frequency and the amplitude value of the vibration image is obtained; according to a preset convolutional neural network model through classification, performing classification calculation on the acquired vibration image and frequency amplitude histogram, obtaining classification results aiming at the vibration image and the frequency amplitude histogram, and finally performing abnormal fault judgment on equipment according to the classification results; the vibration image characteristics of the equipment are analyzed around the improved classified convolution neural network model, the vibration of the equipment is analyzed by a non-contact information acquisition method, and a vibration signal detection means for industrial equipment is expanded.
Referring to fig. 2, a detailed embodiment step flow diagram of a vibration image-based equipment fault detection method provided by an embodiment of the present invention is shown; as shown in fig. 2, the method steps may specifically include:
step 201: a vibration image of the device is acquired.
The step may refer to the above step 101, and this embodiment is not described herein.
In an alternative specific embodiment, the step 201 may further include:
sub-step 2011: and carrying out sample quantity allocation on the vibration image through a sample sampling function, so that the ratio of the sample quantity of the abnormal vibration image sample to the sample quantity of the normal vibration image sample in the vibration image is equal to a first preset proportion threshold value.
It is generally considered that the failure or abnormality of the industrial equipment should be a small probability event, and the vibration image generated by the operation of the industrial equipment is a normal image in most of the daily operation time. For the neural network model, the image characteristic information of the abnormal vibration waveform in the vibration image is accurately extracted and classified, and a large amount of real materials are required to train the model continuously so as to optimize the operation parameters of the classified convolutional neural network.
In practical application, a high-precision camera is used for carrying out an image acquisition process in a unit time, the obtained normal vibration image samples of the industrial motor are about 10000, and the abnormal vibration image samples are about 100. The calculation object of the classified convolutional neural network is against the use intention of anomaly detection, so that the proportional relation between the normal image and the abnormal image in the input image sample is required to be adjusted, and the abnormal image is the main calculation object in the processing sample of the input classified convolutional neural network.
Specifically, all the shot vibration images are extracted by adopting a sample sampling function, so that the proportion of an abnormal sample to a normal sample in one image sample batch is ensured to be kept in a proportion relation of about 10:1.
Step 202: and extracting the characteristics of the vibration image to obtain a frequency amplitude histogram of the vibration image.
The step may refer to the step 102, and the description of this embodiment is omitted here.
Step 203: and fusing the image channel of the vibration image with the image channel of the frequency amplitude histogram to obtain a target fusion image.
Before the classified convolutional neural network calculation is performed, the acquired vibration image and the image channels of the frequency amplitude histogram are required to be fused, and the original image and the image features in the histogram are further combined.
Optionally, the step 203 may specifically include:
sub-step 2031: and obtaining a vibration image matrix collection comprising at least one vibration image matrix according to the image channel information of the vibration image, wherein a single vibration image matrix comprises all content information of the image channel of one vibration image.
First, channel information included therein is acquired by the acquired vibration image. The image channel is used for representing a certain basic image element constituting the image, and the proportion relation is contained in the image.
For example: for primary color images, reference may be made to three primary color channels RGB, RED (RED), GREEN (GREEN) and BLUE (BLUE), which make up the complete image content, which, in combination, are capable of restoring all content information in the original image. Wherein the proportion relation of the current element in the image is represented in the respective primary color channels; for example, the greater the overall number of red pixels in a given original image, the greater the number of pixels that are reflected in a single color channel, i.e., in the display red channel.
In the embodiment of the invention, for a captured vibration image generated by vibration of industrial equipment, channel information in the image is represented by an image matrix; each element (number) in the matrix may represent channel information contained in the pixel at the current position in the image. A single image channel corresponds to a single image matrix and the current image may be split into several image channels, i.e. to several image matrices. All image matrices constitute a set of vibration image matrices.
Sub-step 2032: obtaining a frequency amplitude histogram matrix set comprising at least one frequency amplitude histogram matrix from the image channel information of the frequency amplitude histogram, a single frequency amplitude histogram matrix comprising all content information of the image channel of one of the frequency amplitude histograms.
Similarly, for the frequency amplitude histogram, the content mainly displayed in the image channel is two-dimensional statistical information of the frequency amplitude in the vibration image, and the frequency amplitude histogram matrix aggregate can be obtained according to the statistical information.
Sub-step 2033: fusing the vibration image matrix set with the image channels in the frequency amplitude histogram matrix set to obtain a fused matrix set; wherein the fusion matrix set includes all content information of the image channels of the vibration image and the image channels of the frequency amplitude histogram.
And combining the vibration image with all image matrixes of the frequency amplitude histogram to obtain a fusion matrix combination set. For example: the original vibration image can be specifically divided into an image matrix formed by 2 sub-image channels corresponding to frequency and amplitude respectively, and the frequency-amplitude histogram can be specifically divided into an image matrix formed by 1 sub-image channel, so that the final matrix fusion result is a 2+1=3 image matrix.
And (3) carrying out parallel fusion on the image channels of different images to obtain a fused image containing all the image characteristic information of a plurality of images.
Sub-step 2034: and generating the target fusion image according to the fusion matrix collection.
And merging the vibration image with all the image matrixes of the frequency amplitude histogram, and generating a new target fusion image according to the merged fusion matrix set.
As can be seen from the sub-step 2031, the original image can be split into different image channels, and all the image channels are superimposed and combined and then displayed, so that all the information in the original image can be restored.
And similarly, the fused matrix set after fusion comprises all image channel information of the original vibration image and the frequency amplitude histogram, and the images with all image characteristics of the original vibration image and the frequency amplitude histogram can be obtained by superposing and displaying all image matrixes according to the fused matrix set.
Step 204: and obtaining a classification result aiming at the target fusion image according to the target fusion image and a preset classification convolutional neural network model.
The step may refer to step 103, and this embodiment is not described herein.
Sub-step 2041: and evaluating and scoring the target fusion image through a classifier in the classified convolutional neural network model.
The target fusion image obtained after the fusion matrix is restored can enable the model network to capture more abundant characteristic information by using a multi-head selection (multi-head) classification method through the Mobile Net V3 network model used in the embodiment of the invention. Wherein each selection header (head) is an independent classifier with different selection class calculation parameters. The classification result of the two-dimensional image is a specific score generated by a classification algorithm.
Sub-step 2042: an arithmetic average of all scoring results of the classifier is obtained.
And carrying out arithmetic average summation on scoring results of a plurality of classification heads in the model to obtain a final classification result.
For example: aiming at the classification convolutional neural network model adopting 5 classification heads, the scores are as follows in sequence: 0.3, 0.5, 0.7, 0.6, 0.4, the final arithmetic average is classified as: (0.3+0.5+0.5+0.7+0.6+0.4)/5=0.6.
Sub-step 2043: the arithmetic average value is taken as a classification result.
After the arithmetic mean value is obtained, the arithmetic mean value is used as a final classification result, and whether an abnormal factor exists in the vibration waveform of the equipment is determined through the magnitude relation between a preset judging threshold value and the final result, so that whether the industrial equipment has fault abnormality is determined.
Step 205: and determining whether the equipment has faults according to the classification result.
The step may refer to the step 104, and the description of this embodiment is omitted here.
Sub-step 2051: and if the arithmetic average value of the target fusion image in the classification result is greater than or equal to a first preset threshold value, determining that the equipment has faults.
In the embodiment of the invention, if the finally obtained arithmetic mean value is greater than or equal to 0.5, the current industrial equipment can be considered to have equipment abnormality. In practical application, for the specific value of the judging threshold value of the fault abnormality, maintenance personnel can modify according to different application scenes, and the embodiment of the invention is not limited here.
Sub-step 2052: and if the arithmetic average value of the target fusion image is smaller than a first preset threshold value, determining that the equipment has no fault.
If the finally obtained arithmetic average value is smaller than 0.5, the current industrial equipment can be considered to have no equipment abnormality, and the characteristic waveforms contained in the captured vibration images are all vibration waveforms generated in a normal working state.
Step 206: and model training is carried out on the classifier of the classified convolutional neural network model through a loss function, and calculation parameters used by the classifier in classifying the target fusion image are optimized.
Referring to fig. 3, a logic relationship diagram of a complete implementation of a device fault detection method based on a vibration image according to an embodiment of the present invention is shown; as shown in fig. 3, which includes:
firstly, a high-precision camera is used for obtaining a vibration image of equipment to be measured, and a frequency amplitude two-dimensional histogram of the vibration image is extracted. And then carrying out channel fusion on the vibration image and the histogram, and sending the fusion result into an improved Mobile Net V3 classification convolution network model to obtain a classification result. Finally, training the network model through a weighted Sigmoid loss function.
The Sigmoid loss function is also called a Logistic function, is used for hidden layer you to output, has a value range of (0, 1), can map a real number to a section of (0, 1), can be used for classification, and has better effect when the characteristic phase difference is complex or the phase difference is not particularly large.
In the equipment fault detection method based on the vibration image, due to the fact that normal samples of industrial equipment are most and few abnormal samples are, a weighting loss function is sampled during training, the weight of the abnormal samples is increased, and in order to enable the Mobile Net V3 classification network model used in the classification process to accurately extract and classify the characteristic information of the vibration image, a sigmoid function is introduced to conduct optimization training on calculation parameters in the model after each classification judgment.
Specifically, the sigmoid function belongs to one of the logic loss functions, is applicable to two classification tasks, and needs to satisfy one of the assumptions: the data satisfies the bernoulli distribution. h is a θ (x) Representing the probability of the sample being predicted to be of positive class, 1-h θ (x) The probability of predicting a sample as a negative class, the entire model can be expressed as: h is a θ (x, θ) =p, wherein,
where θ is the weight value, x is the input variable, the exponential function represents the output of the model), and finally the final expression of logistic regression is obtained. The loss function of logistic regression is its maximum likelihood function.
In the process of model training by using the Sigmoid loss function, the weight of the abnormal sample is set to be 10, and the weight of the normal sample is set to be 1.
In summary, according to the device fault detection method based on the vibration image provided by the embodiment of the invention, firstly, the vibration image of the device is acquired through image acquisition, and after the feature extraction is performed on the vibration image, a frequency amplitude two-dimensional histogram about the vibration frequency and the amplitude value of the vibration image is obtained; according to a preset convolutional neural network model through classification, performing classification calculation on the acquired vibration image and frequency amplitude histogram, obtaining classification results aiming at the vibration image and the frequency amplitude histogram, and finally performing abnormal fault judgment on equipment according to the classification results; the vibration image characteristics of the equipment are analyzed around the improved classified convolution neural network model, the vibration of the equipment is analyzed by a non-contact information acquisition method, and a vibration signal detection means for industrial equipment is expanded.
Referring to fig. 4, a schematic diagram of a module composition of an apparatus fault detection device based on a vibration image according to an embodiment of the present invention is shown; as shown in fig. 4, the apparatus includes:
an image acquisition module 301, configured to acquire a vibration image of the device;
the feature extraction module 302 is configured to perform feature extraction on the vibration image, and obtain a frequency amplitude histogram of the vibration image;
An image classification module 303, configured to obtain a classification result for the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram, and a preset classification convolutional neural network model;
and the equipment fault determining module 304 is used for determining whether the equipment has faults according to the classification result.
Optionally, the image classification module 303 further includes:
the fusion image acquisition sub-module is used for fusing the image channel of the vibration image with the image channel of the frequency amplitude histogram to obtain a target fusion image;
and the classification algorithm execution sub-module is used for classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result.
Optionally, the fused image acquisition sub-module further includes:
a vibration image channel matrix obtaining unit, configured to obtain a vibration image matrix collection including at least one vibration image matrix according to image channel information of the vibration image, where a single vibration image matrix includes all content information of an image channel of the vibration image;
a frequency amplitude image channel matrix obtaining unit, configured to obtain, according to image channel information of the frequency amplitude histogram, a frequency amplitude histogram matrix aggregate including at least one frequency amplitude histogram matrix, a single frequency amplitude histogram matrix including all content information of an image channel of the frequency amplitude histogram;
The matrix fusion unit is used for fusing the vibration image matrix collection with the image channels in the frequency amplitude histogram matrix collection to obtain a fusion matrix collection; wherein the fusion matrix aggregate includes all content information of the image channels of the vibration image and the image channels of the frequency amplitude histogram;
and the fusion image generation unit is used for generating the target fusion image according to the fusion matrix collection.
Optionally, the classification algorithm execution sub-module further includes:
the classification result acquisition unit is used for evaluating and scoring the target fusion image through a classifier in the classification convolutional neural network model;
a classification result averaging unit, configured to obtain an arithmetic average value of all scoring results of the classifier;
and the classification result generating unit is used for taking the arithmetic average value as a classification result.
Optionally, the apparatus further includes:
the sample proportion allocation module is used for allocating the sample number of the vibration image through a sample sampling function, so that the ratio of the sample number of the abnormal vibration image sample to the sample number of the normal vibration image sample in the vibration image is equal to a first preset proportion threshold value.
Optionally, the apparatus further includes:
and the network training execution module is used for carrying out model training on the classifier of the classified convolutional neural network model through a loss function and optimizing calculation parameters used by the classifier when classifying the target fusion image.
In summary, according to the device for detecting a device fault based on a vibration image provided by the embodiment of the invention, firstly, the vibration image of the device is acquired through image acquisition, and after feature extraction is performed on the vibration image, a frequency-amplitude two-dimensional histogram about the vibration frequency and amplitude value of the vibration image is obtained; according to a preset convolutional neural network model through classification, performing classification calculation on the acquired vibration image and frequency amplitude histogram, obtaining classification results aiming at the vibration image and the frequency amplitude histogram, and finally performing abnormal fault judgment on equipment according to the classification results; the vibration image characteristics of the equipment are analyzed around the improved classified convolution neural network model, the vibration of the equipment is analyzed by a non-contact information acquisition method, and a vibration signal detection means for industrial equipment is expanded.
Additionally, the embodiment of the invention also provides electronic equipment, which comprises: the sample container unloading method comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the computer program is executed by the processor to realize the sample container unloading method.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, realizes the processes of the sample container unloading method embodiment, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
Fig. 5 is a block diagram of an electronic device 600, according to an example embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 5, the electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input/output (I/O) interface 612, a sensor component 614, and a communication component 616.
The processing component 602 generally controls overall operation of the electronic device 600, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 602 can include one or more modules that facilitate interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate interaction between the multimedia component 608 and the processing component 602.
The memory 604 is used to store various types of data to support operations at the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, multimedia, and so forth. The memory 604 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 606 provides power to the various components of the electronic device 600. The power supply components 606 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.
The multimedia component 608 includes a screen between the electronic device 600 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense demarcations of touch or sliding actions, but also detect durations and pressures associated with the touch or sliding operations. In some embodiments, the multimedia component 608 includes a front camera and/or a rear camera. When the electronic device 600 is in an operational mode, such as a shooting mode or a multimedia mode, the front-facing camera and/or the rear-facing camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 610 is for outputting and/or inputting audio signals. For example, the audio component 610 includes a Microphone (MIC) for receiving external audio signals when the electronic device 600 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, audio component 610 further includes a speaker for outputting audio signals.
The I/O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 614 includes one or more sensors for providing status assessment of various aspects of the electronic device 600. For example, the sensor assembly 614 may detect an on/off state of the electronic device 600, a relative positioning of the components, such as a display and keypad of the electronic device 600, the sensor assembly 614 may also detect a change in position of the electronic device 600 or a component of the electronic device 600, the presence or absence of a user's contact with the electronic device 600, an orientation or acceleration/deceleration of the electronic device 600, and a change in temperature of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 616 is utilized to facilitate communication between the electronic device 600 and other devices, either in a wired or wireless manner. The electronic device 600 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 600 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for implementing a vibration image based device fault detection method provided by an embodiment of the invention.
In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as memory 604, including instructions executable by processor 620 of electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
Fig. 6 is a block diagram of an electronic device 700, according to an example embodiment. For example, the electronic device 700 may be provided as a server. Referring to fig. 6, electronic device 700 includes a processing component 722 that further includes one or more processors and memory resources represented by memory 732 for storing instructions, such as application programs, executable by processing component 722. The application programs stored in memory 732 may include one or more modules that each correspond to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform a vibration image-based device fault detection method provided by an embodiment of the present invention.
The electronic device 700 may also include a power supply component 726 configured to perform power management of the electronic device 700, a wired or wireless network interface 750 configured to connect the electronic device 700 to a network, and an input output (I/O) interface 758. The electronic device 700 may operate based on an operating system stored in memory 732, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It is to be understood that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. A method for detecting a device failure based on a vibration image, the method comprising:
acquiring a vibration image of the equipment;
extracting features of the vibration image to obtain a frequency amplitude histogram of the vibration image;
obtaining classification results aiming at the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model;
And determining whether the equipment has faults according to the classification result.
2. The method of claim 1, wherein the obtaining classification results for the vibration image and the frequency amplitude histogram from the vibration image, the frequency amplitude histogram, and a preset classification convolutional neural network model comprises:
fusing the image channel of the vibration image with the image channel of the frequency amplitude histogram to obtain a target fusion image;
and classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result.
3. The method of claim 2, wherein the vibration image and the frequency amplitude histogram are two-dimensional histograms; the fusing the image channel of the vibration image and the image channel of the frequency amplitude histogram to obtain a target fused image comprises the following steps:
obtaining a vibration image matrix collection comprising at least one vibration image matrix according to the image channel information of the vibration image, wherein a single vibration image matrix comprises all content information of an image channel of one vibration image;
Obtaining a frequency amplitude histogram matrix aggregate comprising at least one frequency amplitude histogram matrix according to the image channel information of the frequency amplitude histogram, wherein a single frequency amplitude histogram matrix comprises all content information of the image channel of one frequency amplitude histogram;
fusing the vibration image matrix set with the image channels in the frequency amplitude histogram matrix set to obtain a fused matrix set; wherein the fusion matrix aggregate includes all content information of the image channels of the vibration image and the image channels of the frequency amplitude histogram;
and generating the target fusion image according to the fusion matrix collection.
4. The method of claim 2, wherein classifying the convolutional neural network model comprises at least one classifier; the classifying the target fusion image through a preset classification convolutional neural network model to obtain a classification result comprises the following steps:
evaluating and scoring the target fusion image through a classifier in a classified convolutional neural network model;
obtaining an arithmetic average of all scoring results of the classifier;
the arithmetic average value is taken as a classification result.
5. The method of claim 4, wherein said determining whether a device is malfunctioning based on the classification result comprises:
if the arithmetic average value of the target fusion image in the classification result is larger than or equal to a first preset threshold value, determining that the equipment has faults;
and if the arithmetic average value of the target fusion image is smaller than a first preset threshold value, determining that the equipment has no fault.
6. The method of claim 1, wherein the vibration image comprises: a normal vibration image sample and an abnormal vibration image sample;
before extracting features of the vibration image and obtaining a frequency amplitude histogram of the vibration image, the method further includes:
and carrying out sample quantity allocation on the vibration image through a sample sampling function, so that the ratio of the sample quantity of the abnormal vibration image sample to the sample quantity of the normal vibration image sample in the vibration image is equal to a first preset proportion threshold value.
7. The method of claim 6, wherein after classifying the target fusion image by a predetermined classification convolutional neural network model to obtain a classification result, the method further comprises:
And model training is carried out on the classifier of the classified convolutional neural network model through a loss function, and calculation parameters used by the classifier in classifying the target fusion image are optimized.
8. A vibration image-based device failure detection apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a vibration image of the equipment;
the feature extraction module is used for extracting features of the vibration image and obtaining a frequency amplitude histogram of the vibration image;
the image classification module is used for obtaining classification results aiming at the vibration image and the frequency amplitude histogram according to the vibration image, the frequency amplitude histogram and a preset classification convolutional neural network model;
and the equipment fault determining module is used for determining whether equipment has faults or not according to the classification result.
9. An electronic device, comprising: a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 7.
10. A readable storage medium, characterized in that instructions in the readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733357.1A CN116188846A (en) | 2022-12-30 | 2022-12-30 | Equipment fault detection method and device based on vibration image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733357.1A CN116188846A (en) | 2022-12-30 | 2022-12-30 | Equipment fault detection method and device based on vibration image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116188846A true CN116188846A (en) | 2023-05-30 |
Family
ID=86451662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211733357.1A Pending CN116188846A (en) | 2022-12-30 | 2022-12-30 | Equipment fault detection method and device based on vibration image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116188846A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935245A (en) * | 2023-06-16 | 2023-10-24 | 国网山东省电力公司金乡县供电公司 | Long-distance communication power grid unmanned aerial vehicle inspection system and method |
-
2022
- 2022-12-30 CN CN202211733357.1A patent/CN116188846A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935245A (en) * | 2023-06-16 | 2023-10-24 | 国网山东省电力公司金乡县供电公司 | Long-distance communication power grid unmanned aerial vehicle inspection system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229369B (en) | Image shooting method and device, storage medium and electronic equipment | |
| CN110517185B (en) | Image processing method, device, electronic equipment and storage medium | |
| US20210103733A1 (en) | Video processing method, apparatus, and non-transitory computer-readable storage medium | |
| US11443438B2 (en) | Network module and distribution method and apparatus, electronic device, and storage medium | |
| CN113344901A (en) | Gluing defect detection method and device, storage medium and electronic equipment | |
| CN105138962A (en) | Image display method and image display device | |
| CN105678242B (en) | Focusing method and device under hand-held certificate mode | |
| CN110532956B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN106384348B (en) | The method for detecting abnormality and device of monitoring image | |
| CN109145970B (en) | Image-based question and answer processing method and device, electronic equipment and storage medium | |
| EP3975046B1 (en) | Method and apparatus for detecting occluded image and medium | |
| US11551465B2 (en) | Method and apparatus for detecting finger occlusion image, and storage medium | |
| CN107798314A (en) | Skin color detection method and device | |
| CN111435422B (en) | Action recognition method, control method and device, electronic equipment and storage medium | |
| CN109784327B (en) | Boundary box determining method and device, electronic equipment and storage medium | |
| CN111984803B (en) | Multimedia resource processing method and device, computer equipment and storage medium | |
| CN113139471A (en) | Target detection method and device, electronic equipment and storage medium | |
| CN115909127A (en) | Training method of abnormal video recognition model, abnormal video recognition method and device | |
| CN116188846A (en) | Equipment fault detection method and device based on vibration image | |
| CN112381091A (en) | Video content identification method and device, electronic equipment and storage medium | |
| CN110110742B (en) | Multi-feature fusion method and device, electronic equipment and storage medium | |
| CN109271863B (en) | Face living body detection method and device | |
| CN111507131A (en) | Living body detection method and apparatus, electronic device, and storage medium | |
| CN115408544A (en) | Image database construction method, device, equipment, storage medium and product | |
| CN110659726B (en) | Image processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |