CN111079821A - Derailment automatic braking pull ring falling fault image identification method - Google Patents
Derailment automatic braking pull ring falling fault image identification method Download PDFInfo
- Publication number
- CN111079821A CN111079821A CN201911272332.4A CN201911272332A CN111079821A CN 111079821 A CN111079821 A CN 111079821A CN 201911272332 A CN201911272332 A CN 201911272332A CN 111079821 A CN111079821 A CN 111079821A
- Authority
- CN
- China
- Prior art keywords
- layer
- pull ring
- output
- image
- automatic braking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The utility model provides a derailment automatic braking pull ring drops trouble image recognition method, relates to freight train and detects technical field, in order to solve the problem that the inefficiency and the rate of accuracy are low that current manual detection mode exists, include: acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images; step two, constructing a U-Net network model, and training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model; and step three, inputting the image to be detected into the trained U-Net network model, and judging that the derailment automatic braking pull ring falls off. A deep learning model is built by utilizing a convolutional neural network, automatic identification is carried out on the derailment automatic braking pull ring falling fault, and the identification precision and stability of an identification algorithm are improved.
Description
Technical Field
The invention relates to the technical field of freight train detection, in particular to an image identification method for a derailment automatic braking pull ring falling fault.
Background
Derailment automatic braking device has played a role effectively in many plays railway freight car derailment accidents, greatly reduced the loss that the derailment caused, the fault detection mode before looks over for the people's eye, but derailment automatic braking pull ring is located the freight car bottom, is subject to the vision angle, and the phenomenon of examining, the wrong detection very easily takes place to miss in the inspection process for the personnel of examining the car, seriously threatens the operation safety of freight car.
Therefore, in order to promote the rapid development of the automation degree of railway transportation, the problems of high cost, low efficiency and low accuracy of a manual detection mode are solved, and the realization of the automation of truck fault detection has great practical significance.
Disclosure of Invention
The purpose of the invention is: in order to solve the problem of low identification precision of the existing manual detection mode, the method for identifying the derailment automatic braking pull ring falling fault image is provided.
The technical scheme adopted by the invention to solve the technical problems is as follows:
an image identification method for derailment automatic braking pull ring falling fault comprises the following steps:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images;
step two, constructing a U-Net network model, and training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model;
and step three, inputting the image to be detected into the trained U-Net network model, and judging that the derailment automatic braking pull ring falls off.
Further, acquiring railway wagon images in the first step, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images; the specific process is as follows:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images;
step two, contrast enhancement processing is carried out on the image of the derailed automatic braking pull ring area to obtain an enhanced image of the derailed automatic braking pull ring;
step three, amplifying the gray level image by one or more of rotating, randomly cutting, horizontally turning, vertically turning, stretching and zooming methods to obtain all amplified images;
and step four, marking the pull ring contour in the image amplified in the step three by using a marking tool and generating a corresponding binary image, wherein the marked image and the image amplified in the step three form a sample training set.
Further, the specific steps of constructing a U-Net network model in the second step, and training the constructed U-Net network model by using a training set to obtain a trained deep learning model are as follows:
step two, constructing a U-Net network model:
the U-Net network comprises a contraction path and an expansion path, wherein the contraction path is used for capturing context information in the picture, the expansion path is used for accurately positioning the part needing to be segmented in the picture,
the constricted path comprises: a first layer: 64-channel convolution layer, ReLU activation function and post-connection pooling layer; a second layer: a 128-channel convolutional layer, a ReLU activation function, and a post-pooling layer; and a third layer: 256-channel convolutional layers, a ReLU activation function and a post-pooling layer;
the dilation path comprises: a first layer: upsampling, 256-channel convolutional layer, ReLU activation function; a second layer: upsampling, 128-channel convolutional layer, ReLU activation function; and a third layer: upsampling, 64-channel convolutional layer, ReLU activation function;
secondly, training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model; the specific process is as follows:
the training set image is used as the input of a 64-channel convolution layer in a contraction path and is output from an output layer of the 64-channel convolution layer, the output of the 64-channel convolution layer is used as the input of a first pooling layer after passing through a ReLU activation function and is output from the output layer of the first pooling layer, and the first down sampling is completed;
the output of the first pooling layer is used as the input of the 128-channel convolution layer in the contraction path and is output from the output layer of the 128-channel convolution layer, and the output of the 128-channel convolution layer is used as the input of the second pooling layer after being subjected to a ReLU activation function and is output from the output layer of the second pooling layer, so that second down sampling is completed;
the output of the second pooling layer is used as the input of the 256-channel convolution layer in the contraction path and is output from the output layer of the 256-channel convolution layer, and the output of the 256-channel convolution layer is used as the input of a third pooling layer after being subjected to a ReLU activation function and is output from the output layer of the third pooling layer, so that third down sampling is completed;
the output of the third pooling layer is used as the input of the 512-channel convolution layer and is output from the output layer of the 512-channel convolution layer, and the output of the 512-channel convolution layer is output after passing through a ReLU activation function;
the result of up-sampling output of the 512-channel convolution layer and the result of convolution processing output of the 256 channels in the contraction path are convolved by the 256-channel convolution layer in the expansion path and pass through a ReLU activation function to finish the first up-sampling;
the output of the first up-sampling process is up-sampled, and the result of convolution processing with the output of 128 channels in the contraction path is convoluted by the 128 channel convolution layer in the expansion path, and is subjected to a ReLU activation function to complete second up-sampling;
the output of the second up-sampling process is up-sampled, and the result of convolution processing is carried out on the output of 64 channels in the contraction path, and is convoluted by the 64-channel convolution layer in the expansion path, and is subjected to ReLU activation function to complete the third up-sampling and output a segmentation image;
the sizes of convolution kernels of 64-channel convolution layers, 128-channel convolution layers, 256-channel convolution layers and 512-channel convolution layers in the contraction path are 3 multiplied by 3;
the convolution kernel size of the 256-channel convolution layer, the 128-channel convolution layer and the 64-channel convolution layer in the expansion path is 3 multiplied by 3;
at the last layer of the network, a convolution operation is performed using a convolution kernel of 1 × 1, and each 64-dimensional feature vector is mapped to the output layer of the network.
Further, in the third step, the image to be detected is input into the trained U-Net network model, and the judgment of the derailment automatic braking pull ring falling is carried out; the specific process is as follows:
inputting the collected image of the derailed automatic braking pull ring into a trained U-Net network, predicting the outline of the derailed automatic braking pull ring in the image, converting the predicted image into a binary image, wherein a white area in the binary image is the outline area of the derailed automatic braking pull ring;
step two, calculating the rotation angle of the pull ring according to the profile of the pull ring in the obtained binary image;
if the angular offset of the rotation of the pull ring is larger than or equal to the set threshold, the derailed automatic braking pull ring is considered to fall off, and fault alarm information is output;
if the rotation angle offset of the pull ring is smaller than a set threshold value, the derailed automatic braking pull ring is considered to be not in fault;
step three: and judging whether an unprocessed image exists, if so, executing the step three I and the step three II again, and if not, ending.
Further, the loss function of the deep learning model is as follows:
wherein L is a cross entropy loss function,
predict distribution for current sample label, yiAnd (4) for the current sample label real distribution, i is the ith sample, and N is the total number of samples.
Further, the specific process for judging that the derailing automatic braking pull ring falls off is as follows:
firstly, inputting an acquired image of the derailed automatic braking pull ring into a U-Net network for training to obtain a weight coefficient output by training, then carrying out histogram equalization processing on a sub-image of the derailed automatic braking pull ring, predicting the outline of the derailed automatic braking pull ring in the image by using the weight coefficient output by training, converting the image into a binary image, calculating the rotation angle of the pull ring by using an image processing algorithm on the obtained binary image, if the angle offset is greater than a set threshold value, considering that the derailed automatic braking pull ring is fallen off, outputting fault alarm information, and if the angle offset is less than the set threshold value, considering that the derailed automatic braking pull ring is not in fault, and continuously processing the next image.
The invention has the beneficial effects that:
1. a deep learning model is built by utilizing a convolutional neural network, and the derailment automatic braking pull ring falling fault is automatically identified, so that the identification precision and stability of an identification algorithm are improved;
2. the artificial intelligence replaces the manual labor, so that the accuracy and the efficiency of the detection result are improved while the labor is saved;
3. on the basis of a classic U-Net structure, the model is improved, and the running speed of the model is increased;
4. the width and the height of the image are zoomed to a proper proportion, and the running speed of the model is increased;
5. and contrast enhancement processing is performed on the image, so that program identification precision is improved.
Drawings
Fig. 1 is a flowchart of identifying a tab failure of an automatic derailment brake.
Fig. 2 is a flowchart of weight coefficient calculation.
FIG. 3 is an improved U-Net network model.
Detailed Description
The first embodiment is as follows: specifically describing the present embodiment with reference to fig. 1 to 3, the derailment autobrake tab fall-off failure image recognition method according to the present embodiment includes the following steps:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images;
step two, constructing a U-Net network model, and training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model;
and step three, inputting the image to be detected into the trained U-Net network model, and judging that the derailment automatic braking pull ring falls off. 1. Building a training sample library
The high-definition image acquisition device built near the rail can acquire high-definition gray images of key parts of the truck. Because the gray scale of the derailing automatic braking pull ring changes along with the factors such as the color of the vehicle body, whether rain stain interference exists, the pollution of goods pulled by a carriage or oil stain pollution and the like, and images acquired by different acquisition devices are different, when a sample training library is established, the gray scale images of the pull ring in various states are collected to cover all conditions as much as possible, and the integrity of a data set is ensured.
Since the collected data may not cover the derailed autobrake tab image taken in all cases, the data is enhanced, which may include rotating, randomly cropping, flipping horizontally, flipping vertically, stretching, scaling, etc. the grayscale image to enhance the integrity of the data set.
And marking the pull ring outline in the image by using a marking tool, wherein the marked image and the original gray level image form a sample training set. The gray level image and the marked binary image are in one-to-one correspondence, namely the binary image is a filling image of the position outline of the pull ring in the gray level image, and the two images are consistent in size.
2. Network construction
The U-Net network structure consists of one systolic path and one dilated path, wherein the systolic path follows a typical convolutional network structure, the systolic path consists of two repeated 3 × 3 convolutional kernels (unfilled convolution), and each uses the ReLU activation function and one maximum pooling operation with step size 2 for downsampling (down-sample), and in each downsampling step, the number of feature channels doubles. In the dilation path, each step involves upsampling (up-sample) the feature map, and then performing a convolution operation (up-convolution) with a 3 × 3 convolution kernel to reduce the number of feature channels by half. And then cascading the corresponding cut characteristic graphs in the contraction path, and performing convolution operation by using two convolution kernels of 3 multiplied by 3, wherein the convolution kernels both use a ReLU activation function. Since there is a missing problem with the boundary pixels in each convolution operation, it is necessary to crop the feature map. In the last layer, convolution operations are performed using a 1 × 1 convolution kernel, and each 64-dimensional feature vector is mapped to an output layer of the network.
In order to improve the capability of a model for resisting noise interference in an image, a part of a classic U-Net structure for connecting a contraction path and an expansion path is improved, and a layer corresponding to the contraction path in the classic U-Net structure is directly connected with a layer corresponding to the expansion path after being subjected to convolution operation through a 3 x 3 convolution kernel. In order to increase the running speed of the model, the five-layer structure in the classic U-Net structure is reduced to a four-layer structure, and the corresponding number of channels is adjusted, as shown in fig. 3.
3. Model training
Histogram equalization processing is carried out on the derailment automatic braking pull ring subimage, the contrast ratio of the pull ring and the vehicle body background is enhanced, and the precision of subsequent segmentation can be improved. The width and the height of the acquired image of the derailing automatic braking pull ring are reduced to 1/2 of the original image, and the speed of model detection is improved on the basis of ensuring the segmentation precision.
U-Net uses a novel lossy weighting scheme for each pixel, so that the edges of the segmented object have higher weight. This loss weighting scheme helps the U-Net model to segment the outline of the derailed autobrake tab in the acquired grayscale image in a more robust manner.
The cross entropy loss function (binary _ cross _ entry) using the binary class is a special case of the multi-class (softmax _ cross _ entry), and when the class has only two classes in the multi-class, namely 0 or 1, the multi-class is called binary class.
Wherein L is a cross entropy loss function,
predict distribution for current sample label, yiThe label of the current sample is truly distributed, i is the ith sample, and N is the total number of the samples;
when y isiAnd
when equal, loss is 0; otherwise loss is a positive number; also, the greater the probability difference, the greater the loss.
The optimizer selects an Adam algorithm, can replace an optimization algorithm of the traditional random gradient descent process, and iteratively updates the neural network weight based on training data. Adam's algorithm differs from the traditional stochastic gradient descent, Adam designs independent adaptive learning rates for different parameters by computing first and second moment estimates of the gradient.
4. Derailment automatic braking pull ring falling fault identification
And inputting the collected image of the derailed automatic braking pull ring into a U-Net network for training to obtain a weight coefficient of training output. Histogram equalization processing is carried out on the derailment automatic braking pull ring subimage, the contrast ratio of the pull ring and the vehicle body background is enhanced, and the precision of subsequent segmentation can be improved. And predicting the outline of the derailing automatic braking pull ring in the image by using the weight coefficient output by training, and converting the image into a binary image, wherein a white area in the binary image is the outline area of the derailing automatic braking pull ring. The normal pull ring position is in the horizontal state, and the angle is 0. If the tab is in the disengaged state, an angular deviation occurs in the horizontal direction. And calculating the rotation angle of the pull ring by using an image processing algorithm on the obtained binary image, if the angle offset is greater than a set threshold value, considering that the derailing automatic braking pull ring falls off, outputting fault alarm information, and if the angle offset is less than the set threshold value, considering that the derailing automatic braking pull ring does not have a fault, and continuously processing the next image.
The second embodiment is as follows: the embodiment is a further description of the first specific embodiment, and the difference between the first specific embodiment and the first specific embodiment is that the railway wagon image is collected in the first step, the derailment automatic braking pull ring area is determined from the collected railway wagon image, and the derailment automatic braking pull ring area image is extracted; constructing a training set by using the extracted images; the specific process is as follows:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images;
performing gray processing on the derailed automatic braking pull ring area image to obtain a derailed automatic braking pull ring area gray image;
step three, amplifying the gray level image by one or more of rotating, randomly cutting, horizontally turning, vertically turning, stretching and zooming methods to obtain all amplified images;
and step four, marking the pull ring contour in the image amplified in the step three by using a marking tool and generating a corresponding binary image, wherein the marked image and the image amplified in the step three form a sample training set.
The high-definition image acquisition device built near the rail can acquire high-definition gray images of key parts of the truck. Because the gray scale of the derailing automatic braking pull ring changes along with the factors such as the color of the vehicle body, whether rain stain interference exists, the pollution of goods pulled by a carriage or oil stain pollution and the like, and images acquired by different acquisition devices are different, when a sample training library is established, the gray scale images of the pull ring in various states are acquired, all conditions are covered as much as possible, and the integrity of a data set is ensured.
Since the collected data may not cover the derailed autobrake tab image taken in all cases, the data is enhanced, which may include rotating, randomly cropping, flipping horizontally, flipping vertically, stretching, scaling, etc. the grayscale image to enhance the integrity of the data set.
The original gray level image and the marked binary image are in one-to-one correspondence, namely the marked binary image is a filling image of the position outline of the pull ring in the original gray level image, and the two images are consistent in size.
Amplifying the image by adopting one or more of image rotation, mirror image, zooming and contrast enhancement methods to obtain all amplified images;
and carrying out size normalization on all the amplified images, and taking the normalization result as a training set of the deep learning model.
The third concrete implementation mode: the embodiment is further described with respect to the first embodiment, and the difference between the embodiment and the first embodiment is that the U-Net network model is constructed in the second step, and the constructed U-Net network model is trained by using a training set, so as to obtain a trained deep learning model, which specifically includes:
step two, constructing a U-Net network model:
the U-Net network comprises a contraction path and an expansion path, wherein the contraction path is used for capturing context information in the picture, the expansion path is used for accurately positioning the part needing to be segmented in the picture,
the constricted path comprises: a first layer: 64-channel convolution layer, ReLU activation function and post-connection pooling layer; a second layer: a 128-channel convolutional layer, a ReLU activation function, and a post-pooling layer; and a third layer: 256-channel convolutional layers, a ReLU activation function and a post-pooling layer;
the dilation path comprises: a first layer: upsampling, 256-channel convolutional layer, ReLU activation function; a second layer: upsampling, 128-channel convolutional layer, ReLU activation function; and a third layer: upsampling, 64-channel convolutional layer, ReLU activation function;
secondly, training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model; the specific process is as follows:
the training set image is used as the input of a 64-channel convolution layer in a contraction path and is output from an output layer of the 64-channel convolution layer, the output of the 64-channel convolution layer is used as the input of a first pooling layer after passing through a ReLU activation function and is output from the output layer of the first pooling layer, and the first down sampling is completed;
the output of the first pooling layer is used as the input of the 128-channel convolution layer in the contraction path and is output from the output layer of the 128-channel convolution layer, and the output of the 128-channel convolution layer is used as the input of the second pooling layer after being subjected to a ReLU activation function and is output from the output layer of the second pooling layer, so that second down sampling is completed;
the output of the second pooling layer is used as the input of the 256-channel convolution layer in the contraction path and is output from the output layer of the 256-channel convolution layer, and the output of the 256-channel convolution layer is used as the input of a third pooling layer after being subjected to a ReLU activation function and is output from the output layer of the third pooling layer, so that third down sampling is completed;
the output of the third pooling layer is used as the input of the 512-channel convolution layer and is output from the output layer of the 512-channel convolution layer, and the output of the 512-channel convolution layer is output after passing through a ReLU activation function;
the result of up-sampling output of the 512-channel convolution layer and the result of convolution processing output of the 256 channels in the contraction path are convolved by the 256-channel convolution layer in the expansion path and pass through a ReLU activation function to finish the first up-sampling;
the output of the first up-sampling process is up-sampled, and the result of convolution processing with the output of 128 channels in the contraction path is convoluted by the 128 channel convolution layer in the expansion path, and is subjected to a ReLU activation function to complete second up-sampling;
the output of the second up-sampling process is up-sampled, and the result of convolution processing is carried out on the output of 64 channels in the contraction path, and is convoluted by the 64-channel convolution layer in the expansion path, and is subjected to ReLU activation function to complete the third up-sampling and output a segmentation image;
the sizes of convolution kernels of 64-channel convolution layers, 128-channel convolution layers, 256-channel convolution layers and 512-channel convolution layers in the contraction path are 3 multiplied by 3;
the convolution kernel size of the 256-channel convolution layer, the 128-channel convolution layer and the 64-channel convolution layer in the expansion path is 3 multiplied by 3;
in the last layer of the network, carrying out convolution operation by using a convolution kernel of 1 multiplied by 1, and mapping each 64-dimensional feature vector to an output layer of the network;
and step three, selecting an Adam optimizer to optimize the weight, and obtaining a trained U-Net weight coefficient.
In the last layer of the network, carrying out convolution operation by using a convolution kernel of 1 multiplied by 1, and mapping each 64-dimensional feature vector to an output layer of the network;
wherein the systolic path follows a typical convolutional network structure, the systolic path consists of two repeated 3 × 3 convolution kernels (unfilled convolution), and both use the ReLU activation function and one maximum pooling operation with step size of 2 for downsampling (down-sample), and the number of feature channels is doubled in each downsampling step. In the dilation path, each step involves upsampling (up-sample) the feature map, and then performing a convolution operation (up-convolution) with a 3 × 3 convolution kernel to reduce the number of feature channels by half. And then cascading the corresponding cut characteristic graphs in the contraction path, and performing convolution operation by using two convolution kernels of 3 multiplied by 3, wherein the convolution kernels both use a ReLU activation function. Since there is a missing problem with the boundary pixels in each convolution operation, it is necessary to crop the feature map. In the last layer, convolution operations are performed using a 1 × 1 convolution kernel, and each 64-dimensional feature vector is mapped to an output layer of the network.
U-Net uses a novel lossy weighting scheme for each pixel, so that the edges of the segmented object have higher weight. This loss weighting scheme helps the U-Net model to segment the outline of the derailed autobrake tab in the acquired grayscale image in a more robust manner.
The cross entropy loss function (binary _ cross _ entry) using the binary class in the training is a special case of the multi-class (softmax _ cross _ entry), and when the class has only two classes in the multi-class, namely 0 or 1, the multi-class is the binary class.
Wherein L is a cross entropy loss function,
predict distribution for current sample label, yiThe label of the current sample is truly distributed, i is the ith sample, and N is the total number of the samples;
when y isiAnd
when equal, loss is 0; otherwise loss is a positive number; also, the greater the probability difference, the greater the loss.
The optimizer selects an Adam algorithm, can replace an optimization algorithm of the traditional random gradient descent process, and iteratively updates the neural network weight based on training data. Adam's algorithm differs from the traditional stochastic gradient descent, Adam designs independent adaptive learning rates for different parameters by computing first and second moment estimates of the gradient. The method comprises the steps of obtaining predicted image data through the action of initialized weights and input image data, calculating loss values through a cross entropy loss function, optimizing the weights through an Adam optimizer, calculating new weight coefficients, updating the weight coefficients and completing one training iteration. The training program will repeat the process, and iterate all the images for a specified number of times, and in the iterative process, when the loss function becomes low, the weight is updated until the optimal weight coefficient is found.
The fourth concrete implementation mode: the embodiment is further described with respect to the first embodiment, and the difference between the first embodiment and the second embodiment is that in the third step, the image to be tested is input into the trained U-Net network model, and the derailment automatic braking pull ring is determined to be fallen off; the specific process is as follows:
inputting the collected image of the derailed automatic braking pull ring into a trained U-Net network, predicting the outline of the derailed automatic braking pull ring in the image, converting the predicted image into a binary image, wherein a white area in the binary image is the outline area of the derailed automatic braking pull ring;
step two, calculating the rotation angle of the pull ring according to the profile of the pull ring in the obtained binary image;
if the angular offset of the rotation of the pull ring is larger than or equal to the set threshold, the derailed automatic braking pull ring is considered to fall off, and fault alarm information is output;
if the rotation angle offset of the pull ring is smaller than a set threshold value, the derailed automatic braking pull ring is considered to be not in fault;
step three: and judging whether an unprocessed image exists, if so, executing the step three I and the step three II again, and if not, ending.
The normal pull ring position is in the horizontal state, and the angle is 0. If the tab is in the disengaged state, an angular deviation occurs in the horizontal direction.
The fifth concrete implementation mode: the present embodiment is further described with respect to the first embodiment, and the difference between the present embodiment and the first embodiment is that the loss function of the deep learning model is:
wherein L is a cross entropy loss function,
predict distribution for current sample label, yiAnd (4) for the current sample label real distribution, i is the ith sample, and N is the total number of samples.
It should be noted that the detailed description is only for explaining and explaining the technical solution of the present invention, and the scope of protection of the claims is not limited thereby. It is intended that all such modifications and variations be included within the scope of the invention as defined in the following claims and the description.
Claims (6)
1. A derailment automatic braking pull ring falling fault image identification method is characterized by comprising the following steps:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images;
step two, constructing a U-Net network model, and training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model;
and step three, inputting the image to be detected into the trained U-Net network model, and judging that the derailment automatic braking pull ring falls off.
2. The method for recognizing the derailment autobrake pull-ring fall-off fault image according to claim 1, wherein: acquiring railway wagon images in the first step, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images; constructing a training set by using the extracted images; the specific process is as follows:
acquiring railway wagon images, determining derailment automatic braking pull ring areas from the acquired railway wagon images, and extracting the derailment automatic braking pull ring area images;
step two, contrast enhancement processing is carried out on the image of the derailed automatic braking pull ring area to obtain an enhanced image of the derailed automatic braking pull ring;
step three, amplifying the gray level image by one or more of rotating, randomly cutting, horizontally turning, vertically turning, stretching and zooming methods to obtain all amplified images;
and step four, marking the pull ring contour in the image amplified in the step three by using a marking tool and generating a corresponding binary image, wherein the marked image and the image amplified in the step three form a sample training set.
3. The method for recognizing the derailment autobrake pull-ring fall-off fault image according to claim 1, wherein: the second step of constructing a U-Net network model, and training the constructed U-Net network model by using a training set to obtain a trained deep learning model comprises the following specific steps:
step two, constructing a U-Net network model:
the U-Net network comprises a contraction path and an expansion path, wherein the contraction path is used for capturing context information in the picture, the expansion path is used for accurately positioning the part needing to be segmented in the picture,
the constricted path comprises: a first layer: 64-channel convolution layer, ReLU activation function and post-connection pooling layer; a second layer: a 128-channel convolutional layer, a ReLU activation function, and a post-pooling layer; and a third layer: 256-channel convolutional layers, a ReLU activation function and a post-pooling layer;
the dilation path comprises: a first layer: upsampling, 256-channel convolutional layer, ReLU activation function; a second layer: upsampling, 128-channel convolutional layer, ReLU activation function; and a third layer: upsampling, 64-channel convolutional layer, ReLU activation function;
secondly, training the constructed U-Net network model by using a training set to obtain a well-trained deep learning model; the specific process is as follows:
the training set image is used as the input of a 64-channel convolution layer in a contraction path and is output from an output layer of the 64-channel convolution layer, the output of the 64-channel convolution layer is used as the input of a first pooling layer after passing through a ReLU activation function and is output from the output layer of the first pooling layer, and the first down sampling is completed;
the output of the first pooling layer is used as the input of the 128-channel convolution layer in the contraction path and is output from the output layer of the 128-channel convolution layer, and the output of the 128-channel convolution layer is used as the input of the second pooling layer after being subjected to a ReLU activation function and is output from the output layer of the second pooling layer, so that second down sampling is completed;
the output of the second pooling layer is used as the input of the 256-channel convolution layer in the contraction path and is output from the output layer of the 256-channel convolution layer, and the output of the 256-channel convolution layer is used as the input of a third pooling layer after being subjected to a ReLU activation function and is output from the output layer of the third pooling layer, so that third down sampling is completed;
the output of the third pooling layer is used as the input of the 512-channel convolution layer and is output from the output layer of the 512-channel convolution layer, and the output of the 512-channel convolution layer is output after passing through a ReLU activation function;
the result of up-sampling output of the 512-channel convolution layer and the result of convolution processing output of the 256 channels in the contraction path are convolved by the 256-channel convolution layer in the expansion path and pass through a ReLU activation function to finish the first up-sampling;
the output of the first up-sampling process is up-sampled, and the result of convolution processing with the output of 128 channels in the contraction path is convoluted by the 128 channel convolution layer in the expansion path, and is subjected to a ReLU activation function to complete second up-sampling;
the output of the second up-sampling process is up-sampled, and the result of convolution processing is carried out on the output of 64 channels in the contraction path, and is convoluted by the 64-channel convolution layer in the expansion path, and is subjected to ReLU activation function to complete the third up-sampling and output a segmentation image;
the sizes of convolution kernels of 64-channel convolution layers, 128-channel convolution layers, 256-channel convolution layers and 512-channel convolution layers in the contraction path are 3 multiplied by 3;
the convolution kernel size of the 256-channel convolution layer, the 128-channel convolution layer and the 64-channel convolution layer in the expansion path is 3 multiplied by 3;
at the last layer of the network, a convolution operation is performed using a convolution kernel of 1 × 1, and each 64-dimensional feature vector is mapped to the output layer of the network.
4. The method for recognizing the derailment autobrake pull-ring fall-off fault image according to claim 1, wherein: inputting the image to be tested into the trained U-Net network model in the third step, and judging the derailment automatic braking pull ring falling; the specific process is as follows:
inputting the collected image of the derailed automatic braking pull ring into a trained U-Net network, predicting the outline of the derailed automatic braking pull ring in the image, converting the predicted image into a binary image, wherein a white area in the binary image is the outline area of the derailed automatic braking pull ring;
step two, calculating the rotation angle of the pull ring according to the profile of the pull ring in the obtained binary image;
if the angular offset of the rotation of the pull ring is larger than or equal to the set threshold, the derailed automatic braking pull ring is considered to fall off, and fault alarm information is output;
if the rotation angle offset of the pull ring is smaller than a set threshold value, the derailed automatic braking pull ring is considered to be not in fault;
step three: and judging whether an unprocessed image exists, if so, executing the step three I and the step three II again, and if not, ending.
5. The method for recognizing the derailment autobrake pull-ring fall-off fault image according to claim 1, wherein: the loss function of the deep learning model is as follows:
wherein L is a cross entropy loss function,
predict distribution for current sample label, yiAnd (4) for the current sample label real distribution, i is the ith sample, and N is the total number of samples.
6. The method for recognizing the derailment autobrake pull-ring fall-off fault image according to claim 2, wherein: the specific process for judging the derailment automatic braking pull ring falling off comprises the following steps:
firstly, inputting an acquired image of the derailed automatic braking pull ring into a U-Net network for training to obtain a weight coefficient output by training, then carrying out histogram equalization processing on a sub-image of the derailed automatic braking pull ring, predicting the outline of the derailed automatic braking pull ring in the image by using the weight coefficient output by training, converting the image into a binary image, calculating the rotation angle of the pull ring by using an image processing algorithm on the obtained binary image, if the angle offset is greater than a set threshold value, considering that the derailed automatic braking pull ring is fallen off, outputting fault alarm information, and if the angle offset is less than the set threshold value, considering that the derailed automatic braking pull ring is not in fault, and continuously processing the next image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911272332.4A CN111079821A (en) | 2019-12-12 | 2019-12-12 | Derailment automatic braking pull ring falling fault image identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911272332.4A CN111079821A (en) | 2019-12-12 | 2019-12-12 | Derailment automatic braking pull ring falling fault image identification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111079821A true CN111079821A (en) | 2020-04-28 |
Family
ID=70314059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911272332.4A Pending CN111079821A (en) | 2019-12-12 | 2019-12-12 | Derailment automatic braking pull ring falling fault image identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111079821A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652094A (en) * | 2020-05-21 | 2020-09-11 | 哈尔滨市科佳通用机电股份有限公司 | Railway flat car end plate falling fault image identification method based on deep learning |
| CN112152633A (en) * | 2020-09-30 | 2020-12-29 | 石家庄铁道大学 | Vibration signal compression method and device |
| CN112347952A (en) * | 2020-11-11 | 2021-02-09 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon coupler tail frame supporting plate falling-off identification method |
| CN112734780A (en) * | 2020-12-25 | 2021-04-30 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying deformation fault of pull ring of derailment automatic braking pull ring device |
| CN114549865A (en) * | 2022-02-24 | 2022-05-27 | 中铁第四勘察设计院集团有限公司 | Method and system for automatically detecting state of skirtboard of motor train unit based on computer vision |
| CN114581861A (en) * | 2022-03-02 | 2022-06-03 | 北京交通大学 | Track area identification method based on deep learning convolutional neural network |
| CN115973125A (en) * | 2023-02-15 | 2023-04-18 | 慧铁科技有限公司 | Method for processing fault of automatic derailment braking device of railway wagon |
| CN116091652A (en) * | 2023-01-20 | 2023-05-09 | 哈尔滨市科佳通用机电股份有限公司 | Image generation and detection method for pull ring falling fault of derailment automatic device |
| CN116246114A (en) * | 2023-03-14 | 2023-06-09 | 哈尔滨市科佳通用机电股份有限公司 | Method and device for detecting pull ring falling image abnormality of self-supervision derailment automatic device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| CN107886081A (en) * | 2017-11-23 | 2018-04-06 | 武汉理工大学 | Two-way U Net deep neural network mine down-holes hazardous act is intelligently classified discrimination method |
| CN107958271A (en) * | 2017-12-06 | 2018-04-24 | 电子科技大学 | The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN108510502A (en) * | 2018-03-08 | 2018-09-07 | 华南理工大学 | Melanoma picture tissue segmentation methods based on deep neural network and system |
| CN109035252A (en) * | 2018-06-29 | 2018-12-18 | 山东财经大学 | A kind of super-pixel method towards medical image segmentation |
| CN109711409A (en) * | 2018-11-15 | 2019-05-03 | 天津大学 | A kind of hand-written music score spectral line delet method of combination U-net and ResNet |
| WO2019200740A1 (en) * | 2018-04-20 | 2019-10-24 | 平安科技（深圳）有限公司 | Pulmonary nodule detection method and apparatus, computer device, and storage medium |
-
2019
- 2019-12-12 CN CN201911272332.4A patent/CN111079821A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016665A (en) * | 2017-02-16 | 2017-08-04 | 浙江大学 | A kind of CT pulmonary nodule detection methods based on depth convolutional neural networks |
| CN107977671A (en) * | 2017-10-27 | 2018-05-01 | 浙江工业大学 | A kind of tongue picture sorting technique based on multitask convolutional neural networks |
| CN107886081A (en) * | 2017-11-23 | 2018-04-06 | 武汉理工大学 | Two-way U Net deep neural network mine down-holes hazardous act is intelligently classified discrimination method |
| CN107958271A (en) * | 2017-12-06 | 2018-04-24 | 电子科技大学 | The cutaneous lesions deep learning identifying system of Analysis On Multi-scale Features based on expansion convolution |
| CN108510502A (en) * | 2018-03-08 | 2018-09-07 | 华南理工大学 | Melanoma picture tissue segmentation methods based on deep neural network and system |
| WO2019200740A1 (en) * | 2018-04-20 | 2019-10-24 | 平安科技（深圳）有限公司 | Pulmonary nodule detection method and apparatus, computer device, and storage medium |
| CN109035252A (en) * | 2018-06-29 | 2018-12-18 | 山东财经大学 | A kind of super-pixel method towards medical image segmentation |
| CN109711409A (en) * | 2018-11-15 | 2019-05-03 | 天津大学 | A kind of hand-written music score spectral line delet method of combination U-net and ResNet |
Non-Patent Citations (1)
| Title |
|---|
| 卢婷: "基于几何与形状特征的列车故障图像匹配算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652094A (en) * | 2020-05-21 | 2020-09-11 | 哈尔滨市科佳通用机电股份有限公司 | Railway flat car end plate falling fault image identification method based on deep learning |
| CN112152633A (en) * | 2020-09-30 | 2020-12-29 | 石家庄铁道大学 | Vibration signal compression method and device |
| CN112347952A (en) * | 2020-11-11 | 2021-02-09 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon coupler tail frame supporting plate falling-off identification method |
| CN112347952B (en) * | 2020-11-11 | 2021-05-11 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon coupler tail frame supporting plate falling-off identification method |
| CN112734780A (en) * | 2020-12-25 | 2021-04-30 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying deformation fault of pull ring of derailment automatic braking pull ring device |
| CN112734780B (en) * | 2020-12-25 | 2021-09-14 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying deformation fault of pull ring of derailment automatic braking pull ring device |
| CN114549865A (en) * | 2022-02-24 | 2022-05-27 | 中铁第四勘察设计院集团有限公司 | Method and system for automatically detecting state of skirtboard of motor train unit based on computer vision |
| CN114581861A (en) * | 2022-03-02 | 2022-06-03 | 北京交通大学 | Track area identification method based on deep learning convolutional neural network |
| CN116091652A (en) * | 2023-01-20 | 2023-05-09 | 哈尔滨市科佳通用机电股份有限公司 | Image generation and detection method for pull ring falling fault of derailment automatic device |
| CN115973125A (en) * | 2023-02-15 | 2023-04-18 | 慧铁科技有限公司 | Method for processing fault of automatic derailment braking device of railway wagon |
| CN116246114A (en) * | 2023-03-14 | 2023-06-09 | 哈尔滨市科佳通用机电股份有限公司 | Method and device for detecting pull ring falling image abnormality of self-supervision derailment automatic device |
| CN116246114B (en) * | 2023-03-14 | 2023-10-10 | 哈尔滨市科佳通用机电股份有限公司 | Method and device for detecting pull ring falling image abnormality of self-supervision derailment automatic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079821A (en) | Derailment automatic braking pull ring falling fault image identification method | |
| CN112967243B (en) | Deep learning chip packaging crack defect detection method based on YOLO | |
| CN107609525B (en) | Remote sensing image target detection method for constructing convolutional neural network based on pruning strategy | |
| CN110555368B (en) | Fall-down behavior identification method based on three-dimensional convolutional neural network | |
| CN110414417B (en) | Traffic sign board identification method based on multi-level fusion multi-scale prediction | |
| CN112232349A (en) | Model training method, image segmentation method and device | |
| CN111079631A (en) | Method and system for identifying falling fault of hook lifting rod of railway wagon | |
| CN109886200B (en) | Unmanned lane line detection method based on generative confrontation network | |
| CN112308826B (en) | Bridge structure surface defect detection method based on convolutional neural network | |
| CN106407903A (en) | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method | |
| CN108009518A (en) | A kind of stratification traffic mark recognition methods based on quick two points of convolutional neural networks | |
| CN109117788A (en) | A kind of public transport compartment crowding detection method merging ResNet and LSTM | |
| CN112488025B (en) | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion | |
| CN112464911A (en) | Improved YOLOv 3-tiny-based traffic sign detection and identification method | |
| CN112699967B (en) | Remote airport target detection method based on improved deep neural network | |
| CN113255837A (en) | Improved CenterNet network-based target detection method in industrial environment | |
| CN111080600A (en) | Fault identification method for split pin on spring supporting plate of railway wagon | |
| CN116485717B (en) | Concrete dam surface crack detection method based on pixel-level deep learning | |
| CN114841972A (en) | Power transmission line defect identification method based on saliency map and semantic embedded feature pyramid | |
| CN116110036B (en) | Electric power nameplate information defect level judging method and device based on machine vision | |
| Dorbe et al. | FCN and LSTM based computer vision system for recognition of vehicle type, license plate number, and registration country | |
| CN112733747A (en) | Identification method, system and device for relieving falling fault of valve pull rod | |
| CN116129291A (en) | Unmanned aerial vehicle animal husbandry-oriented image target recognition method and device | |
| CN110991374B (en) | Fingerprint singular point detection method based on RCNN | |
| CN111462090A (en) | Multi-scale image target detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200428 |
|
| RJ01 | Rejection of invention patent application after publication |