CN111652094A - Railway flat car end plate falling fault image identification method based on deep learning - Google Patents
Railway flat car end plate falling fault image identification method based on deep learning Download PDFInfo
- Publication number
- CN111652094A CN111652094A CN202010437704.0A CN202010437704A CN111652094A CN 111652094 A CN111652094 A CN 111652094A CN 202010437704 A CN202010437704 A CN 202010437704A CN 111652094 A CN111652094 A CN 111652094A
- Authority
- CN
- China
- Prior art keywords
- end plate
- network
- image
- inclusion
- fault
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61D—BODY DETAILS OR KINDS OF RAILWAY VEHICLES
- B61D17/00—Construction details of vehicle bodies
- B61D17/04—Construction details of vehicle bodies with bodies of metal; with composite, e.g. metal and wood body structures
- B61D17/10—Floors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M17/00—Testing of vehicles
- G01M17/08—Railway vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Wood Science & Technology (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
Abstract
A railway flat car end plate falling fault image identification method based on deep learning belongs to the technical field of railway flat car end plate falling fault image identification. The invention solves the problems of low accuracy and efficiency of the existing railway flat car end plate falling fault identification. The method comprises the following specific steps: acquiring end plate images of the left side and the right side of a carriage; step two, processing the acquired image to obtain a sample data set; loading an inclusion v2 network model, inputting the obtained sample data set into an inclusion v2 network for training to obtain a trained inclusion v2 network; step four, optimizing the trained inclusion v2 network by using TensorRT; and step five, utilizing the optimized inclusion v2 network to perform fault identification on the acquired to-be-detected end plate image. The method can be applied to the identification of the falling fault of the end plate of the railway flat car.
Description
Technical Field
The invention belongs to the technical field of railway flat car end plate falling fault image identification, and particularly relates to a railway flat car end plate falling fault image identification method based on deep learning.
Background
At present, fault detection of trucks generally adopts a manual troubleshooting mode to carry out fault maintenance. Since the detection operation is greatly influenced by factors such as the quality of service, the responsibility and the labor intensity of the operators, the conditions of missing detection or simplified operation are easy to occur. And manual detection work efficiency is low, and in case the operation quality problem appears, be unfavorable for looking for the reason that produces the problem in the operation process and the time that the problem takes place.
The end plates are positioned at the head end and the tail end of a carriage of the railway flat car and are connected with a car body through car end hinges, and the end plates fall off when the hinges on one side or two sides are damaged. The single-side camera cannot directly shoot the complete image, so the double-side linear array camera needs to shoot the left image and the right image respectively, and then the images are detected respectively to judge the end plate falling fault.
And due to the influence of external factors such as a camera light source and weather, the original image may have the defects of overexposure, underexposure, uneven brightness and the like. This will directly affect the identification of the failure of the end plate falling off in the image, resulting in lower efficiency and accuracy of the identification of the failure of the end plate falling off in the image.
Disclosure of Invention
The invention aims to solve the problems of low accuracy and low efficiency of the existing identification of the falling fault of the end plate of the railway flat car, and provides a deep learning-based image identification method of the falling fault of the end plate of the railway flat car.
The technical scheme adopted by the invention for solving the technical problems is as follows: a railway flatbed end plate falling fault image identification method based on deep learning comprises the following steps:
firstly, respectively acquiring end plate images on the left side and the right side of a carriage by using a linear array camera;
step two, carrying out self-adaptive histogram equalization processing on the image acquired in the step one to obtain an image subjected to self-adaptive histogram equalization; amplifying the obtained image, and marking different areas of the image after amplification to obtain a sample data set;
loading an Inceposition v2 network model, inputting the sample data set obtained in the step two into an Inceposition v2 network for training, and obtaining a trained Inceposition v2 network after training for 500 epochs;
step four, optimizing the inclusion v2 network trained in the step three by using TensorRT to obtain an optimized inclusion v2 network;
and step five, utilizing the optimized inclusion v2 network to perform fault identification on the acquired to-be-detected end plate image.
The invention has the beneficial effects that: the invention provides a railway flat car end plate falling fault image identification method based on deep learning. The deep learning method is applied to component positioning and fault detection, and robustness and accuracy of the algorithm can be effectively improved. After the loaded network model is trained, the network is accelerated by adopting TensorRT during prediction, so that the prediction accuracy and the operation efficiency are ensured.
Drawings
Fig. 1 is a flowchart of an image recognition method for a railway flat car end plate falling fault based on deep learning according to the invention.
Detailed Description
The first embodiment is as follows: this embodiment will be described with reference to fig. 1. The method for identifying the railway flat car end plate falling fault image based on deep learning in the embodiment is realized by the following steps:
firstly, respectively acquiring end plate images on the left side and the right side of a carriage by using a linear array camera;
step two, carrying out self-adaptive histogram equalization processing on the image acquired in the step one to obtain an image subjected to self-adaptive histogram equalization; amplifying the obtained image, and marking different areas of the image after amplification to obtain a sample data set;
loading an Inceposition v2 network model, inputting the sample data set obtained in the step two into an Inceposition v2 network for training, and obtaining a trained Inceposition v2 network after training for 500 epochs;
step four, optimizing the inclusion v2 network trained in the step three by using a TensorRT (high performance model optimizer) to obtain an optimized inclusion v2 network;
and step five, utilizing the optimized inclusion v2 network to perform fault identification on the acquired to-be-detected end plate image.
With the great improvement of the processing performance of chip hardware, a foundation is provided for the complex computation of a deep network. The deep learning is widely applied to the field of image processing, and compared with the traditional mode, the deep learning integrates the feature learning into the process of establishing the model, so that the accuracy and efficiency of fault detection can be effectively improved.
Because the area occupied by the end plate falling fault in the end plate image is large, the deep learning SSD is adopted to detect whether the falling fault exists in the end plate image. Compared with the target detection networks such as FasterRCNN and Yolo, the SSD has higher accuracy and detection speed. In order to further improve the efficiency of the SSD network in the process of predicting the image, the tensorRT is adopted to accelerate the prediction process of the SSD network, and the prediction speed of the network is improved.
The second embodiment is as follows: the first difference between the present embodiment and the specific embodiment is: and in the second step, amplifying the obtained image in a mode of stretching, rotating and mirror image transformation.
The third concrete implementation mode: the second embodiment is different from the first embodiment in that: the inclusion v2 network trained in the third step is optimized by using TensorRT to obtain an optimized inclusion v2 network, which specifically comprises the following steps:
the Float32 data type is changed to the Float16 data type by TensorRT merging the convolutional and pooling layers of the inclusion v2 network.
The convolutional layer and the pooling layer are merged through TensorRT, the data type precision in the prediction process is reduced, the data type of Float32 is changed into the data type of Float16, less GPU video memory is occupied, and the calculation is faster.
The fourth concrete implementation mode: the third difference between the present embodiment and the specific embodiment is that: the method comprises the following steps of utilizing an optimized inclusion v2 network to identify faults of collected images of the end plate to be detected, and the specific process is as follows:
for the collected images of the end plate to be detected, after the images of the end plate to be detected are subjected to adaptive histogram equalization processing, the images subjected to the adaptive histogram equalization processing are input into an optimized inclusion v2 network, and the inclusion v2 network outputs the identification result of the end plate faults;
and if the identification result is the end plate fault, judging the position of the fault according to the fault detection frame position predicted by the inclusion v2 network, and judging the type of the fault according to the type probability predicted by the inclusion v2 network.
And after the fault detection of the image to be detected is finished, uploading the image to be detected to a network. For manual review. And the staff performs corresponding processing according to the image recognition result to ensure the safe operation of the locomotive.
The fifth concrete implementation mode: the first difference between the present embodiment and the specific embodiment is: before the image after the adaptive histogram equalization processing is input into the optimized inclusion v2 network, the images of the end plates to be identified need to be adjusted to a uniform size.
Examples
The following describes the specific implementation of the method of the present invention in detail:
step 1, end plate image preprocessing
The method has the advantages that images of the cameras on two sides are collected, self-adaptive histogram equalization is carried out on the images, image contrast is enhanced, and the image recognition effect is improved.
Common histogram equalization algorithms use the same histogram transformation for the pixels of the entire image, and the algorithm works well for those images where the distribution of pixel values is relatively equalized. Then, if the image includes portions that are significantly darker or lighter than other areas of the image, the contrast in these portions will not be effectively enhanced. Adaptive histogram equalization algorithms change the above problem by performing responsive histogram equalization on local regions. In its simplest form, each pixel is equalized by a histogram of pixels within a rectangular range of its perimeter. The equalization method is completely the same as the common equalization algorithm: the transformation function is proportional to the cumulative histogram function around the pixel. Pixels at the edges of the image require special handling because the field of edge pixels is not completely inside the image and needs to be addressed by mirroring the row or column pixels at the edges of the image.
Step 2, establishing a sample data set
And (3) acquiring a large number of images of the end plate through high-definition linear array camera equipment, and acquiring high-contrast end plate images through the step 1. And collecting images shot in different places, different times, different vehicle types and different weather conditions into a sample data set. The collected images include real failure images and normal endplate images. In order to enrich the variety and the quantity of samples, the collected images are subjected to transformation such as stretching, rotation, mirror image and the like, and the existing data set is amplified. The collection of the end plate images shot under different conditions is beneficial to enriching sample data, and the robustness and the adaptability of the training result are improved.
And marking the edge area of the end plate in a sub-area mode in the image of the end plate, wherein the sub-area mode comprises the edge area of the end plate, the area of a vehicle body, the area of a manual brake and the like, marking names for different areas according to whether faults exist in each area, and generating corresponding label files. And taking the end plate images and the corresponding label files as a training data set of the SSD target detection network.
Step 3, training the data set and calculating the weight
The feature extraction network of the SSD employs inclusion v 2. The inclusion v2 pre-trained network model was loaded for training. The Inception v2 has higher running speed compared with a feature extraction network such as the resnet.
And inputting a training set, and inputting the whole image into increment v2 for feature extraction. And after the training is finished, optimizing the trained model by using the tensorrT, and improving the prediction speed of the network. Data type accuracy in the prediction process is reduced by combining and replacing some network layers by the tensorRT, and the FP32 is changed into the FP 16.
Step 4, failure prediction
And after acquiring the images of the end plates, adjusting the images to be uniform in size, predicting, and judging whether faults exist and the types of the faults according to the bottom positions and the scores of the predicted fault frames.
Since the size of the image of the end plate is large and the size of the feature of the end plate falling off is also large, the size of the image can be reduced to improve the detection speed. And the tensorRT is used for optimizing a training model and the like, so that the program running efficiency can be further improved on the premise of ensuring the detection precision.
The above-described calculation examples of the present invention are merely to explain the calculation model and the calculation flow of the present invention in detail, and are not intended to limit the embodiments of the present invention. It will be apparent to those skilled in the art that other variations and modifications of the present invention can be made based on the above description, and it is not intended to be exhaustive or to limit the invention to the precise form disclosed, and all such modifications and variations are possible and contemplated as falling within the scope of the invention.
Claims (5)
1. A railway flatbed end plate falling fault image identification method based on deep learning is characterized by comprising the following steps:
firstly, respectively acquiring end plate images on the left side and the right side of a carriage by using a linear array camera;
step two, carrying out self-adaptive histogram equalization processing on the image acquired in the step one to obtain an image subjected to self-adaptive histogram equalization; amplifying the obtained image, and marking different areas of the image after amplification to obtain a sample data set;
loading an Incepotion v2 network model, inputting the sample data set obtained in the step two into an Incepotion v2 network for training, and obtaining a trained Incepotion v2 network after training for 500 epochs;
step four, optimizing the inclusion v2 network trained in the step three by using TensorRT to obtain an optimized inclusion v2 network;
and step five, utilizing the optimized inclusion v2 network to perform fault identification on the acquired to-be-detected end plate image.
2. The method for identifying the railway flat car end plate falling fault image based on the deep learning of claim 1, wherein in the second step, the obtained image is amplified, and the amplification mode comprises stretching, rotating and mirror image transformation of the image.
3. The deep learning-based image identification method for the end plate falling fault of the railway flat car according to claim 2, wherein the Incep v2 network trained in the third step is optimized by TensrT to obtain an optimized Incep v2 network, which specifically comprises:
the Float32 data type is changed to the Float16 data type by TensorRT merging the convolutional and pooling layers of the inclusion v2 network.
4. The method for identifying the railway flat car end plate falling fault image based on the deep learning of claim 3, wherein the collected end plate image to be detected is subjected to fault identification by using the optimized inclusion v2 network, and the specific process is as follows:
for the collected images of the end plate to be detected, after the images of the end plate to be detected are subjected to adaptive histogram equalization processing, the images subjected to the adaptive histogram equalization processing are input into an optimized inclusion v2 network, and the inclusion v2 network outputs the identification result of the end plate faults;
and if the identification result is the end plate fault, judging the position of the fault according to the fault detection frame position predicted by the inclusion v2 network, and judging the type of the fault according to the type probability predicted by the inclusion v2 network.
5. The method for identifying the image of the end plate falling fault of the railway flat car based on the deep learning of claim 1, wherein the image to be identified needs to be adjusted to a uniform size before the image subjected to the adaptive histogram equalization processing is input into an optimized inclusion v2 network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010437704.0A CN111652094A (en) | 2020-05-21 | 2020-05-21 | Railway flat car end plate falling fault image identification method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010437704.0A CN111652094A (en) | 2020-05-21 | 2020-05-21 | Railway flat car end plate falling fault image identification method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111652094A true CN111652094A (en) | 2020-09-11 |
Family
ID=72342754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010437704.0A Pending CN111652094A (en) | 2020-05-21 | 2020-05-21 | Railway flat car end plate falling fault image identification method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111652094A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017112591A1 (en) * | 2015-12-20 | 2017-06-29 | Prophecy Sensors, Llc | Machine fault detection based on a combination of sound capture and on spot feedback |
| CN110223266A (en) * | 2019-03-08 | 2019-09-10 | 湖南工业大学 | A kind of Railway wheelset tread damage method for diagnosing faults based on depth convolutional neural networks |
| CN111079820A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Image recognition-based rail wagon fire-proof plate fault recognition method |
| CN111079821A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Derailment automatic braking pull ring falling fault image identification method |
| CN111080601A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying fault image of pull ring grinding shaft of derailment brake device of railway wagon |
| CN111080614A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying damage to rim and tread of railway wagon wheel |
-
2020
- 2020-05-21 CN CN202010437704.0A patent/CN111652094A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017112591A1 (en) * | 2015-12-20 | 2017-06-29 | Prophecy Sensors, Llc | Machine fault detection based on a combination of sound capture and on spot feedback |
| CN110223266A (en) * | 2019-03-08 | 2019-09-10 | 湖南工业大学 | A kind of Railway wheelset tread damage method for diagnosing faults based on depth convolutional neural networks |
| CN111079820A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Image recognition-based rail wagon fire-proof plate fault recognition method |
| CN111079821A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Derailment automatic braking pull ring falling fault image identification method |
| CN111080601A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying fault image of pull ring grinding shaft of derailment brake device of railway wagon |
| CN111080614A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying damage to rim and tread of railway wagon wheel |
Non-Patent Citations (1)
| Title |
|---|
| 赵琳 等: "《海洋环境下的计算机视觉技术》", 31 October 2015, 国防工业出版社 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080620B (en) | Road disease detection method based on deep learning | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN111738342B (en) | Pantograph foreign matter detection method, storage medium and computer equipment | |
| CN111079820B (en) | Image recognition-based rail wagon fire-proof plate fault recognition method | |
| CN106683073B (en) | License plate detection method, camera and server | |
| CN111260629A (en) | Pantograph structure abnormity detection algorithm based on image processing | |
| CN109409404A (en) | A kind of high iron catenary radix saposhnikoviae bracing wire fault detection method based on deep learning | |
| CN104990925A (en) | Defect detecting method based on gradient multiple threshold value optimization | |
| CN111080650B (en) | Method for detecting looseness and loss faults of small part bearing blocking key nut of railway wagon | |
| CN111080614A (en) | Method for identifying damage to rim and tread of railway wagon wheel | |
| CN112966665A (en) | Pavement disease detection model training method and device and computer equipment | |
| CN116665080B (en) | Unmanned aerial vehicle deteriorated insulator detection method and system based on target recognition | |
| CN114926407A (en) | Steel surface defect detection system based on deep learning | |
| CN112419261B (en) | Visual acquisition method and device with abnormal point removing function | |
| CN115131353B (en) | Flat screen printing textile production abnormity identification and positioning method and system | |
| CN111476712B (en) | Trolley grate image shooting and detecting method and system of sintering machine | |
| CN114782410A (en) | Insulator defect detection method and system based on lightweight model | |
| CN114708532A (en) | Monitoring video quality evaluation method, system and storage medium | |
| CN112102280B (en) | Method for detecting loosening and loss faults of small part bearing key nut of railway wagon | |
| CN114549414A (en) | Abnormal change detection method and system for track data | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| CN113869433A (en) | Deep learning method for rapidly detecting and classifying concrete damage | |
| CN116542962A (en) | Improved Yolov5m model-based photovoltaic cell defect detection method | |
| CN113077423A (en) | Laser selective melting pool image analysis system based on convolutional neural network | |
| CN111652094A (en) | Railway flat car end plate falling fault image identification method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200911 |