CN115527170A - Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device - Google Patents
Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device Download PDFInfo
- Publication number
- CN115527170A CN115527170A CN202211260741.4A CN202211260741A CN115527170A CN 115527170 A CN115527170 A CN 115527170A CN 202211260741 A CN202211260741 A CN 202211260741A CN 115527170 A CN115527170 A CN 115527170A
- Authority
- CN
- China
- Prior art keywords
- image
- braking device
- automatic
- fault
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 94
- 238000012549 training Methods 0.000 claims description 43
- 238000012216 screening Methods 0.000 claims description 25
- 238000011176 pooling Methods 0.000 claims description 13
- 230000003321 amplification Effects 0.000 claims description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 7
- 150000007523 nucleic acids Chemical class 0.000 claims description 3
- 102000039446 nucleic acids Human genes 0.000 claims description 3
- 108020004707 nucleic acids Proteins 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000001629 suppression Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000011002 quantification Methods 0.000 description 2
- 208000006440 Open Bite Diseases 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/52—Scale-space analysis, e.g. wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for identifying closing faults of a plug handle of an automatic derailment braking device of a truck, relates to a method and a system for detecting faults of parts of the braking device based on a fault detection model, and aims to solve the problems that the plug handle and the fault and omission detection problems of the automatic derailment braking valve device are manually detected at present, and the method specifically comprises the following steps: s1, acquiring an image to be detected containing a cock handle component of an automatic derailment braking device; s2, detecting the plug handle part of the derail automatic braking device in the image to be detected by using a fault detection model, detecting to obtain the position of the plug handle part of the derail automatic braking device, judging whether the plug handle part of the derail automatic braking device is in a closed state or not, and executing a third step if the plug handle part of the derail automatic braking device is in the closed state; s3, generating fault information; the fault information includes the position of the door handle member of the derailment automatic brake.
Description
Technical Field
The invention relates to a method and a system for detecting faults of brake device components based on a fault detection model.
Background
Train derailment is a very serious accident in the running process of railway vehicles, and in order to avoid the occurrence of the accident, a derailment automatic braking device is often arranged on a railway wagon. However, in the application, the plug door handle closing fault of the derailment automatic brake valve device caused by the misoperation of related maintainers is found, so that not only can the basic performance of the railway wagon be influenced, but also the potential safety hazard can be brought to the running of the wagon. The existing car inspection operation mode of manually looking at the pictures one by one has the problems of influence of personnel quality and responsibility, error and omission detection, difficulty in ensuring the operation quality, huge labor cost, low efficiency and the like.
Disclosure of Invention
The invention aims to solve the problems that the conventional manual detection of the derailment automatic brake valve device has a door plugging hand and the detection is wrong and missed, and provides a method and a system for identifying the closing fault of the door plugging hand of the freight car derailment automatic brake device.
The invention provides a method for identifying a closing fault of a cock handle of a freight car derailment automatic braking device, which specifically comprises the following steps:
s1, acquiring an image to be detected containing a cock handle part of an automatic derailing braking device;
s2, detecting the plug handle part of the derail automatic brake device in the image to be detected by using a fault detection model, detecting to obtain the position of the plug handle part of the derail automatic brake device, judging whether the plug handle part of the derail automatic brake device is in a closed state or not,
if the door closing handle part of the derailing automatic braking device is in a closed state, executing a step three;
the fault detection model is Faster R-CNN, and the characteristic graph is pooled by adopting multi-scale pooling RoIAlign in the Faster R-CNN;
s3, generating fault information;
the fault information includes the position of the door handle member of the derailment automatic brake.
The training method of the fault detection model in the step two is as follows:
s21, acquiring a local area image containing a door closing hand component of the derailing automatic braking device, marking the door closing hand component of the derailing automatic braking device, and generating an original sample data set;
s22, amplifying the original sample data set to obtain a training data set;
s23, training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
In step 22, the amplification method is to amplify by improving the mosaic data enhancement method, and the specific steps are as follows
S221, randomly selecting 4 local area images in the original sample data set to carry out 2 x 2-standard splicing to obtain a spliced image m 4 (ii) a 2 x 2 standard stitching is that each row and each column in a stitched image are stitched with 2 local area images;
s222, randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3 standard stitching is that 3 local area images are stitched in each row and each column in the stitched image;
s223, when the image m 1 Splicing image m 4 And stitching the image m 9 After the quantity of the target nucleic acid reaches a set proportion, completing amplification;
the image m1 is a local area image without stitching.
In the second step, a screening confidence score S is obtained through a Gaussian attenuation score formula i ', and according to a screening confidence score S i ' screening overlapping prediction boxes; the prediction frame is generated in the fault detection model detection process;
wherein, sigma is a hyper-parameter, i is a category number, M is a prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence score i D is the set of processed prediction frames, S i Is the raw confidence score of the prediction box being compared.
In step S21, a specific method for acquiring a local area image is as follows:
step 211, performing line scanning on the bottom of the truck, and splicing to obtain a large truck bottom image;
and step 212, preliminarily positioning the plug handle position of the derailing automatic braking device by utilizing the truck wheel base information and the truck type information, and capturing a local area image containing the plug handle part of the derailing automatic braking device from a large truck bottom image.
The invention also provides a system for identifying the closing fault of the cock handle of the automatic braking device for the derailment of the truck, which comprises the following components:
the device comprises an image acquisition module to be detected, a fault detection module and a fault detection module, wherein the image acquisition module to be detected is used for acquiring an image to be detected containing a cock handle part of an automatic derailing brake device and sending the image to be detected to the fault detection module;
the fault detection module is used for detecting the derailing automatic brake device cock handle part in the image to be detected by using the fault detection model, detecting to obtain the position of the derailing automatic brake device cock handle part, judging whether the derailing automatic brake device cock handle part is in a closed state or not,
if the door handle part of the derailing automatic braking device is in a closed state, the position of the corresponding door handle part of the derailing automatic braking device is sent to the fault information generation module;
the fault detection model is Faster R-CNN, and the characteristic graph is pooled by adopting multi-scale pooling RoIAlign in the Faster R-CNN;
the fault information generating module is used for generating fault information;
the fault information includes the position of the door handle member of the derailment automatic brake.
Wherein, the fault detection model in the fault detection module is obtained through a model generation module, and the model generation module comprises: a
The sample image acquisition module is used for acquiring a local area image containing a derailment automatic braking device cock hand component, marking the derailment automatic braking device cock hand component, generating an original sample data set and sending the original sample data set to the sample enhancement module;
the sample enhancement module is used for amplifying the original sample data set to obtain a training data set and sending the training data set to the model training module;
the model training module is used for training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
Wherein, the sample enhancement module comprises an improved mosaic data enhancement module, and the improved mosaic data enhancement module comprises:
image stitching channel I for generating image m 1 (ii) a Image m 1 The local area images are not spliced;
and the second image splicing channel is used for randomly selecting 4 local area images in the original sample data set to carry out 2 x 2-specification splicing to obtain a spliced image m 4 (ii) a 2 x 2 standard stitching is that each row and each column in a stitched image are stitched with 2 local area images;
and the image splicing channel III is used for randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3 standard stitching has 3 local area images stitched for each row and column in the stitched image.
The fault detection module also comprises a prediction box screening module;
a prediction box screening module for obtaining a screening confidence score S by a Gaussian attenuation score formula i ', and according to a screening confidence score S i ' for overlapping predictionLine screening; the prediction frame is generated in the fault detection model detection process;
wherein sigma is a hyper-parameter, i is a category number, M is a prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence score i D is the set of processed prediction frames, S i Is the raw confidence score of the prediction box being compared.
Wherein, sample image acquisition module includes:
the scanning and splicing module is used for scanning the bottom of the truck in a line mode, splicing to obtain a large truck bottom image and sending the large truck bottom image to the local area intercepting module;
and the local area intercepting module is used for preliminarily positioning the position of the plug handle of the derailment automatic braking device by utilizing the truck wheel base information and the truck type information, and intercepting a local area image containing the plug handle part of the derailment automatic braking device from a large truck bottom image.
The beneficial effects of the invention are:
the automatic detection method has important significance for the automatic detection of the derailment automatic braking device when the derailment automatic braking valve plug handle is closed. By combining image processing and template matching methods, automatic fault identification and alarm are realized, and the quality and efficiency of vehicle inspection operation are effectively improved.
1. The automatic identification technology is introduced into truck fault detection, automatic fault identification and alarm are realized, only the alarm result needs to be confirmed manually, the labor cost is effectively saved, and the operation quality and the operation efficiency are improved.
2. On the basis of a fault detection model, namely, faster R-CNN, the detection capability of the target with different sizes is adjusted by adopting multi-scale roilign (region of interest alignment) pooling, the robustness of the target detection model to the detection of the small-scale target is enhanced, the missing detection rate is reduced, and the detection accuracy is improved.
Drawings
FIG. 1 is a flow chart of a method for identifying a closing fault of a plug handle of an automatic braking device for a freight car derailment according to the present invention;
fig. 2 is a schematic diagram of the principle of an improved mosaic data enhancement method in the identification method of the closing fault of the door stopper handle of the automatic wagon derailment braking device.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
Detailed description of the invention
The method for identifying the closing fault of the cock handle of the automatic freight car derailment braking device specifically comprises the following steps:
s1, acquiring an image to be detected containing a cock handle part of an automatic derailing braking device;
s2, detecting the plug handle part of the derail automatic brake device in the image to be detected by using a fault detection model, detecting to obtain the position of the plug handle part of the derail automatic brake device, judging whether the plug handle part of the derail automatic brake device is in a closed state or not,
if the door closing handle part of the derailing automatic braking device is in a closed state, executing a step three;
the fault detection model is Faster R-CNN, and the characteristic graph is pooled by adopting multi-scale pooling RoIAlign in the Faster R-CNN;
s3, generating fault information;
the fault information includes the position of the door handle member of the derailment automatic brake.
Detailed description of the invention
In another specific embodiment, further describing the first specific embodiment, the training method of the fault detection model in the second step of the second specific embodiment is as follows:
s21, acquiring a local area image containing a derailment automatic braking device door stopping hand component, marking the derailment automatic braking device door stopping hand component, and generating an original sample data set;
s22, amplifying the original sample data set to obtain a training data set;
s23, training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
The other technical solutions in the second embodiment are the same as those in the first embodiment.
Detailed description of the invention
In the third embodiment, the second embodiment is further described, and in the step 22 of the third embodiment, the amplification method is performed by improving the mosaic data enhancement method, and the specific steps are as follows
S221, randomly selecting 4 local area images in the original sample data set to carry out 2 x 2-standard splicing to obtain a spliced image m 4 (ii) a 2, splicing in a 2 x 2 specification is to splice 2 local area images in each row and each column in a spliced image;
s222, randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3 standard stitching is that 3 local area images are stitched in each row and each column in the stitched image;
s223, when the image m 1 Splicing image m 4 And stitching the image m 9 After the quantity of the target nucleic acid reaches a set proportion, completing amplification;
the image m1 is a local area image without stitching.
The other technical solutions in the third embodiment are the same as those in the second embodiment.
Detailed description of the invention
In the second step of the fourth embodiment, the screening confidence score S is obtained by using a gaussian attenuation score formula i ', and according to a screening confidence score S i ' screening overlapping prediction boxes; the prediction frame is generated in the fault detection model detection process;
wherein, sigma is a hyper-parameter, i is a category number, M is a prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence scores i D is the processed prediction frame set, S i Is the original confidence score of the prediction box being compared.
The other technical solutions in the fourth embodiment are the same as those in the third embodiment.
Detailed description of the invention
The fifth specific embodiment is further described with respect to the fourth specific embodiment, and in step S21 of the fifth specific embodiment, a specific method for acquiring the local area image is as follows:
step 211, performing line scanning on the bottom of the truck, and splicing to obtain a large truck bottom image;
and step 212, preliminarily positioning the position of the plug handle of the derailment automatic braking device by utilizing the wheelbase information and the vehicle type information of the truck, and capturing a local area image containing the plug handle part of the derailment automatic braking device from a large bottom image of the truck.
The other technical solutions in the fifth embodiment are the same as those in the fourth embodiment.
Detailed description of the invention
In a sixth embodiment, a system for identifying a closing failure of a door stopper handle of a truck derailing automatic braking device includes:
the device comprises an image acquisition module to be detected, a fault detection module and a fault detection module, wherein the image acquisition module to be detected is used for acquiring an image to be detected containing a cock handle part of an automatic derailing brake device and sending the image to be detected to the fault detection module;
the fault detection module is used for detecting the derailing automatic brake device cock handle part in the image to be detected by using the fault detection model, detecting to obtain the position of the derailing automatic brake device cock handle part, judging whether the derailing automatic brake device cock handle part is in a closed state or not,
if the door handle part of the derailing automatic braking device is in a closed state, the position of the corresponding door handle part of the derailing automatic braking device is sent to the fault information generation module;
the fault detection model is Faster R-CNN, and the characteristic graph is pooled by adopting multi-scale pooling RoIAlign in the Faster R-CNN;
the fault information generating module is used for generating fault information;
the fault information includes the position of the door handle member of the derailment automatic brake.
Detailed description of the invention
A seventh specific embodiment is to further explain the sixth specific embodiment, in the sixth specific embodiment, the fault detection model in the fault detection module is obtained through a model generation module, and the model generation module includes:
the sample image acquisition module is used for acquiring a local area image containing a door closing hand component of the derailing automatic braking device, marking the door closing hand component of the derailing automatic braking device, generating an original sample data set and sending the original sample data set to the sample enhancement module;
the sample enhancement module is used for amplifying the original sample data set to obtain a training data set and sending the training data set to the model training module;
the model training module is used for training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
The other technical solutions in the seventh embodiment are the same as those in the sixth embodiment.
Detailed description of the invention
An eighth specific embodiment is a further description of the seventh specific embodiment, in the eighth specific embodiment, the sample enhancement module includes an improved mosaic data enhancement module, and the improved mosaic data enhancement module includes:
image stitching channel I for generating image m 1 (ii) a Image m 1 The local area images are not spliced;
and the second image splicing channel is used for randomly selecting 4 local area images in the original sample data set to carry out 2 x 2-specification splicing to obtain a spliced image m 4 (ii) a 2, splicing in a 2 x 2 specification is to splice 2 local area images in each row and each column in a spliced image;
and the image splicing channel III is used for randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3 standard stitching has 3 local area images stitched for each row and column in the stitched image.
The other technical solutions in the eighth embodiment are the same as those in the seventh embodiment.
Detailed description of the invention
Ninth in the present specific embodiment, the eighth specific embodiment is further described, and in the ninth specific embodiment, the fault detection module further includes a prediction box screening module;
a prediction box screening module for obtaining a screening confidence score S by a Gaussian attenuation score formula i ' and according to a screening confidence score S i ' screening overlapping prediction boxes; the prediction frame is generated in the fault detection model detection process;
wherein, sigma is a hyper-parameter, i is a category number,m is the prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence score i Cross-over ratio of (1), as a set of processed prediction boxes, S i Is the raw confidence score of the prediction box being compared.
The other technical solutions in the ninth embodiment are the same as those in the eighth embodiment.
Detailed description of the preferred embodiment
A tenth specific embodiment is further described with respect to the ninth specific embodiment, in the tenth specific embodiment, the sample image acquisition module includes:
the scanning and splicing module is used for scanning the bottom of the truck in a line mode, splicing to obtain a large truck bottom image and sending the large truck bottom image to the local area intercepting module;
and the local area intercepting module is used for preliminarily positioning the position of the plug handle of the derailment automatic braking device by utilizing the truck wheelbase information and the truck type information, and intercepting a local area image containing the plug handle part of the derailment automatic braking device from a large bottom image of the truck.
The other technical solutions in the tenth embodiment are the same as those in the ninth embodiment.
The embodiment is as follows:
the method applies the deep learning algorithm to the automatic identification of the closing fault of the cock handle of the derailment automatic braking device, and has higher accuracy and stability compared with the existing machine vision detection method.
1. Linear array image acquisition
High-definition equipment is respectively built at the bottom of a truck track, a truck passing at a high speed is shot, and an image of the bottom of the truck is obtained. By adopting line scanning, seamless splicing of images can be realized, and a two-dimensional image with a large visual field and high precision is generated.
2. Coarse positioning of parts
The position of a cock handle of the derailment automatic braking device is roughly positioned according to the truck wheel base information and the truck type information, and a local area image containing components is captured from a large bottom image, so that the time required by fault identification can be effectively reduced, and the identification accuracy can be improved.
3. Establishing an original sample data set
The truck parts can be influenced by natural conditions such as rainwater, mud, oil, black paint and the like or artificial conditions. Also, there may be differences in the images taken at different sites. Therefore, in the process of collecting the training image data set, the diversity is ensured, and the images of different sites under various conditions are collected as much as possible.
Each sample image dataset includes: an original image set and a label information set. The original image set is a rough positioning image which is shot by the equipment and contains a door handle part of the derailing automatic braking device. The mark information set is information of a rectangular subarea containing the part and is obtained by a manual marking mode. There is a one-to-one correspondence between the original image set and the marker information data set, i.e. one marker data per image.
4. Data set augmentation
Although the creation of the sample data set includes images under various conditions, data amplification of the sample data set is still required to improve the stability of the algorithm. The amplification form comprises operations of rotation, translation, zooming, mirror image and the like of the image, and each operation is performed under random conditions, so that the diversity and applicability of the sample can be ensured to the greatest extent.
5. Model training
After the training data set is established, a deep learning target detection algorithm is selected to detect the cock handle component of the derailment automatic braking device, and the model training process is shown in figure 1.
Compared with the target detection of other truck parts, the plug door handle of the derailment automatic braking device is often positioned behind a bogie, an axle and a pipeline, and obvious shielding exists, so that the deep learning detection model cannot learn complete characteristics and detection omission is caused. The invention adopts an improved mosaic data enhancement method, a multi-scale RoIAlign and an improved non-maximum suppression algorithm from three angles of a data enhancement method, a target identification algorithm and a non-maximum suppression algorithm, and solves the problem of poor detection effect of the plug door handle of the derailment automatic braking device caused by the shielding condition.
1. Improved mosaic data enhancement method
The improved mosaic data enhancement method is shown in fig. 2, the original mosaic method is feature enhancement performed by two channels, namely, an upper channel and a middle channel in fig. 2, and the improved mosaic method is added with the lower channel on the basis of the original method and performs feature enhancement by adopting 3 channels. The output of the 3 rd lower channel is obtained by increasing the number of images arranged in each row and column compared with the two above channels. For convenience of description, a combination mode of newly generating a nine-in-one picture in a 3 × 3 specification as m9, generating a four-in-one picture in a 2 × 2 specification as m4, and generating a picture in a 1 × 1 specification without merging as m1, wherein the ratio of m1, m4 and m9 is o: p: q makes the scale change characteristics of the training data set more diverse to some extent, thereby further enriching the data set and increasing the complexity of the background. Therefore, the network can concentrate on extracting the characteristics of the target object under the interference of a complex background, and the robustness of the network is enhanced.
The improved mosaic data enhancement method is adopted to splice and convert the training pictures into 3 different scales, and the scales are input into the network for training according to different proportions, so that the learning capability of the detection algorithm on the local features is enhanced.
2. Multi-scale RoIAlign
The existing Faster R-CNN framework applies a RoIPooling operation of pooling size 7 × 7 to each RoI's profile generated by the RPN. However, the RoI quantizes the RoI of the floating-point number, divides the quantized RoI into a plurality of intervals, quantizes the intervals themselves, and finally aggregates the eigenvalues of each interval. These quantization operations cause the feature map to lose much of the detail features in the pooling process and lead to mismatch problems between the RoI and the extracted features. While these quantifications do little to detect the robustness of large targets, they severely impact the pixel-level accuracy target boxes that need to be reached to detect small targets. Therefore, the multi-scale pooling operation RoIAlign is adopted to replace the single-scale pooling operation RoIPooling so as to enhance the robustness of the model to the small-scale target detection.
RoIAlign computes the value of each sample point by bilinear interpolation from the adjacent grid points on the signature without quantifying the RoI or any coordinates involved in the sample point. The RoIAlign avoids any quantification of RoI boundaries or subdivision intervals, calculates input characteristic values of 4 regular sampling points in each RoI by a bilinear interpolation method, and aggregates characteristic graphs by utilizing maximum pooling operation.
3. Improved non-maximum suppression algorithm
In order to delete the redundant prediction boxes generated by the network, most detectors are processed by using an NMS algorithm, and under the condition of non-occlusion or sparse target distribution, the NMS can effectively delete the redundant prediction boxes, so that each target is ensured to have a unique optimal proposal box corresponding to the target. However, under occlusion conditions, the overlapping of targets may result in a large overlap between the best suggested frames of different targets, resulting in NMS false suppression and thus missed detection.
Aiming at the defects of the existing NMS algorithm, the invention screens the overlapped check boxes in a Gaussian attenuation score mode, avoids mistakenly deleting the prediction boxes of the shielded targets as much as possible, and thus improves the recall capability of the algorithm to the shielded targets. For the prediction box with the intersection ratio of the prediction box with the highest confidence score exceeding the threshold, adopting a smoother processing mode instead of directly setting the confidence score of the prediction box to be 0, and utilizing a Gaussian function to perform the confidence score S on the prediction box i And processing, wherein the greater the intersection ratio of the prediction frame with the highest confidence score is, the more the confidence score of the prediction frame is reduced:
in the formula, sigma is a hyper-parameter, generally takes a value of 0.5, i is a category number, M is a prediction box with a larger confidence score, and b i For the compared target prediction boxes, iou (M, b) i ) Is M and prediction box b i D is the processed prediction frame set; s i Raw confidence scores, S, of the compared prediction boxes i ' is the screening confidence score, i.e. the confidence score after gaussian processing.
And aiming at the shielded target, an NMS algorithm of attenuation confidence score with a continuous function of Gaussian distribution as weight is used to reduce the risk of mistakenly deleting the shielded target frame, thereby improving the recognition capability of the algorithm on the shielded target.
6. Fault detection
The method comprises the following steps: coarse positioning of parts
And taking out a sub-region image containing the part according to the bogie type information and the prior knowledge of the region where the part is located.
Step two: loading model
And loading a deep learning detection model.
Step three: component detection
Detecting the derailment automatic braking device door handle component in the sub-area image.
Step four: failure determination
And logically analyzing the position and the form of the component to judge whether a fault exists or not.
Step five: upload alarm
And generating a message according to the fault information and the fault position and fault category, and uploading the message to an alarm platform.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features from different dependent claims and herein may be combined in ways other than those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other embodiments.
Claims (10)
1. The method for identifying the closing fault of the cock handle of the automatic freight car derailment braking device is characterized by comprising the following steps:
s1, acquiring an image to be detected containing a cock handle part of an automatic derailing braking device;
s2, detecting the plug handle part of the derail automatic braking device in the image to be detected by using a fault detection model, detecting to obtain the position of the plug handle part of the derail automatic braking device, judging whether the plug handle part of the derail automatic braking device is in a closed state or not,
if the door closing handle part of the derailing automatic braking device is in a closed state, executing a step three;
the fault detection model is Faster R-CNN, and the characteristic graph in the Faster R-CNN is pooled by adopting a multi-scale pooling operation RoIAlign;
s3, generating fault information;
the fault information includes the position of the derailment autobrake door handle member.
2. The method for identifying the closing fault of the plug handle of the automatic truck derailment braking device according to claim 1, wherein the method for training the fault detection model in the second step is as follows:
s21, acquiring a local area image containing a door closing hand component of the derailing automatic braking device, marking the door closing hand component of the derailing automatic braking device, and generating an original sample data set;
s22, amplifying the original sample data set to obtain a training data set;
s23, training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
3. The method for identifying the closing fault of the cock handle of the automatic truck derailment braking device according to claim 2, wherein the step 22 is implemented by amplifying the data by improving a mosaic data enhancement method, and comprises the following steps
S221, randomSelecting 4 local area images in the original sample data set to carry out 2 x 2-specification splicing to obtain a spliced image m 4 (ii) a The 2 × 2-specification splicing is that 2 local area images are spliced in each row and each column in the spliced images;
s222, randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3-specification stitching is that 3 local area images are stitched in each row and each column of the stitched images;
s223, when the image m 1 And splicing the images m 4 And stitching the image m 9 After the quantity of the target nucleic acid reaches a set proportion, completing amplification;
the image m1 is a local area image without stitching.
4. The method for identifying the closing fault of the plug handle of the automatic truck derailment brake device according to claim 3, wherein in the second step, the screening confidence score S is obtained by a Gaussian attenuation score formula i ' and according to a screening confidence score S i ' screening overlapping prediction boxes; the prediction box is generated in the fault detection model detection process;
wherein, sigma is a hyper-parameter, i is a category number, M is a prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence scores i D is the set of processed prediction frames, S i Is the raw confidence score of the prediction box being compared.
5. The method for identifying the closing fault of the cock handle of the automatic truck derailment braking device according to claim 4, wherein in the step S21, the specific method for acquiring the local area image is as follows:
step 211, performing line scanning on the bottom of the truck, and splicing to obtain a large truck bottom image;
and step 212, preliminarily positioning the plug handle position of the derailing automatic braking device by utilizing the truck wheel base information and the truck type information, and capturing a local area image containing the plug handle part of the derailing automatic braking device from a large truck bottom image.
6. Freight train derailment automatic braking device cock hand (hold) closes trouble identification system, its characterized in that includes:
the device comprises an image acquisition module to be detected, a fault detection module and a fault detection module, wherein the image acquisition module to be detected is used for acquiring an image to be detected containing a cock handle part of an automatic derailing brake device and sending the image to be detected to the fault detection module;
a fault detection module for detecting the derail automatic brake device cock handle part in the image to be detected by using a fault detection model, detecting to obtain the position of the derail automatic brake device cock handle part, and judging whether the derail automatic brake device cock handle part is in a closed state or not,
if the plug handle part of the derailing automatic braking device is in a closed state, the position of the corresponding plug handle part of the derailing automatic braking device is sent to the fault information generation module;
the fault detection model is FasterR-CNN, and the characteristic graph is subjected to pooling by adopting a multi-scale pooling operation RoIAlign in the FasterR-CNN;
the fault information generating module is used for generating fault information;
the fault information includes the position of the derailment autobrake door handle member.
7. The method for identifying the closing fault of the plug handle of the automatic truck derailment braking device according to claim 6, wherein the fault detection model in the fault detection module is obtained through a model generation module, and the model generation module comprises:
the sample image acquisition module is used for acquiring a local area image containing a door closing hand component of the derailing automatic braking device, marking the door closing hand component of the derailing automatic braking device, generating an original sample data set and sending the original sample data set to the sample enhancement module;
the sample enhancement module is used for amplifying the original sample data set to obtain a training data set and sending the training data set to the model training module;
the model training module is used for training the fault detection model to be convergent through a training data set to obtain a trained fault detection model and weight; and finishing the training of the fault detection model.
8. The method of identifying a truck derailment autobrake stopper handle closure failure of claim 7, wherein the sample enhancement module comprises a modified mosaic data enhancement module, the modified mosaic data enhancement module comprising:
image stitching channel one for generating image m 1 (ii) a Image m 1 The local area images are not spliced;
and the second image splicing channel is used for randomly selecting 4 local area images in the original sample data set to carry out 2 x 2-specification splicing to obtain a spliced image m 4 (ii) a The 2 × 2-specification splicing is that 2 local area images are spliced in each row and each column in the spliced images;
and the image splicing channel III is used for randomly selecting 4 local area images in the original sample data set to carry out 3 x 3 splicing to obtain a spliced image m 9 (ii) a The 3 × 3 standard stitching is that 3 local area images are stitched in each row and each column in the stitched image.
9. The method for identifying the closing fault of the plug handle of the automatic truck derailment braking device according to claim 8, wherein the fault detection module further comprises a prediction box screening module;
the prediction frame screening module is used for obtaining a screening confidence score S through a Gaussian attenuation score formula i ', and according to a screening confidence score S i ' screening overlapping prediction boxes; the prediction box is generated in the fault detection model detection process;
wherein, sigma is a hyper-parameter, i is a category number, M is a prediction box with the highest confidence score, b i For the prediction boxes being compared, iou (M, b) i ) The prediction box M and the prediction box b with the highest confidence scores i D is the set of processed prediction frames, S i Is the raw confidence score of the prediction box being compared.
10. The method for identifying the closing fault of the door handle of the automatic truck derailment braking device according to claim 9, wherein the sample image acquisition module comprises:
the scanning and splicing module is used for scanning the bottom of the truck in a line mode, splicing to obtain a large truck bottom image and sending the large truck bottom image to the local area intercepting module;
and the local area intercepting module is used for preliminarily positioning the position of the plug handle of the derailment automatic braking device by utilizing the truck wheelbase information and the truck type information, and intercepting a local area image containing the plug handle part of the derailment automatic braking device from a large bottom image of the truck.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211260741.4A CN115527170A (en) | 2022-10-14 | 2022-10-14 | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211260741.4A CN115527170A (en) | 2022-10-14 | 2022-10-14 | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115527170A true CN115527170A (en) | 2022-12-27 |
Family
ID=84701003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211260741.4A Pending CN115527170A (en) | 2022-10-14 | 2022-10-14 | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115527170A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115901299A (en) * | 2023-02-15 | 2023-04-04 | 慧铁科技有限公司 | Method for analyzing and processing faults of buffer component of train coupler hook |
| CN116403163A (en) * | 2023-04-20 | 2023-07-07 | 慧铁科技有限公司 | Method and device for identifying opening and closing states of handles of cut-off plug doors |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080608A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for recognizing closing fault image of automatic brake valve plug handle of railway wagon in derailment |
| CN111091552A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying closing fault image of angle cock handle of railway wagon |
| CN112581430A (en) * | 2020-12-03 | 2021-03-30 | 厦门大学 | Deep learning-based aeroengine nondestructive testing method, device, equipment and storage medium |
| CN113128430A (en) * | 2021-04-25 | 2021-07-16 | 科大讯飞股份有限公司 | Crowd gathering detection method and device, electronic equipment and storage medium |
| CN113436157A (en) * | 2021-06-18 | 2021-09-24 | 西南交通大学 | Vehicle-mounted image identification method for pantograph fault |
| CN113537045A (en) * | 2021-07-14 | 2021-10-22 | 宁夏大学 | Rock picture detection method based on improved FasterR-CNN |
| CN113674216A (en) * | 2021-07-27 | 2021-11-19 | 南京航空航天大学 | Subway tunnel disease detection method based on deep learning |
| CN114140745A (en) * | 2021-11-29 | 2022-03-04 | 上海建科工程咨询有限公司 | Method, system, device and medium for detecting personnel attributes of construction site |
| CN114708566A (en) * | 2022-04-05 | 2022-07-05 | 哈尔滨理工大学 | Improved YOLOv 4-based automatic driving target detection method |
-
2022
- 2022-10-14 CN CN202211260741.4A patent/CN115527170A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080608A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for recognizing closing fault image of automatic brake valve plug handle of railway wagon in derailment |
| CN111091552A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Method for identifying closing fault image of angle cock handle of railway wagon |
| CN112581430A (en) * | 2020-12-03 | 2021-03-30 | 厦门大学 | Deep learning-based aeroengine nondestructive testing method, device, equipment and storage medium |
| CN113128430A (en) * | 2021-04-25 | 2021-07-16 | 科大讯飞股份有限公司 | Crowd gathering detection method and device, electronic equipment and storage medium |
| CN113436157A (en) * | 2021-06-18 | 2021-09-24 | 西南交通大学 | Vehicle-mounted image identification method for pantograph fault |
| CN113537045A (en) * | 2021-07-14 | 2021-10-22 | 宁夏大学 | Rock picture detection method based on improved FasterR-CNN |
| CN113674216A (en) * | 2021-07-27 | 2021-11-19 | 南京航空航天大学 | Subway tunnel disease detection method based on deep learning |
| CN114140745A (en) * | 2021-11-29 | 2022-03-04 | 上海建科工程咨询有限公司 | Method, system, device and medium for detecting personnel attributes of construction site |
| CN114708566A (en) * | 2022-04-05 | 2022-07-05 | 哈尔滨理工大学 | Improved YOLOv 4-based automatic driving target detection method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115901299A (en) * | 2023-02-15 | 2023-04-04 | 慧铁科技有限公司 | Method for analyzing and processing faults of buffer component of train coupler hook |
| CN115901299B (en) * | 2023-02-15 | 2023-06-06 | 慧铁科技有限公司 | Method for analyzing and processing faults of train coupler buffering component |
| CN116403163A (en) * | 2023-04-20 | 2023-07-07 | 慧铁科技有限公司 | Method and device for identifying opening and closing states of handles of cut-off plug doors |
| CN116403163B (en) * | 2023-04-20 | 2023-10-27 | 慧铁科技有限公司 | Method and device for identifying opening and closing states of handles of cut-off plug doors |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079819B (en) | Method for judging state of coupler knuckle pin of railway wagon based on image recognition and deep learning | |
| CN115527170A (en) | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device | |
| CN112434695B (en) | Upper pull rod fault detection method based on deep learning | |
| CN111079746B (en) | Railway wagon axle box spring fault image identification method | |
| CN111080608A (en) | Method for recognizing closing fault image of automatic brake valve plug handle of railway wagon in derailment | |
| CN106778833A (en) | Small object loses the automatic identifying method of failure under a kind of complex background | |
| CN111080611A (en) | Railway wagon bolster spring fracture fault image identification method | |
| CN111079817B (en) | Method for identifying fault image of cross beam of railway wagon | |
| CN111080598A (en) | Bolt and nut missing detection method for coupler yoke key safety crane | |
| CN109489724A (en) | A kind of tunnel safe train operation environment comprehensive detection device and detection method | |
| CN111080650B (en) | Method for detecting looseness and loss faults of small part bearing blocking key nut of railway wagon | |
| CN111079734B (en) | Method for detecting foreign matters in triangular holes of railway wagon | |
| CN111768417B (en) | Railway wagon overrun detection method based on monocular vision 3D reconstruction technology | |
| CN113516629A (en) | Intelligent detection system for TFDS passing operation | |
| CN109934135B (en) | Rail foreign matter detection method based on low-rank matrix decomposition | |
| CN113221839B (en) | Automatic truck image identification method and system | |
| CN111652295A (en) | Railway wagon coupler yoke key joist falling fault identification method | |
| CN113553916B (en) | Orbit dangerous area obstacle detection method based on convolutional neural network | |
| CN111754460A (en) | Method, system and storage medium for automatically detecting gap of point switch | |
| CN111091551A (en) | Method for detecting loss fault of brake beam strut opening pin of railway wagon | |
| CN108648210B (en) | Rapid multi-target detection method and device under static complex scene | |
| CN113788051A (en) | Train on-station running state monitoring and analyzing system | |
| CN115272850A (en) | Railway wagon BAB type brake adjuster pull rod head breaking fault identification method | |
| CN111539421A (en) | Deep learning-based railway locomotive number identification method | |
| CN111652228B (en) | Railway wagon sleeper beam hole foreign matter detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20221227 |
|
| RJ01 | Rejection of invention patent application after publication |