CN117557570A - Rail vehicle abnormality detection method and system - Google Patents
Rail vehicle abnormality detection method and system Download PDFInfo
- Publication number
- CN117557570A CN117557570A CN202410045518.0A CN202410045518A CN117557570A CN 117557570 A CN117557570 A CN 117557570A CN 202410045518 A CN202410045518 A CN 202410045518A CN 117557570 A CN117557570 A CN 117557570A
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- axle
- abnormal
- vehicle bottom
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 64
- 230000005856 abnormality Effects 0.000 title claims description 22
- 230000002159 abnormal effect Effects 0.000 claims abstract description 91
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012795 verification Methods 0.000 claims abstract description 23
- 230000011218 segmentation Effects 0.000 claims abstract description 14
- 238000003709 image segmentation Methods 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 24
- 238000011156 evaluation Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 238000012216 screening Methods 0.000 claims description 12
- 238000013135 deep learning Methods 0.000 claims description 10
- 230000001502 supplementing effect Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 9
- 238000007689 inspection Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Train Traffic Observation, Control, And Security (AREA)
Abstract
The invention provides a method for detecting an anomaly of a railway vehicle, which comprises the steps of data acquisition and summarization, wherein the total data of an image to be detected is obtained; the axle region extraction step, namely carrying out positioning marking on the axle position of the total data of the image to be detected through a YOLO model; an image segmentation and splicing step, namely segmenting the total data of the image to be detected according to a positioned axle and a segmentation strategy to obtain a plurality of equal vehicle bottom authentication images; an image verification registration step, namely sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset vehicle axle template image by taking characteristic point pairing as a basis, and judging whether to analyze the vehicle bottom authentication image according to a registration result; an abnormal region identification step, namely comparing the vehicle bottom authentication image with an image difference detection model, and if the vehicle bottom authentication image has a difference, dividing a corresponding abnormal region; the invention has the advantages that the rail vehicle can be monitored all-weather without being limited by time and environmental conditions, thereby remarkably improving the monitoring efficiency and accuracy.
Description
Technical Field
The invention relates to the technical field of rail vehicle detection, in particular to a rail vehicle abnormality detection method and system.
Background
The safety of the rail vehicle is in great attention, however, the conventional rail vehicle detection method mainly relies on manual inspection, but has the problems of low efficiency, easy interference caused by human, and the like, so that the abnormal detection technology of the rail vehicle based on machine vision is becoming a hot spot in the research and application fields.
In general, thousands of parts are installed at the bottom of a railway vehicle, a plurality of trains are required to be inspected and maintained every day, the safety condition detection of the vehicle parts is a very heavy task, the traditional inspection method mainly relies on manual inspection, workers must inspect each part in sequence after the trains enter a warehouse, and the parts are recorded on an inspection table after faults are found and reported after maintenance is completed.
Compared with the traditional manual inspection method, the technical evolution is widely applied to machine vision for detecting the railway vehicle abnormality, wherein the detection technology utilizes high-resolution image or video data, and realizes automatic detection and identification of railway vehicle behaviors by means of image processing and target detection, but the existing detection and identification means are used for comparing the similarity of targets in images with standards in a database so as to judge whether abnormal objects exist in the images.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention aims to provide the method for detecting the abnormality of the railway vehicle, which can monitor the railway vehicle all-weather without being limited by time and environmental conditions, thereby obviously improving the monitoring efficiency and accuracy.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a method for detecting abnormality of a railway vehicle includes the steps of:
a data acquisition and summarization step, namely acquiring the bottom images shot by cameras at all positions on the portal frame, and summarizing the bottom images to obtain the total data of the images to be detected;
an axle region extraction step, namely screening and extracting total data of an image to be detected through a pre-trained YOLO model to obtain an axle region image;
an image segmentation and splicing step, namely segmenting the total data of the image to be detected according to a positioned axle and a segmentation strategy to obtain a plurality of equal vehicle bottom authentication images;
an image verification registration step, namely sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset vehicle axle template image by taking characteristic point pairing as a basis, if the alignment registration fails, ignoring the vehicle bottom authentication image, and if Ji Peizhun is successful, carrying out anomaly identification;
and an abnormal region identification step, namely constructing an image difference detection model through a model training data set preset by the system by a deep learning strategy, comparing the vehicle bottom authentication image with the image difference detection model, and if the vehicle bottom authentication image has the difference, segmenting out a corresponding abnormal region.
Further, the method also comprises an abnormal type output step and an abnormal type supplementing step;
an abnormal type output step, namely acquiring various abnormal images in a database, converting abnormal conditions in the abnormal images into text meanings, outputting the text meanings to the database, classifying and tabulating the text meanings and the corresponding abnormal images to obtain an abnormal type table, and outputting text meanings corresponding to indexes of the abnormal areas in the abnormal type table as the abnormal conditions;
and an abnormal type supplementing step, namely when the abnormal region does not index the corresponding text meaning, manually identifying and inputting the corresponding text meaning as a supplementing type, and filling the supplementing type into an abnormal type table.
Further, the segmentation strategy comprises an image segmentation sub-step and an image screening sub-step;
the image segmentation sub-step is that the axle region images are cut according to the axle shooting sequence of the railway vehicle to obtain a plurality of sections of vehicle bottom rough images, and each section of vehicle bottom rough image comprises a complete axle;
and the image screening and cutting sub-step is that the vehicle bottom rough image is subjected to equal-size cutting on each section of the vehicle bottom rough image according to the position of the vehicle axle to obtain a vehicle bottom authentication image.
Further, in the image verification registration step, feature points of the vehicle bottom authentication image and the vehicle axle template image are extracted through an ORB feature detector, a matching point pair of the feature points between the vehicle bottom authentication image and the vehicle axle template image is obtained according to violent matching, the matching point pair is optimized through a GMS algorithm, and if the number of the matching points between the vehicle bottom authentication image and the vehicle axle template image is larger than a threshold value, a registered to-be-detected image is obtained through calculating a matrix of image registration transformation.
Further, the image verification registration step includes a verification calibration sub-step, if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is smaller than a threshold value, extracting feature points of the vehicle bottom authentication image and the vehicle axle template image through an AKAZE feature detector, obtaining a matching point pair of the feature points between the vehicle bottom authentication image and the vehicle axle template image according to violent matching, optimizing the matching point pair through a GMS algorithm, if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is larger than the threshold value, obtaining a registered to-be-detected image through calculating a matrix of image registration transformation, and if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is smaller than the threshold value, discarding the vehicle bottom authentication image and the vehicle axle template image.
Further, the abnormal region identification step comprises a model training sub-step, a model optimizing sub-step and a model verifying sub-step;
the model training sub-step is to construct an image difference detection model through a model training data set preset by a system by a deep learning strategy;
the model optimization sub-step is used for obtaining a predicted value and a true value in a model training data set, calculating the predicted value and the true value through a cross entropy loss function arithmetic formula to obtain a loss value, and compensating and optimizing an image difference detection model through the loss value;
and the model verification sub-step is used for defining that a change area in the vehicle bottom authentication image is positive, a background area is negative, obtaining an evaluation score by substituting the number of positive samples in the vehicle bottom authentication image, the number of false negative samples, the number of correctly identified negative samples and the number of missed positive samples into an evaluation calculation formula, outputting the model accurately if the evaluation score is greater than or equal to a threshold value, and outputting the model incorrectly if the evaluation score is smaller than the threshold value.
Further, the cross entropy loss function formula is configured to:
wherein,-1->Predicted value of individual pixels,/>-1->A true value for the individual pixel;
the evaluation formula is configured to:
wherein,-accuracy, & gt>-recall->-number of positive samples identified, +.>-number of false positive negative samples, +.>-number of negative samples correctly identified, +.>-number of positive samples missing.
And further, in the axle region extraction step, denoising and/or adjusting the total data of the image to be detected to obtain processed total data of the image, and inputting the total data of the image into a YOLO model for target detection to obtain an axle region image.
Further, the literal meaning in the anomaly type table includes hanging, missing parts and ectopic parts.
A rail vehicle anomaly detection system, comprising:
the image acquisition module is used for acquiring the bottom images shot by the cameras at all positions on the portal frame and summarizing the bottom images to obtain the total data of the images to be detected;
the image processing module is used for screening and extracting total data of an image to be detected through a pre-trained YOLO model to obtain an axle region image, segmenting the total data of the image to be detected according to a segmentation strategy based on a positioned axle to obtain a plurality of equal vehicle bottom authentication images, sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset axle template image based on characteristic point pairing, if the alignment registration fails, ignoring the vehicle bottom authentication image, if Ji Peizhun is successful, carrying out anomaly identification, constructing an image difference detection model through a model training data set preset by a system by using a deep learning strategy, comparing the vehicle bottom authentication image with the image difference detection model, and segmenting a corresponding anomaly region if a difference exists;
the output interaction module is preset with an abnormal type table, and the text meaning corresponding to the index of the abnormal region in the abnormal type table is used as the current abnormal condition to be output;
and the abnormality alarm module is used for sending out an alarm instruction when receiving the output of the abnormal condition.
The invention has the beneficial effects that: 1. the method comprises the steps of shooting a bottom image of a railway vehicle at multiple angles through a camera on a portal frame, automatically reserving a part with an axle in the image through matching screening of a model, dividing the continuous image in a segmentation mode to obtain an authentication image with the same size and corresponding axle, registering matching points between the authentication image and a preset image, and carrying out anomaly analysis on the image which is successfully registered.
2. Compared with the traditional manual inspection method, the machine vision-based railway vehicle anomaly detection technology utilizes high-resolution image or video data, realizes automatic detection and identification of railway vehicle behaviors by means of technologies such as image processing, target detection and pattern identification, and the like, and firstly, the machine vision method can monitor the railway vehicle all-weather without being limited by time and environmental conditions, so that the monitoring efficiency and accuracy are remarkably improved; and secondly, the machine vision technology can use an advanced target detection and semantic segmentation model to realize accurate detection of the vehicle and accurately find out the conditions of damage, falling-off and the like of the vehicle bottom, the vehicle side and the vehicle roof parts.
Drawings
FIG. 1 is an overall flow chart of anomaly detection in the present invention;
FIG. 2 is a schematic illustration of an image verification registration step in the present invention;
FIG. 3 is a schematic view of a camera setup position of the present invention;
FIG. 4 is a flow chart of a system for anomaly detection in the present invention.
Detailed Description
The invention will now be described in further detail with reference to the drawings and examples. Wherein like parts are designated by like reference numerals. It should be noted that the words "front", "back", "left", "right", "upper" and "lower" used in the following description refer to directions in the drawings, and the words "bottom" and "top", "inner" and "outer" refer to directions toward or away from, respectively, the geometric center of a particular component.
Compared with the traditional manual inspection method, the existing railway vehicle anomaly detection method is widely applied to machine vision for detecting railway vehicle anomalies through technical evolution, wherein the detection technology utilizes high-resolution images or video data, and realizes automatic detection and identification of railway vehicle behaviors by means of image processing and target detection, but the existing detection and identification means are used for comparing the similarity or point cloud mode of targets in images and standards in a database to judge whether anomalies exist in the images, and the method is easy to miss, low in comparison accuracy and time-consuming and difficult to meet the existing detection mode, so that the railway vehicle anomaly detection method is designed, as shown in fig. 1, and comprises the following steps:
a data acquisition and summarization step, namely acquiring the bottom images shot by the high-resolution line scanning cameras on the portal frame, summarizing the bottom images to obtain the total data of the images to be detected, wherein as shown in fig. 3, 3 high-resolution line scanning cameras are arranged at the roof position of the portal frame, 4 high-resolution line scanning cameras are laterally arranged, 2 high-resolution line scanning cameras are arranged at the walking part, 5 high-resolution line scanning cameras are arranged at the bottom of the walking part, each camera can automatically record the images of each angle, and the image data is sent to a background processing system in a high-speed transmission mode to be used as the total data of the images to be detected;
an axle region extraction step, namely screening and extracting total data of an image to be detected through a pre-trained YOLO model to obtain an axle region image; in order to realize accurate detection of axles, the embodiment adopts an advanced YOLO target detection technology, the technology is based on a deep learning algorithm, has the characteristics of high efficiency and accuracy, and can identify and detect the position and the outline of each axle in the railway vehicle through a training model. Then, the present embodiment inputs the processed image into the YOLO model for target detection to identify and locate the position of the axle, and through the above improvement and optimization measures, the present embodiment can obtain a high-resolution image and accurately detect the axle of the railway vehicle using an advanced target detection technique. Therefore, the method and the device can monitor and identify the abnormal behavior of the railway vehicle in time, and improve the running safety and reliability of the railway vehicle.
For example: the line scanning camera is used for scanning the train, when the train passes through the camera, every 4000 pixels are stored into an image, the camera is stopped after the train leaves, in the process, 164 train images with the length of 4000 pixels and the width of 3768 pixels are obtained in total, and then the position of 24 axles of the train is detected by utilizing the YOLO target detection technology and is equivalent to a mark drawing frame.
The image segmentation and splicing step, namely segmenting the total data of the image to be detected according to a positioned axle and a segmentation strategy to obtain a plurality of equal vehicle bottom authentication images; specifically, the problem that subsequent processing is difficult due to oversized images is avoided, and the segmentation strategy comprises an image segmentation sub-step and an image screening sub-step;
the image segmentation sub-step, cutting the axle region image according to the axle shooting sequence of the railway vehicle to obtain a plurality of sections of vehicle bottom rough images, wherein each section of vehicle bottom rough image comprises a complete axle; based on the axle position information, the 164 images need to be spliced and segmented again to obtain 25 images based on axle segmentation,
an image screening sub-step, namely performing equal-size cutting on each section of the vehicle bottom rough image to obtain a vehicle bottom authentication image according to the position of an axle on the vehicle bottom rough image; in order to ensure the effect and speed of subsequent processing, the 25 images are further segmented by using an equal segmentation method, and finally 125 pictures with similar sizes and standard positions are obtained.
An image verification registration step, as shown in fig. 2, of sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset vehicle axle template image by taking characteristic point pairing as a basis, if the alignment registration fails, ignoring the vehicle bottom authentication image, and if Ji Peizhun is successful, carrying out anomaly identification;
specifically, in the image verification registration step, feature points of the vehicle bottom authentication image and the vehicle axle template image are extracted through a ORB (Oriented FAST and Rotated BRIEF) feature detector, a matching point pair of the feature points between the vehicle bottom authentication image and the vehicle axle template image is obtained according to violent matching, the accuracy of the matching point pair is optimized through a GMS (Grid-based Motion Statistics) algorithm, if the matching point number between the vehicle bottom authentication image and the vehicle axle template image is larger than a threshold value, a registered to-be-detected image is obtained through calculating a matrix of image registration transformation, and the template image is preset before the to-be-detected image, wherein in the case, the template image corresponding to the to-be-detected image is reserved.
The image verification registration step comprises a verification calibration sub-step, if the number of matching points between the vehicle bottom verification image and the vehicle axle template image is smaller than a threshold value, the characteristic points of the vehicle bottom verification image and the vehicle axle template image are extracted through an AKAZE (accepted-KAZE) characteristic detector, the matching point pairs of the characteristic points between the vehicle bottom verification image and the vehicle axle template image are obtained according to violent matching, the matching point pairs are optimized through a GMS (Grid-based Motion Statistics) algorithm, if the number of matching points between the vehicle bottom verification image and the vehicle axle template image is larger than the threshold value, a registered to-be-detected image is obtained through calculating a matrix of image registration transformation, the template image corresponding to the to-be-detected image is preset before, and under the condition, if the number of matching points between the vehicle bottom verification image and the vehicle axle template image is smaller than the threshold value, the vehicle bottom verification image and the vehicle axle template image are abandoned.
Through the optimization measures, the accuracy and stability of image registration can be improved, and through the combination of ORB and AKAZE feature detectors, violent matching and GMS algorithm, matching point pairs can be effectively obtained and an image transformation matrix is calculated, so that automation and high efficiency of the registration process are achieved, and therefore the standard position picture to be detected and the template picture can be accurately matched after registration, and a reliable data base is provided for subsequent detection and analysis.
After the registration of the images is completed, the embodiment adopts a neural network model to divide the abnormal regions, an image difference detection model is constructed by a deep learning strategy through a model training data set preset by a system, an under-vehicle authentication image is compared with the image difference detection model, if the differences exist, the corresponding abnormal regions are divided, and the image difference detection model can accurately distinguish the abnormal regions from the normal regions in the images, so that a reliable basis is provided for subsequent abnormal detection and analysis;
specifically, the abnormal region identification step includes a model training sub-step, a model optimizing sub-step and a model verifying sub-step;
a model training sub-step, namely constructing an image difference detection model by a deep learning strategy through a model training data set preset by a system, wherein the model training data set comprises 5000 railway vehicle template images and corresponding 5000 railway vehicle abnormal images, and the model training data set comprises the conditions that parts at the bottom of a thunder vehicle are absent, redundant foreign matters and the colors or shapes of parts are changed;
a model optimization sub-step of obtaining a predicted value and a true value in a model training data set, calculating to obtain a loss value through a cross entropy loss function calculation formula, and compensating and optimizing an image difference detection model through the loss value; the task of anomaly detection requires the model to output a mask of varying regions, thus training the network to compensate for the loss on each pixel by cross entropy loss, where the cross entropy loss function is configured to:
wherein,-1->Predicted value of individual pixels,/>-1->A true value for the individual pixel;
a model verification sub-step of defining a change area in the vehicle bottom authentication image as positive and a background area as negative, obtaining an evaluation score by substituting the number of positive samples in the analysis vehicle bottom authentication image, the number of false negative samples, the number of correctly recognized negative samples and the number of missed positive samples into an evaluation calculation formula, outputting the model accurately if the evaluation score is greater than or equal to a threshold value, and outputting the model in error if the evaluation score is smaller than the threshold value; the model continuously optimizes its own parameters through a back propagation algorithm during training to improve accuracy and robustness, wherein the evaluation formula is configured to:
wherein,-accuracy, & gt>-recall->-number of positive samples identified, +.>-number of false positive negative samples, +.>-number of negative samples correctly identified, +.>-number of positive samples missing report;
in practical application, the embodiment takes the registered image as input, and obtains the segmented image through the reasoning process of the neural network model, wherein the abnormal region and the normal region are clearly distinguished, so the embodiment can easily extract the abnormal region in the image, provides an important data basis for subsequent abnormal detection and judgment, can realize automation and precision improvement in the image segmentation process through the introduction of the neural network model, can accurately segment the abnormal region in the image through learning the characteristic difference between the abnormal region and the normal region in the training data, avoids the artificial subjectivity and the inconsistency in the traditional method, can more accurately divide the abnormal region in the image in a pixel manner relative to the abnormal recognition mode of the existing image point cloud, and provides effective basis sources for subsequent abnormal type analysis, and the introduction of the model greatly improves the accuracy and the efficiency of the abnormal detection.
Because the abnormality detection result is simply output whether an abnormality instruction exists or not, the method further comprises an abnormality type output step and an abnormality type supplementing step in order to facilitate the subsequent processing of maintenance supervision personnel;
an abnormal type output step, namely acquiring various abnormal images in the database, converting abnormal conditions in the abnormal images into text meanings, and outputting the text meanings to the database, namely, assuming 5000 abnormal images with different angles, and storing various abnormal conditions in a text meaning mode through automatic classification and artificial classification of a system, such as: the method comprises the steps of classifying and dividing a table by using text meanings and corresponding abnormal images to obtain an abnormal type table, wherein the text meanings in the abnormal type table comprise suspended objects, missing parts and ectopic parts, and the text meanings corresponding to indexes of an abnormal area in the abnormal type table are output as abnormal conditions of the time, and a supervisor can pertinently make subsequent maintenance processing instructions after receiving a certain condition of an axle is abnormal;
and an exception type supplementing step, wherein when the exception area does not index the corresponding text meaning, the corresponding text meaning is manually identified and input as a supplement type, and the supplement type is filled in the exception type table to expand the exception type table so as to enable the subsequent index to be more accurate.
A rail vehicle anomaly detection system, as shown in fig. 4, comprising:
the image acquisition module is used for acquiring the bottom images shot by the cameras at all positions on the portal frame and summarizing the bottom images to obtain the total data of the images to be detected;
the image processing module is used for screening and extracting total data of an image to be detected through a pre-trained YOLO model to obtain an axle region image, segmenting the total data of the image to be detected according to a segmentation strategy based on a positioned axle to obtain a plurality of equal vehicle bottom authentication images, sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset axle template image based on characteristic point pairing, if the alignment registration fails, ignoring the vehicle bottom authentication image, if Ji Peizhun is successful, carrying out abnormal recognition, constructing an image difference detection model through a model training data set preset by a system by a deep learning strategy, comparing the vehicle bottom authentication image with the image difference detection model, and if the difference exists, segmenting a corresponding abnormal region;
the output interaction module is preset with an abnormal type table, and the text meaning corresponding to the index of the abnormal region in the abnormal type table is used as the current abnormal condition to be output;
and the abnormality alarm module is used for sending out an alarm instruction when receiving the output of the abnormal condition.
Abnormal conditions can be timely handled, and normal operation and safety of the railway vehicle are ensured.
The embodiment can construct a high-efficiency and accurate system for acquiring and processing the image data of the railway vehicle and detecting the abnormal situation in the image data, the image acquisition module adopts a high-speed linear array camera and a laser light source, the quality and the definition of the image are ensured, the image processing module executes a program by utilizing a processor to realize the railway vehicle abnormal detection method, the abnormal situation in the image is accurately judged, the output interaction module can output the detected abnormal image, flexible and various output modes are provided, the abnormal alarm module can timely trigger an alarm and feed the abnormal information back to related personnel, and the abnormal situation is ensured to be timely processed and solved.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (10)
1. The method for detecting the abnormality of the railway vehicle is characterized by comprising the following steps of: the method comprises the following steps:
a data acquisition and summarization step, namely acquiring the bottom images shot by cameras at all positions on the portal frame, and summarizing the bottom images to obtain the total data of the images to be detected;
an axle region extraction step, namely positioning and marking the axle position of the total data of the image to be detected through a pre-trained YOLO model;
an image segmentation and splicing step, namely segmenting the total data of the image to be detected according to a positioned axle and a segmentation strategy to obtain a plurality of equal vehicle bottom authentication images;
an image verification registration step, namely sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset vehicle axle template image by taking characteristic point pairing as a basis, if the alignment registration fails, ignoring the vehicle bottom authentication image, and if Ji Peizhun is successful, carrying out anomaly identification;
and an abnormal region identification step, namely constructing an image difference detection model through a model training data set preset by the system by a deep learning strategy, comparing the vehicle bottom authentication image with the image difference detection model, and if the vehicle bottom authentication image has the difference, segmenting out a corresponding abnormal region.
2. The method for detecting an abnormality of a railway vehicle according to claim 1, characterized in that: the method also comprises an abnormal type output step and an abnormal type supplementing step;
an abnormal type output step, namely acquiring various abnormal images in a database, converting abnormal conditions in the abnormal images into text meanings, outputting the text meanings to the database, classifying and tabulating the text meanings and the corresponding abnormal images to obtain an abnormal type table, and outputting text meanings corresponding to indexes of the abnormal areas in the abnormal type table as the abnormal conditions;
and an abnormal type supplementing step, namely when the abnormal region does not index the corresponding text meaning, manually identifying and inputting the corresponding text meaning as a supplementing type, and filling the supplementing type into an abnormal type table.
3. The method for detecting an abnormality of a railway vehicle according to claim 2, characterized in that: the segmentation strategy comprises an image segmentation sub-step and an image screening sub-step;
the image segmentation sub-step is that the axle region images are cut according to the axle shooting sequence of the railway vehicle to obtain a plurality of sections of vehicle bottom rough images, and each section of vehicle bottom rough image comprises a complete axle;
and the image screening and cutting sub-step is that the vehicle bottom rough image is subjected to equal-size cutting on each section of the vehicle bottom rough image according to the position of the vehicle axle to obtain a vehicle bottom authentication image.
4. A rail vehicle abnormality detection method according to claim 3, characterized in that: and in the image verification registering step, feature points of the vehicle bottom authentication image and the vehicle axle template image are extracted through an ORB feature detector, a matching point pair of the feature points between the vehicle bottom authentication image and the vehicle axle template image is obtained according to violent matching, the matching point pair is optimized through a GMS algorithm, and if the number of the matching points between the vehicle bottom authentication image and the vehicle axle template image is larger than a threshold value, a registered map to be detected is obtained through calculating a matrix of image registration transformation.
5. The method for detecting an abnormality of a railway vehicle according to claim 4, characterized in that: the image verification registration step comprises a verification calibration sub-step, if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is smaller than a threshold value, extracting characteristic points of the vehicle bottom authentication image and the vehicle axle template image through an AKAZE characteristic detector, obtaining a matching point pair of the characteristic points between the vehicle bottom authentication image and the vehicle axle template image according to violent matching, optimizing the matching point pair through a GMS algorithm, if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is larger than the threshold value, obtaining a registered to-be-detected image through calculating a matrix of image registration transformation, and if the number of matching points between the vehicle bottom authentication image and the vehicle axle template image is smaller than the threshold value, discarding the vehicle bottom authentication image and the vehicle axle template image.
6. The abnormality detection method for a railway vehicle according to claim 1 or 5, characterized in that: the abnormal region identification step comprises a model training sub-step, a model optimizing sub-step and a model verifying sub-step;
the model training sub-step is to construct an image difference detection model through a model training data set preset by a system by a deep learning strategy;
the model optimization sub-step is used for obtaining a predicted value and a true value in a model training data set, calculating the predicted value and the true value through a cross entropy loss function arithmetic formula to obtain a loss value, and compensating and optimizing an image difference detection model through the loss value;
and the model verification sub-step is used for defining that a change area in the vehicle bottom authentication image is positive, a background area is negative, obtaining an evaluation score by substituting the number of positive samples in the vehicle bottom authentication image, the number of false negative samples, the number of correctly identified negative samples and the number of missed positive samples into an evaluation calculation formula, outputting the model accurately if the evaluation score is greater than or equal to a threshold value, and outputting the model incorrectly if the evaluation score is smaller than the threshold value.
7. The method for detecting an abnormality of a railway vehicle according to claim 6, characterized in that: the cross entropy loss function formula is configured to:
wherein,-1->Predicted value of individual pixels,/>-1->A true value for the individual pixel;
the evaluation formula is configured to:
wherein,-accuracy, & gt>-recall->-number of positive samples identified, +.>-the number of false positive negative samples,-number of negative samples correctly identified, +.>-number of positive samples missing.
8. The method for detecting an abnormality of a railway vehicle according to claim 7, characterized in that: and the axle region extraction step is to denoise and/or scale-adjust the total image data to be detected to obtain processed total image data, and then input the total image data into a YOLO model for target detection to obtain an axle region image.
9. The method for detecting an abnormality of a railway vehicle according to claim 2, characterized in that: the literal meaning in the anomaly type table includes hanging, missing parts and ectopic parts.
10. An anomaly detection system for a rail vehicle, characterized in that: comprising the following steps:
the image acquisition module is used for acquiring the bottom images shot by the cameras at all positions on the portal frame and summarizing the bottom images to obtain the total data of the images to be detected;
the image processing module is used for screening and extracting total data of an image to be detected through a pre-trained YOLO model to obtain an axle region image, segmenting the total data of the image to be detected according to a segmentation strategy based on a positioned axle to obtain a plurality of equal vehicle bottom authentication images, sequentially carrying out alignment registration on each complete vehicle bottom authentication image and a preset axle template image based on characteristic point pairing, if the alignment registration fails, ignoring the vehicle bottom authentication image, if Ji Peizhun is successful, carrying out anomaly identification, constructing an image difference detection model through a model training data set preset by a system by using a deep learning strategy, comparing the vehicle bottom authentication image with the image difference detection model, and segmenting a corresponding anomaly region if a difference exists;
the output interaction module is preset with an abnormal type table, and the text meaning corresponding to the index of the abnormal region in the abnormal type table is used as the current abnormal condition to be output;
and the abnormality alarm module is used for sending out an alarm instruction when receiving the output of the abnormal condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410045518.0A CN117557570B (en) | 2024-01-12 | 2024-01-12 | Rail vehicle abnormality detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410045518.0A CN117557570B (en) | 2024-01-12 | 2024-01-12 | Rail vehicle abnormality detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117557570A true CN117557570A (en) | 2024-02-13 |
| CN117557570B CN117557570B (en) | 2024-05-24 |
Family
ID=89811508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410045518.0A Active CN117557570B (en) | 2024-01-12 | 2024-01-12 | Rail vehicle abnormality detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117557570B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014059655A (en) * | 2012-09-14 | 2014-04-03 | Toshiba Corp | Road situation-monitoring device, and road situation-monitoring method |
| CN109242835A (en) * | 2018-08-24 | 2019-01-18 | 深圳市兆泰云智能科技有限公司 | Vehicle bottom defect inspection method, device, equipment and system based on artificial intelligence |
| CN110321905A (en) * | 2019-07-11 | 2019-10-11 | 广东工业大学 | Abnormal area detection method, system and associated component based on semantic segmentation |
| CN111080603A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting breakage fault of shaft end bolt of railway wagon |
| CN111091549A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting breakage fault of crossed rod bodies of bottom parts of railway freight cars |
| CN112330646A (en) * | 2020-11-12 | 2021-02-05 | 南京优视智能科技有限公司 | Motor car bottom abnormity detection method based on two-dimensional image |
| CN114913445A (en) * | 2021-10-11 | 2022-08-16 | 广东灏云智能科技有限公司 | Intelligent inspection operation system for airplane route based on image recognition |
| CN115601574A (en) * | 2022-10-21 | 2023-01-13 | 昆明理工大学（Cn） | Unmanned aerial vehicle image matching method for improving AKAZE characteristics |
| CN115908872A (en) * | 2022-11-08 | 2023-04-04 | 中数智科(杭州)科技有限公司 | Train bottom part matching method |
| CN116958933A (en) * | 2023-07-25 | 2023-10-27 | 四川农业大学 | Multi-lane load identification method, system, electronic equipment and storage medium |
| CN117132763A (en) * | 2023-08-28 | 2023-11-28 | 中国南方电网有限责任公司超高压输电公司电力科研院 | Power image anomaly detection method, device, computer equipment and storage medium |
-
2024
- 2024-01-12 CN CN202410045518.0A patent/CN117557570B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014059655A (en) * | 2012-09-14 | 2014-04-03 | Toshiba Corp | Road situation-monitoring device, and road situation-monitoring method |
| CN109242835A (en) * | 2018-08-24 | 2019-01-18 | 深圳市兆泰云智能科技有限公司 | Vehicle bottom defect inspection method, device, equipment and system based on artificial intelligence |
| CN110321905A (en) * | 2019-07-11 | 2019-10-11 | 广东工业大学 | Abnormal area detection method, system and associated component based on semantic segmentation |
| CN111080603A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting breakage fault of shaft end bolt of railway wagon |
| CN111091549A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting breakage fault of crossed rod bodies of bottom parts of railway freight cars |
| CN112330646A (en) * | 2020-11-12 | 2021-02-05 | 南京优视智能科技有限公司 | Motor car bottom abnormity detection method based on two-dimensional image |
| CN114913445A (en) * | 2021-10-11 | 2022-08-16 | 广东灏云智能科技有限公司 | Intelligent inspection operation system for airplane route based on image recognition |
| CN115601574A (en) * | 2022-10-21 | 2023-01-13 | 昆明理工大学（Cn） | Unmanned aerial vehicle image matching method for improving AKAZE characteristics |
| CN115908872A (en) * | 2022-11-08 | 2023-04-04 | 中数智科(杭州)科技有限公司 | Train bottom part matching method |
| CN116958933A (en) * | 2023-07-25 | 2023-10-27 | 四川农业大学 | Multi-lane load identification method, system, electronic equipment and storage medium |
| CN117132763A (en) * | 2023-08-28 | 2023-11-28 | 中国南方电网有限责任公司超高压输电公司电力科研院 | Power image anomaly detection method, device, computer equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| LIU, HZ ET AL: "A Unified Detection Approach for Point and Subsequence Anomaly Data From Train Axle Temperature Sensors", IEEE SENSORS JOURNAL, 15 October 2023 (2023-10-15) * |
| 吴梦迪: "地铁列车轴端螺栓异常状况检测技术研究", 中国优秀硕士学位论文全文数据库工程科技II辑, no. 07, 15 July 2021 (2021-07-15), pages 2 - 4 * |
| 李芮睿: "基于深度学习的高铁车轴定位研究", 中国优秀硕士学位论文全文数据库工程科技II辑, no. 03, 15 March 2022 (2022-03-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117557570B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108280855A (en) | A kind of insulator breakdown detection method based on Fast R-CNN | |
| CN111260629A (en) | Pantograph structure abnormity detection algorithm based on image processing | |
| CN111080691A (en) | Infrared hot spot detection method and device for photovoltaic module | |
| CN111949809A (en) | Intelligent processing method for infrared inspection data of power transmission line | |
| CN104483320A (en) | Digitized defect detection device and detection method of industrial denitration catalyst | |
| CN107886047A (en) | A kind of detecting system and method for vehicle annual test survey report | |
| CN108846331B (en) | Video identification method for judging whether screw fastener of motor train unit chassis falls off or not | |
| CN111591715A (en) | Belt longitudinal tearing detection method and device | |
| CN111582084B (en) | Weak supervision learning-based rail foreign matter detection method and system under empty base view angle | |
| CN112508911A (en) | Rail joint touch net suspension support component crack detection system based on inspection robot and detection method thereof | |
| CN114972177A (en) | Road disease identification management method and device and intelligent terminal | |
| KR20210122429A (en) | Method and System for Artificial Intelligence based Quality Inspection in Manufacturing Process using Machine Vision Deep Learning | |
| CN109543533B (en) | Automatic identification method for airport operation personnel management | |
| CN117314921B (en) | RFID-based starting point detection and treatment method for track inspection equipment | |
| CN117114420B (en) | Image recognition-based industrial and trade safety accident risk management and control system and method | |
| CN117557570B (en) | Rail vehicle abnormality detection method and system | |
| CN111626104B (en) | Cable hidden trouble point detection method and device based on unmanned aerial vehicle infrared thermal image | |
| CN115855276A (en) | System and method for detecting temperature of key components of urban rail vehicle based on deep learning | |
| CN115857040A (en) | Dynamic visual detection device and method for foreign matters on locomotive roof | |
| CN110070520B (en) | Pavement crack detection model construction and detection method based on deep neural network | |
| CN117252840B (en) | Photovoltaic array defect elimination evaluation method and device and computer equipment | |
| CN112686215B (en) | Track tracking monitoring and early warning system and method for carrier vehicle | |
| Posada Moreno et al. | Cargo wagon structural health estimation using computer vision | |
| CN110826473A (en) | Neural network-based automatic insulator image identification method | |
| CN115810010B (en) | Train pantograph online detection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |