CN117495759A - Sleeper crack detection method - Google Patents
Sleeper crack detection method Download PDFInfo
- Publication number
- CN117495759A CN117495759A CN202210872553.0A CN202210872553A CN117495759A CN 117495759 A CN117495759 A CN 117495759A CN 202210872553 A CN202210872553 A CN 202210872553A CN 117495759 A CN117495759 A CN 117495759A
- Authority
- CN
- China
- Prior art keywords
- image
- sleeper
- crack
- area
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 241001669679 Eleotris Species 0.000 title claims abstract description 129
- 238000001514 detection method Methods 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 claims abstract description 62
- 238000005070 sampling Methods 0.000 claims abstract description 49
- 238000003709 image segmentation Methods 0.000 claims abstract description 19
- 239000013598 vector Substances 0.000 claims description 21
- 230000011218 segmentation Effects 0.000 claims description 12
- 239000004575 stone Substances 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000013135 deep learning Methods 0.000 claims description 9
- 238000013145 classification model Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000007637 random forest analysis Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 229910000831 Steel Inorganic materials 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000010959 steel Substances 0.000 description 3
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 235000017166 Bambusa arundinacea Nutrition 0.000 description 1
- 235000017491 Bambusa tulda Nutrition 0.000 description 1
- 241001330002 Bambuseae Species 0.000 description 1
- 235000015334 Phyllostachys viridis Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000011425 bamboo Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000356 contaminant Substances 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012567 pattern recognition method Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30116—Casting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a sleeper crack detection method, which comprises the steps of acquiring a track color image and a depth image by a 3D camera, finding a sleeper region by a target detection method, removing a region which is not easy to generate cracks in the sleeper region, taking the region as a crack detection candidate region, carrying out crack region detection in the crack detection candidate region by adopting an image segmentation or suspected crack detection and dense sampling classification method, and finally counting the crack region to obtain sleeper crack quantity, length and maximum width information.
Description
Technical Field
The invention relates to the technical field of rail traffic disease detection, in particular to a sleeper crack detection method.
Background
With the rapid development of rail traffic, diseases on rail lines are increasing, for example: the sleeper is used as a rail bearing piece, and after long-term use, the sleeper is easy to crack to form cracks, so that the bearing capacity of the rail is affected. Therefore, it is necessary to periodically check the sleeper crack. The sleeper crack detection is carried out by adopting a manual method, so that the problems of low efficiency, difficulty in quantification and the like exist. In recent years, the visual detection technology is widely applied to track defect detection, and the obtained track surface image can be used for sleeper surface crack detection. However, in the conventional method, a pure image method is often used for crack detection. The actual operation circuit comprises: ballastless track and ballasted track. The sleeper of the ballastless track is generally a short sleeper, the short sleeper is positioned below a steel rail and is generally flat, however, interference such as foreign matters and cables exists, and therefore the cable edge and a real track bed crack are difficult to distinguish by an image-based method. The sleeper of the ballast track is generally a long sleeper, spans the whole track, has the interferences of broken stone, foreign matters, signal equipment and the like, and also brings great challenges to sleeper crack detection. Existing image crack and crack detection methods (such as deep crack, IEEE Transactions on Image Processing (Volume: 28,Issue:3,March 2019) deep crack: learning Hierarchical Convolutional Features for Crack Detection) are difficult to cope with these complications, and for this purpose, the present invention proposes a new method for implementing the occipital crack detection method.
Disclosure of Invention
In order to solve the problems in the background technology, the invention provides a sleeper crack detection method, which comprises the following steps:
s1, acquiring a track color image and a depth image by adopting a 3D camera;
s2, detecting sleeper areas on the color images and/or the depth images by adopting a target detection method;
s3, shielding an area which is not easy to generate cracks in the sleeper area, and taking the rest area as a sleeper crack detection candidate area;
s4, detecting a crack region by adopting an image segmentation method on a color image and/or a depth image in a sleeper crack detection candidate region;
and S5, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks.
The 3D cameras include, but are not limited to, line structured light 3D cameras, binocular vision 3D cameras, area structured light 3D cameras, binocular speckle 3D cameras, and the like; the acquired color image includes: color images and RGB color images.
The areas where cracking is not likely to occur include, but are not limited to, fasteners, signaling devices, crushed stone, debris, cables.
The detection method of the sleeper crack detection candidate area comprises the following steps: and marking fasteners, signal equipment, broken stones, sundries and cables on the color image and/or the depth image, training a target detection or image segmentation model, detecting or segmenting the fastener, the signal equipment, the broken stones, the sundries and the cable area by adopting a target detection or image segmentation method, and subtracting the fastener, the signal equipment, the broken stones, the sundries and the cable area from the sleeper area to be used as sleeper crack detection candidate areas.
Further, the method for detecting the sleeper crack detection candidate area further comprises the following steps: detecting fasteners and signal equipment in a sleeper region on the color image and/or the depth image by adopting a target detection method, shielding the regions, detecting a sleeper flat region in the residual region, and taking the detected sleeper flat region as a sleeper crack detection candidate region;
the sleeper flat area detection method comprises the following steps:
the method comprises the steps of 1, marking a sleeper flat area on a color image and/or a depth image, training an image segmentation model, and dividing the sleeper flat area in a residual area;
and 2, filling the holes in the residual area, performing expansion operation after filling the holes, setting a sampling area with the height of h by taking the transverse central line of the sleeper as the center, taking the minimum value of each column of pixels of the depth image in the sampling area to form a sleeper cross section curve, generating a sleeper plane reference image by using the sleeper cross section curve, making a difference between the depth image and the sleeper plane reference image, and finding out an area with the height similar to the sleeper plane by threshold segmentation to be used as a sleeper flat area.
Further, the S4 th step is replaced with the following operations:
s4-1, processing the color image and/or the depth image in a sleeper crack detection candidate area to detect a suspected crack area;
s4-2, performing dense sampling on the color image and the depth image by adopting sliding window operation and scanning suspected crack areas to obtain a dense sampling image block;
s4-3, dividing the densely sampled image blocks into cracks and non-crack negatives by using a classifier; when the densely sampled image block is judged to be non-cracked, deleting the corresponding sampling area; when a densely sampled image block is a crack, the corresponding sampling region is marked as a crack.
The method for detecting the suspected crack area comprises the following steps:
firstly, on a color image, a suspected crack region is extracted, and the specific steps are as follows:
a-1, carrying out Fourier transform on the color image in a sleeper crack detection candidate area to obtain a Fourier frequency domain image;
a-2, carrying out smooth filtering on the frequency domain image, eliminating high-frequency components, and then carrying out inverse transformation to obtain a smooth image;
a-3: subtracting the Fourier smooth image from the original color image to obtain a gradient enhanced image;
a-4, detecting and positioning a linear region by using a linear filter to serve as a suspected crack region;
secondly, on the depth image, extracting suspected crack areas,
the specific method comprises the following steps: method 1: performing expansion operation on the depth image D0 to obtain a depth image D1; taking the region exceeding the set threshold T3 as a suspected crack region by making a difference between the depth image D1 and the depth image D0; method 2: calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region;
and finally, merging the suspected crack areas extracted from the color image and the depth image to obtain a final suspected crack area.
The dense sampling method in S4-2 is as follows: and during sliding window extraction, estimating the main direction of the crack in the sampling window, aligning the sampling window with the main direction, sampling the color image and the depth image, and aligning the main direction of the suspected crack with the horizontal direction or the vertical direction in the sampling image block.
The classification method of the densely sampled image blocks comprises the following steps: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
Further, the dense sampling image block classification method is as follows: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
The densely sampled image block classification method also includes a deep learning based classification method including, but not limited to, VGG or ResNet or VIT or MobileNet classification model.
In the method, the target detection method comprises a method based on image feature extraction and classifier recognition and a target detection method based on deep learning; methods based on image feature extraction+classification recognition include, but are not limited to, hog+svm or DPM; target detection methods based on deep learning, including but not limited to FasterRCNN or Yolov3 or Yolov5 or Yolov7 or Yolox or DETR; the image segmentation methods include, but are not limited to, threshold segmentation or region filling or UNET or DeepLabv3 or SOLOV2.
Further, when the deep learning method is adopted, the color image and the depth image are combined in a channel mode, and target detection and image segmentation are carried out on the multi-channel image.
The beneficial effects of the invention are as follows:
1. the rail color image and the depth image obtained by the 3D camera are adopted, so that the shape information of the height change of the sleeper surface is increased, the crack, the color pollution and the foreign body boundary interference are favorably distinguished, and the crack detection accuracy is further improved
In the practical running track, especially urban subway, the line condition is complex, the track contains accessory equipment such as steel rail, fastener, sleeper, cable, signal equipment, ditch, damping section of thick bamboo, etc. in the colour image that traditional method adopted, the boundary of these equipment easily causes the misjudgement. The boundaries of these regions are all of a certain height difference, so that the depth image provides a new information supplement that can be used to eliminate the effect of these boundaries. In addition, the contaminants of the sleeper, such as greasy dirt, are also easily misjudged as cracks simply from the image, but are easily distinguished from the depth image.
2. Provides a simple and efficient sleeper crack candidate detection area detection method,
After fasteners and signal equipment in the sleeper area are detected by adopting a target detection method, the areas are shielded, flat area detection is carried out in the residual area by utilizing a depth image, and the detected flat area is used as a sleeper crack detection candidate area. And filling the holes in the depth image, performing expansion operation after filling the holes, taking the transverse central line of the sleeper as a center, setting a sampling area with the height of h, taking the minimum value of each column of pixels of the image in the sampling area to form a sleeper cross section curve, generating a sleeper plane reference image by using the sleeper cross section curve, making a difference between the depth image and the sleeper plane reference image, and finding out an area with the height similar to the sleeper plane through threshold segmentation as a sleeper flat area. The sleeper crack detection is carried out in the flat area, so that the crack detection interference caused by broken stone, foreign matters, signal equipment and the like on the sleeper can be effectively avoided, and the sleeper crack detection method has important application value for the ballasted track line.
3. Provides a simple and effective suspected crack judging method
The image blocks are obtained by densely sampling the preliminary determined suspected crack areas, the image blocks are subjected to feature extraction, and the image blocks are classified by adopting a conventional pattern recognition method, so that the calculation complexity can be effectively reduced, and the method is convenient to use in low-power consumption application scenes such as detection trolleys and the like.
Drawings
FIG. 1 is a schematic view of a short sleeper track;
FIG. 2 is a schematic view of a long sleeper rail;
FIG. 3 is a schematic view of sleeper flat area extraction;
fig. 4 is a schematic diagram of suspected crack image sampling, in which a is a layout diagram of a partial sampling window of a crack region, and b is a suspected crack image sampling result;
FIG. 5 shows the effect of positive and negative samples on color images, depth images, and curve distribution shapes after projection with image tiles; wherein a (0) is the effect of the crack on the depth image, a (1) is the effect of the crack on the color image, a (2) is the schematic diagram of the longitudinal projection of the crack dense sampling block on the color image along the main direction, and a (3) is the schematic diagram of the longitudinal projection of the crack dense sampling block on the depth image along the main direction; b (0) is the effect of the foreign matter on the depth image, b (1) is the effect of the foreign matter on the color image, b (2) is the schematic diagram of the longitudinal projection of the dense sample block of the foreign matter boundary on the color image along the main direction, and b (3) is the schematic diagram of the longitudinal projection of the dense sample block of the foreign matter boundary on the depth image along the main direction; c (0) is the effect of the polluted area on the depth image, c (1) is the effect of the polluted area on the color image, c (2) is the schematic diagram of the longitudinal projection of the dense sample block of the polluted area on the color image along the main direction, and c (3) is the schematic diagram of the longitudinal projection of the dense sample block of the polluted area on the depth image along the main direction.
1-rail, 2-sleeper, 3-cable, 4-track bed, 5-sleeper marking frame, 6-crack, 7-signal equipment, 8-sleeper central line, 9-sampling area, 10-sampling window, 11-main direction, 12-pollution area, 13-foreign matter.
Detailed Description
The present invention will be described in detail below with reference to the drawings and examples thereof, but the scope of the present invention is not limited to the examples.
Example 1:
a sleeper crack detection method comprises the following steps:
s1, acquiring a track color image and a depth image by adopting a 3D camera
The 3D camera is a line structured light 3D camera, a binocular vision 3D camera, a surface structured light 3D camera and a binocular speckle 3D camera; the acquired color image includes: color images and RGB color images.
S2, detecting sleeper areas on the color image by adopting a target detection method
Labeling sleeper on the collected color image in advance, and training FasterRCNN or Yolov3 or Yolov5 or Yolov7 or Yolox or DETR target detector; performing target detection on the currently acquired color image, wherein the detected area can be a sleeper area;
as shown in fig. 1, for the short sleeper lines, the rail areas on two sides are marked respectively; as shown in fig. 2, the long sleeper lines are directly marked.
S3, shielding the area which is not easy to generate cracks in the sleeper area, and taking the rest area as a sleeper crack detection candidate area
The method comprises the steps of marking fasteners, signal equipment, broken stones, sundries and cables on a color image, training a FasterRCNN or YOLOV3 or YOLOV5 or YOLOV7 or YOLOX or DETR target detection model, detecting the areas of the fasteners, the signal equipment, the broken stones, the sundries and the cables in the sleeper area, and subtracting the areas from the sleeper area to be used as sleeper crack detection candidate areas.
S4, detecting a crack region in the sleeper crack detection candidate region by adopting an image segmentation method
Dividing and marking sleeper cracks in advance, and training a UNET or SOLOV2 image dividing model on the color image; in the sleeper crack detection candidate region, image segmentation is performed on the color image, and the segmented region is taken as a sleeper crack region.
And S5, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks.
And (3) connecting the sleeper crack areas obtained in the step (S4), counting the number of the communication domains by using a Blob analysis method as the number of cracks, calculating the Ma Hadu distance from the communication domains to the tail as the crack length, and calculating the maximum value of the normal width of the communication domains from the head to the tail as the maximum width of the cracks.
Example 2
On the basis of embodiment 1, the difference is that object detection and image segmentation are performed on the depth image.
Example 3
The difference from embodiment 1 is that object detection and image segmentation are performed on a multi-channel image formed by combining a color image and a depth image.
Example 4
On the basis of example 1, the difference is that the detection method for the sleeper crack detection candidate region in step S3 is: detecting fasteners and signal equipment in a sleeper region on the color image and/or the depth image by adopting a target detection method, shielding the regions, detecting a sleeper flat region in the residual region, and taking the detected sleeper flat region as a sleeper crack detection candidate region;
the sleeper flat area detection method comprises the following steps:
in the method 1, a sleeper flat area is marked on a color image and/or a depth image, an image segmentation model is trained, and the sleeper flat area is segmented in the rest area. Also here, UENT or SOLOV2 image segmentation methods may be employed.
In the method 2, in the remaining area, cavity filling is carried out on the depth image, and then expansion operation is carried out after cavity filling, as shown in fig. 3, a sampling area (9) with the height of h is arranged by taking the transverse central line (8) of the sleeper as the center, the minimum value of each column of pixels of the depth image in the sampling area is taken to form a sleeper cross section curve, a sleeper plane reference image is generated by the sleeper cross section curve, the depth image is differed from the sleeper plane reference image, an area with the height similar to the sleeper plane is found out through threshold segmentation to be used as a sleeper flat area, and the height h is the length of the sleeper along the longitudinal direction of the steel rail.
The flat area obtained by the method can effectively eliminate boundary interference of broken stone, foreign matters and signal equipment on the sleeper, particularly on the long sleeper in a ballasted line, and effectively improve the detection accuracy of the sleeper crack.
Example 5
On the basis of example 1, the difference is that the S4 th step is replaced with the following operation:
s4-1, processing the color image and/or the depth image in a sleeper crack detection candidate area to detect a suspected crack area;
s4-2, performing dense sampling on the color image and the depth image by adopting sliding window operation and scanning suspected crack areas to obtain a dense sampling image block;
s4-3, dividing the densely sampled image blocks into cracks and non-crack negatives by using a classifier; when the densely sampled image block is judged to be non-cracked, deleting the corresponding sampling area; when a densely sampled image block is a crack, the corresponding sampling region is marked as a crack.
The method for detecting the suspected crack area comprises the following steps:
firstly, on a color image, a suspected crack region is extracted, and the specific steps are as follows:
a-1, carrying out Fourier transform on the color image in a sleeper crack detection candidate area to obtain a Fourier frequency domain image;
a-2, carrying out smooth filtering on the frequency domain image, eliminating high-frequency components, and then carrying out inverse transformation to obtain a smooth image;
a-3: subtracting the Fourier smooth image from the original color image to obtain a gradient enhanced image;
a-4, detecting and positioning a linear region by using a linear filter to serve as a suspected crack region;
secondly, on the depth image, extracting suspected crack areas,
the specific method comprises the following steps: method 1: performing expansion operation on the depth image D0 to obtain a depth image D1; taking the region exceeding the set threshold T3 as a suspected crack region by making a difference between the depth image D1 and the depth image D0; method 2: calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region;
and finally, merging the suspected crack areas extracted from the color image and the depth image to obtain a final suspected crack area.
The dense sampling method in S4-2 is as follows: as shown in fig. 4, during sliding window extraction, the main direction of the crack is estimated in the sampling window, after the sampling window is aligned with the main direction, color image and depth image sampling are performed, and the main direction of the suspected crack in the sampling image block is aligned with the horizontal direction or the vertical direction.
The classification method of the densely sampled image blocks comprises the following steps: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
Example 6
On the basis of embodiment 5, the dense sample image block classification method is different in that: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
The main direction as shown in fig. 4 samples Ji Choumi, so that the feature space distribution diversity of the sampled image blocks is reduced, and further, the features of the cracks on the color image and the depth image are effectively obtained through transverse and longitudinal projection. As shown in fig. 5, the slit is projected in the main direction in a concave area (fig. 5 (a 1)) on the flat area, and is projected in the main direction in a concave shape (fig. 5 (a 3)), and is projected in the main direction in a dark stripe area (fig. 5 (a 0)), and is projected in a concave shape (fig. 5 (a 2));
a negative example, such as a foreign matter boundary, is a first-order step curve (fig. 5 (b 3)) after the depth image is projected in the main direction, and is a line-like dark region (fig. 5 (b 0)) after the color image is projected in the main direction, and is a concave-down shape (fig. 5 (b 2));
the negative example, such as tie contamination area, is a flat curve in the longitudinal projection curve after the depth image is projected in the main direction (fig. 5 (c 1)), a flat curve in the longitudinal projection curve after the color image is projected in the main direction (fig. 5 (c 3)), and a concave shape with a large area after the color image is projected in the main direction (fig. 5 (c 0)), and the longitudinal projection curve is projected in the main direction (fig. 5 (c 2)).
Obviously, after the feature extraction, the obtained dense sampling image block features have stronger discriminant, and are favorable for high-precision classification by adopting a simple classifier.
Example 6
On the basis of embodiment 5, the dense sample image block classification method is different in that it adopts a classification method based on deep learning, including but not limited to VGG or ResNet or VIT or MobileNet classification model.
While the principles of the invention have been described in detail in connection with the preferred embodiments thereof, it should be understood by those skilled in the art that the foregoing embodiments are merely illustrative of the implementations of the invention and are not intended to limit the scope of the invention. The details of the embodiments are not to be taken as limiting the scope of the invention, and any obvious modifications based on equivalent changes, simple substitutions, etc. of the technical solution of the invention fall within the scope of the invention without departing from the spirit and scope of the invention.
Claims (10)
1. The sleeper crack detection method is characterized by comprising the following steps of:
s1, acquiring a track color image and a depth image by adopting a 3D camera;
s2, detecting sleeper areas on the color images and/or the depth images by adopting a target detection method;
s3, shielding an area which is not easy to generate cracks in the sleeper area, and taking the rest area as a sleeper crack detection candidate area;
s4, detecting a crack region by adopting an image segmentation method on a color image and/or a depth image in a sleeper crack detection candidate region;
and S5, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks.
2. A method for detecting a sleeper crack as defined in claim 1, wherein said method for detecting a sleeper crack detection candidate region is as follows: and marking fasteners, signal equipment, broken stones, sundries and cables on the color image and/or the depth image, training a target detection or image segmentation model, detecting or segmenting the fastener, the signal equipment, the broken stones, the sundries and the cable area by adopting a target detection or image segmentation method, and subtracting the fastener, the signal equipment, the broken stones, the sundries and the cable area from the sleeper area to be used as sleeper crack detection candidate areas.
3. A method for detecting a sleeper crack as defined in claim 1 or 2, wherein said method for detecting a sleeper crack detection candidate region is as follows: detecting fasteners and signal equipment in a sleeper region on the color image and/or the depth image by adopting a target detection method, shielding the regions, detecting a sleeper flat region in the residual region, and taking the detected sleeper flat region as a sleeper crack detection candidate region;
the sleeper flat area detection method comprises the following steps:
the method comprises the steps of 1, marking a sleeper flat area on a color image and/or a depth image, training an image segmentation model, and dividing the sleeper flat area in a residual area;
and 2, filling the holes in the residual area, performing expansion operation after filling the holes, setting a sampling area with the height of h by taking the transverse central line of the sleeper as the center, taking the minimum value of each column of pixels of the depth image in the sampling area to form a sleeper cross section curve, generating a sleeper plane reference image by using the sleeper cross section curve, making a difference between the depth image and the sleeper plane reference image, and finding out an area with the height similar to the sleeper plane by threshold segmentation to be used as a sleeper flat area.
4. A method of sleeper crack detection as defined in claim 1 wherein step S4 is replaced by the steps of:
s4-1, processing the color image and/or the depth image in a sleeper crack detection candidate area to detect a suspected crack area;
s4-2, performing dense sampling on the color image and the depth image by adopting sliding window operation and scanning suspected crack areas to obtain a dense sampling image block;
s4-3, dividing the densely sampled image blocks into cracks and non-crack negatives by using a classifier; when the densely sampled image block is judged to be non-cracked, deleting the corresponding sampling area; when a densely sampled image block is a crack, the corresponding sampling region is marked as a crack.
5. A method of detecting a tie crack as claimed in claim 4 wherein said method of detecting a suspected crack region is:
firstly, on a color image, a suspected crack region is extracted, and the specific steps are as follows:
a-1, carrying out Fourier transform on the color image in a sleeper crack detection candidate area to obtain a Fourier frequency domain image;
a-2, carrying out smooth filtering on the frequency domain image, eliminating high-frequency components, and then carrying out inverse transformation to obtain a smooth image;
a-3: subtracting the Fourier smooth image from the original color image to obtain a gradient enhanced image;
a-4, detecting and positioning a linear region by using a linear filter to serve as a suspected crack region;
secondly, on the depth image, extracting suspected crack areas,
the specific method comprises the following steps: method 1: performing expansion operation on the depth image D0 to obtain a depth image D1; taking the region exceeding the set threshold T3 as a suspected crack region by making a difference between the depth image D1 and the depth image D0; method 2: calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region;
and finally, merging the suspected crack areas extracted from the color image and the depth image to obtain a final suspected crack area.
6. A method of sleeper crack detection as described in claim 4 wherein said dense sampling method in S4-2 is: and during sliding window extraction, estimating the main direction of the crack in the sampling window, aligning the sampling window with the main direction, sampling the color image and the depth image, and aligning the main direction of the suspected crack with the horizontal direction or the vertical direction in the sampling image block.
7. A tie crack detection method as claimed in claim 4 or 6, wherein the classification of the densely sampled image blocks is: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
8. A tie crack detection method as claimed in claim 7 wherein said densely sampled image block classification method is: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
9. A tie crack detection method as claimed in claim 4 or 6 wherein the densely sampled image block classification method may also employ a deep learning based classification method including but not limited to VGG or res net or VIT or MobileNet classification models.
10. A sleeper crack detection method as described in any one of claims 1-4, wherein said target detection method comprises a method based on image feature extraction + classifier recognition, a target detection method based on deep learning; methods based on image feature extraction+classification recognition include, but are not limited to, hog+svm or DPM; target detection methods based on deep learning, including but not limited to FasterRCNN, YOLOV or YOLOV7 or YOLOX or DETR; the image segmentation methods include, but are not limited to, threshold segmentation or region filling or UNET or deep labv3 or SOLOV2; when the deep learning method is adopted, the color image and the depth image are combined in a channel mode, and target detection and image segmentation are carried out on the multi-channel image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210872553.0A CN117495759A (en) | 2022-07-24 | 2022-07-24 | Sleeper crack detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210872553.0A CN117495759A (en) | 2022-07-24 | 2022-07-24 | Sleeper crack detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117495759A true CN117495759A (en) | 2024-02-02 |
Family
ID=89666503
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210872553.0A Pending CN117495759A (en) | 2022-07-24 | 2022-07-24 | Sleeper crack detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117495759A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952977A (en) * | 2024-03-27 | 2024-04-30 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
-
2022
- 2022-07-24 CN CN202210872553.0A patent/CN117495759A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952977A (en) * | 2024-03-27 | 2024-04-30 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
| CN117952977B (en) * | 2024-03-27 | 2024-06-04 | 山东泉海汽车科技有限公司 | Pavement crack identification method, device and medium based on improvement yolov s |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145161B (en) | Pavement crack digital image processing and identifying method | |
| CN101893580B (en) | Digital image based detection method of surface flaw of steel rail | |
| CN105488454B (en) | Front vehicles detection and ranging based on monocular vision | |
| CN113592861B (en) | Bridge crack detection method based on dynamic threshold | |
| CN110211101A (en) | A kind of rail surface defect rapid detection system and method | |
| CN109272548B (en) | Method for measuring diameter of bubbles in flotation process | |
| CN111539436B (en) | Rail fastener positioning method based on straight template matching | |
| CN108921076B (en) | Pavement crack disease self-adaptive constant false alarm detection method based on image | |
| CN108759670A (en) | A kind of contact line abrasion device for dynamically detecting based on non-contact detection technology | |
| CN110717900B (en) | Pantograph abrasion detection method based on improved Canny edge detection algorithm | |
| CN110009633B (en) | Steel rail surface defect detection method based on reverse Gaussian difference | |
| CN104992429A (en) | Mountain crack detection method based on image local reinforcement | |
| CN117495759A (en) | Sleeper crack detection method | |
| CN108725511A (en) | The real time position modification method of rail corrugation string measuring point | |
| CN112233111A (en) | Tunnel gap detection method based on digital image processing | |
| CN112800974A (en) | Subway rail obstacle detection system and method based on machine vision | |
| CN110189419B (en) | Vehicle-mounted Lidar steel rail point cloud extraction method based on generalized neighborhood height difference | |
| CN110672632A (en) | Tunnel disease identification method | |
| CN107610094B (en) | Three-dimensional crack detection method based on ellipsoid three-dimensional representation | |
| CN111723793A (en) | Real-time rigid contact net positioning point identification method | |
| Kaur et al. | An Efficient Method of Number Plate Extraction from Indian Vehicles Image | |
| CN115541591A (en) | Method and system for detecting abrasion edge of carbon pantograph slider of train | |
| CN100555329C (en) | Based on multi-scale wavelet transform video foreground moving Object Segmentation method | |
| CN117495758A (en) | Ballast bed crack and crack detection method based on 3D camera | |
| CN112949483A (en) | Non-contact rail stretching displacement real-time measurement method based on fast R-CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |