CN117495758A - Ballast bed crack and crack detection method based on 3D camera - Google Patents
Ballast bed crack and crack detection method based on 3D camera Download PDFInfo
- Publication number
- CN117495758A CN117495758A CN202210872552.6A CN202210872552A CN117495758A CN 117495758 A CN117495758 A CN 117495758A CN 202210872552 A CN202210872552 A CN 202210872552A CN 117495758 A CN117495758 A CN 117495758A
- Authority
- CN
- China
- Prior art keywords
- crack
- image
- cracks
- depth image
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 37
- 238000005070 sampling Methods 0.000 claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 42
- 241001669679 Eleotris Species 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 17
- 229910000831 Steel Inorganic materials 0.000 claims description 11
- 239000010959 steel Substances 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 10
- 238000009499 grossing Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 4
- 235000002566 Capsicum Nutrition 0.000 claims description 3
- 239000006002 Pepper Substances 0.000 claims description 3
- 235000016761 Piper aduncum Nutrition 0.000 claims description 3
- 235000017804 Piper guineense Nutrition 0.000 claims description 3
- 235000008184 Piper nigrum Nutrition 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 235000012434 pretzels Nutrition 0.000 claims description 3
- 238000007637 random forest analysis Methods 0.000 claims description 3
- 150000003839 salts Chemical class 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 244000203593 Piper nigrum Species 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 10
- 238000000605 extraction Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 241000722363 Piper Species 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 235000017166 Bambusa arundinacea Nutrition 0.000 description 1
- 235000017491 Bambusa tulda Nutrition 0.000 description 1
- 244000082204 Phyllostachys viridis Species 0.000 description 1
- 235000015334 Phyllostachys viridis Nutrition 0.000 description 1
- 239000011425 bamboo Substances 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012567 pattern recognition method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30116—Casting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a track bed crack and crack detection method based on a 3D camera, which comprises the steps of acquiring a track color image and a depth image by the 3D camera, detecting a candidate area of track bed crack and crack detection in the depth image, detecting a suspected crack and crack area in the candidate area, densely sampling the color image and the depth image in the candidate area, judging whether a densely sampled image block is a real crack and a real crack by adopting an image classifier, connecting the crack or crack area, and counting the number, the length and the maximum width parameters of the crack. According to the method, the interference of the boundary of the cable, the signal equipment and other accessory equipment on the track bed on crack and crack detection is eliminated through the depth image, so that the calculation complexity is reduced, and the effective detection of the track bed crack and crack is realized.
Description
Technical Field
The invention relates to the technical field of rail transit disease detection, in particular to a track bed crack and crack detection method based on a 3D camera.
Background
Along with the rapid development of urban rail transit, the crack of a track bed is taken as an important disease of a railway track, and becomes an important detection object for railway inspection. However, the existing main ballast bed crack and crack detection method generally adopts texture images shot by a linear array camera or an area array camera, and crack detection is carried out on the texture images, so that interference of suspected cracks and cracks in an actual line is easily received. As shown in fig. 1, for urban rails, the lines are provided with objects such as water ditches, cables, signal equipment and the like besides steel rails, fasteners and sleepers, and the edges of the steel rails, the sleepers, the water ditches, the cables, the signal equipment and the like are very similar to cracks and cracks, so that the detection of cracks and the cracks of ballast beds is greatly interfered. In recent years, deep learning technology has advanced, and scholars have proposed crack detection methods based on CNN architecture, which have very high computational complexity and are unfavorable for application to embedded data processors of small platforms such as track inspection trolleys. Therefore, aiming at the problems of track bed cracks and cracks in complex scenes, the invention provides the track bed cracks and cracks detection method based on the 3D camera, which can reduce the computational complexity and realize high-accuracy track bed cracks and cracks detection by combining and applying the traditional image processing and pattern recognition technology.
Disclosure of Invention
In order to solve the problems in the background technology, the invention provides a track bed crack and crack detection method based on a 3D camera, which comprises the following steps:
s1, acquiring a track color image and a depth image by adopting a 3D camera;
s2, detecting a ballast bed crack and a crack detection candidate area in the depth image;
s3, processing the color image in the road bed crack and crack detection candidate area to detect suspected cracks and crack areas; processing the depth image and detecting a suspected crack region;
s4, performing dense sampling on suspected cracks and crack areas on the color image and the depth image to obtain dense sampling image blocks;
s5, dividing the dense sampling image block into a positive sample and a negative sample by using a classifier, wherein the positive sample comprises: cracks, repairing cracks and cracks on cracks, the negative sample is an image other than a positive sample, and the positive sample is further divided into: crack, repairing crack on crack and crack; when the densely sampled image block is judged to be a negative sample, deleting the corresponding sampling area; when the densely sampled image block is a crack, marking the corresponding sampling area as a crack; when the densely sampled image block is a crack, marking the corresponding sampling area as a crack;
s6, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks; the crack areas are connected, and the number, length and maximum width of the cracks are measured.
The 3D camera is a GrayD or RGBD camera and is used for acquiring a track gray level image and a depth image or an RGB color image and a depth image with aligned pixels.
The method for detecting the ballast bed area on the depth image comprises the following steps: taking the mode of each column of pixel values in the depth image along the longitudinal direction of the steel rail to obtain a reference datum line of the track cross section, generating a reference image B with the same size by using the datum line, using the current depth image C to make a difference with the reference image B, and extracting a region R1 with the difference smaller than a set threshold T1; threshold segmentation is carried out on the reference image B, and a region R2 exceeding a set threshold T2 is taken; performing edge detection on the current depth image C, and performing expansion operation on an edge region to obtain a region R3; taking the intersection of R1 and R2 to obtain R4; r4 and R3 are differenced to obtain R5, wherein R5 is a ballast bed crack and crack detection candidate region.
Further, when the reference datum line of the cross section of the track is extracted, images of areas on two sides of the steel rail are taken for transverse projection and longitudinal projection, the longitudinal position of the sleeper is found in the transverse projection according to the height difference between the track bed and the sleeper, the transverse position of the sleeper is found in the longitudinal projection, a sleeper shielding area is arranged in a depth image according to the sleeper position and the sleeper size, and the mode number in the row direction is counted after the sleeper image is shielded.
The method for processing the color image to detect suspected cracks and crack areas comprises the following steps: after salt and pepper noise denoising is carried out on the color image, fourier transformation is carried out, and a Fourier frequency domain image is obtained; smoothing and filtering the frequency domain image to eliminate high-frequency components, and then carrying out inverse transformation to obtain a smooth image; and subtracting the Fourier smoothing image from the pretzel noise denoising image to obtain a gradient enhanced image, and detecting and positioning a linear region by adopting a line filter to serve as a suspected crack region.
The method for processing the depth image to detect the suspected crack region includes but is not limited to: performing expansion operation on the depth image D0 to obtain a depth image D1, wherein the depth image D1 and the depth image D0 are differenced, and the region exceeding a set threshold T3 is taken as a suspected crack region; or calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region.
The method for simultaneously densely sampling suspected cracks and crack areas on the color image and the depth image comprises the following steps: and scanning suspected cracks and crack areas by adopting sliding window operation, and extracting image blocks on the color image and the depth image.
Further, when the sliding window extraction is performed, estimating the local main directions of cracks and fissures, performing color image and depth image sampling after aligning the sampling window with the main directions, and aligning the main directions of suspected cracks and fissures with the horizontal direction or the vertical direction in the sampling image block.
The dense sampling image block classification method comprises the following steps: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
Further, the dense sampling image block classification method is as follows: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
Further, the dense sampling image block classification method is a classification method based on deep learning, and is realized on an embedded platform by adopting a light-weight network.
The ballast bed crack/crack parameter measuring method comprises the following steps: connecting the crack/crack areas, and calculating the number of the cracks/cracks by adopting a Blob analysis method; measuring Manhattan distance from a start point to an end point of each crack/crack as a crack/crack length; traversing from the start point to the end point of the crack/crack, and taking the normal direction maximum width as the crack/crack maximum width.
The beneficial effects of the invention are as follows:
1. the track color image and the depth image obtained by the 3D camera are adopted, so that the reference basis for crack and crack detection judgment is increased, and the crack and crack detection accuracy is improved
In the practical running track, especially urban subway, the line condition is complex, the track contains accessory equipment such as steel rail, fastener, sleeper, cable, signal equipment, ditch, damping section of thick bamboo, etc. in the colour image that traditional method adopted, the boundary of these equipment easily causes the misjudgement. The boundaries of these regions are all of a certain height difference, so that the depth image provides a new information supplement that can be used to eliminate the effect of these boundaries.
2. Provides a simple and effective method for detecting the crack candidate detection area of the ballast bed,
By adopting a method for estimating the track datum line of the depth image, the track bed datum plane (excluding the areas of signal equipment, steel rail, sleeper, fastener, cable, ditch and the like) can be effectively detected by utilizing image comparison threshold segmentation, and boundaries of the steel rail, fastener, sleeper, cable, signal equipment, ditch, vibration reduction barrel and the like can be detected by performing difference after image expansion operation of different scales on the depth image, and the boundary areas are deducted from the track bed datum plane areas, namely the real track bed areas, wherein track bed cracks and cracks can exist in the areas. By the method, the interference of ballast bed accessory equipment on crack and crack detection can be effectively eliminated.
3. Provides a simple and effective suspected crack and crack judging method
The image blocks are obtained by densely sampling the preliminary determined suspected cracks and crack areas, the features of the image blocks are extracted, and the image blocks are classified by using the conventional pattern recognition method, so that the calculation complexity can be effectively reduced, and the method is convenient to use in low-power consumption application scenes such as detection trolleys. Furthermore, as shown in fig. 2 and 3, the extracted main direction is used for sampling Ji Choumi, so that the feature space distribution diversity of the sampled image blocks is reduced, and the features of the ballast bed cracks and the cracks on the color image and the depth image are further obtained through transverse and longitudinal projection.
As shown in fig. 4, the ballast bed crack, after being projected in the main direction, has a downward concave shape (fig. 4.A (3)), and after being projected in the main direction, has a downward concave shape (fig. 4.A (2)), and after being projected in the main direction, has a dark stripe-shaped region (fig. 4.A (0)), has a downward concave shape (fig. 4.A (2));
ballast bed cracks, after the depth image is a flat area (fig. 4.B (1)), the longitudinal projection curve is a flat curve (fig. 4.B (3)), after the depth image is projected in the main direction, the longitudinal projection curve is a concave shape (fig. 4.B (2)) after the color image is projected in the main direction, the color image is a linear dark area (fig. 4.B (0));
a negative example, such as an accessory signal device boundary, is a one-step curve (fig. 4.C (3)) after the depth image is projected in the main direction, and a downward concave shape (fig. 4.C (2)) after the color image is projected in the main direction; in the ballast bed polluted region, the longitudinal projection curve is a flat curve (fig. 4.D (3)) after the depth image is projected in the main direction (fig. 4.D (1)), and the longitudinal projection curve is a concave shape of a large area after the color image is projected in the main direction (fig. 4.D (0)) after the color image is projected in the irregular spot region (fig. 4.D (2)).
Obviously, after the feature extraction, the obtained dense sampling image block features have stronger discriminant, and are favorable for high-precision classification by adopting a simple classifier.
Drawings
FIG. 1 is a schematic view of a ballast bed area;
fig. 2 is a schematic diagram of sampling a suspected crack image according to embodiment 1, wherein a is a layout diagram of a partial sampling window of a crack region, and b is a sampling result of the suspected crack image;
fig. 3 is a schematic diagram of sampling a suspected crack image according to embodiment 5, wherein a is a layout diagram of a partial sampling window of a crack region, and b is a sampling result of the suspected crack image;
FIG. 4 is a graph showing the effect of a crack, a signal device, a contaminated area on a color image, a depth image, and a graph showing the distribution of curves after projection using image blocks, wherein a (0) is the effect of a crack on a depth image, a (1) is the effect of a crack on a color image, a (2) is a graph showing the longitudinal projection of a crack dense sample block on a color image along a main direction, and a (3) is a graph showing the longitudinal projection of a crack dense sample block on a depth image along a main direction; b (0) is the effect of the crack on the depth image, b (1) is the effect of the crack on the color image, b (2) is the schematic diagram of the longitudinal projection of the crack dense sampling block on the color image along the main direction, and b (3) is the schematic diagram of the longitudinal projection of the crack dense sampling block on the depth image along the main direction; c (0) is the effect of the signal equipment on the depth image, c (1) is the effect of the signal equipment on the color image, c (2) is the schematic diagram of the longitudinal projection of the signal equipment boundary dense sampling block on the color image along the main direction, and c (3) is the schematic diagram of the longitudinal projection of the signal equipment boundary dense sampling block on the depth image along the main direction; d (0) is the effect of the polluted area on the depth image, d (1) is the effect of the polluted area on the color image, d (2) is the schematic diagram of the longitudinal projection of the dense sampling block of the polluted area on the color image along the main direction, and d (3) is the schematic diagram of the longitudinal projection of the dense sampling block of the polluted area on the depth image along the main direction;
1-rail, 2-sleeper, 3-signalling device, 4-ballast bed, 5-ditch, 6-cable, 7-crack, 8-sampling window of example 1, 9-sampling window of example 5, 10-crack principal axis direction, 11-crack, 12-contaminated area.
Detailed Description
The present invention will be described in detail below with reference to the drawings and examples thereof, but the scope of the present invention is not limited to the examples.
Example 1:
a ballast bed crack and crack detection method based on a 3D camera comprises the following steps:
s1, acquiring a track color image (namely a gray image) and a depth image by adopting a GrayD camera based on line structured light;
s2, detecting a ballast bed crack and a crack detection candidate area in the depth image
The specific method for detecting the ballast bed area on the depth image comprises the following steps: as shown in fig. 5, the mode of the pixel value of each column (19) in the depth image is taken along the longitudinal direction of the steel rail, and a reference datum line of the track cross section is obtained, and as shown in fig. 7, the datum line is composed of a ditch (18), a track bed (17) and the steel rail (13), and a reference image B with the same size is generated by the datum line. Wherein the concept of mode is: the pixel value participating in statistics is the most frequently occurring pixel value. The specific implementation is to calculate a statistical histogram of pixel values in each column, take the maximum number of corresponding pixel values in the statistical histogram as the mode, and set the bin of the statistical histogram to 32 or 64 or 128.
Using the current depth image C to make a difference with the reference image B, and extracting a region R1 with the difference smaller than a set threshold T1, wherein the value range of T1 is 5-10 mm; threshold segmentation is carried out on the reference image B, a region R2 exceeding a set threshold T2 is taken, the value range of T2 is H-40 mm-H-20 mm, and H is the average height of a ballast bed; performing edge detection on the current depth image C, and performing expansion operation on an edge region to obtain a region R3; taking the intersection of R1 and R2 to obtain R4; r4 and R3 are differenced to obtain R5, wherein R5 is a ballast bed crack and crack detection candidate region. The method for detecting the edge of the current depth image C comprises the steps of firstly performing a closed operation on the depth image, eliminating a cavity area in the image, and then extracting the edge of the depth image by adopting an edge detection operator.
S3, processing the color image in the road bed crack and crack detection candidate area to detect suspected cracks and crack areas, wherein the concrete method comprises the following steps: after salt and pepper noise denoising is carried out on the color image, fourier transformation is carried out, and a Fourier frequency domain image is obtained; smoothing and filtering the frequency domain image to eliminate high-frequency components, and then carrying out inverse transformation to obtain a smooth image; subtracting the Fourier smoothing image from the pretzel noise denoising image to obtain a gradient enhanced image, and detecting and positioning a linear region by adopting a line filter to serve as a suspected crack region; processing the depth image, methods of detecting suspected crack regions include, but are not limited to: performing expansion operation on the depth image D0 to obtain a depth image D1, wherein the depth image D1 and the depth image D0 are differenced, and the region exceeding a set threshold T3 is taken as a suspected crack region; or calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region.
S4, on the color image and the depth image, simultaneously carrying out dense sampling on suspected cracks and crack areas to obtain dense sampling image blocks, wherein the specific method comprises the following steps: as shown in fig. 2, a sliding window operation is adopted to scan pixels in suspected cracks and crack areas on the color image and the depth image to form a dense sampling image block.
S5, dividing the dense sampling image block into a positive sample and a negative sample by using a classifier, wherein the positive sample comprises: cracks, repairing cracks and cracks on cracks, the negative sample is an image other than a positive sample, and the positive sample is further divided into: crack, repairing crack on crack and crack; when the densely sampled image block is judged to be a negative sample, deleting the corresponding sampling area; when the densely sampled image block is a crack, marking the corresponding sampling area as a crack; when the densely sampled image block is a crack, the corresponding sampled region is marked as a crack.
The method for classifying the densely sampled image blocks is as follows: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
S6, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks; the crack areas are connected, and the number, length and maximum width of the cracks are measured.
The specific method for measuring the ballast bed crack/crack parameters is as follows: connecting the crack/crack areas, and calculating the number of the cracks/cracks by adopting a Blob analysis method; measuring Manhattan distance from a start point to an end point of each crack/crack as a crack/crack length; traversing from the start point to the end point of the crack/crack, and taking the maximum width in the direction perpendicular to the local main direction as the maximum width of the crack/crack.
Example 2
The difference from embodiment 1 is that when the reference line of the cross section of the rail is extracted, the images of the areas on both sides of the rail are taken for transverse and longitudinal projection, the longitudinal position of the sleeper is found in the transverse projection according to the height difference between the ballast bed and the sleeper, the transverse position of the sleeper is found in the longitudinal projection, the sleeper shielding area is set in the depth image according to the sleeper position and the sleeper size, and the mode in the row direction is counted after the sleeper image is shielded.
Example 3:
the difference from example 1 is that the method of dense sampling in step S4 is: as shown in fig. 3, during sliding window extraction, the local main directions (10) of cracks and fissures are estimated, after the sampling window (9) is aligned with the main directions (10), color image and depth image sampling is performed, and the main directions of suspected cracks and fissures in a sampling image block are aligned with the horizontal direction or the vertical direction.
However, in the image block of the image sample of the crack obtained by the conventional sampling method shown in fig. 2, the direction of the crack is arbitrary, and from the aspect of feature subspace dimension, the distribution divergence of the image block of the crack sampled in this way in the feature space is large, which is not beneficial to modeling the image block by a classifier model, and further a more complex classifier is required; after the crack direction alignment sampling method provided by the invention is adopted, the crack direction in the acquired crack image block is aligned vertically or horizontally (the horizontal direction alignment is shown in fig. 3), and compared with the sampling method shown in fig. 2, the mode has smaller distribution divergence from the aspect of characteristic subspace dimension, and is more beneficial to modeling by a classifier.
Example 4
The difference from embodiment 1 is that the densely sampled image block classification method is: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
Example 5
The difference from embodiment 4 is that the dense sampling image block classification method is a classification method based on deep learning, and is implemented on an embedded platform by adopting a lightweight network.
Example 6
The difference from embodiment 1 is that the 3D camera is an RGBD camera for acquiring an RGB color image and a depth image of a pixel-aligned track. In classifying suspected cracks and flaws, RGB color images are also used for feature extraction for classification.
While the principles of the invention have been described in detail in connection with the preferred embodiments thereof, it should be understood by those skilled in the art that the foregoing embodiments are merely illustrative of the implementations of the invention and are not intended to limit the scope of the invention. The details of the embodiments are not to be taken as limiting the scope of the invention, and any obvious modifications based on equivalent changes, simple substitutions, etc. of the technical solution of the invention fall within the scope of the invention without departing from the spirit and scope of the invention.
Claims (10)
1. The track bed crack and crack detection method based on the 3D camera is characterized by comprising the following steps of:
s1, acquiring a track color image and a depth image by adopting a 3D camera;
s2, detecting a ballast bed crack and a crack detection candidate area in the depth image;
s3, processing the color image in the road bed crack and crack detection candidate area to detect suspected cracks and crack areas; processing the depth image and detecting a suspected crack region;
s4, performing dense sampling on suspected cracks and crack areas on the color image and the depth image to obtain dense sampling image blocks;
s5, dividing the dense sampling image block into a positive sample and a negative sample by using a classifier, wherein the positive sample comprises: cracks, repairing cracks and cracks on cracks, the negative sample is an image other than a positive sample, and the positive sample is further divided into: crack, repairing crack on crack and crack; when the densely sampled image block is judged to be a negative sample, deleting the corresponding sampling area; when the densely sampled image block is a crack, marking the corresponding sampling area as a crack; when the densely sampled image block is a crack, marking the corresponding sampling area as a crack;
s6, connecting the crack areas, and measuring the number, the length and the maximum width of the cracks; the crack areas are connected, and the number, length and maximum width of the cracks are measured.
2. The 3D camera-based ballast bed crack, crack detection method according to claim 1, wherein the 3D camera is a gray D or RGBD camera for acquiring pixel aligned track gray level images and depth images or RGB color images and depth images.
3. The method for detecting the track bed cracks and the cracks based on the 3D camera according to claim 1, wherein the method for detecting the track bed area on the depth image is as follows: taking the mode of each column of pixel values in the depth image along the longitudinal direction of the steel rail to obtain a reference datum line of the track cross section, generating a reference image B with the same size by using the datum line, using the current depth image C to make a difference with the reference image B, and extracting a region R1 with the difference smaller than a set threshold T1; threshold segmentation is carried out on the reference image B, and a region R2 exceeding a set threshold T2 is taken; performing edge detection on the current depth image C, and performing expansion operation on an edge region to obtain a region R3; taking the intersection of R1 and R2 to obtain R4; r4 and R3 are differenced to obtain R5, wherein R5 is a ballast bed crack and crack detection candidate region; when the reference datum line of the cross section of the track is extracted, images of areas on two sides of the steel rail are taken for transverse projection and longitudinal projection, the longitudinal position of the sleeper is found in the transverse projection according to the height difference between the track bed and the sleeper, the transverse position of the sleeper is found in the longitudinal projection, a sleeper shielding area is arranged in the depth image according to the sleeper position and the sleeper size, and the mode number in the row direction is counted after the sleeper image is shielded.
4. The method for detecting ballast bed cracks and fissures based on a 3D camera according to claim 1, wherein the method for processing the color image to detect suspected cracks and fissure areas is as follows: after salt and pepper noise denoising is carried out on the color image, fourier transformation is carried out, and a Fourier frequency domain image is obtained; smoothing and filtering the frequency domain image to eliminate high-frequency components, and then carrying out inverse transformation to obtain a smooth image; and subtracting the Fourier smoothing image from the pretzel noise denoising image to obtain a gradient enhanced image, and detecting and positioning a linear region by adopting a line filter to serve as a suspected crack region.
5. The method for detecting ballast bed cracks and cracks based on a 3D camera according to claim 1, wherein the method for processing the depth image to detect suspected crack areas includes but is not limited to: performing expansion operation on the depth image D0 to obtain a depth image D1, wherein the depth image D1 and the depth image D0 are differenced, and the region exceeding a set threshold T3 is taken as a suspected crack region; or calculating a gradient intensity image of the depth image D0, performing threshold segmentation on the gradient intensity image, and taking a threshold segmentation result as a suspected crack region.
6. The method for detecting ballast bed cracks and cracks based on a 3D camera according to claim 1, wherein the method for densely sampling suspected cracks and crack areas on a color image and a depth image simultaneously comprises the following steps: and scanning suspected cracks and crack areas by adopting sliding window operation, estimating local main directions of the cracks and the cracks when the sliding window is used for extracting, carrying out color image and depth image sampling after aligning the sampling window with the main directions, and aligning the main directions of the suspected cracks and the cracks with the horizontal direction or the vertical direction in a sampling image block.
7. The 3D camera-based ballast bed crack and flaw detection method according to claim 1, wherein the dense sampling image block classification method is as follows: normalizing the densely sampled color image blocks and depth image blocks; performing PCA dimension reduction to obtain 2 one-dimensional vectors v1 and v2 serving as feature vectors v3= [ v1 and v2]; training an SVM or MLP or KNN or random forest classifier for dense sampling image block classification.
8. The 3D camera-based ballast bed crack and flaw detection method of claim 6, wherein the dense sampling image block classification method is: and carrying out transverse and longitudinal accumulated projection on the densely sampled color image blocks and depth image blocks to obtain 4 one-dimensional vectors v4, v5, v6 and v7, carrying out normalization processing and/or dimension reduction processing on the vectors v4, v5, v6 and v7, and connecting the vectors with a feature vector v3 to obtain new feature vectors v8= [ v3, v4, v5, v6 and v7] for classification.
9. The 3D camera-based ballast bed crack, crack detection method of claim 6, wherein the dense sample image block classification method is a deep learning-based classification method.
10. The ballast bed crack and crack detection method based on the 3D camera as claimed in claim 1, wherein the crack/crack areas are connected, and the number of cracks/cracks is calculated by using a Blob analysis method; measuring Manhattan distance from a start point to an end point of each crack/crack as a crack/crack length; traversing from the start point to the end point of the crack/crack, and taking the normal maximum width as the maximum width of the crack/crack.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210872552.6A CN117495758A (en) | 2022-07-24 | 2022-07-24 | Ballast bed crack and crack detection method based on 3D camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210872552.6A CN117495758A (en) | 2022-07-24 | 2022-07-24 | Ballast bed crack and crack detection method based on 3D camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117495758A true CN117495758A (en) | 2024-02-02 |
Family
ID=89671347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210872552.6A Pending CN117495758A (en) | 2022-07-24 | 2022-07-24 | Ballast bed crack and crack detection method based on 3D camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117495758A (en) |
-
2022
- 2022-07-24 CN CN202210872552.6A patent/CN117495758A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115272334B (en) | Method for detecting tiny defects on surface of steel rail under complex background | |
| Jie et al. | Real-time rail head surface defect detection: A geometrical approach | |
| CN101893580B (en) | Digital image based detection method of surface flaw of steel rail | |
| Dubey et al. | Maximally stable extremal region marking-based railway track surface defect sensing | |
| CN111539436B (en) | Rail fastener positioning method based on straight template matching | |
| CN108596872B (en) | Steel rail disease detection method based on Gabor wavelet and SVM | |
| CN104458764B (en) | Curved uneven surface defect identification method based on large-field-depth stripped image projection | |
| CN109376740A (en) | A kind of water gauge reading detection method based on video | |
| CN107705294B (en) | Cross laser image type roadbed surface settlement monitoring method and monitoring system | |
| CN109489724A (en) | A kind of tunnel safe train operation environment comprehensive detection device and detection method | |
| CN112949482B (en) | Non-contact type rail sleeper relative displacement real-time measurement method based on deep learning and visual positioning | |
| CN111754460A (en) | Method, system and storage medium for automatically detecting gap of point switch | |
| CN112528861A (en) | Foreign matter detection method and device applied to track bed in railway tunnel | |
| CN110717900A (en) | Pantograph abrasion detection method based on improved Canny edge detection algorithm | |
| Fu et al. | Research on image-based detection and recognition technologies for cracks on rail surface | |
| CN112949483B (en) | Non-contact rail stretching displacement real-time measurement method based on fast R-CNN | |
| CN113538421A (en) | Contact network data detection method based on vision technology | |
| CN111723793B (en) | Real-time rigid contact net positioning point identification method | |
| CN117495759A (en) | Sleeper crack detection method | |
| CN117495758A (en) | Ballast bed crack and crack detection method based on 3D camera | |
| CN115170657A (en) | Steel rail identification method and device | |
| Bao | Road distress analysis using 2D and 3D information | |
| CN113963053A (en) | Method for detecting height of contact net | |
| CN111723792B (en) | Real-time positioning point identification method suitable for rigid-flexible contact network | |
| CN113537015A (en) | Method for identifying rail scratches, method for routing inspection and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |