CN113240790B - Rail defect image generation method based on 3D model and point cloud processing - Google Patents
Rail defect image generation method based on 3D model and point cloud processing Download PDFInfo
- Publication number
- CN113240790B CN113240790B CN202110401797.6A CN202110401797A CN113240790B CN 113240790 B CN113240790 B CN 113240790B CN 202110401797 A CN202110401797 A CN 202110401797A CN 113240790 B CN113240790 B CN 113240790B
- Authority
- CN
- China
- Prior art keywords
- defect
- steel rail
- point cloud
- model
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 198
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012545 processing Methods 0.000 title claims abstract description 28
- 229910000831 Steel Inorganic materials 0.000 claims abstract description 135
- 239000010959 steel Substances 0.000 claims abstract description 135
- 238000004088 simulation Methods 0.000 claims abstract description 26
- 238000013507 mapping Methods 0.000 claims abstract description 14
- 238000003672 processing method Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000013528 artificial neural network Methods 0.000 claims abstract description 4
- 230000003416 augmentation Effects 0.000 claims description 16
- 230000003321 amplification Effects 0.000 claims description 13
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 13
- 238000009826 distribution Methods 0.000 claims description 7
- 238000009499 grossing Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 abstract description 15
- 230000007704 transition Effects 0.000 abstract description 8
- 238000001514 detection method Methods 0.000 abstract description 3
- 230000002950 deficient Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 239000000243 solution Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005266 casting Methods 0.000 description 1
- 230000001609 comparable effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000001746 injection moulding Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000004660 morphological change Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a steel rail defect image generation method based on a 3D model and point cloud processing, which comprises the steps of firstly constructing a normal steel rail 3D model and a defect steel rail initial 3D model; then amplifying the number of initial 3D models of the defect steel rail and the types of defects by a point cloud processing method; operating the transition difference between the smooth defect and the background based on the curvature and the reverse curvature; finally, automatic labeling is realized through a label mapping method, and meanwhile, the diversity and complexity of the steel rail simulation data background are increased through a texture replacement method. The method provided by the invention can generate infinite high-quality marked rail defect simulation data. The data can be used for later neural network training or auxiliary training, and the influence of insufficient data samples on the rail defect detection task is effectively improved.
Description
Technical Field
The invention relates to the technical field of data set generation, in particular to a steel rail defect image generation method based on a 3D model and point cloud processing.
Background
China is a large country of railways, and the total length of railways is over 13.1 kilometers. As the length of railways grows, maintenance of the railway conditions becomes increasingly important. In railway state maintenance, automatic positioning and identification of rail surface defects is an important ring.
As an important technical means for detecting the service state of rail transit infrastructure, the steel rail surface defect detection technology based on the deep learning technology trains a neural network model through a large amount of defect image data, so that identification and positioning of various disease defects on the steel rail surface are realized. In an actual scene, the number of defective steel rails is rare due to good maintenance, so that a batch of simulation data is generated according to the characteristics of the defective steel rails to conduct independent research or research together with real data is a good scheme.
There have been many attempts in other fields to explore simulation data generation techniques, but there have been few studies on directly generating simulation data of rail surface defects. In the university of south China, 2014 proposes a method and a system for generating a dendritic shrinkage porosity defect simulation image of a casting, which are used for generating a new false data set by carrying out binarization conversion and histogram matching on a defect image and combining a method of superposing a real background image and diffusing a defect edge area so as to increase test and learning samples, but the method is only used for generating the defect on a two-dimensional image through an image processing method, and the quality of the generated defect is not ideal. In 2019, university of Zhejiang proposes a method for identifying surface image defects of injection molding products based on transfer learning, a permit a leave sample set is generated by using a method for generating an countermeasure network, and in 2019, university of Zhejiang proposes a method for generating drainage pipeline defect detection training data based on PGGAN transfer learning, wherein virtual data is generated by using a technology for generating the countermeasure network. The generation methods based on generating the countermeasure network all have common defects:
(1) The training process of the generator requires a large amount of real data to participate, and if the real data are less, a simulation data set with high quality cannot be obtained;
(2) When generating data, the label information of the label cannot be synchronously acquired, and a great deal of manpower is consumed for marking in the later period;
(3) The type of defect generated is random and the result generated is not controllable.
In addition to the common defects, since the size information of the rail defects in the rail data set is relatively fine relative to the scene in which the rail defects are located, that is, the occupied area in the whole image is relatively small, it is difficult to keep the defect information by generating virtual data through a generated countermeasure network, and therefore the generated rail has relatively real overall effect but lacks critical defect information.
Disclosure of Invention
The embodiment of the invention provides a steel rail defect image generation method based on a 3D model and point cloud processing, which is used for solving the problems in the prior art.
In order to achieve the above purpose, the present invention adopts the following technical scheme.
A steel rail defect image generation method based on a 3D model and point cloud processing comprises the following steps:
constructing a normal steel rail 3D model, and adding initial defect information into data of the normal steel rail 3D model to obtain a defect steel rail initial 3D model;
Amplifying the initial defect information by a point cloud processing method to obtain a plurality of extended defect information, and further obtaining one or more extended 3D models of the defect steel rail;
Smoothing the extended 3D model of the defect steel rail by an operation method based on curvature and reverse curvature to obtain a target 3D model of the defect steel rail;
converting the 3D model of the defect steel rail target after smoothing treatment to obtain a steel rail defect simulation data set;
And mapping the extended defect position information of the defect steel rail target 3D model into a steel rail defect simulation data set by a tag mapping method to obtain target tag information, wherein the target tag information is used for neural network training.
Preferably, the amplifying the initial defect information by a point cloud processing method to obtain a plurality of extended defect information, and further obtaining a plurality of extended 3D models of the defect steel rail includes:
performing format conversion on the point cloud libraries of the normal steel rail 3D model and the defect steel rail initial 3D model to respectively obtain a normal steel rail point cloud model and a defect steel rail point cloud model;
extracting point cloud information of a defect steel rail point cloud model, and performing amplification processing to obtain a plurality of extended defect information;
based on the plurality of extended defect information, a plurality of extended 3D models of the defective steel rail are obtained.
Preferably, extracting the point cloud information of the defect steel rail point cloud model includes:
Through type
DefectPoint＝{(x,y,z)|(x,y,z)∈RailPoint,z≠f(x)} (1)
Acquiring information of points which do not meet the curve function of the surface of the normal steel rail; wherein f (x) is a fitted surface curve function of the normal steel rail;
Acquiring information DefectPoint of all point cloud parts which do not meet the surface distribution relation of the steel rail based on the fitted surface curve function of the normal steel rail, and calculating coordinates X L、YL、XR and Y R of a smallest square frame which can enclose all the point cloud parts which do not meet the surface distribution relation of the steel rail; wherein X L and Y L are each
XL＝min(x|(x,y,z)∈DefectPoint),YL＝min(y|(x,y,z)∈DefectPoint),XR And Y R are respectively
XR＝max(x|(x,y,z)∈DefectPoint),YR＝max(y|(x,y,z)∈DefectPoint)；
Based on X L、YL、XR and Y R, all point cloud information DefectBoxPoint = { (X, Y, z) | (X, Y, z) ∈ RailPoint, X L≤x≤XR,YL≤y≤YR } 2 in the smallest square frame that can enclose all point cloud portions that do not satisfy the rail surface distribution relationship is extracted.
Preferably, the plurality of extended defect information includes volume augmentation defect information, shape augmentation defect information, number augmentation defect information, and position augmentation defect information;
the volume augmentation defect information is obtained by the formula ScalePoint = { α (x, y, z) | (x, y, z) ∈ DefectBoxPoint, α∈ (0, 1] } (3), where α represents a volume scaling factor;
Shape augmentation defect information pass-through
ShapePoint＝{(α1x,α2y,α3z)|(x,y,z)∈DefectBoxPoint,α1,α2,α3∈(0,1]} (4) Obtaining; wherein, alpha 1、α2 and alpha 3 respectively represent scaling factors of three dimensions of coordinate axes x, y and z;
The quantity amplification defect information is obtained by calculating the offset range of the point cloud area of the defect steel rail point cloud model in the square frame;
position amplification defect information pass-through
FinalPoint = { (x+ offsetX, y+ offsetY, z) | (x, y, z) ∈ DefectBoxPoint u ScalePoint u Shapeoint } (5).
Preferably, the smoothing processing is performed on the defect steel rail extension 3D model by an operation method based on curvature and reverse curvature, and obtaining the defect steel rail target 3D model includes:
Stretching the extended defect information to a rail surface coordinate system in a normal rail 3D model by CurvePoint = { (x, y, z+f (x)) | (x, y, z) ∈ FinalPont } (6);
By passing through
AntiCurvePoint
＝{(x,y,z-f(x))|(x,y,z)∈DefectBoxPoint∪ScalePoint∪ShapePoint
FinalPoint＝{(x+offsetX,y+offsetY,z)|(x,y,z)∈AntiCurvePoint}
CurvePoint = { (x, y, z+f (x))| (x, y, z) ∈ FinalPoint } (7) the curvature information of the square frame is eliminated/added.
Preferably, mapping extended defect position information of a defect steel rail target 3D model to a steel rail defect simulation data set by a tag mapping method, and obtaining target tag information includes:
Through type
Obtaining position information and volume information of defect information in the steel rail defect simulation data set;
By the formula ResultPicture = TexturePicture ×r+ OriginalPicture × (1-R), R e (0, 1) (9) replaces the texture of the rail defect simulation dataset.
According to the technical scheme provided by the embodiment of the invention, the method for generating the steel rail defect image based on the 3D model and the point cloud processing is designed, and the point cloud processing method is designed to automatically generate a large number of amplified steel rail defect models with the same comparable effect with the manually-kneaded steel rail defect models according to a small initial manually-kneaded defective steel rail simulation model; meanwhile, in the process of expanding the defects by using the point cloud generating processing method, the curvature of the steel rail is fitted, then the transition between the defects which are smooth by the curvature and the reverse curvature operation and the background information of the 3D model of the steel rail is designed, and the 3D model of the expanded defect steel rail is more real. After the two-dimensional steel rail defect simulation data set is obtained by rendering the 3D model of the defect steel rail, a texture set is constructed by utilizing the real data set of the normal steel rail, and textures of the simulation data are replaced by randomly selecting textures from the inside, so that the background of the generated steel rail defect simulation data set becomes more real. The marking information is extracted while the defect 3D model is generated, and is mapped onto the two-dimensional data set through a mapping method, so that the automatic work of the marking information is realized, and the generating efficiency is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a process flow diagram of a method for generating a rail defect image based on a 3D model and point cloud processing;
FIG. 2 is a schematic diagram of a normal steel rail 3D model in a steel rail defect image generation method based on a 3D model and point cloud processing;
FIG. 3 is a schematic diagram of an initial 3D model of a defective rail in a rail defect image generation method based on a 3D model and point cloud processing;
fig. 4 is a schematic process diagram of a point cloud processing method in a steel rail defect image generation method based on a 3D model and point cloud processing;
FIG. 5 is a schematic view of a rail surface curvature fitting operation in a rail defect image generation method based on a 3D model and point cloud processing;
FIG. 6 is an effect diagram of a steel rail defect image generation method based on a 3D model and point cloud processing after curvature and inverse curvature operations;
FIG. 7 is a schematic diagram of curvature stretching operation in a method for generating a rail defect image based on a 3D model and point cloud processing;
FIG. 8 is a schematic diagram of a reverse curvature stretching operation in a method for generating a rail defect image based on a 3D model and point cloud processing;
Fig. 9 is a schematic diagram of an automatic extraction principle of tag information in a method for generating a rail defect image based on a 3D model and point cloud processing;
fig. 10 is a schematic diagram of an automatic extraction effect of tag information in a method for generating a rail defect image based on a 3D model and point cloud processing;
fig. 11 is a schematic diagram of texture replacement results in a method for generating a rail defect image based on a 3D model and point cloud processing.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
For the purpose of facilitating an understanding of the embodiments of the invention, reference will now be made to the drawings of several specific embodiments illustrated in the drawings and in no way should be taken to limit the embodiments of the invention.
The invention provides a method for generating a steel rail defect image data set, which is used for achieving the following purposes:
1. generating predictable rail defect simulation data without actual defect rail data;
2. the automatic extraction of the labeling information avoids the complexity of manual labeling;
3. Amplifying defects by a point cloud processing method, and improving the speed of generating a defect data set based on a 3d model method;
4. and generating more real rail defect simulation data through texture replacement and curvature fitting.
Referring to fig. 1, the method for generating the steel rail defect image based on the 3D model and the point cloud processing provided by the invention comprises the following steps:
S1, constructing a normal steel rail 3D model, and adding initial defect information into data of the normal steel rail 3D model to obtain a defect steel rail initial 3D model; in the step, modeling tools in the prior art, such as solidworks or other 3D model construction software, can be used for constructing a normal rail 3D model, and the corresponding normal rail 3D model is constructed according to the size and proportion of an actual rail (namely, modeling is performed on the actual rail in a virtual environment); then adding initial defect information on the normal steel rail 3D model according to the defect characteristics (defect information to be generated) to be generated by utilizing wrap or other software, so as to obtain a small batch of initial 3D models of the virtual defective steel rails with defects;
S2, amplifying the initial defect information through a point cloud processing method to obtain a plurality of extended defect information, and further obtaining one or more extended 3D models of the defect steel rail; the method is mainly used for extracting defect information in an initial 3D model of a defective steel rail, and then carrying out morphological, size and position changes on the defect information, so that a large number of virtual 3D models with defects can be amplified;
S3, in order to enable defect information to appear more naturally on the steel rail 3D model, smoothing the 3D model of the defect steel rail extension by an operation method based on curvature and reverse curvature to obtain a defect steel rail target 3D model;
S4, after a large number of defect steel rail target 3D models which are diversified and have smooth fusion of defect characteristics are generated, converting the defect steel rail target 3D models to obtain a steel rail defect simulation data set; specifically, an acquisition tool for acquiring the information of the top surface of the steel rail in batches is developed through 3dmax, and the acquisition tool is used for converting a 3D model set of a defect steel rail target into a two-dimensional steel rail defect simulation data set;
And S5, mapping the extended defect position information of the defect steel rail target 3D model into the steel rail defect simulation data set through a label mapping method, so that target label information which can be used for training a neural network is obtained without manual labeling.
In the preferred embodiment provided by the invention, step S1 is specifically to construct a normal rail 3D model with the length of 500mm according to the size and proportion of a real rail by utilizing 3D modeling software, as shown in FIG. 2. And then adding defect information to the surface of the steel rail by using model fine-tuning software or common 3D modeling software to obtain an initial 3D model of the defective steel rail, as shown in fig. 3.
After the initial 3D model of the defect steel rail is obtained, the number and diversity of the 3D models of the defect steel rail are required to be expanded by using a designed point cloud expansion method. The main steps are as shown in fig. 4, firstly, the point cloud libraries of the normal steel rail 3D model and the defect steel rail initial 3D model are subjected to format conversion, for example, the point cloud libraries are realized through python, and the normal steel rail point cloud model and the defect steel rail point cloud model are respectively obtained. For the defect steel rail point cloud model RailPoint, the defect point cloud part DefectPoint is extracted through the characteristic that defects do not accord with the surface distribution of a normal steel rail, and amplification processing is carried out to obtain a plurality of extended defect information. The method can be operated by the following substeps:
as shown in figure 5 for the rail cross-section,
The upper surface curve of the steel rail can be mainly divided into 5 sections of circular arcs, which are respectively called S1, S2, S3, S4 and S5 from left to right, and the circle center of the circular arc is marked as O 1,O2,O3,O4,O5. Wherein the curvature radius of S1, S5 is R 3, and the curvature radius of S2, S4 is R 2. S3 has a radius of curvature R 1. Other parameters in fig. 1: a 1 is the x-axis length of the first arc, A 2 is the x-axis length of the second arc, and A 3 is the x-axis length of the third arc. The curve function of the surface of the steel rail can be obtained through mathematical derivation:
Whereby the defective point cloud portion is expressed as
DefectPoint = { (x, y, z) | (x, y, z) ∈ RailPoint, z+.f (x) } (1); wherein (x, y, z) is used to represent the coordinates of the points in the three-dimensional point cloud, wherein x, y, z represents the value of each individual dimension in the coordinates, specifically z represents the depth of the rail, the x axis is the transverse axis of the rail, and y is the longitudinal axis of the rail, and the depth is only related to the x axis because the sections of the rail are all the same; firstly, extracting points belonging to RailPoint by a formula (1), wherein among the points, points of which the x coordinate and the z coordinate do not accord with a z=f (x) function relation are removed, and the coordinates of the rest points are assigned to DefectPoint;
Coordinates X L、YL、XR and Y R of the smallest square frame that can enclose these defective point cloud portions are calculated; wherein X L and Y L are each
X L＝min(x|(x,y,z)∈DefectPoint),YL＝min(y|(x,y,z)∈DefectPoint),XR and Y R are
XR＝max(x|(x,y,z)∈DefectPoint),YR＝max(y|(x,y,z)∈DefectPoint)；
The specific processing procedure through the above formula is exemplified by X L: firstly, the x coordinate of the point belonging to DefectPoint is taken out; selecting a minimum value from the X coordinates and assigning the minimum value to X L; other similar matters;
Based on X L、YL、XR and Y R, all point cloud information DefectBoxPoint = { (X, Y, z) | (X, Y, z) ∈ RailPoint, X L≤x≤XR,YL≤y≤YR } (2) within the smallest square frame enclosing the defective rail point cloud portion can be extracted.
For the extracted point cloud information in the square frame, only changing the x coordinate, the y coordinate and the z coordinate or only changing two of the 3 coordinates at the same time can change the shape of the defect, and changing the values of the three coordinates in the same proportion can change the size of the defect. By the method, the diversity can be amplified to obtain the extension defect information.
Further, the extended defect information includes volume augmentation defect information, shape augmentation defect information, number augmentation defect information, and position augmentation defect information.
The volume amplification defect information is obtained by the formula ScalePoint = { alpha (x, y, z) | (x, y, z) ∈ DefectBoxPoint, alpha epsilon (0, 1] } (3), wherein alpha represents a volume scaling factor, the smaller the value of the volume scaling factor is, and the smaller the defect point cloud after amplification is;
Shape augmentation defect information pass-through
ShapePoint＝{(α1x,α2y,α3z)|(x,y,z)∈DefectBoxPoint,α1,α2,α3∈(0,1]} (4) Obtaining;
Wherein, alpha 1、α2 and alpha 3 respectively represent scaling factors of three dimensions of coordinate axes x, y and z, when the scaling factors of the dimensions are different, the scaling factors of the dimensions can lead the stretching degree of the dimensions to be different so as to deform the original defect point cloud, thereby achieving the purpose of shape expansion;
Meanwhile, the deflectable ranges (offsetX, offsetY) of the point clouds in the square frames on the surfaces of the steel rails are calculated, and the point clouds are randomly deflected on any position on the steel rails, so that the amplification in quantity can be realized.
The position amplification defect information is obtained by the formula FinalPoint = { (x+ offsetX, y+ offsetY, z) | (x, y, z) ∈ DefectBoxPoint u ScalePoint u ShapePoint } (5).
And digging out point cloud information at a corresponding position on the normal steel rail point cloud model, filling the point cloud information FinalPoint which is contained in the square frame and has amplified forms and quantity into the point cloud information to obtain a defective steel rail 3D point cloud file, and converting the defective steel rail 3D point cloud file into a 3D model file, thereby realizing the point cloud amplification scheme.
When the point cloud information is filled into the normal steel rail 3D point cloud file, the problem that the transition between the square frame containing the defect point cloud and the background point cloud is unnatural is solved. As shown at left 1 in fig. 6. In response to this problem, the present invention provides a preferred embodiment in which the corresponding operation of the curvature and the reverse curvature is designed to solve this problem. The effects after the curvature and reverse curvature operations are shown as left 2 and left 3, respectively, in fig. 6. Curvature and inverse curvature operations are some optimization operations that take into account the cause of edge transition artifacts. The transition is not natural because the frame of reference of the square frame is not horizontal as the frame of reference when the smallest square frame containing defect information is extracted and filled. When the defect is filled into the normal rail 3D point cloud, the edge of the normal rail 3D point cloud is not horizontal, so if the defect is directly filled, the edge is tilted or dented as shown in left 1 of fig. 7. Therefore, curvature operation is adopted to stretch the defect point cloud information FinalPoint to the coordinate system (f (x) of the rail surface in the normal rail 3D model as a rail surface curve function), and the calculation formula is CurvePoint = { (x, y, z+f (x))| (x, y, z) ∈ FinalPoint }.
As shown in fig. 7 left two. The transition of the square frame containing the defective point cloud and the background point cloud becomes more natural through the curvature operation, but this problem is not completely solved. Therefore, in this embodiment, the operation of the reverse curvature is proposed again, mainly considering that the coordinate system of the square frame of the extracted defect point cloud is not flat, and also has the rail curvature, as shown in fig. 8. Therefore, the curvature information of the square frame is firstly eliminated through the inverse curvature operation, and then the curvature information of the position to be filled is added through the curvature operation during filling, and the calculation formula is as follows
AntiCurvePoint
＝{(x,y,z-f(x))|(x,y,z)∈DefectBoxPoint∪ScalePoint∪ShapePoint}
FinalPoint＝{(x+offsetX,y+offsetY,z)|(x,y,z)∈AntiCurvePoint}
CurvePoint＝{(x,y,z+f(x))|(x,y,z)∈FinalPoint} (7)。
Through the above operation, the generated 3D model of the defective rail target is shown as left 3 in fig. 6.
In a preferred embodiment of the present invention, the operation of the labeling information mapping method in step S5 is as shown in fig. 9, which maps the generated defect location information onto the final two-dimensional image, and the format of the labeling information adopts the target level labeling in YOLO format (the labeling information in YOLO format uses a four-tuple (X, Y, W, H) to label the position of an object in the picture.
First, the coordinates (x 1, y 1) of the upper left corner and the coordinates (x 2, y 2) of the lower right corner of the square frame can be obtained in the point cloud processing process, wherein the coordinates are relative to the modeling origin, and the modeling origin is positioned at the center below the rail surface. In mapping to the labeling information in YOLO format, it is necessary to know the relative value of the center point coordinates of the square frame on the picture, and the relative values of the width and height, and the coordinates to be obtained at this time are the coordinates at the origin of the image. The map information is obtained by calculation as shown in fig. 9. Thereby realizing automatic labeling of labeling information, and fig. 10 shows the effect of automatic labeling.
In a specific embodiment, as shown in fig. 9, first, the size of the finally obtained image is fixed (only including the rail surface area) in the point cloud stage, and is W rail*Hrail, and the frame information is already obtained in the point cloud processing stage, namely (xl, yl), (x 2, y 2).
By calculation
(8) And obtaining the position information and the volume information of a square frame surrounding the defect in the defect steel rail target 3D model.
Wherein (x 1+ x 2)/2+W rail/2 is the abscissa of the center point of the square frame relative to the upper left corner of the rail surface (the position corresponding to the origin of the image), and then divided by W rail, which is the relative coordinate, and the calculation process of y is the same. The relative position of the center point of the final square frame is (X, Y) in the above figure.
At the same time, the relative size of the frame, i.e. the size relative to the rail surface, is consistent with the information at the point cloud stage, so that it is (W, H) in the calculation formula (8).
After the two-dimensional data set is obtained, a texture set is constructed by utilizing the two-dimensional image of the normal steel rail, and then the textures of the virtual data set are replaced in a pixel superposition mode. Calculation formula
ResultPicture = TexturePicture xr + OriginalPicture x (1-R), R in R e (0, 1) (9) is the intensity parameter of texture substitution, texturePicture is a picture randomly extracted from the texture set (scaled to the same size as the virtual image in order to enable pixel level superimposition), and OriginalPicture is the virtual picture. The result after replacement is shown in fig. 11.
In summary, the method for generating the steel rail defect image based on the 3D model and the point cloud processing provided by the invention comprises the following steps:
And manufacturing a normal steel rail 3D model by utilizing a solidworks or other 3D model construction software, and fine-adjusting the normal steel rail 3D model by utilizing a wrap or other software to obtain a batch of defective steel rail 3D models with defects.
And expanding the quantity and diversity of the 3D models of the defective steel rails through a point cloud processing method. For diversity we vary the size, shape of the defect. For quantity, we randomly shift the positions of defect information after morphological changes and original defect information on the rail. Thereby obtaining an infinite and sufficient number of 3D models of defective steel rails. And simultaneously fitting the curvature of the steel rail 3D model, and smoothing the transition of the defect and the background information of the steel rail 3D model through curvature and inverse curvature operation.
Virtual rail surface views of the 3D model of the defect rail generated by us are collected in batch through 3dmax or other software. Meanwhile, in order to make the background information of the virtual rail surface map more complex and diversified, a texture set is constructed through the texture of the normal image, and one texture set is randomly selected to replace the texture information of the rail surface map.
And mapping the defect positions acquired in the point cloud processing process to a final rail surface diagram by a designed labeling information mapping method, and forming a YOLO labeling format.
The method provided by the invention has the following beneficial effects:
the generation of the rail simulation data set can be automatically and rapidly realized without any participation of real data. The generated data can be used for network training or auxiliary training based on deep learning in the later period;
The generated virtual data all have corresponding label information, manual labeling is not needed, the input of human resources is greatly reduced, meanwhile, the speed of the point cloud-based augmentation mode is high, and a large amount of data can be generated;
the defects amplified by the point cloud processing method appear and transition on the steel rail more naturally by adopting curvature and reverse curvature operation, so that the similarity of the manually-kneaded model and the point cloud-expanded model reaches a higher level;
texture information is extracted through real normal steel rail data to replace textures of generated data, so that background diversity and authenticity of the generated defect data are guaranteed.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (1)
1. A method for generating a steel rail defect image based on a 3D model and point cloud processing is characterized by comprising the following steps:
Constructing a normal steel rail 3D model, and adding initial defect information into data of the normal steel rail 3D model to obtain a defect steel rail initial 3D model; the method specifically comprises the following steps:
Amplifying the initial defect information by a point cloud processing method to obtain a plurality of extended defect information, and further obtaining one or more extended 3D models of the defect steel rail; the method specifically comprises the following steps:
performing format conversion on the point cloud libraries of the normal steel rail 3D model and the defect steel rail initial 3D model to respectively obtain a normal steel rail point cloud model and a defect steel rail point cloud model;
extracting point cloud information of the defect steel rail point cloud model, and performing amplification treatment to obtain a plurality of extended defect information; the extracting the point cloud information of the defect steel rail point cloud model comprises the following steps:
Through type
DefectPoint＝{(x,y,z)|(x,y,z)∈RailPoint,z≠f(x)} (1)
Acquiring information of points which do not meet the curve function of the surface of the normal steel rail; wherein f (x) is a fitted surface curve function of the normal steel rail; (x, y, z) represents coordinates of a midpoint of the three-dimensional point cloud, z represents a depth of the steel rail, an x axis is a transverse axis of the steel rail, and y is a longitudinal axis of the steel rail;
Acquiring information DefectPoint of all point cloud parts which do not meet the surface distribution relation of the steel rail based on the fitted surface curve function of the normal steel rail, and calculating coordinates X L、YL、XR and Y R of a smallest square frame which can enclose all the point cloud parts which do not meet the surface distribution relation of the steel rail; wherein X L and Y L are each
X L＝min(x|(x,y,z)∈DefectPoint),YL＝min(y|(x,y,z)∈DefectPoint),XR and Y R are each
XR＝max(x|(x,y,z)∈DefectPoint),YR＝max(y|(x,y,z)∈DefectPoint)；
Extracting all point cloud information DefectBoxPoint = { (X, Y, z) | (X, Y, z) ∈ RailPoint, X L≤x≤XR,YL≤y≤YR } (2) in the smallest square frame capable of enclosing all point cloud parts which do not meet the distribution relation of the steel rail surface based on X L、YL、XR and Y R;
The plurality of extended defect information includes volume augmentation defect information, shape augmentation defect information, number augmentation defect information, and position augmentation defect information;
The volume amplification defect information is obtained through ScalePoint = { alpha (x, y, z) | (x, y, z) ∈ DefectBoxPoint, alpha epsilon (0, 1) } (3), wherein alpha represents a volume scaling factor;
the shape-amplified defect information is passed through
ShapePoint＝{(α1x,α2y,α3z)|(x,y,z)∈DefectBoxPoint,α1,α2,α3∈(0,1]}(4) Obtaining; wherein, alpha 1、α2 and alpha 3 respectively represent scaling factors of three dimensions of coordinate axes x, y and z;
the quantity amplification defect information is obtained by calculating an offsetable range (offsetX, offsetY) of a point cloud area of a defect steel rail point cloud model in the square frame;
the position amplification defect information passes through
FinalPoint = { (x+ offsetX, y+ offsetY, z) | (x, y, z) ∈ DefectBoxPoint u ScalePoint u ShapePoint } (5);
Acquiring a plurality of defect steel rail extension 3D models based on the plurality of extension defect information;
performing smoothing treatment on the defect steel rail extension 3D model by an operation method based on curvature and reverse curvature to obtain a defect steel rail target 3D model; the method specifically comprises the following steps:
Stretching the extended defect information to a rail surface coordinate system in the normal rail 3D model by CurvePoint = { (x, y, z+f (x)) | (x, y, z) ∈ FinalPoint } (6);
By passing through
AntiCurvePoint
＝{(x,y,z-f(x))|(x,y,z)∈DefectBoxPoint∪ScalePoint∪ShapePoint}
FinalPoint＝{(x+offsetX,y+offsetY,z)|(x,y,z)∈AntiCurvePoint}
CurvePoint = { (x, y, z+f (x)) | (x, y, z) ∈ FinalPoint } (7) eliminates/adds curvature information of the square frame;
converting the 3D model of the defect steel rail target after smoothing to obtain a steel rail defect simulation data set;
mapping the extended defect position information of the defect steel rail target 3D model into the steel rail defect simulation data set by a tag mapping method to obtain target tag information; the method specifically comprises the following steps:
Through type
Obtaining position information and volume information of defect information in the steel rail defect simulation data set; wherein (X1, Y1) represents the upper left corner coordinates of the square frame, (X2, Y2) represents the lower right corner coordinates of the square frame, X and Y represent the relative center coordinates of the smallest square frame surrounding the rail, and W and H represent the relative sizes of the square frames;
Replacing the texture of the rail defect simulation dataset by formula ResultPicture = TexturePicture xr + OriginalPicture x (1-R), R e (0, 1) (9);
the target tag information is used for neural network training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110401797.6A CN113240790B (en) | 2021-04-14 | 2021-04-14 | Rail defect image generation method based on 3D model and point cloud processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110401797.6A CN113240790B (en) | 2021-04-14 | 2021-04-14 | Rail defect image generation method based on 3D model and point cloud processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113240790A CN113240790A (en) | 2021-08-10 |
| CN113240790B true CN113240790B (en) | 2024-06-14 |
Family
ID=77128270
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110401797.6A Active CN113240790B (en) | 2021-04-14 | 2021-04-14 | Rail defect image generation method based on 3D model and point cloud processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113240790B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2613879A (en) * | 2021-12-17 | 2023-06-21 | Zeta Motion Ltd | Automated inspection system |
| CN114529689B (en) * | 2022-04-24 | 2022-08-16 | 广州易道智慧信息科技有限公司 | Ceramic cup defect sample amplification method and system based on antagonistic neural network |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107995225A (en) * | 2017-12-26 | 2018-05-04 | 国网河南省电力公司信息通信公司 | A kind of security even analysis method towards complex network |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11948292B2 (en) * | 2019-07-02 | 2024-04-02 | MakinaRocks Co., Ltd. | Systems and methods for detecting flaws on panels using images of the panels |
| CN112070712B (en) * | 2020-06-05 | 2024-05-03 | 湖北金三峡印务有限公司 | Printing defect detection method based on self-encoder network |
| CN111982910B (en) * | 2020-07-06 | 2022-10-25 | 华南理工大学 | Weak supervision machine vision detection method and system based on artificial defect simulation |
-
2021
- 2021-04-14 CN CN202110401797.6A patent/CN113240790B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107995225A (en) * | 2017-12-26 | 2018-05-04 | 国网河南省电力公司信息通信公司 | A kind of security even analysis method towards complex network |
Non-Patent Citations (1)
| Title |
|---|
| 《轮芯缺陷检测及缺陷样本生成技术》;周洲;《全国优秀硕士论文》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113240790A (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113240790B (en) | Rail defect image generation method based on 3D model and point cloud processing | |
| CN107194872A (en) | Remote sensed image super-resolution reconstruction method based on perception of content deep learning network | |
| CN106846272A (en) | A kind of denoising compressing method of point cloud model | |
| CN105095581B (en) | A kind of casting shrinking hole defect image generation method | |
| CN106791773B (en) | A kind of novel view synthesis method based on depth image | |
| CN110223376B (en) | Three-dimensional particle reconstruction method based on single accumulated particle material image | |
| CN114596290A (en) | Defect detection method, defect detection device, storage medium, and program product | |
| CN110544300A (en) | Method for automatically generating three-dimensional model based on two-dimensional hand-drawn image characteristics | |
| CN109345625A (en) | Core image adaptive partition three-dimensional rebuilding method | |
| CN109598782A (en) | Building historical relic restorative procedure, storage medium based on dimensional Modeling Technology | |
| CN111598803B (en) | Point cloud filtering method based on variable-resolution voxel grid and sparse convolution | |
| CN108197555B (en) | Real-time face fusion method based on face tracking | |
| CN112819726A (en) | Light field rendering artifact removing method | |
| CN116205876A (en) | Unsupervised notebook appearance defect detection method based on multi-scale standardized flow | |
| CN115937461A (en) | Multi-source fusion model construction and texture generation method, device, medium and equipment | |
| CN114359269A (en) | Virtual food box defect generation method and system based on neural network | |
| CN111462084B (en) | Image vectorization printing bleeding point prediction system and method based on random forest | |
| CN113011480A (en) | Cambered surface defect image generation method based on cyclic generation countermeasure network | |
| JP2018163468A (en) | Foreground extraction device and program | |
| CN103945206A (en) | Three-dimensional picture synthesis system based on comparison between similar frames | |
| CN115841546A (en) | Scene structure associated subway station multi-view vector simulation rendering method and system | |
| CN115273080A (en) | Lightweight visual semantic odometer method for dynamic scene | |
| CN110717471B (en) | B-ultrasonic image target detection method based on support vector machine model and B-ultrasonic scanner | |
| CN108961384A (en) | three-dimensional image reconstruction method | |
| CN110348458A (en) | A kind of image texture classification method based on local direction extreme value mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |