CN107292870A - Track plug pin fault detection method and system based on image alignment with detection network model - Google Patents
Track plug pin fault detection method and system based on image alignment with detection network model Download PDFInfo
- Publication number
- CN107292870A CN107292870A CN201710424243.1A CN201710424243A CN107292870A CN 107292870 A CN107292870 A CN 107292870A CN 201710424243 A CN201710424243 A CN 201710424243A CN 107292870 A CN107292870 A CN 107292870A
- Authority
- CN
- China
- Prior art keywords
- image
- plug pin
- rail
- network
- track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to computer glitch detection technique field, specially a kind of track plug pin fault detection method and system based on image alignment with detection network model.In the present invention, normal plug pin image is recorded as benchmark image by rail inspection car；Staff is during routine rail inspection, newest rail conditions are shot using inspection car, rail conditions image is obtained as image to be detected, processing is analyzed to current image to be detected and benchmark image, detect problem plug pin component locations in rail, so as to carry out changing problem plug pin in time, it is ensured that the safety of rail operation；Specific steps include：High-speed railway rail image Accurate align based on depth network model and marking area, the fault detect of the track plug pin compensated based on depth network model and environmental change.The present invention carries out image alignment based on depth network model and compensated with environmental change, with more accurate recognition performance and reliability.
Description
Technical field
The invention belongs to computer glitch detection technique field, and in particular to high-speed railway rail fault detection method and system.
Background technology
It is important transportation trade in China railways, rail is related to the life of the people as the carrier of railway transportation
Order property safety.For a long time, the detection of high-speed railway rail uses manual inspection, easily by weather condition, illumination etc.
Influence, and there is potential safety hazard in rail patrol officer.High-speed railway rail fault detect based on image procossing has important grind
Study carefully value.
Because the plug pin of high-speed railway rail failure in practice is less, it is possible to by shooting same position of multiple months
High-speed railway rail and compare the change of plug pin, judge that plug pin whether there is failure with this.
Plug pin fault detect mainly includes two aspects in high-speed railway rail：Track alignment and fault detect.Orbital image pair
Mainly there are three difficult points with detection together：One be rail track difference it is smaller, directly carry out matching easily cause erroneous matching.Two
Be image location information it is inaccurate, rail inspection car is equipped with gps system, but GPS location has error.Three be high-speed railway rail
Influenceed very big by illumination factor, because the time of shooting is different, illumination is different, and the rail of same position has very big in brightness
Difference.
The feature that traditional plug pin detection method is relied on all is artificial constructed by rule of thumb, such as classical HOG, LBP is special
[1-4] is levied etc., traditional disaggregated model is then based on and is judged, performance of such method in feature representation and grader
On often have larger limitation.
Compared with artificial constructed feature, deep learning algorithm can train suitable character representation automatically, if added
Reasonable utilization, can improve the detection performance of plug pin failure in high-speed railway rail.Used in document [5] based on depth network
Disaggregated model carries out the detection of rail surface defect, does not then use the feature representation based on depth network wherein.In text
Offer in [6], author proposes the object detection algorithms based on depth network structure, but it is not included for processing high-speed railway rail
The alignment occurred in image and the counter-measure of environmental change problem.
The present invention solves the above problems, it is proposed that image alignment model and object fault detect mould based on depth network
Type, and modelling and algorithm optimization have been done for the shooting environmental change of high-speed railway rail photo, realize high-precision failure
Detection.The resolving ideas proposed in this programme can not only use the plug pin fault detect in high-speed railway rail, can be applicable to
Unit failure detection in other type high-speed railway rails.
With reference to selected works：
[1] Du Xinyu, height favorable to the people, Wu Nan, Cheng Yu, Wang Xiaodong railway plug pin image detecting methods and device
CN201510752738.8,2016.
[2]E.Deutschl,C.Gasser,A.Niel and J.Werschonig.Defect detection on
rail surfaces by a vision based system.IEEE Intelligent Vehicles Symposium,
2004,pp. 507-511.
[3]Lin Jie,Luo Siwei,Li Qingyong,Zhang Hanqing and Ren Shengwei.Real-
time rail head surface defect detection:A geometrical approach.2009IEEE
International Symposium on Industrial Electronics,Seoul,2009,pp.769-774.
[4] railway plug pins of the Du Xin fine jades based on computer vision technique is automatically positioned algorithm railway communication signals, 52
(9):68-72,2016.
[5] vision-based detection of Tang Xiangna, Wang Yao south rail surface defects and recognizer computer engineering, 39 (3):
25-30, 2013.
[6]Shaoqing Ren,Kaiming He,Ross B Girshick,Jian Sun.Faster R-CNN:
Towards Real-Time Object Detection with Region Proposal Networks.In Advances
in Neural Information Processing Systems,2015,pp.91–99.。
The content of the invention
It is an object of the invention to provide a kind of high accuracy, efficient track plug pin fault detection method and system.
, it is necessary to record normal plug pin image as benchmark image by rail inspection car in the application scenarios of the present invention.
After staff during routine rail inspection, newest rail conditions are shot using inspection car, rail conditions are obtained
These present images (image to be detected) and benchmark image are analyzed by the present invention as image to be detected for image
Reason, detects problem plug pin component locations in rail, so as to carry out changing problem plug pin in time, it is ensured that the safety of rail operation.
The track plug pin fault detection method that the present invention is provided, is comprised the following steps that：
1st, the high-speed railway rail image Accurate align based on depth network model and marking area
Influenceed by GPS location error, rail inspection car is in the image not photographed in the same time to same section of railway track track
In the presence of certain skew, therefore before analyzing the image that detection-phase is photographed, first by present image and reference map
As being alignd.The alignment of general orbital image is location information and characteristic matching by rail to complete, conventional method
Use and matched based on the local feature that SIFT, HOG etc. are extracted mostly, but by the performance impact of traditional characteristic,
The algorithm accuracy of these alignment needs into a raising.The alignment schemes of the present invention are to be based on specific network proposed by the present invention
Model, the network model includes：Object segmentation network, image alignment network, can scheme to the benchmark image and detection of input automatically
Extract and estimate with offset deviation as the key point carried out, realize that fast and accurately orbital image aligns.Consider in rail image
Image segmentation algorithm is used in feature, network model, strengthen track, sleeper, outside track switch the signal portion image such as row's lock weight,
To realize that more accurately alignment is calculated.
The input of network model includes two parts：A part is the rail image currently inputted, i.e., failure to be detected
Rail image；Another part is reference picture, i.e. the image of the plug pin comprising normal mounting.Reference picture is clapped for rail inspection car
The historical photograph in this section taken the photograph, plug pin therein is normal condition after firm install.Current input image and reference picture
In the presence of certain GPS location error and shooting angle error.
Reference picture inputs object segmentation network respectively with image to be detected.The object segmentation network is main by full convolutional layer
The depth network composition of structure, and instructed for the segmentation task of the obvious object such as row's lock outside railroad track, sleeper and track switch
Practice.Row's lock is the important reference object on rail wherein outside track switch, as shown in the part marked in the red frame in left side in Fig. 3.By thing
After body segmentation network, the general information of the obvious object such as row's lock outside railroad track in image, sleeper and track switch is obtained.Split network
Structure mainly use full convolutional network structure, as shown in Figure 4.Network is output as the classification of each pixel, i.e.,：
Wherein,The prediction probability to kth type objects is arranged for the i-th row jth, K is the number for predicting classification.Obtained object
Cut zone is weighted as the weight of original image, wherein notable thing to the characteristics of image figure of convolutional layer in the middle of network
The weight of body region is close to 1, and the weight of non-significant object area is close to 0, so as to reach the effect of noise filtering.Last feature adds
The forms of characterization of power is：
zij=αijxij
Wherein αij∈ [0,1] be the i-th row jth row whether be object weight, xijIt is original characteristic value, zijIt is weighting
Characteristic value afterwards.
The feature of reference picture and image to be aligned with weight inputs follow-up image alignment network respectively, obtains figure
As alignment parameters.The image alignment network is mainly made up of some convolutional layers and full articulamentum, and netting twine finally exports four variables,
Respectively with entering to be fitted horizontal displacement Δ x, vertical displacement amount Δ y, horizontal stretch coefficient gammawWith stretched vertically coefficient gammah, according to
These coefficients correct input picture with regard to that can be alignd after detection image.The operation of alignment is with being mathematically represented by：
X '=γw(x-Δx)
Y '=γh(y-Δy)
(x, y) is the coordinate before alignment in formula, and (x ', y ') is the coordinate after alignment.
2nd, the fault detect of the track plug pin compensated based on depth network model and environmental change
The purpose of middle orbit plug pin fault detect of the present invention is to find plug pin the problem of in track, and it technically passes through
Compare what the plug pin image in detection photo was realized with the change of original normal plug pin image in reference picture, if plug pin shape
State there occurs certain change, then judge that plug pin breaks down.Conventional conventional method combination Manual definition feature (such as HOG, LBP,
Haar etc.) and the grader such as SVM and Adaboost judged, recognition accuracy Shortcomings, by environmental change influenceed compared with
Greatly.Track plug pin detection method in the present invention extracts plug pin feature by depth network model, and in the centre of network model
Get integrated into environment compensated information in layer, to reduce the influence of the environmental factors such as weather, illumination, improves what algorithm was judged failure plug pin
Accuracy.
As in Fig. 5, the image to be detected of the module input of the leftmost side for reference picture and after aliging, wherein, square frame is marked
Position be plug pin position.Reference picture is first fed to object detection network with image to be detected after aliging.The present invention's
Realize and object detection network (the full volume with Region Proposal Network based on deep learning is employed in system
Product network structure) target detection is carried out to input picture, the network structure can quickly and accurately find the specific thing in image
Body, exports position (x, y, w, h) (transverse axis, ordinate of orthogonal axes, width, height) and the feature representation z of object.During training network, I
Network is trained as positive example sample using the plug pin position on rail.By the object detection network, we can tie
Positional information after conjunction alignment obtains the plug pin feature in the reference picture and image to be detected at same position, the comparison for after
Analysis is used.
Meanwhile, whole reference picture is input to an ambient compensation network with the detection image after aliging, environment is obtained
Compensating for variations feature, is influenceed to reduce the rail photo photographed due to different time by environmental change.The ambient compensation
Network use with multiple convolutional layers network structure, system by reference picture with align after detection image subtract each other after residual error
The network structure with multiple convolutional layers is input to, obtains describing the further feature description of overall photo change, this feature will be tired
Plug pin object features to be detected are added to as the compensation rate of environmental change.It is expressed mathematically as
X '=x+xc
Wherein, x is the component feature of script, xcFor the complementary characteristics amount calculated, x ' is the characteristic quantity after overcompensation.
Plug pin object features in reference picture will be defeated with the plug pin object features in image to be detected by environmental change compensation
Enter to the last contiguous function being similar in twins' network and be compared, the contiguous function in the present invention uses European
Distance function.If both difference is more than certain threshold value, then it is assumed that plug pin to be detected has failure, and system sends alarm
Information.
Object detection network, environmental change compensation network, contiguous function in whole fault detection module is in model training
When be integral progress optimized parameter study, the contrastive loss functions used in training, its mathematical form is：
Wherein, E is final loss function value, and the image of component that y=1 represents two groups of contrasts belongs to normal component, y=
There is failure for detection part in what 0 representative was inputted；N is the number of input sample, d=| | an-bn||2It is the difference of two groups of predicted values
Different degree, an、bnIt is the network output vector of reference sample and sample to be checked respectively；Margin is default constant threshold, typically
Take 1.
Corresponding to two steps of above-mentioned track plug pin fault detection method, track plug pin fault detection system includes two
Module, i.e., high-speed railway rail image Accurate align module based on depth network model and marking area, based on depth network model
The fault detection module of the track plug pin compensated with environmental change；Two modules realize the operation of two steps respectively.
The specific implementation step of present system is：
The first step, deployment completes the fault detection system of the present invention.The computing unit of detecting system is installed on calculating service
Device center, carries out telecommunication with the camera of rail inspection car, obtains rail photo, analyzed；
Second step, user records normal rail image by rail inspection car and is used as reference picture.Rail inspection car band
There is GPS location function, while each lengths of rail photo is shot, the geographical location information where current rail is recorded automatically；
3rd step, periodically using rail inspection car, rail carries out the shooting of image to user to along, and remote transmission is extremely counted
Analyzed at calculation center；
Algorithm in 4th step, system combines the GPS location information that rail inspection car is provided in real time, uses the figure in system
As alignment module to current shooting to rail image and reference picture (the normal plug pin comprising same section is gone through in reference picture
History image) carry out alignment operation.Alignment model is based entirely on the depth network structure of the present invention during this, wherein using peculiar
Railroad track, sleeper and track switch outside row lock etc. obvious object segmentation and weighting block.Algorithm is detected in picture first to be occurred
The region of lock is arranged outside railroad track, sleeper and track switch, then in a network interbed to occur outside railroad track, sleeper and track switch arrange
The part in the region of lock is weighted.The characteristics of image being weighted continues through the estimation that network model calculates alignment deviation
Value；
5th step, system calls Fault Model therein to detect plug pin image wherein of problems, such as
Fruit finds wherein there is failure plug pin, and system can give a warning in time.Fault Model is based entirely on the present invention during this
Depth web results, wherein using distinctive ambient compensation mixed-media network modules mixed-media.The main thought of fault detect be it is preceding after alignment
Input picture in detection there is the position of plug pin, feature extraction then is carried out to the image in these plug pin regions.Extracting
After original characteristic value in these plug pin regions, the overall difference of original input picture and reference picture is compared by network model,
Automatically the correction value of character representation brought to environmental change is learnt, the characteristic value in adjustment plug pin region, i.e. environmental change are compensated.
Finally, the plug pin provincial characteristics after overcompensation by sorter network model export whether be normal condition plug pin part it is pre-
Survey；
6th step, maintenance personal can be repaired by system feedback to failure plug pin.After the completion of maintenance, can and
When shoot, update new reference image data.
The main points of the present invention can be summarized as：
(1) the track plug pin fault detection system based on image alignment with detection network model is provided, the system is by image
Alignment module and fault detection module two large divisions composition, realize the track plug pin fault detect based on graphical analysis；
(2) detection image and reference map of rail inspection car shooting are realized by the image alignment network model specially designed
Split submodule by image in the alignment operation of picture, the alignment network to strengthen for row's lock etc. thing outside track, sleeper, track switch
The weight of body, removes the noise information in image, lifts the performance of image alignment algorithm；
(3) position of plug pin in image is found by fault detect network model, while extracting the depth net of plug pin object
Network feature.Environmental change complementary characteristics (as shown in Figure 4) specially are devised, the plug pin in image to be detected after compensating will be worked as
Object is compared with the normal plug pin characteristics of image in benchmark image with position, so as to be made whether the judgement that there is failure.
The rail with the weighting such as row's lock outside middle orbit, sleeper, track switch in above technical scheme in (1) article aligns network
The fault detect network compensated with environmental change in model and (2) article is the central inventive point of the present invention.
The advantage of this programme：
(1) the image alignment model in this programme is all based on depth network model with Fault Model, compared to tradition
Method performance more preferably, with higher robustness；
(2) the organic weight for introducing marking area in the structure of depth network of the image alignment model in this programme,
The influence of noise is eliminated in the study of alignment parameters, so as to obtain higher alignment accuracy；
(3) the organic compensation letter for having incorporated environmental change in the structure of depth network of the Fault Model in this programme
Breath, the Expressive Features expression way under complex environment change is learnt by depth network model automatically, and using similar to double born of the same parents
The structure of tire network carries out normal component and compared with the feature of part to be detected, so as to reach more reliable fault detection accuracy.
Brief description of the drawings
The orbital image example that Fig. 1 photographs for the rail inspection car of the present invention.
Fig. 2 is the high-speed railway rail plug pin fault detection system overview flow chart proposed in the present invention.
Fig. 3 is the image alignment module principle figure proposed in the present invention.
Fig. 4 is the fault detection module schematic diagram proposed in the present invention.
Fig. 5 be the leftmost side module input be reference picture with align after image to be detected.
Embodiment
Technology in the present invention is mainly used in the failure plug pin in automatic detection high ferro railroad track, to reduce artificial detection
Time and cost overhead.
User records normal rail image by rail inspection car first and is used as reference when using the present invention.Rail
Inspection car carries GPS location function, and while each lengths of rail photo is shot, the geographical position where current rail can be recorded automatically
Confidence ceases.As the rail photo captured by rail inspection car as shown in figure 1, white edge tab area is to be detected in the present invention in figure
Plug pin part.After deployment completes the fault detection system of this hair energy, user only needs periodically to use rail inspection car to along
The algorithm that rail is carried out in the shooting of image, system can combine the GPS location information that rail inspection car is provided in real time, use system
In the rail image that is arrived to current shooting of image alignment module and reference picture it is (normal comprising same section in reference picture
Plug pin history image) carry out alignment operation；Then system can call Fault Model therein to plug wherein of problems
Nail image is detected that, if it find that wherein there is failure plug pin, system can give a warning in time.
This hair can implement the overall procedure of fault detection system as shown in Fig. 2 image to be detected is passed through respectively with reference picture
Image alignment module is crossed with after fault detection module, obtaining whether including the result of determination of failure plug pin in image to be detected.Tool
The failure detection steps of body are as follows：
Error present in the positional informations such as GPS location and shooting angle when the 1, due to rail inspection car shooting photo,
Calling figure carries out aliging for current input image and reference picture, general frame such as Fig. 3 institutes of the module as alignment module first
Show.
(a) input of image alignment module is made up of two parts.A portion is the rail image currently inputted, that is, is treated
Detect the rail image of failure；Another part is reference picture, i.e. the image of the plug pin comprising normal mounting.Reference picture is iron
The historical photograph in another section that rail inspection car is shot, plug pin therein is normal condition after firm install.Current input image
There is certain GPS location error and shooting angle error with reference picture；
(b) reference picture inputs object segmentation network respectively with image to be detected.The object segmentation network is main by rolling up entirely
The depth network composition of lamination structure, and enter for the segmentation task of the obvious object such as row's lock outside railroad track, sleeper and track switch
Row training.Row's lock is the important reference object on rail wherein outside track switch, as shown in the part marked in the red frame in left side in Fig. 3.Thing
The general information of the obvious object such as row's lock outside railroad track in image, sleeper and track switch can be obtained after body segmentation network；
(c) the object segmentation region obtained is using as the weight of original image, to the characteristics of image of convolutional layer in the middle of network
Figure is weighted, and wherein the weight in obvious object region is close to 1, and the weight of non-significant object area is close to 0, so as to reach noise
The effect of filtering；
(d) feature of the reference picture with weight and image to be aligned inputs follow-up image alignment network mould respectively
Type, obtains image alignment parameter.The network is mainly made up of some convolutional layers and full articulamentum, and netting twine finally exports four changes
Amount, respectively with entering to be fitted horizontal displacement Δ x, vertical displacement amount Δ y, horizontal stretch coefficient gammawWith stretched vertically coefficient gammah, root
Detection image after correcting input picture with regard to that can be alignd according to these coefficients.
2nd, after the image after being alignd, system further calls fault detection module to plug pin wherein of problems
Image is detected that the module can find plug pin position of problems in input picture automatically, and send alarm.Fault detect
The overall flow of module is as shown in Figure 4.
(a) in Fig. 4 the leftmost side module input for reference picture with align after image to be detected, wherein white box mark
The position of note is plug pin position；
(b) reference picture can be admitted to object detection network first with image to be detected after aliging.The realization of the present invention
Object detection network (the full convolution net with Region Proposal Network based on deep learning is employed in system
Network structure) target detection is carried out to input picture, the network structure can quickly and accurately find the certain objects in image,
Export position (x, y, w, h) (transverse axis, ordinate of orthogonal axes, width, height) and the feature representation z of object.During training network, we
Network is trained as positive example sample using the plug pin position on rail.By the object detection network, we can combine
Positional information after alignment obtains the plug pin feature in the reference picture and image to be detected at same position, the comparison for after point
Analysis is used；
(c) simultaneously, whole reference picture is input to an ambient compensation network mould by us with the detection image after aliging
Type, obtains environmental change complementary characteristics, is influenceed to reduce the rail photo photographed due to different time by environmental change.
The network by reference picture with align after detection image subtract each other after residual error be input to the network structure with multiple convolutional layers,
Obtain describing the further feature description of overall photo change, this feature is used as environment using plug pin object features to be detected are added to
The compensation rate of change；
(d) the plug pin object features in reference picture and the plug pin object in the image to be detected compensated by environmental change
Feature is compared the last contiguous function being similar in twins' network is input into, and the contiguous function in the present invention is adopted
It is Euclidean distance function.If both difference is more than certain threshold value, then it is assumed that plug pin to be detected has failure, is
System sends a warning；
(e) object detection network, environmental change compensation network, contiguous function in whole fault detection module is instructed in model
Progress optimized parameter study, the contrastive loss functions used in training are integral when practicing；
(f) the failure plug in the rail image of 80 kilometers of total length is detected paragraph by paragraph in units of two meters with the system in this patent
Nail, reaches precision ratio 97% after tested, and recall ratio 99%, every section of image averaging detects 0.10 second used time；Compared to traditional characteristic
Method (precision ratio 83%, recall ratio 98%, every section of image averaging detects 0.18 second used time) has a distinct increment.
Maintenance personal can be repaired by system feedback to failure plug pin.After the completion of maintenance, can shoot in time,
Update new reference image data.
Bibliography
[1]Krizhevsky,Alex,Ilya Sutskever,Geoffrey E.Hinton.ImageNet
Classification with Deep Convolutional Neural Networks.In Advances in neural
information processing systems, 2012,pp.1097-1105.
[2]NavneetDalal,Bill Triggs.Histograms of Oriented Gradients for
Human Detection.IEEE Computer Society Conference on Computer Vision and
Pattern Recognition,2005,pp.886-893.
[3] Du Xinyu, height favorable to the people, Wu Nan, Cheng Yu, Wang Xiaodong railway plug pin image detecting methods and device
.CN201510752738.8, 2016.
[4]E.Deutschl,C.Gasser,A.Niel and J.Werschonig.Defect detection on
rail surfaces by a vision based system.IEEE Intelligent Vehicles Symposium,
2004,pp.507-511.
[5]Lin Jie,Luo Siwei,Li Qingyong,Zhang Hanqing and Ren Shengwei.Real-
time rail head surface defect detection:A geometrical approach.2009IEEE
International Symposium on Industrial Electronics,Seoul,2009,pp.769-774.
[6] railway plug pins of the Du Xin fine jades based on computer vision technique is automatically positioned algorithm railway communication signals, 52
(9):68-72,2016.
[7] vision-based detection of Tang Xiangna, Wang Yao south rail surface defects and recognizer computer engineering, 39 (3):
25-30,2013.
[8]Shaoqing Ren,Kaiming He,Ross B Girshick,Jian Sun.Faster R-CNN:
Towards Real-Time Object Detection with Region Proposal Networks.In Advances
in Neural Information Processing Systems,2015,pp.91–99。
Claims (3)
1. track plug pin fault detection method, normal plug pin image is recorded as benchmark image by rail inspection car；Work people
After member during routine rail inspection, newest rail conditions are shot using inspection car, rail conditions image work is obtained
For image to be detected, processing is analyzed to current image to be detected and benchmark image, problem plug pin in rail is detected
Component locations, so as to carry out changing problem plug pin in time, it is ensured that the safety of rail operation；Characterized in that, comprising the following steps that：
（1）High-speed railway rail image Accurate align based on depth network model and marking area
High-speed railway rail image Accurate align uses specific network model, and the network model includes：Object segmentation network, image pair
Neat network, extracts to the key point that benchmark image and image to be detected are carried out and estimates with offset deviation, realize fast and accurately rail
Road image alignment；Wherein：
Object segmentation network is mainly made up of the depth network of full convolution Rotating fields, and for outside railroad track, sleeper and track switch
The segmentation task of the obvious objects such as row's lock is trained；After object segmentation network, obtain railroad track in image, sleeper with
The general information of the obvious object such as row's lock outside track switch；Segmentation network is output as the classification of each pixel, i.e.,：
Wherein,The prediction probability to kth type objects is arranged for the i-th row jth,KIt is the number for predicting classification；Obtained object point
Region is cut as the weight of original image, the characteristics of image figure of convolutional layer in the middle of network is weighted, wherein obvious object area
The weight in domain is close to 1, and the weight of non-significant object area is close to 0, so as to reach the effect of noise filtering；Last characteristic weighing
Forms of characterization is：
Wherein,Be the i-th row jth row whether be object weight,It is original characteristic value,After being weighting
Characteristic value；
The feature of reference picture and image to be aligned with weight inputs follow-up image alignment network respectively, obtains image pair
Neat parameter；The image alignment network is mainly made up of some convolutional layers and full articulamentum, finally exports four variables, respectively with entering
It is fitted horizontal displacement∆x, vertical displacement amount∆y, horizontal stretch coefficientγ w With stretched vertically coefficientγ h , according to these coefficients
Detection image after input picture is corrected with regard to that can be alignd；The operation of alignment is with being mathematically represented as：
In formula, It is the coordinate before alignment, It is the coordinate after alignment；
（2）The fault detect of the track plug pin compensated based on depth network model and environmental change
The fault detect of track plug pin is to extract plug pin feature by depth network model, and is melted in the intermediate layer of network model
Enter ambient compensation information, to reduce the influence of the environmental factors such as weather, illumination, improve the accuracy judged failure plug pin；Institute
Stating depth network model includes：Object detection network, ambient compensation network, contiguous function；Wherein：
Reference picture is first fed into object detection network with image to be detected after aliging；Object detection network, which is used, is based on depth
The object detection network with Region Proposal Network of study carries out target detection, output to input picture
The position of body：Transverse axis, ordinate of orthogonal axes, width, height, and feature representation z；
Meanwhile, whole reference picture is input to ambient compensation network with image to be detected after aliging, environmental change benefit is obtained
Feature is repaid, is influenceed to reduce the rail photo photographed due to different time by environmental change；The ambient compensation network is adopted
With the network structure with multiple convolutional layers, system first subtracts each other reference picture with the detection image after aliging, obtained residual error
The network structure with multiple convolutional layers is input to, obtains describing the further feature description of overall photo change, this feature will be tired
Plug pin object features to be detected are added to as the compensation rate of environmental change；Plug pin object features in reference picture are with passing through ring
Plug pin object features in image to be detected of border compensating for variations are input into the last connection being similar in twins' network
Function is compared；Contiguous function uses Euclidean distance function；If both difference is more than certain threshold value, then it is assumed that to be checked
There is failure in the plug pin of survey, system sends a warning；
Object detection network, ambient compensation network, contiguous function in process fault detection are integral in model training
Optimized parameter study, the contrastive loss functions used in training are carried out, its mathematical form is：
Wherein,EIt is final loss function value,y=1 image of component for representing two groups of contrasts belongs to normal component,y=0 represents
There is failure for detection part in what is inputted；NIt is the number of input sample,It is the diversity factor of two groups of predicted values,a n 、b n It is the network output vector of reference sample and sample to be checked respectively；Margin is default constant threshold.
2. a kind of track plug pin fault detection system based on claim 1 methods described, it is characterised in that including two moulds
Block, i.e., high-speed railway rail image Accurate align module based on depth network model and marking area, based on depth network model with
The fault detection module of the track plug pin of environmental change compensation, realizes two corresponding to the track plug pin fault detection method
Step works.
3. the implementation of track plug pin fault detection system as claimed in claim 2, concrete operations flow is as follows：
The first step, disposes track plug pin fault detection system, i.e., is installed on the computing unit of track plug pin fault detection system
Calculation server center, carries out telecommunication with the camera of rail inspection car, obtains rail photo, analyzed；
Second step, user records normal rail image by rail inspection car and is used as reference picture；Rail inspection car carries GPS
Positioning function, while each lengths of rail photo is shot, records the geographical location information where current rail automatically；
3rd step, periodically using rail inspection car, rail carries out the shooting of image to user to along, and remote transmission is into calculating
The heart is analyzed；
Algorithm in 4th step, road plug pin fault detection system combines the GPS location information that rail inspection car is provided in real time, uses
The rail image that image alignment module in system is arrived to current shooting carries out alignment operation with reference picture；The figure being weighted
As feature continues through the estimate that network model calculates alignment deviation；
5th step, road plug pin fault detection system calls Fault Model therein to enter plug pin image wherein of problems
Row detection, if it find that wherein there is failure plug pin, gives a warning in time；
6th step, maintenance personal is repaired by system feedback information to failure plug pin；After the completion of maintenance, shoot in time, more
New reference image data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710424243.1A CN107292870B (en) | 2017-06-07 | 2017-06-07 | Track plug pin fault detection method and system based on image alignment and detection network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710424243.1A CN107292870B (en) | 2017-06-07 | 2017-06-07 | Track plug pin fault detection method and system based on image alignment and detection network model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107292870A true CN107292870A (en) | 2017-10-24 |
| CN107292870B CN107292870B (en) | 2020-11-20 |
Family
ID=60096070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710424243.1A Active CN107292870B (en) | 2017-06-07 | 2017-06-07 | Track plug pin fault detection method and system based on image alignment and detection network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107292870B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108846841A (en) * | 2018-07-02 | 2018-11-20 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| CN108961288A (en) * | 2018-07-10 | 2018-12-07 | 中国铁路上海局集团有限公司科学技术研究所 | A kind of web of the rail plug pin pin and lead detection image intelligent identification Method |
| CN109116865A (en) * | 2018-09-19 | 2019-01-01 | 苏州傲特欣智能科技有限公司 | Large scale equipment unmanned plane cruising inspection system and its method based on machine vision |
| CN109636812A (en) * | 2018-12-13 | 2019-04-16 | 银河水滴科技（北京）有限公司 | A kind of Rail Surface and contact net surface image dividing method and device |
| CN109658397A (en) * | 2018-12-12 | 2019-04-19 | 广州地铁集团有限公司 | A kind of rail polling method and system |
| CN109948690A (en) * | 2019-03-14 | 2019-06-28 | 西南交通大学 | A kind of high-speed rail scene perception method based on deep learning and structural information |
| CN110782443A (en) * | 2019-10-23 | 2020-02-11 | 四川大学 | Railway track defect detection method and system |
| CN111091554A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon swing bolster fracture fault image identification method |
| CN111197977A (en) * | 2019-12-31 | 2020-05-26 | 北京京天威科技发展有限公司 | Choke transformer box image real-time screening method based on rail web plug pin image recognition |
| CN111340774A (en) * | 2020-02-24 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Image detection method, image detection device, computer equipment and storage medium |
| CN112526959A (en) * | 2019-09-19 | 2021-03-19 | 北京国双科技有限公司 | Oil well pump fault diagnosis method, device, equipment and storage medium |
| CN112997196A (en) * | 2018-11-20 | 2021-06-18 | 北伯林顿铁路公司 | System and method for determining defects of physical objects |
| CN113365896A (en) * | 2018-11-15 | 2021-09-07 | 阿万特国际科技公司 | Image-based track/rail fault monitoring and detection |
| CN114022826A (en) * | 2022-01-05 | 2022-02-08 | 石家庄学院 | Block chain-based rail detection method and system |
| CN118101413A (en) * | 2024-04-25 | 2024-05-28 | 中国电建集团江西省电力设计院有限公司 | Data communication network equipment fault diagnosis method and equipment based on neural network |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426894A (en) * | 2015-11-06 | 2016-03-23 | 中国铁道科学研究院 | Image detection method and device for railroad plug nails |
-
2017
- 2017-06-07 CN CN201710424243.1A patent/CN107292870B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426894A (en) * | 2015-11-06 | 2016-03-23 | 中国铁道科学研究院 | Image detection method and device for railroad plug nails |
Non-Patent Citations (4)
| Title |
|---|
| SHAOQING REN 等: "Faster R-CNN:Towards Real-Time Object Detection with Regin Proposal Networks", 《ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEM》 * |
| 吴楠 等: "高速铁路轨腰塞钉巡检***研究", 《中国铁路》 * |
| 杜馨瑜 等: "基于深度学习的铁道塞钉自动检测算法", 《中国铁道科学》 * |
| 杜馨瑜: "基于计算机视觉技术的铁道塞钉自动定位算法", 《铁道通信信号》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108846841A (en) * | 2018-07-02 | 2018-11-20 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| US11488294B2 (en) | 2018-07-02 | 2022-11-01 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method for detecting display screen quality, apparatus, electronic device and storage medium |
| CN108961288A (en) * | 2018-07-10 | 2018-12-07 | 中国铁路上海局集团有限公司科学技术研究所 | A kind of web of the rail plug pin pin and lead detection image intelligent identification Method |
| CN108961288B (en) * | 2018-07-10 | 2022-03-01 | 中国铁路上海局集团有限公司科学技术研究所 | Intelligent identification method for rail web plug pin and lead detection image |
| CN109116865A (en) * | 2018-09-19 | 2019-01-01 | 苏州傲特欣智能科技有限公司 | Large scale equipment unmanned plane cruising inspection system and its method based on machine vision |
| CN113365896B (en) * | 2018-11-15 | 2024-02-27 | 阿万特国际科技公司 | Image-based track/rail fault monitoring and detection |
| CN113365896A (en) * | 2018-11-15 | 2021-09-07 | 阿万特国际科技公司 | Image-based track/rail fault monitoring and detection |
| CN112997196A (en) * | 2018-11-20 | 2021-06-18 | 北伯林顿铁路公司 | System and method for determining defects of physical objects |
| CN109658397A (en) * | 2018-12-12 | 2019-04-19 | 广州地铁集团有限公司 | A kind of rail polling method and system |
| CN109636812A (en) * | 2018-12-13 | 2019-04-16 | 银河水滴科技（北京）有限公司 | A kind of Rail Surface and contact net surface image dividing method and device |
| CN109948690A (en) * | 2019-03-14 | 2019-06-28 | 西南交通大学 | A kind of high-speed rail scene perception method based on deep learning and structural information |
| CN112526959A (en) * | 2019-09-19 | 2021-03-19 | 北京国双科技有限公司 | Oil well pump fault diagnosis method, device, equipment and storage medium |
| CN110782443A (en) * | 2019-10-23 | 2020-02-11 | 四川大学 | Railway track defect detection method and system |
| CN111091554B (en) * | 2019-12-12 | 2020-08-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon swing bolster fracture fault image identification method |
| CN111091554A (en) * | 2019-12-12 | 2020-05-01 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon swing bolster fracture fault image identification method |
| CN111197977A (en) * | 2019-12-31 | 2020-05-26 | 北京京天威科技发展有限公司 | Choke transformer box image real-time screening method based on rail web plug pin image recognition |
| CN111340774A (en) * | 2020-02-24 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Image detection method, image detection device, computer equipment and storage medium |
| CN111340774B (en) * | 2020-02-24 | 2023-03-21 | 腾讯科技（深圳）有限公司 | Image detection method, image detection device, computer equipment and storage medium |
| CN114022826A (en) * | 2022-01-05 | 2022-02-08 | 石家庄学院 | Block chain-based rail detection method and system |
| CN118101413A (en) * | 2024-04-25 | 2024-05-28 | 中国电建集团江西省电力设计院有限公司 | Data communication network equipment fault diagnosis method and equipment based on neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107292870B (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292870A (en) | Track plug pin fault detection method and system based on image alignment with detection network model | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN107657226B (en) | People number estimation method based on deep learning | |
| CN105373135B (en) | A kind of method and system of aircraft docking guidance and plane type recognition based on machine vision | |
| CN104183127B (en) | Traffic surveillance video detection method and device | |
| CN103324913B (en) | A kind of pedestrian event detection method of Shape-based interpolation characteristic sum trajectory analysis | |
| CN108921875A (en) | A kind of real-time traffic flow detection and method for tracing based on data of taking photo by plane | |
| CN104268538A (en) | Online visual inspection method for dot matrix sprayed code characters of beverage cans | |
| CN111611874B (en) | Face mask wearing detection method based on ResNet and Canny | |
| CN110033002A (en) | Detection method of license plate based on multitask concatenated convolutional neural network | |
| CN103886325B (en) | Cyclic matrix video tracking method with partition | |
| CN104504897A (en) | Intersection traffic flow characteristic analysis and vehicle moving prediction method based on trajectory data | |
| CN106682586A (en) | Method for real-time lane line detection based on vision under complex lighting conditions | |
| CN105184271A (en) | Automatic vehicle detection method based on deep learning | |
| CN105787482A (en) | Specific target outline image segmentation method based on depth convolution neural network | |
| CN110084165A (en) | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations | |
| CN103425967A (en) | Pedestrian flow monitoring method based on pedestrian detection and tracking | |
| CN108230254A (en) | A kind of full lane line automatic testing method of the high-speed transit of adaptive scene switching | |
| CN103473570A (en) | Self-adaptive video scene pedestrian detection method | |
| CN105205486A (en) | Vehicle logo recognition method and device | |
| CN103886589A (en) | Goal-oriented automatic high-precision edge extraction method | |
| CN110516633A (en) | A kind of method for detecting lane lines and system based on deep learning | |
| CN111862029A (en) | Fault detection method for bolt part of vertical shock absorber of railway motor train unit | |
| CN106096504A (en) | A kind of model recognizing method based on unmanned aerial vehicle onboard platform | |
| CN104732236B (en) | A kind of crowd's abnormal behaviour intelligent detecting method based on layered shaping |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |