CN109064444B - Track slab disease detection method based on significance analysis - Google Patents

Track slab disease detection method based on significance analysis Download PDF

Info

Publication number
CN109064444B
CN109064444B CN201810687093.8A CN201810687093A CN109064444B CN 109064444 B CN109064444 B CN 109064444B CN 201810687093 A CN201810687093 A CN 201810687093A CN 109064444 B CN109064444 B CN 109064444B
Authority
CN
China
Prior art keywords
frame
video
value
color
monitoring
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810687093.8A
Other languages
Chinese (zh)
Other versions
CN109064444A (en
Inventor
姚莉
吴琼颖
吴含前
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201810687093.8A priority Critical patent/CN109064444B/en
Publication of CN109064444A publication Critical patent/CN109064444A/en
Application granted granted Critical
Publication of CN109064444B publication Critical patent/CN109064444B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Abstract

The invention discloses a track slab disease detection method based on significance analysis, which mainly comprises the following steps: (1) and carrying out illumination preprocessing on the video, and extracting the spatial significance of the vehicle-mounted video by fusing the color, brightness and direction characteristics. (2) On one hand, sparse sampling is carried out on the track video in the monitoring period, the current monitoring video frame is compared with the corresponding sampling video frame, and the video frame with large change is selected as a reliable frame. On the other hand, the track video frame with the abrupt change characteristic relative to the upper frame and the lower frame is selected as the reliable frame. (3) And obtaining rough space-time saliency map processing by fusing the spatial saliency map and the temporal saliency map. And (3) considering background prior, central prior and space compactness factors for the rough significant map to obtain an improved space-time significant map. (4) And obtaining a track disease significant weight according to the significant graph, and identifying and classifying the track diseases of the high-speed railway by using the weight and adopting a semi-supervised classification method based on a significance weighting model. The invention can detect the track diseases in real time.

Description

Track slab disease detection method based on significance analysis
Technical Field
The invention relates to the field of computer vision and image processing, in particular to a track slab disease detection method based on significance analysis.
Background
Smooth passage of high-speed trains relies on high-quality high-speed railway tracks, and therefore, the track quality is directly related to the safety of operation. However, the high-speed railway track is frequently damaged due to exposure outdoors and influence of various environments. And the high-speed railway is put into operation on a large scale only in recent decades, so that the development of related disease detection work is relatively delayed, and even a high-cost artificial detection mode is still required for the disease detection work in many places. In recent years, the continuous development of intelligent detection systems makes intelligent track disease detection methods possible. The GJ-6 type track detection system is developed by adopting a laser camera measurement technology in China, but the detection methods have low identification accuracy for detecting surface layer diseases of high-speed railway tracks.
At present, the rail disease detection method at home and abroad still has the problems to be solved:
(1) human inspection and traditional detection systems do not address the track disease characteristics in a timely manner, and results obtained at high cost do not display the disease results well.
(2) The existing detection method for processing video data is to check each frame of video frame, and the method has the disadvantages of high complexity, time-consuming operation, difficult application to occasions with high real-time requirements and failure in highlighting key disease areas.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems in the prior art, the invention provides a track slab disease detection method based on significance analysis.
The technical scheme is as follows: the track slab disease detection method based on significance analysis is characterized by comprising the following steps: the method comprises the following steps:
(1) acquiring a track slab monitoring video in a monitoring period, and performing illumination preprocessing on the monitoring video to enable the illumination intensity of the monitoring video to be uniform;
(2) extracting the characteristics in color, brightness and direction from the monitoring video frame in the monitoring period after the preprocessing by adopting an Itti algorithm, and fusing to obtain a spatial saliency map;
(3) sparse sampling is carried out on the historical monitoring video, difference calculation is carried out on the sampled video frame and the monitoring period monitoring video frame preprocessed in the step (1), and the monitoring period monitoring video frame with the difference larger than a preset threshold value is selected as a reliable frame;
(4) selecting a frame with characteristic mutation relative to the upper frame and the lower frame as a reliable frame for the monitoring period monitoring video preprocessed in the step (1);
(5) calculating a time saliency value according to all reliable frames by adopting a self-adaptive transmission method to serve as a time saliency map;
(6) fusing the space saliency map and the time saliency map to obtain a rough space-time saliency map;
(7) carrying out image processing considering background prior, center prior and space compactness factors on the rough space-time saliency map to obtain an improved space-time saliency map;
(8) and obtaining a track disease significance weight according to the improved space-time significance map, and identifying and classifying the track diseases of the high-speed railway by using the weight and adopting a semi-supervised classification method based on a significance weighting model.
Further, the step (1) specifically comprises:
(1.1) acquiring a monitoring video of the track slab;
(1.2) intercepting an image of a key focus position in each frame in a monitoring video by using a Range function in an OpenCV function library, and ignoring some dark areas formed by light shadows at the bottom;
and (1.3) equalizing the histograms of the intercepted images by using a cvEqualizeHist function in an OpenCV function library to realize uniform illumination intensity.
Further, the step (3) specifically comprises:
(3.1) carrying out sparse sampling on the historical monitoring video to obtain a plurality of historical sampling video frames;
(3.2) dividing the preprocessed monitoring period monitoring video into a plurality of video segments to obtain a video segment set V ═ Vi1, …, n is the number of the divided video segments;
(3.3) for video segments viScreening out the significant blocks from all frames;
(3.4) projecting the RGB color values of each significant block into a three-dimensional space color model of 10 x 10, and performing Gaussian diffusion on the three-dimensional space color model;
(3.5) forming a one-dimensional vector by using the color values of all points in the three-dimensional space, and taking the one-dimensional vector as the color distribution vector of the current significant block;
(3.6) forming a color matrix by the color distribution vectors of all the significant blocks belonging to one video frame according to the sequence, and performing sparse low-rank matrix decomposition on the color matrix by adopting RPCA to obtain a sparse matrix;
(3.7) calculating the deviation value of the sparse matrix obtained by decomposition and each historical sampling video frame, accumulating the deviation values, and selecting a deviation accumulated value ESUMGreater than a predetermined thresholdVideo frame as video segment viI-1, …, n.
Further, the step (3.3) specifically comprises:
(3.3.1) for video segment viEach frame of which is divided into a plurality of image blocks of a × a pixels, a being the size of the divided image block and 0<a<10；
(3.3.2) eliminating image blocks positioned at the boundary from the divided image blocks;
(3.3.3) calculating the average relevant difference value of each image block and four adjacent image blocks which are remained after the processing of the step (3.3.2);
and (3.3.4) taking the image block with the average correlation difference value larger than the preset threshold value as the significant block.
Further, the step (3.4) specifically comprises:
(3.4.1) equally dividing R, G, B value range 0-255 of the RGB color model into 10 intervals respectively, thereby forming a 10 x 10 three-dimensional space color model; i.e. a 10 x 10 cube is formed, which has a total of 1000 points.
(3.4.2) projecting the RGB color values of each pixel of the saliency block into the three-dimensional space color model and three-dimensionally diffusing the projections using a Gaussian sphere model.
Wherein, the coordinate conversion formula of the projection is as follows:
Figure GDA0003145613480000031
in the formula (R)p,Gp,Bp) The RGB color value of pixel p in the saliency block, (x)p,yp,zp) The coordinate value of the pixel point p projected into the three-dimensional space color model,
Figure GDA0003145613480000032
for coordinates (x) in a three-dimensional space color modelp,yp,zp) The color value at the position, ceil () represents a ceiling function.
Wherein, because 10 × 10 × 10 is larger than the number of the significant blocks, in order to prevent the situation that the color distribution is sparse, the projection is three-dimensionally diffused by adopting a gaussian sphere model, and the specific diffusion formula is as follows:
Figure GDA0003145613480000033
in the formula, Cx,y,zIs the color value at the (x, y, z) coordinate position after diffusion, CposFor a color value at the post-projection position pos, d represents the manhattan distance between position pos and position (x, y, z).
Further, the step (4) specifically comprises:
(4.1) acquiring the monitoring video in the monitoring period preprocessed in the step (1);
(4.2) for each frame of the monitoring video in the monitoring period, respectively calculating the brightness difference value, the color difference value and the texture difference value of the frame with the previous frame and the next frame according to the following formulas:
Li＝(l(Pi)-l(Pi-1))+(l(Pi)-l(Pi+1))
Ci＝(c(Pi)-c(Pi-1))+(c(Pi)-c(Pi+1))
Ti＝(t(Pi)-t(Pi-1))+(t(Pi)-t(Pi+1))
in the formula, PiRepresenting the current video frame, Pi-1、Pi+1Respectively representing a previous frame and a next frame, Li、Ci、TiRespectively representing the current video frame PiThe brightness difference value, the color difference value and the texture difference value of the previous frame and the next frame, wherein l (-) represents the brightness value of the video frame, c (-) represents the color value of the video frame, and t (-) represents the texture value of the video frame;
(4.3) normalizing the brightness difference value, the color difference value and the texture difference value of each frame and the previous frame and the next frame to obtain the integral characteristic difference value of the frame
Figure GDA0003145613480000041
Figure GDA0003145613480000042
In the formula (I), the compound is shown in the specification,
Figure GDA0003145613480000043
representing video frames PiDifference of global characteristics from upper and lower frames, deltaL、δc、δTWeights representing luminance, color, and texture features, respectively;
(4.4) difference of global characteristics
Figure GDA0003145613480000044
And considering the video frames with the characteristics suddenly changed as reliable frames when the video frames with the characteristics suddenly changed are larger than a preset threshold.
Further, the step (5) specifically comprises:
(5.1) acquiring all reliable frames, and calculating deviation accumulated value E of each reliable frame and historical framesSUM,jAnd the difference of the global characteristics of the upper and lower frames
Figure GDA0003145613480000045
(5.2) adopting a self-adaptive transmission method, and calculating the time significance value of each reliable frame by adopting the following formula:
Figure GDA0003145613480000046
in the formula, SjRepresents the temporal saliency value of the jth reliable frame, WSUM represents the final normalized weight, and
Figure GDA0003145613480000047
Wkrepresenting the weight value of the kth reliable frame.
(5.3) the temporal saliency values of all reliable frames form a temporal saliency map.
Has the advantages that: compared with the prior art, the invention has the following remarkable advantages:
(1) the invention provides a method for detecting the track diseases by analyzing the video significance, so that the track disease characteristics are highlighted, and the effect of intelligent track disease detection is achieved.
(2) The method improves the traditional video significance algorithm, ensures that the significant disease area is extracted, and improves the calculation efficiency and the detection rate aiming at the specific track disease.
(3) In the invention, the fused rough space-time apparent image is optimized, and a result image which better accords with human eye attention and highlights disease characteristics is obtained by considering background prior factors, central prior factors and space compactness factors.
(4) The invention provides a semi-supervised classification method based on a significance weighting model, which is used for identifying and classifying diseases.
Drawings
FIG. 1 is a schematic flow diagram of one embodiment of the present invention.
Detailed Description
The embodiment provides a method for detecting a track slab fault based on significance analysis, as shown in fig. 1, the method includes:
the method comprises the steps of firstly, acquiring a track slab monitoring video in a monitoring period, and carrying out illumination preprocessing on the monitoring video to enable the illumination intensity of the monitoring video to be uniform. The method specifically comprises the following steps:
(1.1) acquiring a monitoring video of the track slab;
(1.2) intercepting an image of a key focus position in each frame in a monitoring video by using a Range function in an OpenCV function library, and ignoring some dark areas formed by light shadows at the bottom;
and (1.3) equalizing the histograms of the intercepted images by using a cvEqualizeHist function in an OpenCV function library to realize uniform illumination intensity.
And step two, extracting the characteristics of color, brightness and direction from the monitoring video frame in the post-preprocessing monitoring period by adopting an Itti algorithm, and fusing to obtain a spatial saliency map.
The main steps can be summarized into three main modules:
(2.1) extraction of local features
First, an input video image is represented as a 9-level gaussian pyramid. Wherein the layer 0 is an input image, and the layer 1 to 8 images are formed by filtering and sampling the input image with a 5 × 5 gaussian filter, and have sizes in the range of 1/2 to 1/256 of the input image, respectively. Then, various luminance features, I, R, G, B, Y, and four directions (0 °, 45 °, 90 °, 135 °, respectively), are extracted for each layer of the pyramid, forming a luminance pyramid, a chrominance pyramid, and a direction pyramid. The color histogram method and the brightness histogram method are adopted to respectively reflect the main distribution of color and brightness in the image for extracting the color and brightness characteristics.
(2.2) calculation of multiscale-center surround contrast
The Itti algorithm is a simulated center-surround structure, and differences are made between different scales of a feature pyramid for various features.
Suppose that the center corresponds to the feature map pixel (c ∈ [2,3,4]) of the scale c, and the peripheral area corresponds to the feature map pixel of the scale s (s ═ c + δ, δ ∈ [3,4 ]). Since the resolution of the feature maps of different scales is different, after interpolation is required to make the two images have the same size, the process can be represented by θ by performing point-to-point subtraction on the two images. I (c, s) represents a luminance characteristic diagram, indicating the contrast of luminance. Other features are similarly expressed. The feature contrast of the center (scale c) and the periphery (scale s) can then be obtained to quantify the local directional feature contrast of the center and the periphery.
I(c，s)＝|I(c)|Θ|I(s)|
(2.3) fusion of the comparison plots
To fuse (2.2) feature maps of multiple different scales and different features, the Itti model proposes a normalized function N (·). Firstly, normalizing the significant value of each pixel point in a graph to an interval [0, M ] for each feature graph, so as to eliminate the influence generated by different intervals of the distribution of the significant values of different features; secondly, searching a global maximum value M in the feature map, and calculating the average value M of all other local maximum values; finally, multiply each position in the feature map by (M-M) 2. This enlarges the positions of the potentially salient regions in each feature map so that the salient values for those positions are more prominent relative to the background.
Firstly, normalizing the feature maps of different scales of each feature to form a comprehensive saliency map related to the feature, and then normalizing the saliency maps of different features to obtain the final visual saliency map.
And step three, sparse sampling is carried out on the historical monitoring video, difference calculation is carried out on the sampled video frames and the monitoring period monitoring video frames preprocessed in the step (1), and the monitoring period monitoring video frames with the difference larger than a preset threshold value are selected as reliable frames.
The method comprises the steps of comparing a current monitoring video frame with a historical video frame and a position frame, sampling a monitoring period video by sparse sampling, and performing targeted treatment on track diseases by considering the gradual change characteristic of the track diseases globally. In addition, the reliable frame is selected, so that the calculation amount can be reduced, and the calculation efficiency is improved.
The method specifically comprises the following steps:
(3.1) carrying out sparse sampling on the historical monitoring video to obtain a plurality of historical sampling video frames;
(3.2) dividing the preprocessed monitoring period monitoring video into a plurality of video segments to obtain a video segment set V ═ Vi1, …, n is the number of the divided video segments;
(3.3) for video segments viScreening out the significant blocks from all frames;
the specific process for screening the significant blocks comprises the following steps: (3.3.1) for video segment viEach frame of which is divided into a plurality of image blocks of a × a pixels, a being the size of the divided image block and 0<a<10; in this embodiment, a is 4. (3.3.2) eliminating image blocks positioned at the boundary from the divided image blocks; (3.3.3) calculating the average relevant difference value of each image block and four adjacent image blocks which are remained after the processing of the step (3.3.2); and (3.3.4) taking the image block with the average correlation difference value larger than the preset threshold value as the significant block.
These significant blocks need to be processed next since the foreground will be contained in them. Since for a sequence of video frames the foreground objects are generally invariant throughout the video sequence, their variation in color space is relatively small. Therefore, their foreground part color distribution can be considered to have a low rank characteristic in the entire video sequence. Then, the video frames with low sparsity in the video sequence can be regarded as having higher accuracy of the preliminary result relative to the surrounding frames.
(3.4) projecting the RGB color values of each significant block into a 10 x 10 three-dimensional space color model and performing Gaussian diffusion on the three-dimensional space color model.
The step (3.4) specifically comprises the following steps:
(3.4.1) equally dividing R, G, B value range 0-255 of the RGB color model into 10 intervals respectively, thereby forming a 10 x 10 three-dimensional space color model; i.e. a 10 x 10 cube is formed, which has a total of 1000 points.
(3.4.2) projecting the RGB color values of each pixel of the saliency block into the three-dimensional space color model and three-dimensionally diffusing the projections using a Gaussian sphere model.
Wherein, the coordinate conversion formula of the projection is as follows:
Figure GDA0003145613480000071
in the formula (R)p,Gp,Bp) The RGB color value of pixel p in the saliency block, (x)p,yp,zp) The coordinate value of the pixel point p projected into the three-dimensional space color model,
Figure GDA0003145613480000072
for coordinates (x) in a three-dimensional space color modelp,yp,zp) The color value at the position, ceil () represents a ceiling function.
Wherein, because 10 × 10 × 10 is larger than the number of the significant blocks, in order to prevent the situation that the color distribution is sparse, the projection is three-dimensionally diffused by adopting a gaussian sphere model, and the specific diffusion formula is as follows:
Figure GDA0003145613480000081
in the formula, Cx,y,zIs the color value at the (x, y, z) coordinate position after diffusion, CposFor a color value at position pos, d represents the manhattan distance between position pos and position (x, y, z).
And (3.5) forming a one-dimensional vector by using the color values of all points in the three-dimensional space, and taking the one-dimensional vector as the color distribution vector of the current significant block.
Wherein, because the three-dimensional space color is 10 × 10 × 10, there are 1000 points, and the statistics of 1000 points form a 1000 one-dimensional vector.
(3.6) forming a color matrix by the color distribution vectors of all the significant blocks belonging to one video frame according to a sequence, and performing sparse low-rank matrix decomposition on the color matrix by adopting RPCA (robust Principal Component analysis) to obtain a sparse matrix;
for example, if there are 3 significant blocks of a video frame, K1, K2, and K3, and their corresponding color distribution vectors are cl1, cl2, and cl3, respectively, then the formed color matrix is G ═ cl1, cl2, and cl 3.
RPCA is a matrix decomposition method, where a matrix G is a + B, a is a low rank part, and B is a sparse part.
(3.7) calculating the deviation value of the sparse matrix obtained by decomposition and each historical sampling video frame, accumulating the deviation values, and selecting a deviation accumulated value ESUMThe video frame larger than the preset threshold value is taken as the video segment viI-1, …, n.
Wherein the content of the first and second substances,
Figure GDA0003145613480000082
Elindicating the ith historical sample video frame and LM is the number of historical sample video frames.
And step four, selecting a frame with characteristic mutation relative to the upper frame and the lower frame as a reliable frame for the preprocessed monitoring video in the monitoring period. The method specifically comprises the following steps
(4.1) acquiring the monitoring video in the monitoring period preprocessed in the step (1);
(4.2) for each frame of the monitoring video in the monitoring period, respectively calculating the brightness difference value, the color difference value and the texture difference value of the frame with the previous frame and the next frame according to the following formulas:
Li＝(l(Pi)-l(Pi-1))+(l(Pi)-l(Pi+1))
Ci＝(c(Pi)-c(Pi-1))+(c(Pi)-c(Pi+1))
Ti＝(t(Pi)-t(Pi-1))+(t(Pi)-t(Pi+1))
in the formula, PiRepresenting the current video frame, Pi-1、Pi+1Respectively representing a previous frame and a next frame, Li、Ci、TiRespectively representing the current video frame PiThe brightness difference value, the color difference value and the texture difference value of the previous frame and the next frame, wherein l (-) represents the brightness value of the video frame, c (-) represents the color value of the video frame, and t (-) represents the texture value of the video frame;
(4.3) normalizing the brightness difference value, the color difference value and the texture difference value of each frame and the previous frame and the next frame to obtain the integral characteristic difference value of the frame
Figure GDA0003145613480000091
Figure GDA0003145613480000092
In the formula (I), the compound is shown in the specification,
Figure GDA0003145613480000093
representing video frames PiDifference of global characteristics from upper and lower frames, deltaL、δc、δTWeights representing luminance, color, and texture features, respectively;
(4.4) general characterizationDifference value
Figure GDA0003145613480000094
And considering the video frames with the characteristics suddenly changed as reliable frames when the video frames with the characteristics suddenly changed are larger than a preset threshold.
And step five, calculating to obtain a time saliency value as a time saliency map according to all reliable frames by adopting a self-adaptive transmission method. The method specifically comprises the following steps:
(5.1) acquiring all reliable frames, and calculating deviation accumulated value E of each reliable frame and historical framesSUM,jAnd the difference of the global characteristics of the upper and lower frames
Figure GDA0003145613480000095
(5.2) adopting a self-adaptive transmission method, and calculating the time significance value of each reliable frame by adopting the following formula:
Figure GDA0003145613480000096
in the formula, SjRepresents the temporal saliency value of the jth reliable frame, WSUM represents the final normalized weight, and
Figure GDA0003145613480000097
Wkrepresenting the weight value of the kth reliable frame.
(5.3) the temporal saliency values of all reliable frames form a temporal saliency map.
And step six, fusing the space saliency map and the time saliency map to obtain a rough space-time saliency map.
And seventhly, performing image processing on the rough space-time saliency map by considering background prior, center prior and space compactness factors to obtain an improved space-time saliency map.
(1) Background prior factors:
during the processing, the suppression of background areas is an important means for obtaining accurate significance values. We can measure the difference between the background region and other regions, and the difference is larger and more significant, and vice versa. A Markov absorption chain method is adopted to calculate the background prior value of each super pixel, the edge super pixels are taken as absorption nodes, and meanwhile, the average absorption time of other nodes is calculated as a significant value.
The Markov process analysis is to use the state transition probability p between statesijReflecting the dynamic change of the system state and representing the probability of transferring from the ith state to the jth state in one step. The matrix with the state transition probability as an element is called a one-step state transition probability matrix of the Markov chain, and is called a transition matrix for short. If p isiiWhen 1, i is referred to as the absorption state. Then a markov chain with at least one absorption state is called an absorption chain. Assume that an absorption chain has r absorption states and t transition states. The standard form of the transition matrix can be expressed as:
Figure GDA0003145613480000101
wherein Q ∈ [0,1 ]]t×tRepresenting the transition probability between transition states, R ∈ [0,1 ]]t×tIndicating the transition probability from the transition state to the absorbing state. 0 denotes the zero matrix r × t, and I is the transition probability between absorbing states, which is an identity matrix. Then we can get the basic matrix for P as:
N＝(I-Q)-1
wherein the element N in NijCan be regarded as the expected time for the chain to go from transition state i to transition state j, so the expected time in the chain to go from each transition state to the absorbing state can be expressed as:
Y＝N×c
where c represents a unit column vector of dimension t. The weight defining the boundary between neighboring node i and node j is:
wij＝exp(-||xi-xj||/δ2)
wherein xiAnd xjRespectively representing the characteristic mean values of the super-pixels i and j in the CIELab space, and delta is a constant for controlling the weight. Therefore we canThe element definition of the incidence matrix a can be derived:
Figure GDA0003145613480000102
where N (i) is the set of nodes connected to node i. The diagonal matrix D represents the sum of the weights connecting each node, defined as:
D＝diag(∑jaij)
the transition matrix can thus be expressed as:
P＝D-1+A
according to the formula N ═ (I-Q)-1The value of N can be derived, and the average absorption time per super-pixel, i.e. the expected value SiaI.e. the background prior value is quantized, thereby obtaining a background prior map.
(2) Central prior factor:
an important measure of the central priors is the distance of each pixel from the central pixel, and the following distance can be defined to measure the size of the central priors:
Figure GDA0003145613480000111
wherein (x)0，y0) The center of the image is represented by (W/2, H/2). For the case of division into superpixels, the positional saliency of a superpixel may be calculated using an isotropic gaussian kernel function.
Figure GDA0003145613480000112
Wherein (x)c，yc) Indicating the region ri(x) of (C)0，y0) Is the center of the field of vision and,
Figure GDA0003145613480000113
and
Figure GDA0003145613480000114
are variables in the x and y directions.
(3) Calculating the space compactness:
the spatial compactness reflects the spatial distribution of the superpixels in the image, and the salient objects are obtained through the distribution condition. Calculating superpixel viThe compact spatial density value is Ds (i) with the formula
Figure GDA0003145613480000115
Assume four directions on the original depth map boundary: the upper left, the upper right, the lower left and the lower right are provided with a vertex, wherein
Figure GDA0003145613480000116
Representing the non-normalized Laplace matrix form at four vertices, njRepresenting a superpixel vjInner pixel number, spatial mean
Figure GDA0003145613480000117
Is defined as:
Figure GDA0003145613480000118
Figure GDA0003145613480000121
in conclusion, the background prior factor can well inhibit the background area, the central prior factor and the spatial compactness can well highlight the brightness of the salient object, so that three considered factors can be applied to the rough space-time salient image processing to obtain a final space-time salient value:
S(i)=norm(Sia*exp(1-norm(Sib+Sic))).
and step eight, obtaining a track disease significance weight according to the improved space-time significance diagram, and identifying and classifying the track diseases of the high-speed railway by using the weight and adopting a semi-supervised classification method based on a significance weighting model.
By analyzing the orbit video space-time saliency before, an improved space-time saliency map is obtained. The significance regions can be subjected to weight division by the significance map, for the regions with high significance, the classification contrast weight of the regions with high significance is correspondingly increased, and the most significant disease regions are considered to be most similar to the existing class disease maps. The extracted diseases can be identified by utilizing a trained classifier, and the probability p (Y | X) of the diseases to be classified in each category is calculated according to the color histogram, the texture feature and the SIFT feature of the diseases to be classified; then, classification is performed according to the probability p (Y | X): if the posterior probability p (Y | X) of the sample is the greatest under which class distribution function, it belongs to which class.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims (6)

1. A track slab disease detection method based on significance analysis is characterized by comprising the following steps: the method comprises the following steps:
(1) acquiring a track slab monitoring video in a monitoring period, and performing illumination preprocessing on the monitoring video to enable the illumination intensity of the monitoring video to be uniform;
(2) extracting the characteristics in color, brightness and direction from the monitoring video frame in the monitoring period after the preprocessing by adopting an Itti algorithm, and fusing to obtain a spatial saliency map;
(3) sparse sampling is carried out on the historical monitoring video, difference calculation is carried out on the sampled video frame and the monitoring period monitoring video frame preprocessed in the step (1), and the monitoring period monitoring video frame with the difference larger than a preset threshold value is selected as a reliable frame;
(4) selecting a frame with characteristic mutation relative to the upper frame and the lower frame as a reliable frame for the monitoring period monitoring video preprocessed in the step (1); the method specifically comprises the following steps:
(4.1) acquiring the monitoring video in the monitoring period preprocessed in the step (1);
(4.2) for each frame of the monitoring video in the monitoring period, respectively calculating the brightness difference value, the color difference value and the texture difference value of the frame with the previous frame and the next frame according to the following formulas:
Li＝(l(Pi)-l(Pi-1))+(l(Pi)-l(Pi+1))
Ci＝(c(Pi)-c(Pi-1))+(c(Pi)-c(Pi+1))
Ti＝(t(Pi)-t(Pi-1))+(t(Pi)-t(Pi+1))
in the formula, PiRepresenting the current video frame, Pi-1、Pi+1Respectively representing a previous frame and a next frame, Li、Ci、TiRespectively representing the current video frame PiThe brightness difference value, the color difference value and the texture difference value of the previous frame and the next frame, wherein l (-) represents the brightness value of the video frame, c (-) represents the color value of the video frame, and t (-) represents the texture value of the video frame;
(4.3) normalizing the brightness difference value, the color difference value and the texture difference value of each frame and the previous frame and the next frame to obtain the integral characteristic difference value of the frame
Figure FDA0003145613470000011
Figure FDA0003145613470000012
In the formula (I), the compound is shown in the specification,
Figure FDA0003145613470000013
representing video frames PiDifference of global characteristics from upper and lower frames, deltaL、δc、δTWeights representing luminance, color, and texture features, respectively;
(4.4) difference of global characteristics
Figure FDA0003145613470000014
The video frames larger than the preset threshold value are considered to have the characteristic mutation, and the video frames with the characteristic mutation are used as reliable frames;
(5) calculating a time saliency value according to all reliable frames by adopting a self-adaptive transmission method to serve as a time saliency map;
(6) fusing the space saliency map and the time saliency map to obtain a rough space-time saliency map;
(7) carrying out image processing considering background prior, center prior and space compactness factors on the rough space-time saliency map to obtain an improved space-time saliency map, wherein a space-time saliency value S (i) in the improved space-time saliency map is as follows:
S(i)＝norm(Sia*exp(1-norm(Sib+Sic))
Siafor mean absorption time of superpixels, i.e. for quantifying background prior values, SibIs the position significant value of the super pixel, SicIs the spatial compactness value of the super-pixel;
(8) and obtaining a track disease significance weight according to the improved space-time significance map, and identifying and classifying the track diseases of the high-speed railway by using the weight and adopting a semi-supervised classification method based on a significance weighting model.
2. The track slab disease detection method based on significance analysis according to claim 1, characterized in that: the step (1) specifically comprises the following steps:
(1.1) acquiring a monitoring video of the track slab;
(1.2) intercepting an image of a key focus position in each frame in a monitoring video by using a Range function in an OpenCV function library, and ignoring some dark areas formed by light shadows at the bottom;
and (1.3) equalizing the histograms of the intercepted images by using a cvEqualizeHist function in an OpenCV function library to realize uniform illumination intensity.
3. The track slab disease detection method based on significance analysis according to claim 1, characterized in that: the step (3) specifically comprises the following steps:
(3.1) carrying out sparse sampling on the historical monitoring video to obtain a plurality of historical sampling video frames;
(3.2) dividing the preprocessed monitoring period monitoring video into a plurality of video segments to obtain a video segment set V ═ Vi1, …, n is the number of the divided video segments;
(3.3) for video segments viScreening out the significant blocks from all frames;
(3.4) projecting the RGB color values of each significant block into a three-dimensional space color model of 10 x 10, and performing Gaussian diffusion on the three-dimensional space color model;
(3.5) forming a one-dimensional vector by using the color values of all points in the three-dimensional space, and taking the one-dimensional vector as the color distribution vector of the current significant block;
(3.6) forming a color matrix by the color distribution vectors of all the significant blocks belonging to one video frame according to the sequence, and performing sparse low-rank matrix decomposition on the color matrix by adopting RPCA to obtain a sparse matrix;
(3.7) calculating the deviation value of the sparse matrix obtained by decomposition and each historical sampling video frame, accumulating the deviation values, and selecting a deviation accumulated value ESUMThe video frame larger than the preset threshold value is taken as the video segment viI-1, …, n.
4. The track slab disease detection method based on significance analysis according to claim 3, characterized in that: the step (3.3) specifically comprises the following steps:
(3.3.1) for video segment viEach frame of which is divided into a plurality of image blocks of a × a pixels, a being the size of the divided image block and 0<a<10；
(3.3.2) eliminating image blocks positioned at the boundary from the divided image blocks;
(3.3.3) calculating the average relevant difference value of each image block and four adjacent image blocks which are remained after the processing of the step (3.3.2);
and (3.3.4) taking the image block with the average correlation difference value larger than the preset threshold value as the significant block.
5. The track slab disease detection method based on significance analysis according to claim 3, characterized in that: the step (3.4) specifically comprises the following steps:
(3.4.1) equally dividing R, G, B value range 0-255 of the RGB color model into 10 intervals respectively, thereby forming a 10 x 10 three-dimensional space color model;
(3.4.2) projecting the RGB color values of each pixel of the saliency block into the three-dimensional space color model and three-dimensionally diffusing the projections using a Gaussian sphere model;
wherein, the coordinate conversion formula of the projection is as follows:
Figure FDA0003145613470000031
in the formula (R)p,Gp,Bp) The RGB color value of pixel p in the saliency block, (x)p,yp,zp) The coordinate value of the pixel point p projected into the three-dimensional space color model,
Figure FDA0003145613470000032
for coordinates (x) in a three-dimensional space color modelp,yp,zp) Color value at position, ceil () represents an rounding-up function;
wherein, the three-dimensional diffusion formula is as follows:
Figure FDA0003145613470000033
in the formula, Cx,y,zIs the color value at the (x, y, z) coordinate position after diffusion, CposFor the color value at the post-projection position pos, d represents the Manhattan distance between position pos and position (x, y, z).
6. The track slab disease detection method based on significance analysis according to claim 1, characterized in that: the step (5) specifically comprises the following steps:
(5.1) obtaining all reliable frames and calculating eachDeviation accumulated value E of a reliable frame and a historical frameSUM,jAnd the difference of the global characteristics of the upper and lower frames
Figure FDA0003145613470000041
(5.2) adopting a self-adaptive transmission method, and calculating the time significance value of each reliable frame by adopting the following formula:
Figure FDA0003145613470000042
in the formula, SjRepresents the temporal saliency value of the jth reliable frame, WSUM represents the final normalized weight, and
Figure FDA0003145613470000043
Wka weight value representing a kth reliable frame;
(5.3) the temporal saliency values of all reliable frames form a temporal saliency map.
CN201810687093.8A 2018-06-28 2018-06-28 Track slab disease detection method based on significance analysis Active CN109064444B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810687093.8A CN109064444B (en) 2018-06-28 2018-06-28 Track slab disease detection method based on significance analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810687093.8A CN109064444B (en) 2018-06-28 2018-06-28 Track slab disease detection method based on significance analysis

Publications (2)

Publication Number Publication Date
CN109064444A CN109064444A (en) 2018-12-21
CN109064444B true CN109064444B (en) 2021-09-28

Family

ID=64817728

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810687093.8A Active CN109064444B (en) 2018-06-28 2018-06-28 Track slab disease detection method based on significance analysis

Country Status (1)

Country Link
CN (1) CN109064444B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635790A (en) * 2019-01-28 2019-04-16 杭州电子科技大学 A kind of pedestrian's abnormal behaviour recognition methods based on 3D convolution
CN110827193B (en) * 2019-10-21 2023-05-09 国家广播电视总局广播电视规划院 Panoramic video significance detection method based on multichannel characteristics

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101103378A (en) * 2005-01-10 2008-01-09 汤姆森许可贸易公司 Device and method for creating a saliency map of an image
CN103020965A (en) * 2012-11-29 2013-04-03 奇瑞汽车股份有限公司 Foreground segmentation method based on significance detection
CN105303571A (en) * 2015-10-23 2016-02-03 苏州大学 Time-space saliency detection method for video processing
CN106611427A (en) * 2015-10-21 2017-05-03 中国人民解放军理工大学 A video saliency detection method based on candidate area merging

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101103378A (en) * 2005-01-10 2008-01-09 汤姆森许可贸易公司 Device and method for creating a saliency map of an image
CN103020965A (en) * 2012-11-29 2013-04-03 奇瑞汽车股份有限公司 Foreground segmentation method based on significance detection
CN106611427A (en) * 2015-10-21 2017-05-03 中国人民解放军理工大学 A video saliency detection method based on candidate area merging
CN105303571A (en) * 2015-10-23 2016-02-03 苏州大学 Time-space saliency detection method for video processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
融合背景先验与中心先验的显著性目标检测;周帅骏等;《中国图象图形学报》;20170516(第05期);全文 *

Also Published As

Publication number Publication date
CN109064444A (en) 2018-12-21

Similar Documents

Publication Publication Date Title
CN110598610B (en) Target significance detection method based on neural selection attention
Dudhane et al. Cdnet: Single image de-hazing using unpaired adversarial training
Matsushita et al. Illumination normalization with time-dependent intrinsic images for video surveillance
Dudhane et al. C^ 2msnet: A novel approach for single image haze removal
CN110929593B (en) Real-time significance pedestrian detection method based on detail discrimination
CN111723693B (en) Crowd counting method based on small sample learning
CN105930822A (en) Human face snapshot method and system
CN115294117B (en) Defect detection method and related device for LED lamp beads
CN109685045A (en) A kind of Moving Targets Based on Video Streams tracking and system
CN114758288A (en) Power distribution network engineering safety control detection method and device
AU2020272936B2 (en) Methods and systems for crack detection using a fully convolutional network
Wang et al. Background extraction based on joint gaussian conditional random fields
CN111274964B (en) Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle
CN109064444B (en) Track slab disease detection method based on significance analysis
Babu et al. An efficient image dahazing using Googlenet based convolution neural networks
Shit et al. An encoder‐decoder based CNN architecture using end to end dehaze and detection network for proper image visualization and detection
Gardziński et al. Crowd density estimation based on voxel model in multi-view surveillance systems
CN111832508A (en) DIE _ GA-based low-illumination target detection method
Bolten et al. Evaluation of Deep Learning based 3D-Point-Cloud Processing Techniques for Semantic Segmentation of Neuromorphic Vision Sensor Event-streams.
Zhu Image quality assessment model based on multi-feature fusion of energy Internet of Things
Zhu et al. Multi-feature fusion algorithm in VR panoramic image detail enhancement processing
CN111862098B (en) Individual matching method, device, equipment and medium based on light field semantics
Nair et al. Benchmarking single image dehazing methods
Zhang et al. A unified saliency detection framework for visible and infrared images
CN117078608B (en) Double-mask guide-based high-reflection leather surface defect detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant