CN114821351A

CN114821351A - Railway hazard source identification method and device, electronic equipment and storage medium

Info

Publication number: CN114821351A
Application number: CN202210280934.XA
Authority: CN
Inventors: 孙旭; 潘旭冉; 杨丽娜; 蔡丹路; 高连如
Original assignee: China State Railway Group Co Ltd; Aerospace Information Research Institute of CAS
Current assignee: China State Railway Group Co Ltd; Aerospace Information Research Institute of CAS
Priority date: 2022-03-21
Filing date: 2022-03-21
Publication date: 2022-07-29

Abstract

The invention provides a railway hazard source identification method, a railway hazard source identification device, electronic equipment and a storage medium. The method comprises the following steps: obtaining remote sensing images along the railway; inputting the remote sensing image into a dangerous source identification model to obtain a dangerous source identification result; the training process of the danger source identification model comprises the following steps: obtaining a remote sensing image of a sample along a railway and generating a danger source label; establishing a TE-ResUnet model, inputting the remote sensing images of samples along the railway into the TE-ResUnet model to obtain a plurality of dangerous source prediction identification results with different scales; and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results and the risk source labels at different scales, and completing end-to-end training by utilizing back propagation to obtain an optimal TE-ResUnet model. The invention improves the identification precision of the dangerous source and the identification capability of the small target of the model, and can realize accurate and effective railway dangerous source identification.

Description

Railway hazard source identification method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of remote sensing identification, in particular to a railway hazard source identification method, a railway hazard source identification device, electronic equipment and a storage medium.

Background

Dangerous source electrodes such as aged color steel roofs, unfixed dust screens and mulching films along the high-speed railway lines are easy to blow up and fall on the rails or the high-speed railway net, so that the normal operation of the high-speed railway lines and trains is hindered, and even larger safety accidents are caused. Conventionally, a ground investigation method is a main means for detecting a hazard around a railway, and the ground investigation informs relevant department units to reinforce the hazard so as to effectively prevent accidents. However, the limitations of manpower and material resources affect the accuracy and efficiency of ground survey, so that the survey result is poor. However, the high-resolution satellite remote sensing image has the advantages of large range, space-time continuity, periodicity, easiness in acquisition, low cost and the like, and the spatial resolution of the high-resolution remote sensing image reaches centimeter level, so that a railway hazard source can be clearly presented. Therefore, the high-resolution remote sensing image ground object identification technology can be used for accurately identifying and dynamically monitoring the relevant dangerous source information along the railway.

However, the dangerous sources along the high-speed railway are sparsely distributed on the high-resolution remote sensing image, the number of small targets is large, and the intra-class difference of the same dangerous source is large, which brings certain technical difficulty for the identification of the dangerous sources of the high-resolution remote sensing image and puts higher requirements on the accuracy and the stability of the identification method. At present, the high-resolution remote sensing image ground feature identification and classification are generally carried out by using a semantic segmentation technology, and a deep convolutional neural network has a good effect in the field of natural image semantic segmentation, so that the deep convolutional neural network is widely applied to the task. However, although the existing ground feature identification method based on the deep convolutional neural network improves the ground feature identification accuracy within a certain range, the problems of low ground feature boundary identification accuracy and low small target identification accuracy still exist.

Disclosure of Invention

In order to solve the above problems, the present invention provides a method and an apparatus for identifying a railway hazard source, an electronic device, and a storage medium.

The invention provides a railway hazard source identification method, which comprises the following steps:

obtaining remote sensing images along the railway;

inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows:

step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label;

step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales;

and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model.

Optionally, the texture enhancement module is configured to perform texture enhancement processing on the two feature maps at the shallower scale output by the contracted path module before the expanded path module performs clipping and copying on the two feature maps at the shallower scale output by the contracted path module through a skip connection layer and performs upsampling processing on the two feature maps at the shallower scale output by the contracted path module, and the texture enhancement processing includes:

the texture enhancement module quantizes and counts texture information of the two feature maps of the shallow scale output by the contraction path module by using a histogram quantization method, each level of quantization represents a texture, and a learnable map network is constructed to reconstruct each quantization level so as to enhance the details of the texture.

Optionally, the calculating the multi-scale Lov _ sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, and completing end-to-end training by using back propagation to obtain an optimal TE-ResUnet model, including:

performing first Lov a sz-Softmax loss calculation on a dangerous source prediction identification result obtained after the first characteristic diagram is subjected to dimensionality reduction processing by the dimensionality reduction module and the dangerous source label, performing second Lov a sz-Softmax loss calculation on a dangerous source prediction identification result obtained after the second characteristic diagram is subjected to dimensionality reduction processing by the dimensionality reduction module and the dangerous source label, and performing third Lov a sz-Softmax loss calculation on a dangerous source prediction identification result obtained after the second characteristic diagram is subjected to sampling processing by the expanded path module and dimensionality reduction processing by the dimensionality reduction module and the dangerous source label;

and taking the weighted sum result of the first Lov-sz-Softmax loss, the second Lov-sz-Softmax loss and the third Lov-sz-Softmax loss as a target loss, optimizing the TE-ResUnet model according to the target loss, and finishing training when the target loss is less than a preset threshold value so as to obtain an optimal TE-ResUnet model.

Optionally, the target loss L _Total Comprises the following steps:

wherein λ is ₁ 、λ ₂ 、λ ₃ Weight of the loss function in three scales, Δ J _L Is Lov-sz-Softmax loss function, y is a hazard label,

the risk source prediction identification result output for the TE-ResUnet,

for the feature map subjected to dimension reduction in the extended path module, y ^D2 Down-sampling the original hazard label by a factor of 2,

and reducing the dimension of the feature map subjected to the texture enhancement processing in the extended path module.

Optionally, before the remote sensing image is input to the hazard source identification model, the method further includes:

and processing the remote sensing image with the image size larger than the preset threshold value by using a sliding window method.

The invention also provides a railway hazard source identification device, comprising:

the acquisition module is used for acquiring remote sensing images along the railway;

the processing module is used for inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows:

obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label;

establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales;

and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model.

Optionally, the processing module is further specifically configured to:

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor executes the program to implement the steps of the railway hazard source identification method as described in any one of the above.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for identifying a railway hazard source as defined in any one of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method for identifying a source of railway hazards as defined in any one of the preceding claims.

The invention provides a railway hazard source identification method, a railway hazard source identification device, electronic equipment and a storage medium. Firstly, obtaining a remote sensing image along a railway, and then inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model. Therefore, the high-resolution remote sensing image railway hazard source identification TE-ResUnet model is established, on one hand, texture details are enhanced on low-level features through the texture enhancement module, so that the hazard source identification precision, particularly the hazard source boundary identification precision, is improved, on the other hand, the model is optimized by utilizing a multi-scale Lov a sz-Softmax loss function, so that the small target identification capacity is improved, and meanwhile, the stability of the model is also improved.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart of a railway hazard source identification method provided by the present invention;

FIG. 2 is a schematic diagram of the training of the hazard source identification model provided by the present invention;

FIG. 3 is a schematic diagram of a railway hazard source identification apparatus provided by the present invention;

fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, the method for identifying a railway hazard source provided by the invention comprises the following steps:

step 101: obtaining remote sensing images along the railway;

in this step, optionally, the remote sensing image along the railway obtained by the method is from a high-resolution remote sensing satellite.

Step 102: inputting the remote sensing image into a dangerous source identification model to obtain a dangerous source identification result of the high-speed railway; the training process of the danger source identification model is as follows:

In the step, in the training process of the danger source identification model, firstly, a danger source identification data set of the high-resolution remote sensing image is obtained, and a danger source identification data set along the railway is manufactured, wherein the data set comprises a plurality of remote sensing images with danger source labels. And then, taking the UNet model as a basic framework, using a residual error module as a contraction path, introducing a texture enhancement module, and establishing a TE-ResUnet model. And training and verifying the TE-ResUnet model by using a dangerous source identification data set along the railway, and completing end-to-end training by calculating the multiscale Lov & ltsb & gt-Softmax loss between a prediction result and a label and utilizing reverse propagation to obtain the optimal TE-ResUnet model. And finally, inputting the high-resolution remote sensing image to be tested into the optimal TE-ResUnet model to obtain a railway hazard source identification result of the high-resolution remote sensing image.

Wherein, the TE-ResUnet model comprises: the system comprises a contraction path module, a storage module and a processing module, wherein the contraction path module comprises an input layer, a maximum pooling layer and a plurality of residual error modules and is used for carrying out feature recognition and dimension reduction on an input remote sensing image; optionally, the residual error modules have 50 layers, and perform 2-time down-sampling 5 times to generate feature maps of 5 scales; the expansion path module comprises a convolution layer and an up-sampling layer and is used for up-sampling the feature map from the contraction path module to the size of an original image and simultaneously performing dimension reduction to obtain an identification result; the expansion path module cuts and copies low-layer characteristics in the contraction path module through the jump connection layer and is used for carrying out up-sampling processing in the expansion path module; the texture enhancement module comprises two scales and is used for respectively carrying out texture enhancement processing on the feature maps of the two scales of the shallower layer in the contraction path module; and the dimension reduction module comprises two convolution layers and is used for reducing dimensions of two scale feature maps at a higher layer in the extended path module to obtain prediction results of different scales and calculating multi-scale loss.

It should be noted that the number of feature maps in the shallower scale may be N, where N is greater than or equal to 2, and is determined according to the number of feature maps output by the narrowing-down path module, and when the narrowing-down path module outputs 5 feature maps in different scales, the number of feature maps in the shallower scale may be 2.

The railway hazard source identification method provided by the invention comprises the steps of firstly obtaining remote sensing images along a railway, and then inputting the remote sensing images into a hazard source identification model to obtain a high-speed railway hazard source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model. Therefore, the high-resolution remote sensing image railway hazard source identification TE-ResUnet model is established, on one hand, texture details are enhanced on low-level features through the texture enhancement module, so that the hazard source identification precision, particularly the hazard source boundary identification precision, is improved, on the other hand, the model is optimized by utilizing a multi-scale Lov a sz-Softmax loss function, so that the small target identification capacity is improved, and meanwhile, the stability of the model is also improved.

Based on the content of the foregoing embodiment, in this embodiment, the texture enhancement module is configured to perform texture enhancement processing on the two feature maps at the shallower scale before the extended path module performs clipping to duplicate the two feature maps at the shallower scale output by the contracted path module through a skip connection layer and performs upsampling processing on the two feature maps at the shallower scale output by the contracted path module, and the texture enhancement module includes:

In this embodiment, it should be noted that, the idea of histogram equalization is used to quantize and count the texture information of the two scale low-level features at the shallowest level, each level of quantization represents a texture, and then a learnable graph network is constructed to quantize and reconstruct each level, so as to enhance the detail enhancement of the original texture. Specifically, the input feature map A of the input texture enhancement module _C×H×W The specific treatment method is as follows: first global average pooling is performed to convert to g _C×1 And can be regarded as the average value of all pixels of a. Each element dimension of A is C multiplied by 1 and H multiplied by W in total, and S is obtained by calculating cosine similarity of each element and the mean value g _H×W In which S is _ij The calculation method of (a) is shown as follows:

then, the element values in S are quantized, the quantization series is N, and a quantization coding graph E is obtained _N×HW E and L together represent a histogram, where L is the abscissa representing the number of quantized gray levels, E is the ordinate representing the weight corresponding to each bin, and the final histogram uses C _N×2 Denotes concat (E, L). C is subjected to MLP conversion and global average pooling output and is connected according to depth to serve as final output D _C1×N (histogram).

Then, according to the histogram quantization method, a new quantization level L' needs to be obtained by D. Each new series should be obtained by sensing all the statistics of the original levels and can be treated as a graph. To this end, a graph is constructed to propagate information from various levels. The statistical characteristic of each quantization series is defined as a node. In a conventional histogram quantization algorithm, a neighboring matrix is a manually defined diagonal matrix, which is expanded to a learning matrix as follows:

X＝Softmax(φ ₁ (D) ^T ·φ ₂ (D))

wherein phi ₁ 、φ ₂ Representing two different 1 x 1 convolutional layers and performing Softmax in a first dimension as a non-linear normalization function, then updating each node to obtain a reconstructed quantization series by fusing the characteristics of all other nodes

L′＝φ ₃ (D)·X

Wherein phi ₃ Represents another 1 x 1 convolutional layer, and then maps E E R using quantization coding ^n×hw The reconstruction series L' is assigned to each pixel, resulting in a final output R, since E may reflect the original quantization level of each pixel. R is obtained by:

R＝L′·E

then R is reconstructed into

I.e. the texture enhanced feature map.

Based on the content of the foregoing embodiment, in this embodiment, the calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source tag, and completing end-to-end training by using back propagation to obtain an optimal TE-ResUnet model includes:

and taking the weighted sum of the first Lov-sz-Softmax loss, the second Lov-sz-Softmax loss and the third Lov-Softmax loss as a target loss, optimizing the TE-ResUnet model according to the target loss, and finishing training when the target loss is less than a preset threshold value, thereby obtaining an optimal TE-ResUnet model.

In this embodiment, it should be noted that, in the model training phase, all training data are input into the TE-ResUnet model in batches, the loss between the output result of the TE-ResUnet model and the tag is continuously calculated iteratively, and the network model parameters are adjusted by using back propagation. Because the important characteristics of the hazard source data set are the sparsity and the category imbalance of the hazard source samples, the characteristics ensure that the model cannot better focus on pixels with smaller areas and easy confusion in the optimization process, and cannot better balance the influence of the loss of target and background pixels on the optimization process. When the classical cross entropy loss function is used for the problem, the model can be seriously biased to the background with larger pixel ratio, and the effect is poor. Lov sz-Softmax loss is a metric-based measure, and such metric-based loss functions focus more on metrics of the entire image, rather than on individual pixels for cross-entropy loss, i.e., without considering sample distribution equalization, which is more effective for the two-classification problem where the foreground pixel proportion is much smaller than the background. In order to better supervise two texture enhancement modules, multi-scale Lov-sz-Softmax loss is used, loss function calculation is carried out on the basis of feature maps of three scales, besides the final prediction result and the loss of the label are calculated, Lov-sz-Softmax loss calculation is also carried out on the feature map subjected to texture enhancement processing in the expanded path module and the label subjected to down-sampling respectively, Lov-sz-Softmax loss calculation is carried out on the feature map subjected to dimension reduction processing in the expanded path module and the label, and finally the loss of the three parts is weighted and taken as the total loss.

Based on the aboveContents of examples in the present example, the target loss L _Total Comprises the following steps:

wherein λ is ₁ 、λ ₂ 、λ ₃ Weight Δ J as a function of loss of three scales _L Is Lov-sz-Softmax loss function, y is a hazard label,

the risk source prediction identification result output for the TE-ResUnet,

reducing the dimension of the feature map subjected to the texture enhancement processing in the extended path module; optionally, the training image is cut into image blocks with the size of 512 × 512, and the learning rate is set to 10 ^-3 The batch size was set to 4 and the SGD momentum was set to 0.9. Three weights λ of the loss function ₁ 、λ ₂ 、λ ₃ Set to 0.7, 0.2, 0.1, respectively. The iteration stops when the training is satisfied that the 10 epoch losses do not continue to decrease.

Based on the content of the foregoing embodiment, in this embodiment, in the testing stage, before the remote sensing image is input to the hazard source identification model, the method further includes:

and processing the remote sensing image with the image size larger than the preset threshold value by using a sliding window method to avoid video memory overflow, and taking the average value of the original image and the identification result of the turnover image as the final identification result of the dangerous source of the high-speed railway.

In this embodiment, it should be noted that the test image input into the optimal model TE-ResUnet may be any size; the method comprises the steps of obtaining an original image and a reversed image identification result, wherein for a test image with a larger size and larger than a preset size threshold value, a sliding window method is used for avoiding video memory overflow, and the average value of the original image and the reversed image identification result is used as a final identification result, so that the test precision is improved.

The following is illustrated by specific examples:

the first embodiment is as follows:

in this embodiment, optionally, the data of the present invention is from a high-resolution second satellite, the original image of the high-resolution second data set is a fused image of a panchromatic waveband and a multispectral waveband with a spatial resolution of 0.8 m, and three wavebands of red, green, and blue are used. The label manufacturing method comprises the step of marking the danger source along the railway pixel by pixel in a professional visual interpretation mode, wherein the danger source pixel is a label 1, and the background is a label 0. The data includes dangerous source types of the high-speed railway, such as a color steel house and a dust screen.

The color steel house is a typical temporary building in a city, takes a color-coated steel plate as a raw material, has the main colors of blue, white and red, has larger hue distinction, and has the common characteristic of high color brightness degree in a high-resolution remote sensing image, and most of the shapes are in rectangular or rectangular combination on the whole. The main functions of the dust screen are dust control and environmental protection, the exposed earth surface of a city and materials easy to raise dust are generally required to be covered with the dust screen, and the dust screen is generally made of polyethylene mesh and is light green in color. And marking the image pixel by pixel through professional visual interpretation, endowing a color steel house and a dust screen in the original image with a label 1, endowing the background with a label 0, and finally obtaining a sample label.

The data set contained 153 image slices of 1000 x 1000 color steel house, 139 image slices containing dust screens. In the data preprocessing step, data enhancement is performed through rotation of 90 degrees, 180 degrees and 270 degrees, horizontal and vertical inversion and random cropping, and 512-by-512 image slices are obtained, namely 3672 pieces and 3336 pieces respectively.

In this embodiment, as shown in fig. 2, the TE-ResUnet model of the present invention includes a contraction path module 1, which includes an input layer C1 and a plurality of residual modules, for performing feature recognition and dimension reduction on an input remote sensing image; preferably, the contraction path module has 50 layers, and performs 5 times of 2-time down-sampling to generate a feature map with 5 scales; the expansion path module 2 comprises a convolution layer and an up-sampling layer and is used for up-sampling the feature map from the contraction path module to the size of an original image and simultaneously performing feature dimension reduction to obtain an identification result; a jump connection layer 3, wherein the four dimensions of the expansion path module are cut and copied by the jump connection layer to the low-layer features in the contraction path module, and are used for performing up-sampling processing in the expansion path module; the texture enhancement module 4 comprises two scales and is used for respectively carrying out texture enhancement processing on the feature maps of the shallower layer in the contraction path module; and the dimension reduction module 5 comprises three convolution layers and is used for reducing dimensions of two scale characteristic graphs of a higher layer in the extended path module to obtain prediction results of different scales for calculating the multi-scale Lov & sz-Softmax loss.

In the present embodiment, the high-resolution remote sensing image is first input to a contraction path module 1, which includes an input layer C1, a residual module D1, D2, D3, and D4, and an input layer C1 includes 64 convolution kernels of 7 × 7 with a step size of 2, thereby generating a feature map F1. The module D1 includes 1 maximum pooling layer with step size 2 and 3 tri-layer residual modules, and includes 64 1 × 1 convolution kernels with step size 1, 64 3 × 3 convolution kernels with step size 1, and 256 1 × 1 convolution kernels with step size 1, respectively, to generate the feature map F2. Module D2 contains 4 tri-layer residual modules, the 1 st residual module containing 128 1 × 1 convolution kernels with step size 1, 128 3 × 3 convolution kernels with step size 2, 512 1 × 1 convolution kernels with step size 1; the remaining 3 residual error modules respectively include 128 1 × 1 convolution kernels with step size 1, 128 3 × 3 convolution kernels with step size 1, and 512 1 × 1 convolution kernels with step size 1, and generate a feature map F3. Module D3 contains 6 tri-layer residual modules, the 1 st residual module containing 256 step size 1 × 1 convolution kernels, 256 step size 2 3 × 3 convolution kernels, 1024 step size 1 × 1 convolution kernels; the remaining 5 residual error modules respectively include 256 1 × 1 convolution kernels with step size 1, 256 3 × 3 convolution kernels with step size 1, and 1024 1 × 1 convolution kernels with step size 1, and generate a feature map F4. Module D4 contains 3 tri-level residual modules, the 1 st residual module containing 512 1 × 1 convolution kernels with step size 1, 512 3 × 3 convolution kernels with step size 2, 2048 1 × 1 convolution kernels with step size 1; the remaining 2 residual error modules respectively include 512 1 × 1 convolution kernels with step size 1, 512 3 × 3 convolution kernels with step size 1, and 2048 1 × 1 convolution kernels with step size 1, and generate a feature map F5.

In this embodiment, the feature map F5 passing through the contraction path module is down-sampled to 1/16 of the original image, and then input to the expansion path module 2 for up-sampling. F5 is first input into block U1, which contains 1 convolutional layer containing 192 convolutional kernels of step size 1 and 1 transposed convolutional layer containing 128 4 × 4 convolutional kernels of step size 2. The modules U2, U3, and U4 also include 1 convolutional layer and 1 transposed convolutional layer, the convolutional layers respectively include 128, 96, and 48 convolution kernels with a step size of 1, the transposed convolutional layers respectively include 96, 64, and 32 convolution kernels with a step size of 2, and finally the feature map output by U4 is restored to the resolution of the original image, and is output to the dimensionality reduction module 5 to be subjected to dimensionality reduction by 2 convolution kernels with a step size of 1 × 1 in the C2 layer. Feature maps F1, F2, F3, F4 and F5 of 5 scales generated by a contraction path module are cut and copied through the jump connection layer 3, each scale is fused with a corresponding feature in the expansion path module 2, and the functions of restoring a dangerous source detail feature and refining an edge are achieved. Therefore, the F1 and the F2 are input into the texture enhancement module 4 for texture detail enhancement before entering the jump connection layer. The dimension reduction module 5 comprises C2, C3 and C4 which respectively comprise 21 x 1 convolution kernels with the step length of 1 and are used for reducing dimensions of feature graphs of different scales, further generating prediction results of different scales and calculating multi-scale Lov a sz-Softmax loss, and meanwhile obtaining a final prediction result.

In this embodiment, optionally, the operating environment is based on an intel (r) xeon (r) Gold 5218CPU and a GeForce RTX 2080Ti GPU, the operating system is ubuntu18.04, the deep learning framework used for model construction is Pytorch, and the programming language is Python.

In this embodiment, in the model training phase, all training data are input into the TE-ResUnet model in batches, the loss between the output result of the TE-ResUnet model and the tag is continuously calculated iteratively, and the network model parameters are adjusted by using back propagation. Because the important characteristics of the dangerous source data set are the sparsity and the category imbalance of the dangerous source samples, the characteristics ensure that the model cannot better focus on pixels with smaller areas and easy confusion in the optimization process, and better balance the influence of the loss of the target and background pixels on the optimization process. When the classical cross entropy loss function is used for the problem, the model can be seriously biased to the background with larger pixel ratio, and the effect is poor. Lov-sz-Softmax loss is a measure based on a metric, and such a measure based on a metric loss function focuses on a single pixel unlike cross entropy loss, but focuses more on a metric index of the whole image, i.e., without considering the problem of sample distribution balance, which is more effective for the classification problem that the proportion of foreground pixels is far smaller than that of the background. In order to better supervise two texture enhancement modules, multi-scale Lov-sz-Softmax loss is used, loss function calculation is carried out on the basis of characteristic graphs of three scales, besides the final prediction result and the loss of a label are calculated, dimension reduction is also carried out on an expanded path F8, Lov-sz-Softmax loss calculation is also carried out on the label after down-sampling, Lov-sz-Softmax loss calculation is carried out on the label after the output of U4 is subjected to dimension reduction, finally the loss of the three parts is weighted and taken as the total target loss, the TE-ResUnet model is optimized according to the target loss, and training is finished when the target loss is smaller than a preset threshold value, so that the optimal TE-UResnet model is obtained. Therefore, the TE-ResUnet is optimized by using the multi-scale Lov-sz-Softmax loss function, so that the small target identification capacity and the stability of the model are improved, and the optimal model capable of directly generating the hazard source identification result is obtained finally; and carrying out hazard source identification on the remote sensing image to be identified by utilizing the trained optimal model. The method can realize accurate and effective identification of the dangerous source of the high-speed railway, can ensure higher identification precision for the dangerous source with smaller area, and solves the problem of poor effectiveness of the traditional ground investigation method.

As shown in fig. 3, the present invention also provides a railway hazard source identification apparatus, comprising:

the acquisition module 1 is used for acquiring remote sensing images along the railway;

the processing module 2 is used for inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows:

In the embodiment, in the training process of the danger source identification model, firstly, a danger source identification data set of the high-resolution remote sensing image is obtained, and a danger source identification data set along the railway is manufactured, wherein the data set comprises a plurality of remote sensing images with danger source labels. And then, taking the UNet model as a basic framework, using a residual error module as a contraction path, introducing a texture enhancement module, and establishing a TE-ResUnet model. And training and verifying the TE-ResUnet model by using a dangerous source identification data set along the railway, and completing end-to-end training by calculating the multiscale Lov & ltsb & gt-Softmax loss between a prediction result and a label and utilizing reverse propagation to obtain the optimal TE-ResUnet model. And finally, inputting the high-resolution remote sensing image to be tested into the optimal TE-ResUnet model to obtain a railway hazard source identification result of the high-resolution remote sensing image.

The railway hazard source identification device provided by the invention comprises the steps of firstly obtaining remote sensing images along a railway, and then inputting the remote sensing images into a hazard source identification model to obtain a railway hazard source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model. Therefore, the high-resolution remote sensing image railway hazard source identification TE-ResUnet model is established, on one hand, texture details are enhanced on low-level features through the texture enhancement module, so that the hazard source identification precision, particularly the hazard source boundary identification precision, is improved, on the other hand, the model is optimized by utilizing a multi-scale Lov a sz-Softmax loss function, so that the small target identification capacity is improved, and meanwhile, the stability of the model is also improved.

Based on the content of the foregoing embodiment, in this embodiment, the processing module is further specifically configured to:

Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. Processor 410 may invoke logic instructions in memory 430 to perform a railway hazard source identification method comprising: obtaining remote sensing images along the railway; inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov & sz-Softmax loss between the risk source prediction identification results of the plurality of different scales and the risk source label, completing end-to-end training by utilizing reverse propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model.

In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, a computer is capable of executing the railway hazard source identification method provided by the above methods, the method comprising: obtaining remote sensing images along the railway; inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for identifying a railroad hazard provided by the above methods, the method comprising: obtaining remote sensing images along the railway; inputting the remote sensing image into a dangerous source identification model to obtain a railway dangerous source identification result; the training process of the danger source identification model is as follows: step 1: obtaining a sample remote sensing image along a railway, and carrying out pixel-by-pixel labeling on a dangerous source in the sample remote sensing image to generate a dangerous source label; step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each characteristic diagram output by the contraction path module through a jump connection layer, and performing up-sampling processing on each characteristic diagram to obtain a plurality of processed characteristic diagrams; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales; and step 3: and calculating the multi-scale Lov-sz-Softmax loss between the risk source prediction identification results of the multiple different scales and the risk source label, completing end-to-end training by utilizing backward propagation to obtain an optimal TE-ResUnet model, and taking the optimal TE-ResUnet model as the risk source identification model.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on the understanding, the above technical solutions substantially or otherwise contributing to the prior art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying a source of railway hazards, comprising:

obtaining remote sensing images along the railway;

step 2: establishing a TE-ResUnet model, wherein the TE-ResUnet model comprises a contraction path module, an expansion path module, a texture enhancement module and a dimension reduction module; the contraction path module is used for carrying out feature recognition and down-sampling on the sample remote sensing image to generate a plurality of feature maps from a shallow scale to a deep scale; the expansion path module is used for cutting and copying each feature graph output by the contraction path module through a jump connection layer, and performing up-sampling processing on each feature graph to obtain a plurality of processed feature graphs; the texture enhancement module is used for performing texture enhancement processing on the two feature maps with the shallower scale output by the contraction path module before the expansion path module cuts and copies the two feature maps with the shallower scale output by the contraction path module through a jump connection layer and performs upsampling processing on the two feature maps with the shallower scale output by the contraction path module; wherein the two feature maps of the shallower scale comprise a first feature map and a second feature map, the scale of the first feature map being smaller than the scale of the second feature map; the dimensionality reduction module is used for performing dimensionality reduction processing on the two feature maps with the shallow scales cut and copied by the extended path module and performing dimensionality reduction processing on the second feature map subjected to sampling processing on the extended path module to obtain a plurality of dangerous source prediction identification results with different scales;

2. The railway hazard source identification method according to claim 1, wherein the texture enhancement module is configured to perform texture enhancement processing on the two feature maps at the shallower scale before the extended path module performs texture clipping to duplicate the two feature maps at the shallower scale output by the contracted path module through a skip connection layer and performs upsampling processing on the two feature maps at the shallower scale output by the contracted path module, and includes:

3. The railway hazard identification method of claim 1, wherein the calculating of the multi-scale Lov-sz-Softmax loss between the plurality of different-scale hazard source predictive identification results and the hazard source label, and performing end-to-end training using back propagation to obtain an optimal TE-ResUnet model comprises:

4. The railway hazard source identification method of claim 3, wherein the target loss L _Total Comprises the following steps:

the risk source prediction identification result output for the TE-ResUnet,

5. The method for identifying a railway hazard source of claim 1, further comprising, before inputting the remote sensing image into a hazard source identification model:

6. A railway hazard source identification device, comprising:

acquiring a remote sensing image of a sample along a railway, and marking a danger source in the remote sensing image pixel by pixel to generate a danger source label;

7. The railway hazard source identification device of claim 6, wherein the processing module is further specifically configured to:

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of the railway hazard source identification method of any one of claims 1 to 5.

9. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the railway hazard source identification method according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the steps of the railway hazard source identification method of any one of claims 1 to 5.