CN115546629A - Remote sensing image workshop identification method and system based on deep learning - Google Patents
Remote sensing image workshop identification method and system based on deep learning Download PDFInfo
- Publication number
- CN115546629A CN115546629A CN202211109160.0A CN202211109160A CN115546629A CN 115546629 A CN115546629 A CN 115546629A CN 202211109160 A CN202211109160 A CN 202211109160A CN 115546629 A CN115546629 A CN 115546629A
- Authority
- CN
- China
- Prior art keywords
- semantic segmentation
- sensing image
- remote sensing
- module
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013135 deep learning Methods 0.000 title claims abstract description 27
- 230000011218 segmentation Effects 0.000 claims abstract description 73
- 238000005520 cutting process Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 28
- 238000009826 distribution Methods 0.000 claims abstract description 12
- 238000000605 extraction Methods 0.000 claims abstract description 8
- 238000012216 screening Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 23
- 238000003860 storage Methods 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 150000001875 compounds Chemical class 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 239000006185 dispersion Substances 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000012423 maintenance Methods 0.000 abstract description 2
- 230000005540 biological transmission Effects 0.000 description 23
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013526 transfer learning Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002087 whitening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/16—Image acquisition using multiple overlapping images; Image stitching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
A remote sensing image factory building identification method and system based on deep learning are disclosed, the method comprises the following steps: step 1, collecting remote sensing image data, screening out a region with a factory building, and cutting out a series of sub-images as original samples of the factory building; step 2, marking and cutting original samples of the factory building to form a semantic segmentation data set, and expanding the semantic segmentation data set; step 3, building a semantic segmentation network model; step 4, training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set; step 5, extracting a workshop by using the trained semantic segmentation network model; and 6, splicing the prediction images of all the sub-images according to the coordinates of the initial points during cutting to obtain a plant semantic segmentation result image of the complete remote sensing image, and determining whether a plant exists according to the pixel distribution of the plant semantic segmentation result image. The accurate extraction of the factory building is realized, the geographic coordinates are obtained, and the risk level of the factory building can be evaluated according to the operation and maintenance route.
Description
Technical Field
The invention relates to the field of power transmission, in particular to a remote sensing image workshop identification method and system based on deep learning.
Background
Electricity is the most basic energy source in modern society, and plays a great role in both resident life and industrial production. In order to ensure stable and safe power supply, the method is particularly important for monitoring the running state of the high-voltage power transmission tower, regularly inspecting the power transmission line and inspecting the dangerous source. For the power grid form with long power transmission distance and severe equipment environment, the manual inspection monitoring efficiency is low and the danger is high. Compared with the prior art, the satellite remote sensing observation technology has the advantages of wide observation range, dynamic and timely state, no influence of natural conditions and obvious advantages.
Some existing transmission line detection technologies based on remote sensing images have high requirements on data, or are too simple and rough, and have low robustness. If the MIT Lincoln laboratory detects the electric tower based on whitening filtering on the SAR image with high resolution, sarabandi et al, michigan university, utilizes the SAR image with millimeter polarization to extract the transmission line. In addition, optical-based power transmission line detection is also developed to a certain extent, for example, the peak characteristics of the wire in the Cluster Randon (CR) frequency domain space are constructed, and the power transmission wire is extracted from the visible light remote sensing image; and the electric tower detection and the like are realized by utilizing a target detection technology of deep learning. There is no targeted method for detecting dangerous sources distributed along the transmission line.
The power transmission line danger source-plant identification is different from other target objects, has the characteristics of small sample size, easy confusion and the like, but has obvious characteristics, such as the plants are generally positioned in suburbs, are distributed in pieces, have flat upper surfaces, regular shapes and special colors, and can be extracted by adopting a deep learning method. According to the external characteristics of the factory building, the semantic segmentation method is selected for factory building identification, but overfitting and poor generalization are easily caused by directly using the temporarily marked small sample training model. If a fine tuning method is adopted, generally, the selected pre-training models are trained from the natural image data set, and due to the difference between the natural image and the remote sensing image, the model trained by using the natural image data set has poor fine tuning effect on the remote sensing image data set.
Disclosure of Invention
In order to solve the defects in the prior art, the invention aims to provide a remote sensing image workshop identification method and system based on deep learning.
The invention adopts the following technical scheme.
A remote sensing image factory building identification method based on deep learning comprises the following steps:
step 1, collecting remote sensing image data, screening out a region with a factory building, and cutting out a series of sub-images as original samples of the factory building;
step 2, marking and cutting original samples of the factory building to form a semantic segmentation data set, and expanding the semantic segmentation data set;
step 3, building a semantic segmentation network model;
step 4, training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set;
step 5, extracting a workshop by using the trained semantic segmentation network model;
and 6, splicing the prediction images of all the sub-images according to the coordinates of the initial points during cutting to obtain a plant semantic segmentation result image of the complete remote sensing image, and determining whether a plant exists according to the pixel distribution of the plant semantic segmentation result image.
Preferably, in the step 2, the original samples of the factory building are cut by a sliding window method, starting from the upper left corner, for the unmarked subgraphs, the subgraphs are selectively reserved as negative samples according to a set step length, and the rest are discarded, so that the number of the negative samples is controlled; and for the sub-images with the labels, randomly translating for multiple times around the label targets to generate a plurality of different sub-images as positive samples, expanding the positive samples, sequentially traversing the complete images to form a semantic segmentation data set, wherein the random translation times around the label targets are determined according to the distribution density of the factory buildings in the images.
In step 2, the expanding the semantic segmentation data set comprises: rotation, flipping, saturation, brightness, aspect warping, scaling, translation, and cropping enhancement operations.
Preferably, in step 3, the semantic segmentation network model includes: the system comprises a feature backbone network module, an output layer pixel point classification module and an objective function module.
Preferably, the feature backbone network module is a U-shaped symmetric network which performs down-sampling and up-sampling, fills the middle, extracts image abstract features of different levels from the whole network, integrates the image abstract features of different levels in a feature stacking manner, ensures that edge information is not lost in the down-sampling and up-sampling processes, and simultaneously directly connects all layers at the top of the network to an output layer to realize deep supervision.
Preferably, the output layer pixel point classification module uses a sigmoid function, the input of the sigmoid function is the output of the basic network, the output layer is used for mapping the value of the basic network output layer with a dispersion value to (0,1), then a threshold value is set, if the value is greater than the threshold value, the foreground is determined and the pixel is set to be 1, and if the value is greater than the threshold value, the other pixels are set to be 0, the network outputs a final segmentation result graph, and the calculation formula is as follows:
in the formula (I), the compound is shown in the specification,
x represents the input of the activation function, the output matrix of the backbone network module.
Preferably, using Dice Loss in combination with BCE Loss, the calculation formula is as follows:
in the formula (I), the compound is shown in the specification,
a denotes a network output layer and a network output layer,
b denotes a mask generated by the label,
y i representing a predicted value corresponding to each pixel point of the input image in the output layer,
ω i representing a weight parameter.
The utility model provides a remote sensing image factory building identification system based on deep learning, includes: an acquisition module, a data preprocessing module, a model building module, a model training module, an extraction module and a judgment module, wherein,
the acquisition module is used for acquiring remote sensing image data, screening out an area with a factory building, and cutting out a series of sub-images as original samples of the factory building;
the data preprocessing module is used for labeling and cutting original samples of the factory building to form a semantic segmentation data set and expanding the semantic segmentation data set;
the model building module is used for building a semantic segmentation network model;
the model training module is used for training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set;
the extraction module is used for extracting the plant by using the trained semantic segmentation network model;
the judgment module is used for splicing the prediction images of all the sub-images according to the initial point coordinates during cutting to obtain a plant semantic segmentation result image of the complete remote sensing image, and whether a plant exists is determined according to pixel distribution of the plant semantic segmentation result image.
A terminal comprising a processor and a storage medium; the storage medium is used for storing instructions;
the processor is used for operating according to the instruction to execute the steps of the remote sensing image plant identification method based on deep learning.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method for remote-sensing image plant identification based on deep learning.
The invention has the advantages that compared with the prior art,
(1) The method adopts transfer learning, and simultaneously uses various enhancing means to enhance data, thereby effectively solving the problem of small sample size of the plant target in the remote sensing image;
(2) According to the method, a new sample sliding window cutting strategy is adopted, the number of negative samples is controlled, the number of positive samples is increased, and the problem of unbalance of the positive and negative samples is effectively solved from the perspective of a data set;
(3) According to the method, a multi-scale feature fusion and deep supervision scheme is used, deep network output of local features such as attention details and the like is fused with shallow network output of global features, accurate segmentation of plant targets is achieved, the positions of plants are accurately determined, and the problems that foreground targets are easy to be confused with other backgrounds in remote sensing image scenes with limited resolution and the like are solved.
(4) The invention also provides a strategy for determining the risk region division and the risk level based on the transmission line hazard source (such as a factory building), scientifically inspects the transmission line hazard source, and provides powerful guarantee for improving the safety of power transmission and power utilization.
Drawings
FIG. 1 is a flow chart of a method for identifying a remote sensing image plant based on deep learning according to the present invention;
FIG. 2 is a diagram of a semantic segmentation network architecture;
FIG. 3 illustrates a process of training a semantic segmentation network model;
fig. 4 shows an example of extraction of a plant.
Detailed Description
The present application is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present application is not limited thereby.
Example 1.
The remote sensing image plant identification method based on deep learning comprises the following steps:
step 1, collecting remote sensing image data, screening out an area with a factory building, and cutting out a series of sub-images as original samples of the factory building.
Remote sensing image data is collected from an open source remote sensing data platform (such as water via injection), images are previewed in remote sensing processing software (such as PIE-Map), areas with plants are screened out, and a series of sub-images are cut out by using a cutting tool carried by the software to serve as original samples of the plants.
And 2, marking and cutting the original samples of the factory building to form a semantic segmentation data set, and expanding the semantic segmentation data set.
In this embodiment, preferably, the original sample is labeled by using labeling software (e.g., label me) or remote sensing image processing software (e.g., arcGis), and each image correspondingly generates a mask with the factory building as the foreground. Then, a proper size (for example 1024 × 1024) is set according to the plant distribution size and the hardware condition, and the original samples with different sizes and the corresponding masks are cut into subgraphs with uniform sizes according to a certain overlapping rate. When in cutting, a sliding window method is adopted, the subgraph without labels is selectively reserved as a negative sample according to a set step length (for example, the step length is set to be 2, namely the sliding window is stored for two times), and the rest is discarded, so that the quantity of the negative samples is controlled; for the subgraphs with labels, randomly translating for multiple times around the label target (determined according to the distribution density of the plant in the image) to generate multiple different subgraphs as positive samples, expanding the positive samples, and traversing the complete image in sequence to form a semantic segmentation data set.
And (3) performing enhancement operations such as rotation, overturning, saturation, brightness, length and width distortion, scaling, translation, cutting and the like on the data set obtained in the step (1), expanding the data set, improving the diversity of the sample and being beneficial to improving the generalization capability of the training model.
And 3, building a semantic segmentation network model, wherein the semantic segmentation network structure mainly comprises a feature backbone network, a pixel point classification module and an objective function module.
The backbone network module mainly comprises a down-sampling link and an up-sampling link, wherein the down-sampling link is equivalent to encoding, the feature of an input image is abstractly expressed, the up-sampling link is equivalent to decoding, and the abstracted feature is restored and decoded to the size of an original image.
The basic network of the module consists of an input layer, a convolutional layer, a long connecting layer and an output layer, the network corresponds to the improvement of the feature level and the increase of the receptive field from shallow to deep, all shallow features are connected with deep features of the same dimensionality through long connection, the feature layer stacking is adopted to integrate the shallow features and the deep features, the global features and the local features of the shallow features are ensured to be concerned by the network and are finally carried to the output layer, and the width and the height of the output layer are consistent with those of an input image.
In the embodiment, a U-shaped symmetrical network which performs downsampling and then upsampling is preferably selected, the middle is filled, the whole network captures features of different layers, and the features are integrated in a feature stacking mode, so that edge information is ensured not to be lost in the downsampling and upsampling processes. In addition, all layers (the width and height of the feature diagram are the same as those of the output layer) on the top of the network are directly connected to the output layer, so that deep supervision is realized, and the structure is shown in fig. 2.
The output layer pixel point classification module uses a sigmoid function, the input of the sigmoid function is the output of a basic network, the output layer is used for mapping the value of the basic network output layer with a dispersion value to (0,1), then a threshold value (such as 0.5) is set, the threshold value is larger than the threshold value, the foreground is judged and the pixel is set to be 1, the other is the background and the pixel is set to be 0, and the network outputs the final segmentation result image. Calculating the formula:
x represents the input of the activation function and the output matrix of the backbone network module;
sigmoid (x) denotes the activation function name.
The objective function is used for calculating the deviation between the prediction output of the basic network and the marking mask in the process of training the model, and performing gradient descent on the objective function while continuously updating network parameters during reverse transmission so as to achieve the purpose of training.
This embodiment preferably uses Dice Loss in combination with BCE Loss. Calculating the formula:
in the formula:
a denotes a network output layer and a network output layer,
b denotes the mask generated by the label.
Calculating the formula:
in the formula:
y i for the predicted value in the output layer corresponding to each pixel point of the input image,
ω i the weight parameters are self-learning during training.
And 4, training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set.
Preferably, in this embodiment, the basic model is selected from a model obtained by training an existing remote sensing image semantic segmentation data set, such as an open source semantic segmentation data set of a building, a road, a water body, and the like.
As shown in fig. 3, the training process: using a factory building data set to perform transfer learning, initializing a network by using trained basic model weight parameters, dividing the data set into a training set and a verification set, performing one-time training and verification each time of iteration, feeding sample images in the training set into the network for completing parameter initialization in batches in a training stage, obtaining an output layer matrix after forward propagation of all layers is completed, then calculating a target function value, namely loss of the batch, and then executing gradient descent in backward propagation to update network parameters until the network converges; in the verification stage, only forward propagation is performed, and evaluation indexes such as the intersection ratio of a network output layer and an image labeling mask are calculated. And (5) carrying out the next round after each round of training and verification is finished until the network convergence and the loss are not obviously reduced any more, and finishing the training.
And 5, extracting a workshop by using the trained semantic segmentation network model, cutting the remote sensing image to be processed into subgraphs with preset sizes through a sliding window, feeding the subgraphs into the trained model for prediction, and outputting a result, wherein a workshop extraction example is shown in fig. 4.
And 6, splicing the prediction images of all the sub-images according to the coordinates of the initial points during cutting to obtain a plant semantic segmentation result of the complete remote sensing image, and finally determining whether a plant exists and the distribution position of the plant according to the pixel distribution of the result image.
And determining the risk level, namely defining a risk area at the periphery of the operation and maintenance line set along the transmission line according to the distance, and sequentially arranging a key area, a secondary key area and a general area from near to far. The factory building is used as one of the power transmission line danger sources, the grade needs to be determined according to the risk area to which the factory building position belongs, and the factory buildings located in the key area, the secondary key area and the general area respectively correspond to the first-grade risk grade, the second-grade risk grade and the third-grade risk grade.
Example 2.
Remote sensing image factory building identification system based on degree of deep learning includes: the collection module, data preprocessing module, the module is built to the model, and model training module draws the module, judges the module, wherein:
the acquisition module is used for acquiring remote sensing image data, screening out a region with a factory building, and cutting out a series of sub-images as original samples of the factory building;
the data preprocessing module is used for labeling and cutting original samples of the factory building to form a semantic segmentation data set and expanding the semantic segmentation data set;
the model building module is used for building a semantic segmentation network model;
the model training module is used for training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set;
the extraction module is used for extracting the plant by using the trained semantic segmentation network model;
the judgment module is used for splicing the prediction images of all the sub-images according to the initial point coordinates during cutting to obtain a plant semantic segmentation result image of the complete remote sensing image, and whether a plant exists is determined according to pixel distribution of the plant semantic segmentation result image.
Example 3.
The third embodiment of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a program is stored, where the program, when executed by a processor, implements the steps in the method for identifying a remote sensing image factory building based on deep learning according to the first embodiment of the present invention.
The detailed steps are the same as those of the remote sensing image factory building identification method based on deep learning provided in embodiment 1, and are not repeated herein.
Example 4.
Embodiment 4 of the present invention provides an electronic device.
An electronic device includes a memory, a processor, and a program stored in the memory and capable of running on the processor, where the processor executes the program to implement the steps in the remote sensing image plant identification method based on deep learning according to an embodiment of the present invention.
The invention has the advantages that compared with the prior art,
(1) The method adopts transfer learning, and simultaneously uses various enhancing means to enhance data, thereby effectively solving the problem of small sample size of the plant target in the remote sensing image;
(2) According to the method, a new sample sliding window cutting strategy is adopted, the number of negative samples is controlled, the number of positive samples is increased, and the problem of imbalance of the positive samples and the negative samples is effectively solved from the perspective of a data set;
(3) According to the method, a multi-scale feature fusion and deep supervision scheme is used, deep network output of local features such as attention details and the like is fused with shallow network output of global features, accurate segmentation of plant targets is achieved, the positions of plants are accurately determined, and the problems that foreground targets are easy to be confused with other backgrounds in remote sensing image scenes with limited resolution and the like are solved.
(4) The invention also provides a strategy for determining the risk region division and the risk level based on the transmission line hazard source (such as a factory building), scientifically inspects the transmission line hazard source, and provides powerful guarantee for improving the safety of power transmission and power utilization.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (10)
1. A remote sensing image factory building identification method based on deep learning is characterized by comprising the following steps:
step 1, collecting remote sensing image data, screening out an area with a factory building, and cutting out a series of sub-images as original samples of the factory building;
step 2, marking and cutting original samples of the factory building to form a semantic segmentation data set, and expanding the semantic segmentation data set;
step 3, building a semantic segmentation network model;
step 4, training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set;
step 5, extracting a workshop by using the trained semantic segmentation network model;
and 6, splicing the prediction graphs of all the subgraphs according to the initial point coordinates during cutting to obtain a workshop semantic segmentation result graph of the complete remote sensing image, and determining whether a workshop exists according to the pixel distribution of the workshop semantic segmentation result graph.
2. The remote-sensing image plant identification method based on deep learning of claim 1,
in step 2, cutting the original samples of the factory building by adopting a sliding window method, starting from the upper left corner, selectively reserving the unmarked subgraphs as negative samples according to a set step length, and discarding the rest subgraphs so as to control the number of the negative samples; and for the sub-images with the labels, randomly translating for multiple times around the label targets to generate a plurality of different sub-images as positive samples, expanding the positive samples, sequentially traversing the complete images to form a semantic segmentation data set, wherein the random translation times around the label targets are determined according to the distribution density of the factory buildings in the images.
3. The remote-sensing image plant identification method based on deep learning of claim 1,
in step 2, the expanding the semantic segmentation data set comprises: rotation, flipping, saturation, brightness, length and width warping, scaling, translation, and cropping enhancement operations.
4. The remote-sensing image plant identification method based on deep learning of claim 1,
in step 3, the semantic segmentation network model comprises: the system comprises a feature backbone network module, an output layer pixel point classification module and an objective function module.
5. The remote-sensing image plant identification method based on deep learning of claim 4, wherein,
the feature backbone network module is a U-shaped symmetrical network which firstly performs down sampling and then performs up sampling, the middle is filled, the whole network extracts image abstract features of different levels, the image abstract features of different levels are integrated in a feature stacking mode, edge information is guaranteed not to be lost in the down sampling and up sampling processes, and meanwhile, all layers on the top of the network are directly connected to an output layer, so that deep supervision is achieved.
6. The method for identifying remote sensing image plants based on deep learning of claim 4,
the output layer pixel point classification module uses a sigmoid function, the input of the sigmoid function is the output of a basic network, the output layer is used for mapping the value of the basic network output layer with a dispersion value to (0,1), then a threshold value is set, if the value is larger than the threshold value, the foreground is judged and the pixel is set to be 1, if the value is larger than the threshold value, the other pixels are set to be 0, the network outputs a final segmentation result graph, and the calculation formula is as follows:
in the formula (I), the compound is shown in the specification,
x represents the input of the activation function, the output matrix of the backbone network module.
7. The remote-sensing image plant identification method based on deep learning of claim 1,
in step 3, the Dice Loss and the BCE Loss are combined, and the calculation formula is as follows:
in the formula (I), the compound is shown in the specification,
a denotes a network output layer and a network output layer,
b denotes a mask generated by the label,
y i representing a predicted value corresponding to each pixel point of the input image in the output layer,
ω i representing a weight parameter.
8. A remote sensing image plant identification system based on deep learning by using the method of any one of claims 1 to 7, comprising: an acquisition module, a data preprocessing module, a model building module, a model training module, an extraction module and a judgment module, which is characterized in that,
the acquisition module is used for acquiring remote sensing image data, screening out an area with a factory building, and cutting out a series of sub-images as original samples of the factory building;
the data preprocessing module is used for labeling and cutting original samples of the factory building to form a semantic segmentation data set and expanding the semantic segmentation data set;
the model building module is used for building a semantic segmentation network model;
the model training module is used for training a semantic segmentation network model by using the existing remote sensing image semantic segmentation data set;
the extraction module is used for extracting the plant by using the trained semantic segmentation network model;
the judgment module is used for splicing the prediction images of all the sub-images according to the initial point coordinates during cutting to obtain a plant semantic segmentation result image of the complete remote sensing image, and whether a plant exists is determined according to pixel distribution of the plant semantic segmentation result image.
9. A terminal comprising a processor and a storage medium; the method is characterized in that:
the storage medium is used for storing instructions;
the processor is used for operating according to the instruction to execute the steps of the remote sensing image plant identification method based on deep learning according to any one of claims 1-8.
10. Computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the steps of a method for identifying remote sensing image plants based on deep learning according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211109160.0A CN115546629A (en) | 2022-09-13 | 2022-09-13 | Remote sensing image workshop identification method and system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211109160.0A CN115546629A (en) | 2022-09-13 | 2022-09-13 | Remote sensing image workshop identification method and system based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115546629A true CN115546629A (en) | 2022-12-30 |
Family
ID=84725299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211109160.0A Pending CN115546629A (en) | 2022-09-13 | 2022-09-13 | Remote sensing image workshop identification method and system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115546629A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117475365A (en) * | 2023-12-26 | 2024-01-30 | 国网天津市电力公司高压分公司 | Power transmission channel risk identification method, system, equipment and storage medium |
-
2022
- 2022-09-13 CN CN202211109160.0A patent/CN115546629A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117475365A (en) * | 2023-12-26 | 2024-01-30 | 国网天津市电力公司高压分公司 | Power transmission channel risk identification method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596886A (en) | Aerial Images insulator based on deep learning falls piece fault rapid detecting method | |
| CN113780296A (en) | Remote sensing image semantic segmentation method and system based on multi-scale information fusion | |
| CN114943876A (en) | Cloud and cloud shadow detection method and device for multi-level semantic fusion and storage medium | |
| CN115546630A (en) | Construction site extraction method and system based on remote sensing image characteristic target detection | |
| CN110807376A (en) | Method and device for extracting urban road based on remote sensing image | |
| CN112990086A (en) | Remote sensing image building detection method and device and computer readable storage medium | |
| CN113537070B (en) | Detection method, detection device, electronic equipment and storage medium | |
| CN114898097B (en) | Image recognition method and system | |
| CN113947188A (en) | Training method of target detection network and vehicle detection method | |
| CN116110036B (en) | Electric power nameplate information defect level judging method and device based on machine vision | |
| Li et al. | Gated auxiliary edge detection task for road extraction with weight-balanced loss | |
| CN114187505A (en) | Detection method and device for falling-off of damper of power transmission line, medium and terminal equipment | |
| CN115546629A (en) | Remote sensing image workshop identification method and system based on deep learning | |
| CN115619719A (en) | Pine wood nematode infected wood detection method based on improved Yolo v3 network model | |
| CN116452850A (en) | Road ponding area identification method based on data mining and deep learning | |
| CN115546664A (en) | Cascaded network-based insulator self-explosion detection method and system | |
| CN113902793B (en) | Method, system and electronic equipment for predicting end-to-end building height based on single-vision remote sensing image | |
| CN118038153A (en) | Method, device, equipment and medium for identifying external damage prevention of distribution overhead line | |
| CN113536944A (en) | Distribution line inspection data identification and analysis method based on image identification | |
| CN117456356A (en) | Urban waterlogging video recognition early warning method based on deep learning | |
| CN117197571A (en) | Photovoltaic module fault detection method and device, electronic equipment and storage medium | |
| Gan et al. | Water level classification for flood monitoring system using convolutional neural network | |
| CN115937492A (en) | Transformer equipment infrared image identification method based on feature identification | |
| CN116071299A (en) | Insulator RTV spraying defect detection method and system | |
| CN109377483A (en) | Porcelain insulator crack detecting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |