CN109859209B - Remote sensing image segmentation method and device, storage medium and server - Google Patents
Remote sensing image segmentation method and device, storage medium and server Download PDFInfo
- Publication number
- CN109859209B CN109859209B CN201910015168.2A CN201910015168A CN109859209B CN 109859209 B CN109859209 B CN 109859209B CN 201910015168 A CN201910015168 A CN 201910015168A CN 109859209 B CN109859209 B CN 109859209B
- Authority
- CN
- China
- Prior art keywords
- pixel
- sub
- image
- neural network
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Analysis (AREA)
Abstract
The application relates to the technical fields of image detection and image classification, and provides a remote sensing image segmentation method, which comprises the following steps: acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels; traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images; and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image. According to the application, the object is more fully represented through the multi-scale supercolumn, the structural hierarchy of the convolutional neural network in the application is deeper, the parameters are reduced while effective information is ensured, the performance of the convolutional neural network can be improved through the continuously deepened network structure, the expression relationship is more complex through complex full connection, the precise splitting of pixels is further realized, and the precise splitting of remote sensing images is realized.
Description
Technical Field
The application relates to the technical fields of image detection and image classification, in particular to a remote sensing image segmentation method and device, a storage medium and a server.
Background
With the development of the data acquisition technology of the aerospace remote sensing sensor, people can acquire ultra-large-scale high-resolution remote sensing image data, such as QuickBird satellites, worldView-II satellites, GEOEye-I satellites and CBERS-2B satellites, in a very short time, and can acquire images of 373, 2708, 1145 and 120 megapixels (megapixels) in each minute. Moreover, more and more online systems require real-time processing of remote sensing image data, such as military target identification and terrain matching, weather forecast of weather, emergency disasters, and the like. The remote sensing image data is also used as basic data in a geographic information system (Geographic information System, GIS), a global positioning system (Global Positioning System, GPS) and a remote sensing mapping technology (remote sensing system, RS) three-space information technology, and is widely applied to various fields such as environment monitoring, resource investigation, land utilization, city planning, natural disaster analysis, military and the like. In recent years, with the development of high-resolution remote sensing satellite, imaging radar and unmanned aerial vehicle (Unmanned Aerial Vehicle) technologies, remote sensing image data further presents characteristics of mass, complexity and high resolution, and the realization of fine segmentation has important research significance and application value for promoting accurate extraction and data sharing of remote sensing image information.
Disclosure of Invention
In order to solve the above technical problems, particularly the problem that the existing remote sensing image cannot be accurately segmented due to inaccurate information extraction, the following technical scheme is specifically provided:
the remote sensing image segmentation method provided by the embodiment of the application comprises the following steps:
acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels;
traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images;
and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image.
Optionally, traversing pixels in all the sub-images based on the depth convolutional neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
determining a pixel classification model based on the deep convolutional neural network;
and traversing each pixel in the sub-image one by one through the pixel classification model, and determining a first pixel class of the sub-image.
Optionally, the determining the pixel classification model based on the depth convolutional neural network includes:
determining a second pixel class of the sub-image by the deep convolutional neural network;
determining a loss function of the pixel classification model based on a labeling image and a second pixel class of the sub-image, wherein the labeling image is the target image and the target image has semantic labeling;
determining the pixel classification model by the deep convolutional neural network and the loss function.
Optionally, the first pixel class and the second pixel class are center pixel classes of the sub-image.
Optionally, traversing pixels in all the sub-images based on the depth convolutional neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
extracting depth characteristic information of the sub-image through the depth convolution neural network;
constructing a multi-scale super-column of the same pixel in the sub-image according to the depth characteristic information, wherein the multi-scale super-column is a one-dimensional vector of the same pixel at different depths;
a first pixel class of the sub-image is determined based on the multi-scale super-column.
Optionally, the constructing the multiscale super-column of the same pixel in the sub-image according to the depth characteristic information includes:
convolving the sub-image through the deep convolutional neural network to determine a center pixel of the sub-image;
extracting all pixel characteristic points of the central pixel in a convolution layer and a pooling layer;
and constructing the multi-scale super-column according to the positions of all the pixel characteristic points in the convolution layer and the pooling layer.
Optionally, the labeling the target image according to the pixel class of the sub-image includes:
acquiring position information on the target image corresponding to each sub-image;
and labeling the target image according to the position information and the pixel type in turn.
The embodiment of the application also provides a remote sensing image segmentation device, which comprises:
the segmentation module is used for acquiring a target image and segmenting the target image into a plurality of sub-images of preset pixels;
the first pixel category determining module is used for traversing pixels in all the sub-images based on the depth convolution neural network and sequentially determining the first pixel category of each pixel in the sub-images;
and the labeling module is used for labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium is stored with a computer program, and the program realizes the remote sensing image segmentation method according to any technical scheme when being executed by a processor.
The embodiment of the application also provides a server, which comprises:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the steps of the remote sensing image segmentation method according to any of the claims.
Compared with the prior art, the application has the following beneficial effects:
1. the remote sensing image segmentation method provided by the embodiment of the application comprises the following steps: comprising the following steps: acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels; traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images; and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image. The convolutional neural network structure is complex, and consists of thirteen convolutional layers, a multiscale fused supercolumn layer, three full-connection layers and an output layer. A convolution kernel of 3*3 and a pooling kernel of 2 x 2 were all used. The expression relationship of the network is also more complex through complex full connection. The neural network comprises five sections of convolution layers, 2-3 continuous convolution layers are arranged in each section, as the convolution layers are deeper, the extracted characteristics gradually change from local to global, the continuous convolution layers can obtain more nonlinear transformation, the tail part comprises a maximum pooling layer to extract the maximum value in the area to represent the characteristics, the effective information is ensured, the parameters are reduced, and the performance of the convolutional neural network can be improved through a continuously deepened network structure. The convolutional neural network structure in the application has deeper hierarchy, reduces parameters while guaranteeing effective information, can be improved through the continuously deepened network structure, and has more complex expression relationship through complex full connection, so that accurate splitting of pixels is realized, and fine splitting of remote sensing images is realized.
2. The remote sensing image segmentation method provided by the embodiment of the application comprises the steps of traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel class of each pixel in the sub-images, wherein the first pixel class comprises the following steps: extracting depth characteristic information of the sub-image through the depth convolution neural network; constructing a multi-scale super-column of the same pixel in the sub-image according to the depth characteristic information, wherein the multi-scale super-column is a one-dimensional vector of the same pixel at different depths; on the basis of the multi-scale supercolumn determining that the first pixel class of the sub-image is based on a depth convolutional neural network, a multi-scale supercolumn is built from bottom to top by adopting self-learning multi-scale depth features according to an image pyramid structure, wherein the multi-scale supercolumn is a one-dimensional vector with the same pixel and different depths, the vector comprises the features of each layer of one pixel, and objects can be fully represented by predicting image blocks and extracting the features of each pixel layer in the sub-image, so that the classification of the sub-image pixel level is realized, and the fine segmentation of the remote sensing image is obtained.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic flow chart of an implementation of a remote sensing image segmentation method according to an exemplary embodiment of the present application;
FIG. 2 is a schematic diagram of a medium-depth convolutional neural network of the remote sensing image segmentation method of the present application;
FIG. 3 is a schematic diagram of a remote sensing image segmentation apparatus according to an exemplary embodiment of the present application;
fig. 4 is a schematic structural diagram of an embodiment of a server according to the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, but do not preclude the presence or addition of one or more other features, integers, steps, operations.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
It will be appreciated by those skilled in the art that references to "application," "application program," "application software," and similar concepts herein are intended to be equivalent concepts well known to those skilled in the art, and refer to computer software, organically constructed from a series of computer instructions and related data resources, suitable for electronic execution. Unless specifically specified, such naming is not limited by the type, level of programming language, nor by the operating system or platform on which it operates. Of course, such concepts are not limited by any form of terminal.
In one implementation manner of the remote sensing image segmentation method provided by the embodiment of the present application, as shown in fig. 1, the method includes: s100, S200, S300.
S100: acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels;
s200: traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images;
s300: and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image.
The remote sensing image segmentation method provided by the application is mainly used for realizing the segmentation of the remote sensing image based on the deep convolutional neural network. According to the method, an original target image is firstly obtained, the target image is divided into a plurality of sub-images with 32 pixels, and the sub-image pixel levels can be conveniently classified through a deep convolutional neural network. The target image is a special image shot by a satellite, an aircraft or the like. In the present application, the pixels of the sub-image are preferably 32×32, and the target image can be more clearly and finely divided at the pixel level of 32×32 in the pixel of the combined target image itself, and in other embodiments, the target image can be further divided into sub-images of 16×16, 64×64, and other pixels based on the pixel level of the original remote sensing image.
After obtaining the sub-images, traversing pixels in all the sub-images through the depth convolution neural network, namely traversing pixels in each sub-image one by one aiming at each sub-image input into the depth convolution neural network, and judging a first pixel type of each pixel in the sub-images. The first pixel class is determined primarily by a pixel classification model, wherein the pixel classification model is derived primarily by sub-image training. After the pixel classification model is obtained, the original sub-image is input into the pixel classification model, and the class of the central pixel in the sub-image, that is, the first pixel class, is determined by a deep convolutional neural network algorithm in the pixel classification model, and the detailed process is described in the following, and is not repeated here. After the center pixel class of the sub-image is obtained, the target image is segmented (namely marked) based on the position of the sub-image in the original target image, namely, objects with different semantic classes in the target image are covered by adopting different colors, and further the segmented remote sensing image is obtained. Optionally, the labeling the target image according to the pixel class of the sub-image includes: acquiring position information on the target image corresponding to each sub-image; and labeling the target image according to the position information and the pixel type in turn. Specifically, since each pixel has a certain position, the position of the sub-image in the target image can be determined based on the pixels in the sub-image, when the target image is segmented, the position information of the sub-image in the target image can be obtained based on the pixels, and since the pixel type of the sub-image is already determined, the target image is segmented according to the position information and the pixel type of the sub-image.
Optionally, traversing pixels in all the sub-images based on the depth convolutional neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
determining a pixel classification model based on the deep convolutional neural network;
and traversing each pixel in the sub-image one by one through the pixel classification model, and determining a first pixel class of the sub-image.
In connection with the foregoing, it will be appreciated that in the present application, a pixel classification model is first determined from a target image and a sub-image, and then a first pixel class of the sub-image is determined by traversing each pixel in the sub-image one by one through a pixel splitting model. Wherein the pixel classification model is determined by a self-learning method. The detailed process is as follows:
optionally, the determining the pixel classification model based on the depth convolutional neural network includes:
determining a second pixel class of the sub-image by the deep convolutional neural network;
determining a loss function of the pixel classification model based on a labeling image and a second pixel class of the sub-image, wherein the labeling image is the target image and the target image has semantic labeling;
determining the pixel classification model by the deep convolutional neural network and the loss function.
In the implementation process of the present application, as shown in fig. 2, the convolutional neural network of the present application includes: a-input, B-convolution layer (convlion+ReLU), C-max pooling layer (max-pooling) D-, full connection layer (full connection+ReLU), and E-softmax regression algorithm layer. In a specific implementation process, the convolutional neural network is composed of thirteen convolutional layers, a multiscale fused supercolumn layer, three full-connection layers and an output layer. A convolution kernel of 3*3 and a pooling kernel of 2 x 2 were all used. The method comprises five sections of convolution layers, wherein 2-3 continuous convolution layers are arranged in each section, as the convolution layers are deeper, the extracted features gradually change from local to global, the continuous convolution layers can obtain more nonlinear transformation, and the tail part comprises a maximum pooling layer to extract the maximum value in the area to represent the features, so that the parameters are reduced while effective information is ensured. The performance of the network is improved by the ever-increasing network architecture.
Optionally, traversing pixels in all the sub-images based on the depth convolutional neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
extracting depth characteristic information of the sub-image through the depth convolution neural network;
constructing a multi-scale super-column of the same pixel in the sub-image according to the depth characteristic information, wherein the multi-scale super-column is a one-dimensional vector of the same pixel at different depths;
a first pixel class of the sub-image is determined based on the multi-scale super-column.
In the pixel classification model generation step and the sub-image pixel classification process through the pixel classification model, a depth convolution neural network is adopted to extract multi-scale depth characteristic information in the sub-image, then a multi-scale super-column is constructed from bottom to top according to an image pyramid structure, and the multi-scale super-column forms a vector from the lower end to the top of the pyramid, wherein the vector contains the characteristics of each layer of pixels, namely the depth characteristic information. Therefore, in the present application, depth characteristic information of the sub-image, that is, pixels at different depth levels, is extracted through the depth convolutional neural network, and then a multi-scale super-column is constructed according to the pixels at different depth levels of the same pixel, where multi-scale super-column extraction and construction are implemented in a structure that is as shown in fig. 2 after input a-input and is in a column with input. In a subsequent process, the sub-image pixel class may be further determined by multi-scale super-column. Because the segmentation process is to traverse the pixels one by one to judge the category, the super column is a one-dimensional vector corresponding to one pixel, and the multi-scale super column is a one-dimensional vector of one pixel at different depths. After the one-dimensional vectors with different depths are obtained, the characteristics of the same pixel point under different levels can be extracted through the convolutional neural network, so that the object can be fully represented as described above. The multi-scale super-column based on the method can also more accurately determine the first pixel category in the sub-image through calculation of the convolutional neural network.
Optionally, the constructing the multiscale super-column of the same pixel in the sub-image according to the depth characteristic information includes:
convolving the sub-image through the deep convolutional neural network to determine a center pixel of the sub-image;
extracting all pixel characteristic points of the central pixel in a convolution layer and a pooling layer;
and constructing the multi-scale super-column according to the positions of all the pixel characteristic points in the convolution layer and the pooling layer.
By combining the above processes, the sub-images are convolved through the deep convolutional neural network, the central pixel of the sub-images is determined, and then after the convolution is finished, all the characteristic points corresponding to the central pixel of the input sub-images in the convolved layer and the pooling layer are extracted to form a new one-dimensional vector.
The final part is three full connection layers, the convolution result is further classified, and the more complex the full connection is, the more complex the expressed relationship is. And finally, performing two classifications on the pixel points in the sub-images by adopting a soft-max regression method, and further determining a second pixel classification of the sub-images.
The network parameters are shown in the following table:
specifically, the loss function in the pixel classification model is mainly determined by the second pixel class of the labeling image and the sub-image, and specifically, the labeling image is an original target image with a semantic class, such as a two-class labeling image, and the image contains two colors, wherein the colors can be set by a user in a self-defining way, one color covers a target object, one color covers a non-target, and the labeling is at a pixel level. The labeling of the sub-images is to divide the target image with semantic labeling, calculate the proportion of the target pixels in the sub-images to label the corresponding categories of the sub-images, and finally finely divide the target image according to the pixel categories of the sub-images. The sum of the cross entropy cost function and the regularization term is employed. Wherein the cross entropy function is as follows:
where y is the desired output, α is the actual output of the neural network, and x is the input value. The cross entropy loss function can measure the similarity between the actual output and the expected output of the neural network, and can avoid the problem of reduced learning rate of the mean square error loss function when the gradient of the cross entropy loss function is reduced. Meanwhile, the cross entropy is used as a logarithmic function, and the gradient value is still higher when approaching to the upper boundary, so that the convergence speed of the model is not slowed down.
The regular term is used for attenuating parameters of the deep convolutional neural network, generating sparsity and reducing feature vectors, so that complexity of a model is reduced, and the deep convolutional neural network is prevented from being over fitted in the training process. And (3) integrating the cross entropy cost function and the regularization term, wherein the loss function is as follows:
where λ is the coefficient of the regularization term, θ is the model parameter, and k is the number of parameters.
Given the model loss function, the Adam method is adopted to carry out iterative optimization of model parameters. The formula is as follows:
wherein t is the iteration number, learning_rate is the initial learning rate 0.001, lr t In order to adapt the learning rate after the adaptation,is the loss function C with respect to θ t Partial derivative D t Average value, m, over a training batch sample set t And n t First and second moment estimates for the gradient, respectively, < >>And->The first moment unbiased estimation and the second moment unbiased estimation of the gradient are respectively carried out, mu and v are respectively the attenuation speeds of the first moment estimation and the second moment estimation, the attenuation speeds are initialized to 0.9 and 0.999, and epsilon is a small constant 1e-8 with stable numerical value.
After the foregoing loss function determination, the values are input to a deep convolutional neural network to generate the pixel classification model in the present application.
Preferably, the first pixel class and the second pixel class are center pixel classes of the sub-image. Because the sub-image is divided into smaller pixels, in order to accurately identify each object type in the target image on the basis of finely dividing the target image, the target image is divided by taking the center pixel type of the sub-image as a reference.
The embodiment of the present application further provides a remote sensing image segmentation apparatus, in one implementation manner, as shown in fig. 3, the remote sensing image segmentation apparatus includes: the segmentation module 100, the first pixel class determination module 200, the labeling module 300:
the segmentation module 100 is configured to obtain a target image, and segment the target image into a plurality of sub-images with preset pixels;
a first pixel class determining module 200, configured to sequentially determine a first pixel class of each pixel in the sub-image based on traversing pixels in all the sub-images by the deep convolutional neural network;
the labeling module 300 is configured to label the target image according to the first pixel class of the sub-image, and obtain a segmented target image.
Further, as shown in fig. 3, the apparatus for remote sensing image segmentation method provided in the embodiment of the present application further includes: a first pixel classification model determining unit 210 for determining a pixel classification model based on the deep convolutional neural network; a traversing unit 220, configured to traverse each pixel in the sub-image one by one through the pixel classification model, and determine a first pixel class of the sub-image. A second pixel class determining unit 211 for determining a second pixel class of the sub-image through the deep convolutional neural network; a loss function determining unit 212, configured to determine a loss function of the pixel classification model based on a labeling image and a second pixel class of the sub-image, where the labeling image is the target image, and the target image has a semantic label; a second pixel classification model determination unit 213 for determining the pixel classification model by means of the deep convolutional neural network and the loss function. A depth feature information extraction unit 230 for extracting depth feature information of the sub-image through the depth convolutional neural network; a one-dimensional vector construction unit 240, configured to construct a multi-scale super-column of the same pixel in the sub-image according to the depth feature information, where the multi-scale super-column is a one-dimensional vector of the same pixel at different depths; a first pixel class determination unit 250 for determining a first pixel class of the sub-image based on the multi-scale super-column. A central pixel determining unit 241, configured to determine a central pixel of the sub-image by convolving the sub-image with the deep convolutional neural network; a pixel feature point extracting unit 242, configured to extract all pixel feature points of the central pixel in the convolution layer and the pooling layer; a multi-scale super-column construction unit 243, configured to construct the multi-scale super-column according to the positions of all the pixel feature points in the convolution layer and the pooling layer. A position information obtaining unit 310, configured to obtain position information on the target image corresponding to each of the sub-images; and the labeling unit 320 is configured to label the target image sequentially according to the position information and the pixel type.
The remote sensing image segmentation method device provided by the embodiment of the application can realize the embodiment of the remote sensing image segmentation method, and specific function realization is shown in the description of the embodiment of the method and is not repeated herein.
The embodiment of the application provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the remote sensing image segmentation method according to any one of the technical schemes. The computer readable storage medium includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random AcceSS Memory, random access memories), EPROMs (EraSable Programmable Read-Only memories), EEPROMs (Electrically EraSable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a storage device includes any medium that stores or transmits information in a form readable by a device (e.g., computer, cell phone), and may be read-only memory, magnetic or optical disk, etc.
According to the embodiment of the application, on the basis of dividing a remote sensing image to obtain a plurality of sub-images, the deep convolutional neural network is firstly trained in a self-learning manner, and a pixel classification model based on the deep convolutional neural network is determined, so that sub-images can be classified again through the model, and the remote sensing image can be divided based on the pixel types of the sub-images. In the self-learning training of the application, the one-dimensional vectors with different depths of the same pixel are constructed by adopting the multi-scale depth features, so that the different features with different depths of the same pixel can be obtained, the full representation of the object is realized, and the target image can be segmented more accurately; the remote sensing image segmentation method provided by the embodiment of the application comprises the following steps: acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels; traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images; and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image. The remote sensing image segmentation method provided by the application is mainly used for realizing the segmentation of the remote sensing image based on the deep convolutional neural network. According to the method, an original target image is firstly obtained, the target image is divided into a plurality of sub-images with 32 pixels, and the sub-image pixel levels can be conveniently classified through a deep convolutional neural network. The target image is a special image shot by a satellite, an aircraft or the like. In the present application, the pixels of the sub-image are preferably 32×32, and the target image can be more clearly and finely divided at the pixel level of 32×32 in the pixel of the combined target image itself, and in other embodiments, the target image can be further divided into sub-images of 16×16, 64×64, and other pixels based on the pixel level of the original remote sensing image. After obtaining the sub-images, traversing pixels in all the sub-images through the depth convolution neural network, namely traversing pixels in each sub-image one by one aiming at each sub-image input into the depth convolution neural network, and judging a first pixel type of each pixel in the sub-images. The first pixel class is determined primarily by a pixel classification model, wherein the pixel classification model is derived primarily by sub-image training. After the pixel classification model is obtained, the original sub-image is input into the pixel classification model, and the class of the central pixel in the sub-image, that is, the first pixel class, is determined by a deep convolutional neural network algorithm in the pixel classification model, and the detailed process is described in the following, and is not repeated here. After the center pixel class of the sub-image is obtained, the target image is segmented (namely marked) based on the position of the sub-image in the original target image, namely, objects with different semantic classes in the target image are covered by adopting different colors, and further the segmented remote sensing image is obtained. Optionally, the labeling the target image according to the pixel class of the sub-image includes: acquiring position information on the target image corresponding to each sub-image; and labeling the target image according to the position information and the pixel type in turn. Specifically, since each pixel has a certain position, the position of the sub-image in the target image can be determined based on the pixels in the sub-image, when the target image is segmented, the position information of the sub-image in the target image can be obtained based on the pixels, and since the pixel type of the sub-image is already determined, the target image is segmented according to the position information and the pixel type of the sub-image.
In addition, in another embodiment, the present application further provides a server, as shown in fig. 4, where the server processor 503, the memory 505, the input unit 507, the display unit 509, and other devices. Those skilled in the art will appreciate that the structural elements shown in fig. 4 do not constitute a limitation on all servers, and may include more or fewer components than shown, or may combine certain components. The memory 505 may be used to store an application 501 and various functional modules, and the processor 503 runs the application 501 stored in the memory 505 to perform various functional applications and data processing of the device. The memory 505 may be an internal memory or an external memory, or include both internal and external memories. The internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, or random access memory. The external memory may include a hard disk, floppy disk, ZIP disk, U-disk, tape, etc. The disclosed memory includes, but is not limited to, these types of memory. The memory 505 of the present disclosure is by way of example only and not by way of limitation.
The input unit 507 is used for receiving input of signals, as well as personal information and related physical condition information input by a user. The input unit 507 may include a touch panel and other input devices. The touch panel can collect touch operations on or near the client (such as operations of the client on or near the touch panel using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a preset program; other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., play control keys, switch keys, etc.), a trackball, mouse, joystick, etc. The display unit 509 may be used to display information input by a client or information provided to the client and various menus of the computer device. The display unit 509 may take the form of a liquid crystal display, an organic light emitting diode, or the like. The processor 503 is the control center of the computer device, connecting the various parts of the overall computer using various interfaces and lines, performing various functions and processing data by running or executing software programs and/or modules stored in the memory 503, and invoking data stored in the memory. The one or more processors 503 shown in fig. 4 are capable of executing, implementing, the functions of the segmentation module 100, the functions of the first pixel class determination module 200, the functions of the labeling module 300, the functions of the first pixel classification model determination unit 210, the functions of the traversal unit 220, the functions of the second pixel class determination unit 211, the functions of the loss function determination unit 212, the functions of the second pixel classification model determination unit 213, the functions of the depth feature information extraction unit 230, the functions of the one-dimensional vector construction unit 240, the functions of the first pixel class determination unit 250, the functions of the center pixel determination unit 241, the functions of the pixel feature point extraction unit 242, the functions of the multi-scale super-column construction unit 243, the functions of the position information acquisition unit 310, the functions of the labeling unit 320 shown in fig. 3.
In one embodiment, the server includes one or more processors 503 and one or more memories 505, one or more applications 501, wherein the one or more applications 501 are stored in the memory 505 and configured to be executed by the one or more processors 503, and the one or more applications 301 are configured to perform the remote sensing image segmentation method described in the above embodiments.
According to the embodiment of the application, on the basis of dividing the remote sensing image to obtain a plurality of sub-images, the self-learning training is carried out on the deep convolutional neural network through the first sub-images, and the pixel classification model based on the deep convolutional neural network is determined, so that the sub-images can be classified again through the model, and the remote sensing image can be divided based on the pixel types of the sub-images. In the self-learning training of the application, the one-dimensional vectors with different depths of the same pixel are constructed by adopting the multi-scale depth features, so that the different features with different depths of the same pixel can be obtained, the full representation of the object is realized, and the target image can be segmented more accurately; the remote sensing image segmentation method provided by the embodiment of the application comprises the following steps: acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels; traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images; and labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image. The remote sensing image segmentation method provided by the application is mainly used for realizing the segmentation of the remote sensing image based on the deep convolutional neural network. According to the method, an original target image is firstly obtained, the target image is divided into a plurality of sub-images with 32 pixels, and the sub-image pixel levels can be conveniently classified through a deep convolutional neural network. The target image is a special image shot by a satellite, an aircraft or the like. In the present application, the pixels of the sub-image are preferably 32×32, and the target image can be more clearly and finely divided at the pixel level of 32×32 in the pixel of the combined target image itself, and in other embodiments, the target image can be further divided into sub-images of 16×16, 64×64, and other pixels based on the pixel level of the original remote sensing image. After obtaining the sub-images, traversing pixels in all the sub-images through the depth convolution neural network, namely traversing pixels in each sub-image one by one aiming at each sub-image input into the depth convolution neural network, and judging a first pixel type of each pixel in the sub-images. The first pixel class is determined primarily by a pixel classification model, wherein the pixel classification model is derived primarily by sub-image training. After the pixel classification model is obtained, the original sub-image is input into the pixel classification model, and the class of the central pixel in the sub-image, that is, the first pixel class, is determined by a deep convolutional neural network algorithm in the pixel classification model, and the detailed process is described in the following, and is not repeated here. After the center pixel class of the sub-image is obtained, the target image is segmented (namely marked) based on the position of the sub-image in the original target image, namely, objects with different semantic classes in the target image are covered by adopting different colors, and further the segmented remote sensing image is obtained. Optionally, the labeling the target image according to the pixel class of the sub-image includes: acquiring position information on the target image corresponding to each sub-image; and labeling the target image according to the position information and the pixel type in turn. Specifically, since each pixel has a certain position, the position of the sub-image in the target image can be determined based on the pixels in the sub-image, when the target image is segmented, the position information of the sub-image in the target image can be obtained based on the pixels, and since the pixel type of the sub-image is already determined, the target image is segmented according to the position information and the pixel type of the sub-image.
The server provided by the embodiment of the present application can implement the embodiment of the remote sensing image segmentation method provided above, and specific functional implementation is referred to the description in the method embodiment and will not be repeated here.
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.
Claims (6)
1. The remote sensing image segmentation method is characterized by comprising the following steps of:
acquiring a target image, and dividing the target image into a plurality of sub-images of preset pixels;
traversing pixels in all the sub-images based on a depth convolution neural network, and sequentially determining a first pixel category of each pixel in the sub-images;
labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image;
the traversing of pixels in all the sub-images based on the depth convolution neural network sequentially determines a first pixel category of each pixel in the sub-images, including:
determining a pixel classification model based on the deep convolutional neural network;
traversing each pixel in the sub-image one by one through the pixel classification model, and determining a first pixel class of the sub-image;
the determining a pixel classification model based on the depth convolution neural network comprises the following steps:
determining a second pixel class of the sub-image by the deep convolutional neural network;
determining a loss function of the pixel classification model based on a labeling image and a second pixel class of the sub-image, wherein the labeling image is the target image and the target image has semantic labeling;
determining the pixel classification model by the deep convolutional neural network and the loss function;
wherein the loss function comprises a sum of a cross entropy cost function and a regularization term;
the traversing of pixels in all the sub-images based on the depth convolution neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
extracting depth characteristic information of the sub-image through the depth convolution neural network;
constructing a multi-scale super-column of the same pixel in the sub-image according to the depth characteristic information, wherein the multi-scale super-column is a one-dimensional vector of the same pixel at different depths;
determining a first pixel class of the sub-image based on the multi-scale super-column;
the constructing the multiscale super-column of the same pixel in the sub-image according to the depth characteristic information comprises the following steps:
convolving the sub-image through the deep convolutional neural network to determine a center pixel of the sub-image;
extracting all pixel characteristic points of the central pixel in a convolution layer and a pooling layer;
and constructing the multi-scale super-column according to the positions of all the pixel characteristic points in the convolution layer and the pooling layer.
2. The method of claim 1, wherein the first pixel class and the second pixel class are center pixel classes of the sub-images.
3. The remote sensing image segmentation method according to claim 1, wherein the labeling the target image according to the pixel class of the sub-image comprises:
acquiring position information on the target image corresponding to each sub-image;
and labeling the target image according to the position information and the pixel type in turn.
4. A remote sensing image segmentation apparatus, comprising:
the segmentation module is used for acquiring a target image and segmenting the target image into a plurality of sub-images of preset pixels;
the first pixel category determining module is used for traversing pixels in all the sub-images based on the depth convolution neural network and sequentially determining the first pixel category of each pixel in the sub-images;
the labeling module is used for labeling the target image according to the first pixel category of the sub-image to obtain a segmented target image;
the traversing of pixels in all the sub-images based on the depth convolution neural network sequentially determines a first pixel category of each pixel in the sub-images, including:
determining a pixel classification model based on the deep convolutional neural network;
traversing each pixel in the sub-image one by one through the pixel classification model, and determining a first pixel class of the sub-image;
the determining a pixel classification model based on the depth convolution neural network comprises the following steps:
determining a second pixel class of the sub-image by the deep convolutional neural network;
determining a loss function of the pixel classification model based on a labeling image and a second pixel class of the sub-image, wherein the labeling image is the target image and the target image has semantic labeling;
determining the pixel classification model by the deep convolutional neural network and the loss function;
wherein the loss function comprises a sum of a cross entropy cost function and a regularization term;
the traversing of pixels in all the sub-images based on the depth convolution neural network sequentially determines a first pixel class of each pixel in the sub-images, including:
extracting depth characteristic information of the sub-image through the depth convolution neural network;
constructing a multi-scale super-column of the same pixel in the sub-image according to the depth characteristic information, wherein the multi-scale super-column is a one-dimensional vector of the same pixel at different depths;
determining a first pixel class of the sub-image based on the multi-scale super-column;
the constructing the multiscale super-column of the same pixel in the sub-image according to the depth characteristic information comprises the following steps:
convolving the sub-image through the deep convolutional neural network to determine a center pixel of the sub-image;
extracting all pixel characteristic points of the central pixel in a convolution layer and a pooling layer;
and constructing the multi-scale super-column according to the positions of all the pixel characteristic points in the convolution layer and the pooling layer.
5. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when executed by a processor, the program implements the remote sensing image segmentation method according to any one of claims 1 to 3.
6. A server, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the steps of the remote sensing image segmentation method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910015168.2A CN109859209B (en) | 2019-01-08 | 2019-01-08 | Remote sensing image segmentation method and device, storage medium and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910015168.2A CN109859209B (en) | 2019-01-08 | 2019-01-08 | Remote sensing image segmentation method and device, storage medium and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859209A CN109859209A (en) | 2019-06-07 |
| CN109859209B true CN109859209B (en) | 2023-10-17 |
Family
ID=66894159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910015168.2A Active CN109859209B (en) | 2019-01-08 | 2019-01-08 | Remote sensing image segmentation method and device, storage medium and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859209B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021102947A1 (en) * | 2019-11-29 | 2021-06-03 | 深圳市大疆创新科技有限公司 | Image signal processing apparatus and method, camera, and mobile platform |
| CN111339315B (en) * | 2020-02-21 | 2023-05-02 | 南京星火技术有限公司 | Knowledge graph construction method, system, computer readable medium and electronic equipment |
| CN111444365B (en) * | 2020-03-27 | 2023-12-05 | Oppo广东移动通信有限公司 | Image classification method, device, electronic equipment and storage medium |
| CN111860250B (en) * | 2020-07-14 | 2024-04-26 | 中南民族大学 | Image recognition method and device based on fine-grained character features |
| CN112884764A (en) * | 2021-03-24 | 2021-06-01 | 深圳前海微众银行股份有限公司 | Method and device for extracting land parcel in image, electronic equipment and storage medium |
| CN112906648A (en) * | 2021-03-24 | 2021-06-04 | 深圳前海微众银行股份有限公司 | Method and device for classifying objects in land parcel and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408482A (en) * | 2014-12-08 | 2015-03-11 | 电子科技大学 | Detecting method for high-resolution SAR (Synthetic Aperture Radar) image object |
| CN107527352A (en) * | 2017-08-09 | 2017-12-29 | 中国电子科技集团公司第五十四研究所 | Remote sensing Ship Target contours segmentation and detection method based on deep learning FCN networks |
| CN107784657A (en) * | 2017-09-29 | 2018-03-09 | 西安因诺航空科技有限公司 | A kind of unmanned aerial vehicle remote sensing image partition method based on color space classification |
| CN108090911A (en) * | 2018-01-08 | 2018-05-29 | 北京航空航天大学 | A kind of offshore naval vessel dividing method of remote sensing image |
| CN108229479A (en) * | 2017-08-01 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method and device of semantic segmentation model, electronic equipment, storage medium |
| CN108596248A (en) * | 2018-04-23 | 2018-09-28 | 上海海洋大学 | A kind of classification of remote-sensing images model based on improvement depth convolutional neural networks |
| CN108830322A (en) * | 2018-06-15 | 2018-11-16 | 联想(北京)有限公司 | A kind of image processing method and device, equipment, storage medium |
| CN108985238A (en) * | 2018-07-23 | 2018-12-11 | 武汉大学 | The high-resolution remote sensing image impervious surface extracting method and system of combined depth study and semantic probability |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3392798A1 (en) * | 2017-04-21 | 2018-10-24 | Delphi Technologies, Inc. | A method for the semantic segmentation of an image |
| CN109801293B (en) * | 2019-01-08 | 2023-07-14 | 平安科技（深圳）有限公司 | Remote sensing image segmentation method and device, storage medium and server |
-
2019
- 2019-01-08 CN CN201910015168.2A patent/CN109859209B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408482A (en) * | 2014-12-08 | 2015-03-11 | 电子科技大学 | Detecting method for high-resolution SAR (Synthetic Aperture Radar) image object |
| CN108229479A (en) * | 2017-08-01 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method and device of semantic segmentation model, electronic equipment, storage medium |
| CN107527352A (en) * | 2017-08-09 | 2017-12-29 | 中国电子科技集团公司第五十四研究所 | Remote sensing Ship Target contours segmentation and detection method based on deep learning FCN networks |
| CN107784657A (en) * | 2017-09-29 | 2018-03-09 | 西安因诺航空科技有限公司 | A kind of unmanned aerial vehicle remote sensing image partition method based on color space classification |
| CN108090911A (en) * | 2018-01-08 | 2018-05-29 | 北京航空航天大学 | A kind of offshore naval vessel dividing method of remote sensing image |
| CN108596248A (en) * | 2018-04-23 | 2018-09-28 | 上海海洋大学 | A kind of classification of remote-sensing images model based on improvement depth convolutional neural networks |
| CN108830322A (en) * | 2018-06-15 | 2018-11-16 | 联想(北京)有限公司 | A kind of image processing method and device, equipment, storage medium |
| CN108985238A (en) * | 2018-07-23 | 2018-12-11 | 武汉大学 | The high-resolution remote sensing image impervious surface extracting method and system of combined depth study and semantic probability |
Non-Patent Citations (1)
| Title |
|---|
| 基于改进全卷积神经网络的航拍图像语义分类方法;易盟 等;《计算机工程》;第43卷(第10期);第216-221页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859209A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109801293B (en) | Remote sensing image segmentation method and device, storage medium and server | |
| CN109859209B (en) | Remote sensing image segmentation method and device, storage medium and server | |
| Dong et al. | Object detection in high resolution remote sensing imagery based on convolutional neural networks with suitable object scale features | |
| Yin et al. | Airport detection based on improved faster RCNN in large scale remote sensing images | |
| CN111507271B (en) | Airborne photoelectric video target intelligent detection and identification method | |
| US11587249B2 (en) | Artificial intelligence (AI) system and methods for generating estimated height maps from electro-optic imagery | |
| US11747468B2 (en) | System using a priori terrain height data for interferometric synthetic aperture radar (IFSAR) phase disambiguation and related methods | |
| US11238307B1 (en) | System for performing change detection within a 3D geospatial model based upon semantic change detection using deep learning and related methods | |
| Liu et al. | Deep learning in forest structural parameter estimation using airborne lidar data | |
| Ichim et al. | Road detection and segmentation from aerial images using a CNN based system | |
| CN114219963A (en) | Multi-scale capsule network remote sensing ground feature classification method and system guided by geoscience knowledge | |
| Li et al. | Sea–sky line detection using gray variation differences in the time domain for unmanned surface vehicles | |
| Shi et al. | Fine object change detection based on vector boundary and deep learning with high-resolution remote sensing images | |
| Bagwari et al. | A comprehensive review on segmentation techniques for satellite images | |
| Zhang et al. | Semantic segmentation for remote sensing based on RGB images and lidar data using model-agnostic meta-learning and partical Swarm optimization | |
| CN113139556B (en) | Manifold multi-view image clustering method and system based on self-adaptive composition | |
| Chen et al. | BARS: A benchmark for airport runway segmentation | |
| US11636649B2 (en) | Geospatial modeling system providing 3D geospatial model update based upon predictively registered image and related methods | |
| Shi et al. | Convolutional neural networks applied on weather radar echo extrapolation | |
| Wang et al. | Design of an Image Segmentation System Based on Hierarchical Density-Based Spatial Clustering of Applications with Noise for Off-Road Unmanned Ground Vehicles | |
| Yang et al. | AAE-Dpeak-SC: A novel unsupervised clustering method for space target ISAR images based on adversarial autoencoder and density peak-spectral clustering | |
| Yang et al. | A new framework for GEOBIA: accurate individual plant extraction and detection using high-resolution RGB data from UAVs | |
| Tahir et al. | Rapid detection of aircrafts in satellite imagery based on deep neural networks | |
| Liu et al. | VL-MFL: UAV Visual Localization Based on Multi-Source Image Feature Learning | |
| KR102615412B1 (en) | Apparatus and method for performing visual localization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |