CN115272529B - Layout-first multi-scale decoupling ocean remote sensing image coloring method and system - Google Patents
Layout-first multi-scale decoupling ocean remote sensing image coloring method and system Download PDFInfo
- Publication number
- CN115272529B CN115272529B CN202211186781.9A CN202211186781A CN115272529B CN 115272529 B CN115272529 B CN 115272529B CN 202211186781 A CN202211186781 A CN 202211186781A CN 115272529 B CN115272529 B CN 115272529B
- Authority
- CN
- China
- Prior art keywords
- scale
- decoupling
- features
- layout
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004040 coloring Methods 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000004927 fusion Effects 0.000 claims abstract description 18
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 15
- 230000011218 segmentation Effects 0.000 claims description 17
- 238000011176 pooling Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 238000012935 Averaging Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 150000001875 compounds Chemical class 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 239000000463 material Substances 0.000 claims description 2
- 238000005728 strengthening Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 10
- 238000005070 sampling Methods 0.000 abstract description 6
- 238000000638 solvent extraction Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002352 surface water Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and discloses a layout-first multi-scale decoupling ocean remote sensing image coloring method and system, wherein an input original gray-scale image is sampled into a plurality of gray-scale images with different scales, and the gray-scale images are input into a multi-scale decoupling characteristic extraction module to extract multi-scale decoupling characteristics; inputting the multi-scale decoupling features into a multi-scale feature fusion module with a priority layout, guiding semantic features by using enhanced layout dividing features, then guiding coloring features by using semantic features including layout constraints, and fusing the extracted multi-scale decoupling features; finally, generating a color image; the generated color image and the original color image are discriminated by a discriminator, and a discrimination result can be output. The method solves the problem of consistency of spatial layout of the ocean remote sensing image, the problem that a large amount of noise is contained in the large-scale feature extraction process after down sampling, and the problem that the large-scale constraint is weak to the small-scale constraint in the multi-scale information utilization process.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a layout-first multi-scale decoupling ocean remote sensing image coloring method and system.
Background
The coloring of the ocean remote sensing image is a process of generating a color remote sensing image by utilizing the gray remote sensing image, and the colorization of the gray remote sensing image can increase the interpretation and analysis capability of the remote sensing image. Traditional coloring methods use a single-scale generation network to implement the coloring process, making coloring lack spatial consistency. As the ocean remote sensing image has the characteristic of unbalanced spatial layout, the remote sensing image coloring method of the front edge takes generation of a countermeasure network as a basic framework, and coloring is realized by designing a multi-scale generator based on U-net. The method has the advantages that the macroscale in the multi-scale generator is utilized to constrain the microscale, and the consistency of the image space layout is ensured to a certain extent.
However, the conventional method has the following problems:
first, the geospatial layout consistency constraint is ignored. Although the existing coloring method adopts a multi-scale modeling mode, the space consistency of large-scale target depiction can be improved, but the consistency of the spatial layout of ocean remote sensing images with continuous regions with ultra-large areas cannot be ensured due to the lack of consistency constraint of the layout (sky, sea surface and land). For example, on a continuous sea surface area, light spots caused by strong reflection of a sea surface water body are easily classified as ships or ice surfaces by mistake, and the layout information is not fully utilized for disambiguation in the existing method, so that coloring is wrong.
Second, no scale decoupling is performed, and a large amount of noise is contained in the large scale. The large scale directly extracts coloring features from the gray scale image after down-sampling, so that the features extracted by the large scale still contain a large amount of small scale information, and the small scale information belongs to redundant noise for the large scale and is not beneficial to ensuring the space consistency of the ocean remote sensing image.
Thirdly, in the process of multi-scale information fusion, most of the existing methods consider a plurality of scale features simultaneously to realize coloring, and the influence of different scales on coloring results under different scenes cannot be considered. For example, the pixel of the ship is on a small scale, and the pixel is easy to be judged as the sea on a large scale, so that the guiding effect of the small scale is stronger; however, an ambiguous pixel such as a speckle pixel is misjudged as a ship on a small scale (high-precision remote sensing image), but is eliminated after down-sampling, so that the pixel is judged as an ocean on a large scale (low-precision remote sensing image), and the large scale is more guiding. Since large-scale scenes are more common in marine remote sensing images, attention should be paid to large-scale features. The existing method only adds corresponding elements of two scale features to realize fusion operation, and does not consider the dominant effect of large-scale features in the coloring process of the ocean remote sensing image.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a layout-first multi-scale decoupling ocean remote sensing image coloring method and system, and the semantic features are constrained by a layout dividing module to generate so as to constrain coloring and ensure the coloring consistency; by the multi-scale decoupling method, the problem of noise introduction in the layout division module and the semantic constraint module is solved, and the initial features for layout division and semantic division are obtained, so that efficient layout division and semantic division features are generated for guiding the coloring of the image, and the consistency of a coloring space is ensured.
In order to solve the technical problems, the invention adopts the technical scheme that:
the invention firstly provides a layout-preferred multi-scale decoupling ocean remote sensing image coloring method, which is based on a generation countermeasure network architecture, generates a color image through a generator G, and discriminates a real image and a generated image through a discriminator D, and specifically comprises the following steps:
step 1, inputting an image: the input image is an original gray scale image
And downsampling to two gray-scale maps of different scales
、
Wherein
、
、
Are sequentially reduced;
step 2, designing a multi-scale decoupling feature extraction module, processing the input image in the step 1, and extracting multi-scale decoupling features; the multi-scale decoupling feature extraction module comprises a multi-scale feature decoupling module, a layout division module, a semantic constraint module and an image coloring module, wherein the multi-scale feature decoupling module comprises a multi-scale feature decoupling module I and a multi-scale feature decoupling module II which have the same structure and are used for carrying out multi-scale feature decoupling; when extracting the multi-scale decoupling features, the following are concrete steps:
step 2.1, inputting the input images with different scales in the step 1 into a multi-scale feature decoupling module for multi-scale feature decoupling, and respectively generating decoupling features aiming at scale tasks;
step 2.2, the decoupling features generated in the step 2.1 are subjected to layout division, semantic constraint and image coloring processing to generate layout division features, semantic features and coloring features, wherein the features obtained by the original gray level map are directly input into an image coloring module, the decoupling features output by the multi-scale feature decoupling module I are input into a semantic constraint module, and the decoupling features output by the multi-scale feature decoupling module II are input into a layout division module;
step 4, generating a color image;
and 5, distinguishing the color image generated in the step 4 from the original color image through a discriminator D, and outputting a distinguishing result.
Further, in step 2.1, the multi-scale feature decoupling module performs multi-scale feature decoupling, specifically operating as follows:
first, for the original gray-scale image of step 1
And downsampling to two grayscale images of different scales
、
Respectively carrying out convolution operation to obtain initial coloring characteristic
Initial semantic features
And initial layout features
；
Secondly, completing multi-scale feature decoupling by using the feature map, specifically, in the multi-scale feature decoupling module I, features
Obtaining features after average pooling
By the features
Subtracting features
To obtain a decoupling characteristic
(ii) a Similarly, will be characterized
Upsampling to obtain features
By the features
Subtracting features
To obtain a decoupling characteristic
(ii) a Will be provided with
And
adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of a representation pair
Performing upsampling operation;
characteristic of a representation pair
Carrying out average pooling operation;
same, characteristic
And features of
The characteristic decoupling is carried out by a multi-scale characteristic decoupling module II which has the same structure as the multi-scale characteristic decoupling module I, in particular, in the multi-scale characteristic decoupling module II, the characteristics
Obtaining features after average pooling
By the features
Subtracting features
To obtain a decoupling characteristic
(ii) a Similarly, will be characterized
Upsampling to obtain features
By using characteristics
Subtracting characteristics
To obtain a decoupling characteristic
(ii) a Will be provided with
And with
Adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of expression pair
An upsampling operation;
characteristic of expression pair
And (4) averaging pooling operation.
Further, the layout partitioning module in step 2.2 includes two operations, one of which is to extract semantic features through a pre-trained U-net network for semantic segmentation tasks, and the other of which is to calculate and merge similar semantic regions according to the correlation to generate the layout segmentation graph.
Further, the specific operation of the layout dividing module in step 2.2 is as follows:
firstly, generating a semantic segmentation graph through a pre-trained U-Net network, and extracting the last layer of features of the pre-trained U-Net network
Taking out, is characterized in
Averaging the characteristic values contained in the corresponding position of each semantic area in the semantic segmentation graph, and calculating the centroid of each semantic area; in addition, will be characterized
Divided into q 2 /r 2 Blocks of size r, q being characteristic
R is the size of each block, r equals the feature
The common factor of the size is represented by the centroid of the semantic region to which each block belongs, and the centroid of the block P and the centroids of the surrounding 8 blocks are respectively subjected to subtraction to obtain an absolute value, so that 8-dimensional feature representation of the block P is obtained;
secondly, merging similar semantic regions to generate layout division characteristics: if the two adjacent blocks A and B do not belong to the same semantic meaning, calculating the cosine similarity between the two blocks A and B, wherein the calculation formula is as follows:
wherein,
a z-th dimension vector representing the block a,
representing the z-th dimension vector of the block B, calculating the similarity of the blocks A and B by calculating cosine similarity through a formula, wherein z is the number dimension on the 8-dimension vector; if cosine similarity
Greater than a threshold value
And merging semantic areas to which the two blocks belong to generate layout division characteristics, wherein the concrete merging method comprises the following steps: the mean z of the centroids of the two semantic regions is calculated, and the pixels contained in the two semantic regions are set to z to obtain a feature representing the layout division.
Further, the layout-first multi-scale feature fusion module in step 3 specifically operates as follows:
step 3.1, carrying out Sigmoid function processing on the output characteristics of the layout division module, and strengthening the layout division characteristics with weak significance to prevent the loss of the layout division characteristics;
step 3.2, multiplying the features processed in the step 3.1 by corresponding elements of the output features of the semantic constraint module, thereby generating semantic features through layout constraint;
and 3.3, carrying out Tanh function processing on the semantic features with the layout constraint obtained in the step 3.2 to realize feature mapping, and then multiplying the semantic features with the generated features of the image coloring module by the constraint to finally color.
Further, the network is optimized by a game method of the generator G and the discriminator D, which specifically comprises the following steps:
first, training generator G, where the discriminator is fixed, has the following formula:
in the formula (1), x 1 For large-scale samples, x 2 Is a mesoscale sample, x 3 Is a sample with a small scale, and the sample is,
is a function of
With respect to X 1 ~P(X 1 )，X 2 ~P(X 2 )，X 3 ~P(X 3 ) P () represents a probability distribution,
for the generated color image sample, y is the original image,
a representation discriminator D discriminates the generated image;
a probability that the generated image is true is discriminated by the discriminator; when the generator G is trained, the generator G,
the smaller the better;
in the formula (2), the first and second groups,
is an L1 loss function for generating a color image by minimizing the sum of absolute differences of corresponding pixel values of a generated image and a real image, lambda being a hyper-parameter,
in the formula (3)
For cross-entropy loss, a semantic segmentation map for mesoscale generation, wherein,
representing the output of the mesoscale network, i.e. the probability that the class is i, i is the semantic class, m is the total number of all semantic classes, y 2 Is the truth value of the mesoscale semantic segmentation result; thereby, an objective function (4) is obtained,
in the formula (4), the first and second groups,
are the model parameters of the discriminator D,
is a small-scale model parameter,
Is a mesoscale model parameter,
Is a model parameter of a large scale,
loss of generator G;
then, discriminant D is trained, with the generator stationary, and during discriminant D training, the following equation is applied:
in the formula (5), the first and second groups,
is a function of
With respect to the expected value of y, P (.) represents a probability distribution,
representation arbiter D on the real image y 1 The judgment is carried out, and the judgment is carried out,
discriminating the real image y for the discriminator D 1 A probability of being true, the larger the better;
in the formula (6), the first and second groups,
is a function of
With respect to X 1 ~P(X 1 )，X 2 ~P(X 2 )，X 3 ~P(X 3 ) P () represents a probability distribution,
the presentation discriminator discriminates the generated image,
the probability that the generated image is true is judged for the discriminator D, and the larger the probability, the better the probability; thereby, an objective function (7) is obtained,
in the formula (7), the first and second groups of the compound,
is the loss of discriminator D.
The invention also provides a layout-prior multi-scale decoupling marine remote sensing image coloring system, which is used for implementing the layout-prior multi-scale decoupling marine remote sensing image coloring method and comprises an input module, a generator and a discriminator, wherein the input module inputs gray-scale images with different scales into the generator, the generator is used for generating a color image and comprises a multi-scale decoupling feature extraction module, a layout-prior multi-scale feature fusion module and a final coloring module, the multi-scale decoupling feature extraction module comprises a multi-scale feature decoupling module, a layout partitioning module, a semantic constraint module and an image coloring module, the multi-scale feature decoupling module is used for carrying out multi-scale feature decoupling on the input image to respectively generate decoupling features aiming at scale tasks, and the decoupling features, the layout partitioning features, the semantic features and the coloring features are respectively processed by the layout partitioning module, the semantic constraint module and the image coloring module; the multi-scale feature fusion module with the prior layout utilizes the enhanced layout division features to guide semantic features and utilizes the semantic features containing layout constraints to guide coloring features, fuses the multi-scale decoupling features extracted by the multi-scale decoupling feature extraction module, and generates a color image through the final coloring module; and the discriminator is used for discriminating and comparing the color image generated by the generator with the original color image and outputting a discrimination result.
Compared with the prior art, the invention has the advantages that:
(1) The invention designs a layout division module, generates constraint coloring by utilizing constraint semantic features of the layout division module, and ensures coloring consistency. Dividing the semantic features into a plurality of blocks with the same size, wherein each block is represented by the centroid of the semantic region to which the block belongs, and each block is characterized by calculating the relation with the surrounding blocks (the centroid of one block and the centroid of the surrounding 8 blocks are respectively subjected to subtraction to obtain an absolute value to obtain 8-dimensional feature representation of the block). Compared with a method of only self-characterization, the method of utilizing the ambient environment characterization establishes the associated description between the features, and has higher robustness. Secondly, calculating the correlation between adjacent blocks belonging to different semantic regions, merging the related semantic regions, and generating layout division characteristics in an unsupervised mode, so that the problem that the layout division lacks labels is solved, and meanwhile, the generalization of the network is improved.
(2) The invention uses a multi-scale feature decoupling method to solve the problem of noise introduction in a layout dividing bit module and a semantic constraint module so as to mine potential features of each scale for modeling aiming at scale tasks. Removing redundant noise in the initial semantic features by using the initial coloring features to obtain decoupling semantic features; and removing redundant noise by utilizing the decoupling semantic features in the initial layout features to obtain decoupling layout features. By the multi-scale decoupling method, initial features used for layout division and semantic division are obtained, so that efficient layout division and semantic division features are generated to guide image coloring, the consistency of a coloring space is guaranteed, namely the difference between a small-scale feature and an up-sampled large-scale feature is calculated, meanwhile, the difference between the down-sampled small-scale feature and the up-sampled large-scale feature is calculated, and two different modes are used for fully decoupling a macroscopic feature to better remove noise.
(3) The invention realizes multi-scale feature fusion by a layout priority method and enhances the leading action of layout features. Firstly, sigmoid function activation is carried out on layout division characteristics, and the layout division characteristics are multiplied by semantic characteristics to carry out fusion operation, so that layout information is strengthened, generation of semantic characteristics is restrained, rationality of generated semantics is ensured, and leading effect of large-scale characteristics in a coloring process of a marine remote sensing image is enhanced; secondly, tanh function processing is carried out on the semantic features, more accurate semantic information is reserved, and the processed features are multiplied by the coloring features for final coloring. The method for fusing the emphatic layout is favorable for ensuring the space consistency of the ocean remote sensing images containing more large-scale scenes.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a structural diagram of a layout-first multi-scale decoupling marine remote sensing image coloring system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a multi-scale feature decoupling module according to an embodiment of the invention.
Detailed Description
The invention is further described with reference to the following figures and specific embodiments.
Example 1
With reference to fig. 1-2, the present embodiment designs a layout-first multi-scale decoupling marine remote sensing image coloring method, which generates a color image through a generator G based on a generation countermeasure network architecture, and discriminates a real image and a generated image through a discriminator D, and specifically includes the following steps:
step 1, inputting an image: the input image is an original gray scale image
And downsampling to two grayscale images of different scales
、
Wherein
、
、
In turn decreases in size.
In use, for an input raw gray scale map
For convenience of understanding, the present embodiment only takes three different scales as an example, the grayscale images adopted in the present embodiment are the original grayscale images and the grayscale images downsampled to the original grayscale images 2/1 and 4/1, and the input sizes of the images in the three scales are 256 × 256, 128 × 128, and 64 × 64, respectively.
Step 2, designing a multi-scale decoupling feature extraction module, processing the input image in the step 1, and extracting multi-scale decoupling features; the multi-scale decoupling feature extraction module comprises a multi-scale feature decoupling module, a layout division module, a semantic constraint module and an image coloring module, wherein the image coloring module and the semantic constraint module adopt a U-net structure, the prior art can be referred to, the description is omitted here, and the layout division module is introduced in step 2.2. The multi-scale characteristic decoupling module comprises a multi-scale characteristic decoupling module I and a multi-scale characteristic decoupling module II which have the same structure, and is used for carrying out multi-scale characteristic decoupling; when extracting the multi-scale decoupling features, the following are concrete steps:
and 2.1, inputting the input images with different scales in the step 1 into a multi-scale feature decoupling module for multi-scale feature decoupling, and respectively generating decoupling features aiming at scale tasks. The step realizes characteristic decoupling by using different scale characteristics, and solves the problem of introducing a large amount of noise in a macro scale.
In step 2.1, the multi-scale feature decoupling module performs multi-scale feature decoupling, and the specific operations are as follows:
firstly, respectively performing convolution operation on the original gray-scale image, the gray-scale image with the 2 times of down-sampling and the gray-scale image with the 4 times of down-sampling in the step 1 to obtain initial coloring characteristics
Initial semantic features
And initial layout features
；
Secondly, completing multi-scale feature decoupling by using the feature map, specifically, in a multi-scale feature decoupling module I, features
Obtaining features after average pooling
By the features
Subtracting characteristics
To obtain a decoupling characteristic
(ii) a Similarly, will be characterized
Upsampling to obtain features
By using characteristics
Subtracting characteristics
To obtain a decoupling characteristic
(ii) a Will be provided with
And with
Adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of expression pair
An upsampling operation;
characteristic of expression pair
Carrying out average pooling operation;
same, characteristic
And features of
The characteristic decoupling is carried out by a multi-scale characteristic decoupling module II which has the same structure as the multi-scale characteristic decoupling module I, in particular, in the multi-scale characteristic decoupling module II, the characteristics
Obtaining features after average pooling
By the features
Subtracting features
To obtainTo a decoupling feature
(ii) a Similarly, will be characterized
Upsampling to obtain features
By the features
Subtracting features
To obtain a decoupling characteristic
(ii) a Will be provided with
And
adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of a representation pair
Performing upsampling operation;
characteristic of expression pair
And (4) averaging pooling operation.
And 2.2, performing layout division, semantic constraint and image coloring on the decoupling features generated in the step 2.1 to generate layout division features, semantic features and coloring features, wherein the features obtained by the original gray level map are directly input into the image coloring module, the decoupling features output by the multi-scale feature decoupling module I are input into the semantic constraint module, and the decoupling features output by the multi-scale feature decoupling module II are input into the layout division module. In each scale, the input and output of each module of the image coloring module, the semantic constraint module and the layout dividing module are the same as the input size of the scale, and the input and output sizes of the image coloring module, the semantic constraint module and the layout dividing module are 256 × 256, 128 × 128 and 64 × 64, respectively.
The layout partitioning module in step 2.2 includes two operations, one of which is to extract semantic features through a pre-trained U-net network for semantic segmentation tasks, and the other of which is to calculate and merge similar semantic regions according to the correlation to generate a layout segmentation graph.
The specific operation of the layout partitioning module is as follows:
firstly, generating a semantic segmentation graph through a pre-trained U-Net network, and extracting the last layer of features of the pre-trained U-Net network
Taking out, is characterized in
In the method, the feature values contained in the corresponding positions of each semantic region in the semantic segmentation graph are averaged, and the centroid of each semantic region is calculated. In addition, will be characterized by
Divided into q 2 /r 2 Blocks of size r, q being characteristic
R is the size of each block, r equals the feature
The common factor of the size (the side length changes with the change of the input size of the third scale) is used to ensure that the whole graph can be exactly divided, in this embodiment, 64 × 64, each block is represented by the centroid of the semantic region to which the block belongs, and the centroid of the block P and the centroids of the surrounding 8 blocks are respectively subjected to difference to obtain an absolute value, so as to obtain an 8-dimensional feature representation for the block P. For example, if the centroid of the block p is (x, y), i is the centroid, (x-1, y-1), (x-1, y + 1), (x +1, y-1), (x +1, y + 1), (x +1, y + 1) is a, b, c, d, e, f, g, h, then the block p is represented as an 8-dimensional vector (| i-a |, | i-b |, | i-c |, | i-d |, | i-e |, | i-f |, | i-g |, | i-h |).
Secondly, merging similar semantic regions to generate layout division characteristics: if two adjacent blocks A and B do not belong to the same semantic meaning, calculating the cosine similarity between the two blocks A and B, wherein the calculation formula is as follows:
wherein, among others,
a z-th dimension vector representing the block a,
representing a z-dimension vector of the block B, calculating the similarity of the blocks A and B by calculating cosine similarity through a formula, wherein z is a dimension on the 8-dimension vector; if cosine similarity
Greater than a threshold value
Disclosure of the inventionAnd the semantic regions to which the two blocks belong, the specific merging method is as follows: the mean z of the centroids of the two semantic regions is calculated, and the pixels contained in the two semantic regions are set to z to obtain a feature representing the layout division.
And 3, designing a multi-scale feature fusion module with a priority layout, guiding semantic features by using the enhanced layout dividing features, guiding coloring features by using the semantic features including layout constraints, and fusing the multi-scale decoupling features extracted in the step 2. The specific operation is as follows:
step 3.1, performing Sigmoid function processing on output features of the layout division module, and reinforcing the layout division features with weak significance to prevent loss of the layout division features, so that the leading effect of large-scale features in the coloring process of the ocean remote sensing image is enhanced;
step 3.2, multiplying the corresponding elements of the features processed in the step 3.1 and the output features of the semantic constraint module, thereby generating reasonable semantic features through a layout constraint network;
and 3.3, carrying out Tanh function processing on the semantic features with the layout constraint obtained in the step 3.2 to realize conventional mapping of the features, multiplying the conventional mapping features by the generated features of the image coloring module to restrict and finally coloring. By utilizing different activation functions, the leading effect of large-scale features in the coloring process of the ocean remote sensing image is strengthened, and therefore the coloring spatial consistency is guaranteed.
And 4, generating a color image.
And 5, distinguishing the color image generated in the step 4 from the original color image through a discriminator D, and outputting a distinguishing result.
The following describes the training and penalty functions of the generator G and the discriminator D according to the invention. The invention optimizes the network by a game method of a generator G and a discriminator D, which comprises the following steps:
first, training generator G, where the discriminator is fixed, has the following formula:
in the formula (1), x 1 For large-scale samples, x 2 Is a mesoscale sample, x 3 Is a sample with a small scale, and the sample is,
in the formula (4), the first and second groups of the chemical reaction are shown in the specification,
are the model parameters of the discriminator D,
is a small-scale model parameter,
Is a mesoscale model parameter,
Is a parameter of the model with a large scale,
in order to minimize the loss of the generator G, the objective function (4) is to minimize the score given to the generated image by the discriminator D and minimize the corresponding pixel difference between the generated image and the real image;
then, discriminant D is trained, with the generator stationary, and during discriminant D training, the following equation is applied:
in the formula (5), the first and second groups of the chemical reaction materials are selected from the group consisting of,
is a function of
With respect to the expected value of y, P () represents a probability distribution,
representation discriminator D for real image y 1 The judgment is carried out, and the judgment is carried out,
discriminating the real image y for the discriminator D 1 Probability of being true, so the larger the better;
in the formula (6), the first and second groups of the compound,
is a function of
With respect to X 1 ~P(X 1 )，X 2 ~P(X 2 )，X 3 ~P(X 3 ) P () represents a probability distribution,
the presentation discriminator discriminates the generated image,
the probability that the generated image is true is judged by a discriminator D, and the larger the probability, the better the probability; thereby, an objective function (7) is obtained,
in the formula (7), the first and second groups,
the objective function (7) is to maximize the score given by the discriminator D to the real image and minimize the score given by the discriminator D to the generated picture, for the loss of the discriminator D.
The training process and the calculation of each loss function of the present invention can refer to the prior art, and are only briefly described here, and will not be described in detail.
As a preferred embodiment, the present embodiment uses the NWPU-reic 45 dataset. The data set consisted of 45 scene categories, each consisting of 700 images, for a total of 31,500 images of size 256 x 256. 500, 100 and 100 images were used in each category as training set, test machine and validation set, respectively. Using an Adam optimizer, the hyperparameter λ was set to 100. The network adopts an end-to-end batch training method, and each batch is set to be 8.
Example 2
The embodiment provides a layout-prioritized multi-scale decoupling marine remote sensing image coloring system, which comprises an input module, a generator and a discriminator, wherein the input module inputs gray-scale images of different scales into the generator, the generator is used for generating a color image and comprises a multi-scale decoupling feature extraction module, a multi-scale feature fusion module with prioritized layout and a final coloring module, the multi-scale decoupling feature extraction module comprises a multi-scale feature decoupling module, a layout dividing module, a semantic constraint module and an image coloring module, wherein the multi-scale feature decoupling module performs multi-scale feature decoupling on the input image, respectively generates decoupling features aiming at scale tasks, and respectively generates layout dividing features, semantic features and coloring features through the processing of the layout dividing module, the semantic constraint module and the image coloring module; the layout-first multi-scale feature fusion module utilizes the enhanced layout division features to guide semantic features and utilizes semantic features containing layout constraints to guide coloring features, fuses the multi-scale decoupling features extracted by the multi-scale decoupling feature extraction module, and generates a color image through the final coloring module; and the discriminator is used for discriminating and comparing the color image generated by the generator with the original color image and outputting a discrimination result. The system is used for implementing the layout-first multi-scale decoupling ocean remote sensing image coloring method, the functions of all modules and the coloring method steps can be recorded as in the embodiment 1, and the description is omitted here.
In summary, the invention designs a multi-scale decoupling feature extraction module and a multi-scale feature fusion module with a preferential layout, and for input gray-scale images with different scales, firstly, decoupling features aiming at scale tasks are respectively generated by the multi-scale feature decoupling module; secondly, generating a layout division characteristic, a semantic characteristic and a coloring characteristic by the decoupling characteristic through a layout division module, a semantic constraint module and an image coloring module; then, the layout-first multi-scale feature fusion module guides semantic features by using the enhanced layout division features, and then guides coloring features by using semantic features containing layout constraints; and finally, generating a color chart by using a final coloring module. The method solves the problem of consistency of spatial layout of the ocean remote sensing image, the problem that a large amount of noise is contained in the large-scale feature extraction process after down sampling, and the problem that the large-scale constraint is weak to the small-scale constraint in the multi-scale information utilization process.
It will be understood that the foregoing description is not intended to limit the invention, and that the invention is not limited to the examples described above, and that various changes, modifications, additions and substitutions which may be made by one of ordinary skill in the art without departing from the spirit of the invention are therefore intended to be included within the scope of the invention.
Claims (7)
1. A layout-preferred multi-scale decoupling ocean remote sensing image coloring method is based on a generation countermeasure network architecture, a color image is generated through a generator G, a real image and a generated image are distinguished through a discriminator D, and the method is characterized by specifically comprising the following steps:
step 1, inputting an image: the input image is an original gray scale image
And downsampling to two grayscale images of different scales
、
In which
、
、
The sizes of (a) and (b) are sequentially reduced;
step 2, designing a multi-scale decoupling feature extraction module, processing the input image in the step 1, and extracting multi-scale decoupling features; the multi-scale decoupling feature extraction module comprises a multi-scale feature decoupling module, a layout division module, a semantic constraint module and an image coloring module, wherein the multi-scale feature decoupling module comprises a multi-scale feature decoupling module I and a multi-scale feature decoupling module II which have the same structure and are used for carrying out multi-scale feature decoupling; when extracting the multi-scale decoupling features, the following are specific:
step 2.1, inputting the input images with different scales in the step 1 into a multi-scale feature decoupling module for multi-scale feature decoupling, and respectively generating decoupling features aiming at scale tasks;
step 2.2, the decoupling features generated in the step 2.1 are subjected to layout division, semantic constraint and image coloring processing to generate layout division features, semantic features and coloring features, wherein the features obtained by the original gray-scale image are directly input into an image coloring module, the decoupling features output by the multi-scale feature decoupling module I are input into a semantic constraint module, and the decoupling features output by the multi-scale feature decoupling module II are input into a layout division module;
step 3, designing a multi-scale feature fusion module with a preferential layout, inputting the multi-scale decoupling features extracted in the step 2 into the multi-scale feature fusion module with the preferential layout, guiding semantic features by utilizing enhanced layout division features, then guiding coloring features by utilizing the semantic features containing layout constraints, and fusing the multi-scale decoupling features extracted in the step 2;
step 4, generating a color image;
and 5, distinguishing the color image generated in the step 4 from the original color image through a discriminator D, and outputting a distinguishing result.
2. The layout-first multi-scale decoupling marine remote sensing image coloring method according to claim 1, characterized in that in step 2.1, a multi-scale feature decoupling module performs multi-scale feature decoupling, and the specific operations are as follows:
first, the original gray-scale map of step 1
And downsampling to two gray-scale maps of different scales
、
Respectively carrying out convolution operation to obtain initial coloring characteristics
Initial semantic features
And initial layout features
；
Secondly, completing multi-scale feature decoupling by using the feature map, specifically, in the multi-scale feature decoupling module I, features
Obtaining features after average pooling
By using characteristics
Subtracting characteristics
Obtaining decoupling characteristics
(ii) a Similarly, will be characterized
Upsampling to obtain features
By using characteristics
Subtracting features
To obtain a decoupling characteristic
(ii) a Will be provided with
And
adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of a representation pair
Performing upsampling operation;
characteristic of expression pair
Carrying out average pooling operation;
same, characteristic of
And features of
Performing characteristic decoupling by a multi-scale characteristic decoupling module II with the same structure as the multi-scale characteristic decoupling module I, specifically, in the multi-scale characteristic decoupling module II, performing characteristic decoupling
Obtaining features after average pooling
By the features
Subtracting features
To obtain a decoupling characteristic
(ii) a Similarly, will be characterized
Upsampling to obtain features
By the features
Subtracting features
Obtaining decoupling characteristics
(ii) a Will be provided with
And
adding to obtain decoupling characteristics
(ii) a The formula is expressed as:
wherein,
characteristic of expression pair
An upsampling operation;
characteristic of expression pair
Average poolAnd (5) carrying out chemical operation.
3. The layout-first multi-scale decoupling marine remote sensing image coloring method according to claim 1, wherein the layout division module in step 2.2 comprises two operations, one is to extract semantic features through a pre-trained U-net network for semantic segmentation tasks, and the other is to calculate and merge similar semantic regions according to correlation to generate the layout division map.
4. The layout-first multi-scale decoupling marine remote sensing image coloring method according to claim 3, wherein the layout division module in step 2.2 specifically operates as follows:
firstly, generating a semantic segmentation graph through a pre-trained U-Net network, and extracting the last layer of features of the pre-trained U-Net network
Taking out, is characterized in
Averaging the characteristic values contained in the corresponding position of each semantic area in the semantic segmentation graph, and calculating the centroid of each semantic area; in addition, will be characterized
Divided into q 2 /r 2 Blocks of size r, q being characteristic
R is the size of each block, r equals the feature
The common factor of the size is represented by the centroid of the semantic region to which each block belongs, the centroid of the block P and the centroids of the surrounding 8 blocks are respectively subjected to difference to obtain an absolute value, and an 8-dimensional feature representation of the block P is obtained；
Secondly, merging similar semantic regions to generate layout division characteristics: if two adjacent blocks A and B do not belong to the same semantic meaning, calculating the cosine similarity between the two blocks A and B, wherein the calculation formula is as follows:
wherein,
a z-th dimension vector representing the block a,
representing a z-dimension vector of the block B, calculating the similarity of the blocks A and B by calculating cosine similarity through a formula, wherein z is a dimension on the 8-dimension vector; if cosine similarity
Greater than a threshold value
Merging the semantic regions to which the two blocks belong to generate layout division characteristics, wherein the specific merging method comprises the following steps: the mean z of the centroids of the two semantic regions is calculated, and the pixels contained in the two semantic regions are set as z to obtain a feature representing the layout division.
5. The layout-first multi-scale decoupling ocean remote sensing image coloring method according to claim 1, wherein the layout-first multi-scale feature fusion module in step 3 specifically operates as follows:
step 3.1, carrying out Sigmoid function processing on the output characteristics of the layout division module, and strengthening the layout division characteristics with weak significance to prevent the loss of the layout division characteristics;
step 3.2, multiplying the features processed in the step 3.1 by corresponding elements of the output features of the semantic constraint module, thereby generating semantic features through layout constraint;
and 3.3, carrying out Tanh function processing on the semantic features with the layout constraint obtained in the step 3.2 to realize feature mapping, and then multiplying the semantic features with the generated features of the image coloring module by the constraint for final coloring.
6. The layout-first multi-scale decoupling marine remote sensing image coloring method according to claim 1, characterized in that a network is optimized by a game method of a generator G and a discriminator D, and the method specifically comprises the following steps:
first, training generator G, where the discriminator is fixed, has the following formula:
in the formula (1), x 1 Is a large scale sample, x 2 Is a mesoscale sample, x 3 Is a sample with a small scale, and the sample is,
is a function of
About
， 
， 
Is expected toThe value, P (), represents the probability distribution,
for the generated color image sample, y is the original image,
a representation discriminator D discriminates the generated image;
a probability that the generated image is true is discriminated by the discriminator; when the generator G is trained, the generator G,
the smaller the better;
in the formula (2), the first and second groups of the compound,
is an L1 loss function for generating a color image by minimizing the sum of absolute differences of corresponding pixel values of a generated image and a real image, lambda being a hyper-parameter,
in the formula (3)
For cross-entropy loss, a semantic segmentation map for mesoscale generation, wherein,
representing the output of the mesoscale network, i.e. the probability that the class is i, i is the semantic class, m is the total number of all semantic classes, y 2 Is the truth value of the mesoscale semantic segmentation result; thereby, an objective function (4) is obtained,
in the formula (4), the first and second groups of the chemical reaction are shown in the specification,
are the model parameters of the discriminator D,
is a small-scale model parameter,
Is a mesoscale model parameter,
Is a parameter of the model with a large scale,
loss of generator G;
then, discriminant D is trained, with the generator stationary, and during discriminant D training, the following equation is applied:
in the formula (5), the first and second groups of the chemical reaction materials are selected from the group consisting of,
is a function of
With respect to the expected value of y,
representation discriminator D for real image y 1 The judgment is carried out, and the judgment is carried out,
discriminating the real image y for the discriminator D 1 A probability of being true, the larger the better;
formula (6) In the step (1), the first step,
is a function of
About
， 
， 
P () represents a probability distribution,
the presentation discriminator discriminates the generated image,
the probability that the generated image is true is judged for the discriminator D, and the larger the probability, the better the probability; thereby, an objective function (7) is obtained,
in the formula (7), the first and second groups of the compound,
is the loss of discriminator D.
7. The layout-first multi-scale decoupling ocean remote sensing image coloring system is characterized by being used for implementing the layout-first multi-scale decoupling ocean remote sensing image coloring method according to any one of claims 1 to 6, and comprising an input module, a generator and a discriminator, wherein the input module inputs gray-scale images of different scales into the generator, the generator comprises a multi-scale decoupling feature extraction module, a layout-first multi-scale feature fusion module and a final coloring module, and the final coloring module is used for generating a color image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186781.9A CN115272529B (en) | 2022-09-28 | 2022-09-28 | Layout-first multi-scale decoupling ocean remote sensing image coloring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186781.9A CN115272529B (en) | 2022-09-28 | 2022-09-28 | Layout-first multi-scale decoupling ocean remote sensing image coloring method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115272529A CN115272529A (en) | 2022-11-01 |
| CN115272529B true CN115272529B (en) | 2022-12-27 |
Family
ID=83757722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211186781.9A Active CN115272529B (en) | 2022-09-28 | 2022-09-28 | Layout-first multi-scale decoupling ocean remote sensing image coloring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115272529B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116311253B (en) * | 2023-05-18 | 2023-07-21 | 中国海洋大学 | Ocean remote sensing image semantic segmentation method and system based on scale separation |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104050722A (en) * | 2014-06-06 | 2014-09-17 | 北京航空航天大学 | Indoor three-dimensional scene layout and color transfer generation method driven by image contents |
| CN107369158A (en) * | 2017-06-13 | 2017-11-21 | 南京邮电大学 | The estimation of indoor scene layout and target area extracting method based on RGB D images |
| CN114782568A (en) * | 2022-03-10 | 2022-07-22 | 中国海洋大学 | Multi-scale stage feature progressive fusion remote sensing image coloring method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10657623B2 (en) * | 2018-08-10 | 2020-05-19 | Apple Inc. | Two stage multi-scale processing of image data |
| US11386589B2 (en) * | 2020-08-04 | 2022-07-12 | Ping An Technology (Shenzhen) Co., Ltd. | Method and device for image generation and colorization |
| CN114627006B (en) * | 2022-02-28 | 2022-12-20 | 复旦大学 | Progressive image restoration method based on depth decoupling network |
-
2022
- 2022-09-28 CN CN202211186781.9A patent/CN115272529B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104050722A (en) * | 2014-06-06 | 2014-09-17 | 北京航空航天大学 | Indoor three-dimensional scene layout and color transfer generation method driven by image contents |
| CN107369158A (en) * | 2017-06-13 | 2017-11-21 | 南京邮电大学 | The estimation of indoor scene layout and target area extracting method based on RGB D images |
| CN114782568A (en) * | 2022-03-10 | 2022-07-22 | 中国海洋大学 | Multi-scale stage feature progressive fusion remote sensing image coloring method and system |
Non-Patent Citations (4)
| Title |
|---|
| MIGN: Multiscale Image Generation Network for Remote Sensing Image Semantic Segmentation;Jie Nie et al.;《IEEE Transactions on Multimedia》;20220808;全文 * |
| Scanning Thermal Microscopy for Fast Multiscale Imaging and Manipulation;Rachel J. Cannara et al.;《IEEE Transactions on Nanotechnology》;20100318;全文 * |
| 一种基于非对称逆布局模型的彩色图像表示方法;郑运平等;《软件学报》;20071115(第11期);全文 * |
| 基于联合一致循环生成对抗网络的人像着色;刘昌通等;《计算机工程与应用》;20200815(第16期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115272529A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109919108B (en) | Remote sensing image rapid target detection method based on deep hash auxiliary network | |
| CN110335290B (en) | Twin candidate region generation network target tracking method based on attention mechanism | |
| CN110276316B (en) | Human body key point detection method based on deep learning | |
| CN111291714A (en) | Vehicle detection method based on monocular vision and laser radar fusion | |
| CN111753677B (en) | Multi-angle remote sensing ship image target detection method based on characteristic pyramid structure | |
| CN112184577A (en) | Single image defogging method based on multi-scale self-attention generation countermeasure network | |
| CN112991350B (en) | RGB-T image semantic segmentation method based on modal difference reduction | |
| CN112767418B (en) | Mirror image segmentation method based on depth perception | |
| CN112950645A (en) | Image semantic segmentation method based on multitask deep learning | |
| CN113139550B (en) | Remote sensing image coastline extraction method based on deep semantic segmentation network | |
| CN115272529B (en) | Layout-first multi-scale decoupling ocean remote sensing image coloring method and system | |
| CN113379707A (en) | RGB-D significance detection method based on dynamic filtering decoupling convolution network | |
| CN110633727A (en) | Deep neural network ship target fine-grained identification method based on selective search | |
| WO2022109922A1 (en) | Image matting implementation method and apparatus, and device and storage medium | |
| CN115482518A (en) | Extensible multitask visual perception method for traffic scene | |
| CN111899203A (en) | Real image generation method based on label graph under unsupervised training and storage medium | |
| CN115908772A (en) | Target detection method and system based on Transformer and fusion attention mechanism | |
| Zhao et al. | CRAS-YOLO: A novel multi-category vessel detection and classification model based on YOLOv5s algorithm | |
| CN113297961A (en) | Target tracking method based on boundary feature fusion twin circulation neural network | |
| CN111723632A (en) | Ship tracking method and system based on twin network | |
| Fernandez-Labrador et al. | Panoroom: From the sphere to the 3d layout | |
| CN115049921A (en) | Method for detecting salient target of optical remote sensing image based on Transformer boundary sensing | |
| CN113313668B (en) | Subway tunnel surface disease feature extraction method | |
| CN112801029B (en) | Attention mechanism-based multitask learning method | |
| CN111476226B (en) | Text positioning method and device and model training method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |