WO2020238558A1 - 一种图像超分辨方法和*** - Google Patents

一种图像超分辨方法和*** Download PDF

Info

Publication number
WO2020238558A1
WO2020238558A1 PCT/CN2020/088215 CN2020088215W WO2020238558A1 WO 2020238558 A1 WO2020238558 A1 WO 2020238558A1 CN 2020088215 W CN2020088215 W CN 2020088215W WO 2020238558 A1 WO2020238558 A1 WO 2020238558A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
module
neural network
convolutional neural
feature
Prior art date
Application number
PCT/CN2020/088215
Other languages
English (en)
French (fr)
Inventor
夏树涛
戴涛
李清
林栋�
汪漪
Original Assignee
鹏城实验室
清华大学深圳国际研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 鹏城实验室, 清华大学深圳国际研究生院 filed Critical 鹏城实验室
Publication of WO2020238558A1 publication Critical patent/WO2020238558A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the field of artificial intelligence technology, in particular to an image super-resolution method and system.
  • Image super-resolution is a very important preprocessing link in computer vision systems such as video surveillance, logistics, and face recognition.
  • Image super-resolution can reconstruct super-resolution images from low-resolution images to meet user browsing requirements. Thanks to the powerful nonlinear expression capabilities of Convolutional Neural Networks (CNN), the super-resolution model of Convolutional Neural Networks can be used to achieve super-resolution processing of low-resolution images to obtain super-resolution images.
  • CNN Convolutional Neural Networks
  • the super-resolution processing performance of the existing convolutional neural network super-division model is often only related to the depth or width of the convolutional neural network structure.
  • the super-resolution processing performance improvement effect brought by increasing the depth or width of the convolutional neural network structure is very limited, and cannot meet the high standard requirements of users for super-resolution images. Therefore, the super-resolution images generated by the existing convolutional neural network super-division model still have many shortcomings.
  • the embodiments of the present invention provide an image super-resolution method and system to solve the problem that the image details of the super-resolution image generated through the existing convolutional neural network super-resolution model are not obvious.
  • the first aspect of the embodiments of the present invention discloses an image super-resolution method.
  • the image super-resolution method includes:
  • the image to be processed is used as the input of the pre-built convolutional neural network super-division model
  • the convolutional neural network super-division model is composed of four successively connected execution modules, and the second execution module is superimposed and embedded in the second-order channel attention
  • the residual module composition of the module
  • the image to be processed is processed through the first execution module in the convolutional neural network super-division model, and the first processed image is obtained as the input of the second execution module.
  • the size of the image to be processed is the same;
  • the second processed image is processed to obtain a third processed image whose size meets the preset scale factor, and the The third processed image is used as the input of the fourth execution module in the convolutional neural network super-division model;
  • the pre-built process of the convolutional neural network super-division model includes:
  • the training set including a low-resolution image and a high-resolution image corresponding to the low-resolution image
  • the preset loss function and optimization algorithm are used to train the preset convolutional neural network model until The preset convolutional neural network model outputs a high-resolution image corresponding to the low-resolution image, and it is determined that the currently trained convolutional neural network model is a convolutional neural network super-division model;
  • the convolutional neural network super-division model is composed of four execution modules connected in sequence, and the second execution module is composed of a residual module superimposed and embedded in a second-order channel attention module.
  • the process of superimposing and embedding the residual module of the second-order channel attention module to form the second execution module includes:
  • embedding the second-order channel attention module into the residual module to obtain the residual module with weighted features includes:
  • the first convolutional layer, the activation layer, the second convolutional layer, the first residual unit, the second-order channel attention module, and the second residual unit are sequentially connected in order to obtain the residual module with weighted features.
  • the process of presetting the second-order channel attention module includes:
  • mapping processing on the first feature according to the matrix reorganization method to obtain the second feature Performing mapping processing on the first feature according to the matrix reorganization method to obtain the second feature
  • Dimensionality reduction learning and dimensionality increase learning are sequentially performed on the row mean vector based on the depth dimension to obtain the first weight
  • the second-order channel attention module is constructed using the weighted features.
  • the second aspect of the embodiments of the present invention discloses an image super-resolution system, which includes:
  • the input unit is used to use the image to be processed as the input of the pre-built convolutional neural network super-division model
  • the convolutional neural network super-division model is composed of four sequentially connected execution modules, and the second execution module is superimposed and embedded The residual module composition of the second-order channel attention module;
  • the first execution unit is configured to process the to-be-processed image via the first execution module in the convolutional neural network super-division model, and obtain the first processed image as the input of the second execution module.
  • a size of the processed image is the same as the size of the image to be processed;
  • the second execution unit is configured to use the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the first in the convolutional neural network super-division model 3. Input of execution module;
  • the third execution unit is configured to process the second processed image based on the third execution module and the preset scale factor of the preset input image and output image size to obtain a second processed image whose size meets the preset scale factor Three processing images, using the third processing image as the input of the fourth execution module in the convolutional neural network super-division model;
  • the fourth execution unit is configured to perform mapping processing on the third processed image via the fourth execution module, and output a super-resolution image corresponding to the image to be processed.
  • the input unit includes:
  • a constructing subunit for constructing a training set including a low-resolution image and a high-resolution image corresponding to the low-resolution image;
  • the processing subunit is used to input the low-resolution image into a preset convolutional neural network model for feature extraction, feature amplification, and feature mapping, to obtain a processed image;
  • the training subunit is used to train the preset volume based on the low-resolution image, the high-resolution image corresponding to the low-resolution image, and the processed image, using a preset loss function and optimization algorithm Convolutional neural network model until the preset convolutional neural network model outputs a high-resolution image corresponding to the low-resolution image, and determining that the currently trained convolutional neural network model is a convolutional neural network superdivision model;
  • the convolutional neural network super-division model is composed of four execution modules connected in sequence, and the second execution module is composed of a residual module superimposed and embedded in a second-order channel attention module.
  • the above image super-resolution system it further includes:
  • the first construction unit is used to embed the preset second-order channel attention module into the residual module to obtain the residual module with weighted characteristics, and determine the weighted characteristic of the residual module required to construct the second execution module
  • the number of modules is sequentially stacked to obtain the second execution module.
  • the second-order channel attention module is embedded in the residual module to obtain the first building unit of the residual module with weighted characteristics, which is specifically configured to sequentially connect the first The convolutional layer, the activation layer, the second convolutional layer, the second-order channel attention module, and the residual are used to obtain the residual module with weighted features.
  • the above-mentioned image super-resolution system further includes: a second construction unit, and the second construction unit includes:
  • the acquiring subunit is configured to acquire the first feature of any layer in the convolutional layer of the convolutional neural network based on the first processed image as input;
  • the mapping subunit is used to perform mapping processing on the first feature according to the matrix reorganization method to obtain the second feature;
  • the feature calculation subunit is configured to calculate a sample variance matrix based on the transposition of the first feature and the second feature;
  • the matrix normalization subunit is used to normalize the sample variance matrix to obtain a covariance matrix
  • the matrix calculation subunit is configured to calculate a row mean vector based on the depth dimension based on the covariance matrix
  • the learning subunit is used to sequentially perform dimensionality reduction learning and dimensionality increase learning on the row mean vector based on the depth dimension to obtain the first weight;
  • the weight normalization subunit is used to normalize the first weight to obtain the second weight
  • the feature weighting subunit is used to obtain a weighted feature based on the first feature and the second weight;
  • the construction subunit is used to construct a second-order channel attention module using the weighted feature.
  • the image to be processed is used as the input of the pre-built convolutional neural network super-division model, and the convolutional neural network super-division model consists of four sequentially connected
  • the second execution module is composed of a residual module that is sequentially superimposed and embedded in a second-order channel attention module; the first execution module in the convolutional neural network super-division model processes the image to be processed, Obtain a first processed image as the input of the second execution module; use the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the convolution
  • the processed image is subjected to mapping processing, and a super-resolution image corresponding to the image to be processed is output.
  • the convolutional neural network super-division model sets weighted features for the image to be processed, and determines important features in the image to be processed by learning the weighted features, and performs super-resolution based on the important features Processing, thereby improving the feature expression ability of the super-division model of the convolutional neural network, so that the detail quality of the super-resolution image obtained after the super-resolution processing is greatly improved.
  • FIG. 1 is a schematic flowchart of an image super-resolution method provided by an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a method for constructing a convolutional neural network super-division model provided by an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for constructing a second execution module according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a residual module with weighted features provided by an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of a method for constructing a second-order channel attention module according to an embodiment of the present invention
  • FIG. 6 is a schematic structural diagram of an image super-resolution system provided by an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of another image super-resolution system provided by an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of another image super-resolution system provided by an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of another image super-resolution system provided by an embodiment of the present invention.
  • the terms “include”, “include” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes no Other elements clearly listed, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence “including a" does not exclude the existence of other same elements in the process, method, article, or equipment including the element.
  • FIG. 1 it is a schematic flowchart of an image super-resolution method provided by an embodiment of the present invention. The method includes the following steps:
  • S101 Use an image to be processed as an input of a pre-built convolutional neural network super-division model.
  • the convolutional neural network super-division model is composed of four execution modules connected in sequence, and the execution modules include a first execution module, a second execution module, a third execution module, and a fourth execution module.
  • the second execution module is composed of a residual module that is sequentially superimposed and embedded in a second-order channel attention module.
  • the second-order channel attention module utilizes the second-order statistical characteristics of the features of the input image in the convolutional neural network superdivision model, so that the convolutional neural network superdivision model adaptively learns features The importance of better focus on using useful features to improve the feature expression ability of convolutional neural networks, thereby improving the processing effect of image super-resolution.
  • S102 Process the image to be processed via the first execution module in the convolutional neural network super-division model, and obtain the first processed image as the input of the second execution module.
  • the size of the first processed image is the same as the size of the image to be processed.
  • the first execution module includes a convolution layer, and the first execution module performs a convolution operation on the image to be processed based on the convolution layer to obtain the first processed image.
  • S103 Use the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the input of the third execution module in the convolutional neural network super-division model.
  • the second execution module uses its own multiple residual modules with second-order channel attention modules to weight the features corresponding to the first processed image multiple times, thereby determining the first One processes the more important features in the image, and obtains a second processed image that contains weighted features.
  • the second processed image contains weighted features
  • the importance of the image to be processed is determined Features, and super-resolution processing is performed according to important features, thereby improving the feature expression ability of the convolutional neural network super-division model.
  • the size of the second processed image is the same as the size of the image to be processed.
  • S104 Process the second processed image based on the third execution module and the preset scale factors of the input image and output image that are preset to obtain a third processed image whose size meets the preset scale factor.
  • the fourth execution module in the convolutional neural network super division model.
  • the second processed image is input to the third execution module, and the third execution module amplifies the second processed image according to a preset scale factor, and sends it to the fourth execution module
  • the third processed image is output.
  • the third execution module includes a convolutional layer, and based on the convolutional layer, the third execution module performs feature amplification on the second processed image to obtain the first image whose size meets the preset scale factor. Three processing images.
  • the specific value of the preset scale factor can be set by a technician according to actual conditions.
  • S105 Perform mapping processing on the third processed image via the fourth execution module, and output a super-resolution image corresponding to the image to be processed.
  • the size of the super-resolution image is the same as the size of the image to be processed.
  • the fourth execution module includes a convolution layer, and the fourth execution module maps the third processed image based on the convolution layer to obtain and output the super-resolution image.
  • the image to be processed is used as the input of the pre-built convolutional neural network superdivision model.
  • the convolutional neural network superdivision model is composed of four execution modules connected in sequence, and the second execution module is The residual module of the second-order channel attention module is superimposed and embedded; the image to be processed is processed through the first execution module in the convolutional neural network super-division model to obtain the first processed image as the second Input of the execution module; using the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the third execution module of the convolutional neural network super-division model Based on the third execution module, and the preset scale factor of the input image and output image preset in advance, the second processed image is processed to obtain a third processed image whose size meets the preset scale factor, The third processed image is used as the input of the fourth execution module in the convolutional neural network super-division model; the third processed image is mapped through the fourth execution
  • the convolutional neural network super-division model sets weighted features for the image to be processed, and determines important features in the image to be processed by learning the weighted features, and performs super-resolution based on the important features Processing, thereby improving the feature expression ability of the super-division model of the convolutional neural network, so that the detail quality of the super-resolution image obtained after the super-resolution processing is greatly improved.
  • FIG. 2 is a schematic flowchart of a method for constructing a convolutional neural network super-division model provided by an embodiment of the present invention. The method includes the following steps:
  • the training set includes a low-resolution image and a high-resolution image corresponding to the low-resolution image.
  • S202 Input the low-resolution image into a preset convolutional neural network model to perform feature extraction, feature amplification, and feature mapping, to obtain a processed image.
  • S203 Based on the low-resolution image, the high-resolution image corresponding to the low-resolution image, and the processed image, use the preset loss function and optimization algorithm to train the preset convolutional neural network model until the preset convolution
  • the neural network model outputs a high-resolution image corresponding to the low-resolution image, and it is determined that the currently trained convolutional neural network model is a convolutional neural network superdivision model.
  • the convolutional neural network super-division model is composed of four execution modules connected in sequence, wherein the second execution module is composed of a residual module superimposed and embedded in a second-order channel attention module.
  • the loss function includes but is not limited to L1-norm
  • the optimization algorithm includes but is not limited to a stochastic gradient descent algorithm.
  • the training set includes a low-resolution image and a high-resolution image corresponding to the low-resolution image; the low-resolution image is input to a preset convolutional nerve
  • the network model performs feature extraction, feature magnification, and feature mapping to obtain processed images; based on low-resolution images, high-resolution images corresponding to low-resolution images, and processed images, using preset loss functions and
  • the optimization algorithm trains the preset convolutional neural network model until the preset convolutional neural network model outputs a high-resolution image corresponding to the low-resolution image, and determines that the currently trained convolutional neural network model is a convolutional neural network Network super-division model. Based on the embodiment of the present invention, a convolutional neural network superdivision model can be effectively constructed.
  • the second execution module is composed of a residual module superimposed and embedded in the second-order channel attention module.
  • FIG. 3 is an example provided by the embodiment of the present invention.
  • a schematic flow chart of a method for constructing a second execution module the method includes the following steps:
  • S301 Embedding a preset second-order channel attention module into the residual module to obtain a residual module with weighted features.
  • the residual module includes two convolutional layers, one activation layer and two residual units.
  • the first convolutional layer, the activation layer, the second convolutional layer, the first residual unit, the second-order channel attention module, and the second residual unit are sequentially connected in order to obtain the weighted feature Residual module.
  • the specific structure of the residual module with weighted features can refer to FIG. 4.
  • S302 Determine the number of residual modules with weighted features required to construct the second execution module.
  • the number of residual modules with weighted features required by the second execution module can be set by a technician according to actual conditions, and is not limited in the embodiment of the present invention.
  • a convolutional neural network structure can be constructed by stacking multiple residual modules.
  • the second execution module is the convolutional neural network structure. Since the convolutional neural network structure is formed by stacking each of the residual modules with weighted features, the convolutional neural network structure can set weighted features for the input image. It can be seen from this that the second execution module can generate a second processed image including a weighted feature based on the first processed image.
  • the second-order channel attention module is embedded in the residual module to obtain the residual module with weighted characteristics; the number of residual modules with weighted characteristics required to construct the second execution module is determined ; Sequentially stack each of the residual modules with weighted features to obtain the second execution module. Based on the embodiment of the present invention, a second execution module with a second-order channel attention mechanism can be effectively constructed.
  • the process of the preset second-order channel attention module is in specific implementation.
  • FIG. 5 it is a method for constructing second-order channel attention provided by the embodiment of the present invention. Schematic diagram of the flow of the module method, the method includes the following steps:
  • S501 Obtain the first feature of any layer in the convolutional layer of the convolutional neural network based on the first processed image as the input.
  • the first feature is specifically a H ⁇ W ⁇ C feature map, and the H ⁇ W ⁇ C feature map is marked as x, where H is the height of the convolutional layer, and W is the convolution The width of the layer, C is the depth of the convolutional layer.
  • S502 Perform mapping processing on the first feature according to the matrix reorganization method to obtain the second feature.
  • the H ⁇ W ⁇ C feature map x is mapped to the (H*W) ⁇ C feature X according to the matrix reorganization method, and X is output, and X is the second feature.
  • S503 Calculate the sample variance matrix based on the transposition of the first feature and the second feature.
  • the sample variance matrix ⁇ is calculated according to formula (1).
  • I refers to the identity matrix of (H*W) ⁇ (H*W), and 1 refers to a matrix with all 1 elements.
  • ⁇ diag( ⁇ 1 ,..., ⁇ C )
  • U refers to an orthogonal matrix
  • ⁇ refers to a diagonal matrix whose elements are eigenvalues ⁇ i , and each eigenvalue in the diagonal matrix ⁇ i is sorted in descending order
  • ⁇ is a positive integer
  • i refers to the number of columns of the diagonal matrix.
  • the mean value of the elements in the jth row of the covariance matrix Y is calculated according to formula (3), and the C-dimensional row mean vector is determined according to the mean value of the elements in the jth row.
  • j refers to the number of rows of the diagonal matrix.
  • S506 Perform dimensionality reduction learning and dimensionality increase learning sequentially on the row mean vector based on the depth dimension to obtain the first weight.
  • the C-dimensional row mean vector is used as the input of the preset first fully connected network, and 1 ⁇ 1 ⁇ C/r is output to obtain the output result m.
  • the output result m is used as the input of the preset second fully connected network, and 1 ⁇ 1 ⁇ C is output to obtain the first weight.
  • S507 Perform normalization processing on the first weight to obtain the second weight.
  • a Sigmoid function is used to perform normalization calculation on the first weight to obtain the second weight.
  • S508 Obtain a weighted feature based on the first feature and the second weight.
  • the first feature of any layer in the convolutional layer of the convolutional neural network is obtained based on the first processed image as input; the first feature is mapped according to the matrix reorganization method to obtain the second Feature; based on the first feature and the second feature, calculate the sample variance matrix; normalize the sample variance matrix to obtain the covariance matrix; based on the covariance matrix, calculate the row based on the depth dimension Mean vector; performing dimensionality reduction learning and dimensionalization learning on the row mean vector based on the depth dimension to obtain a first weight; normalizing the first weight to obtain a second weight; based on the first feature And the second weight to obtain a weighted feature, and use the weighted feature to construct a second-order channel attention module.
  • the second-order channel attention module can be effectively constructed.
  • the embodiment of the present invention also provides an image super-resolution system. As shown in FIG. 6, it is the image super-resolution system provided by the embodiment of the present invention. Structure diagram, the system includes:
  • the input unit 100 is configured to use the image to be processed as the input of the pre-built convolutional neural network super-division model, the convolutional neural network super-division model is composed of four sequentially connected execution modules, and the second execution module is superimposed and The residual module is composed of the second-order channel attention module embedded.
  • the first execution unit 200 is configured to process the image to be processed via the first execution module in the convolutional neural network super-division model to obtain the first processed image as the input of the second execution module, and
  • the size of the first processed image is the same as the size of the image to be processed.
  • the second execution unit 300 is configured to use the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the convolutional neural network super-division model The input of the third execution module.
  • the third execution unit 400 is configured to process the second processed image based on the third execution module and preset scale factors of the preset input image and output image to obtain a size meeting the preset scale factor
  • the third processed image is used as the input of the fourth execution module in the convolutional neural network super-division model.
  • the fourth execution unit 500 is configured to perform mapping processing on the third processed image via the fourth execution module, and output a super-resolution image corresponding to the image to be processed.
  • the image to be processed is used as the input of the pre-built convolutional neural network superdivision model.
  • the convolutional neural network superdivision model is composed of four execution modules connected in sequence, and the second execution module is The residual module of the second-order channel attention module is superimposed and embedded; the image to be processed is processed through the first execution module in the convolutional neural network super-division model to obtain the first processed image as the second Input to the execution module; use the second execution module to perform feature extraction and feature processing on the first processed image, and output a second processed image containing weighted features as the third execution module of the convolutional neural network super-division model Based on the third execution module, and the preset scale factor of the input image and output image preset in advance, the second processed image is processed to obtain a third processed image whose size meets the preset scale factor, The third processed image is used as the input of the fourth execution module in the convolutional neural network super-division model; the third processed image is mapped through the fourth execution
  • the convolutional neural network super-division model sets weighted features for the image to be processed, and determines important features in the image to be processed by learning the weighted features, and performs super-resolution based on the important features Processing, thereby improving the feature expression ability of the super-division model of the convolutional neural network, so that the detail quality of the super-resolution image obtained after the super-resolution processing is greatly improved.
  • FIG. 7 it is a schematic structural diagram of another image super-resolution system provided by an embodiment of the present invention, and the input unit 100 includes:
  • the construction subunit 101 is configured to construct a training set, the training set including a low-resolution image and a high-resolution image corresponding to the low-resolution image.
  • the processing subunit 102 is configured to input the low-resolution image into a preset convolutional neural network model for feature extraction, feature amplification, and feature mapping, to obtain a processed image.
  • the training subunit 103 is configured to train the preset loss function and optimization algorithm based on the low resolution image, the high resolution image corresponding to the low resolution image, and the processed image.
  • Convolutional neural network model until the preset convolutional neural network model outputs a high-resolution image corresponding to the low-resolution image, and it is determined that the currently trained convolutional neural network model is a convolutional neural network super-division model ;
  • the convolutional neural network super-division model is composed of four sequentially connected execution modules, and the second execution module is composed of a residual module superimposed and embedded in a second-order channel attention module.
  • the training set includes a low-resolution image and a high-resolution image corresponding to the low-resolution image; the low-resolution image is input to a preset convolutional nerve
  • the network model performs feature extraction, feature magnification, and feature mapping to obtain processed images; based on low-resolution images, high-resolution images corresponding to low-resolution images, and processed images, using preset loss functions and
  • the optimization algorithm trains the preset convolutional neural network model until the preset convolutional neural network model outputs a high-resolution image corresponding to the low-resolution image, and determines that the currently trained convolutional neural network model is a convolutional neural network Network super-division model. Based on the embodiment of the present invention, a convolutional neural network superdivision model can be effectively constructed.
  • FIG. 6 it is a schematic structural diagram of another image super-resolution system provided by an embodiment of the present invention, and the system further includes:
  • the first construction unit 600 is used to embed the second-order channel attention module into the residual module to obtain a residual module with weighted features, and determine the number of residual modules with weighted features required to construct the second execution module , Stacking each of the residual modules with weighted features in turn to obtain the second execution module.
  • the first construction unit 600 is specifically configured to sequentially connect the first convolutional layer, the activation layer, the second convolutional layer, the second-order channel attention module, and the residual in order to obtain the residual weighted feature. Poor module.
  • the second-order channel attention module is embedded in the residual module to obtain the residual module with weighted characteristics; the number of residual modules with weighted characteristics required to construct the second execution module is determined ; Sequentially stack each of the residual modules with weighted features to obtain the second execution module. Based on the embodiment of the present invention, a second execution module with a second-order channel attention mechanism can be effectively constructed.
  • FIG. 8 and FIG. 9 it is a schematic structural diagram of another image super-resolution system according to an embodiment of the present invention.
  • the system further includes: a second construction unit 700.
  • the second construction unit 700 includes:
  • the obtaining subunit 701 is configured to obtain the first feature of any layer in the convolutional layer of the convolutional neural network based on the first processed image as input.
  • the mapping subunit 702 is configured to perform a mapping process on the first feature according to a matrix reorganization method to obtain a second feature.
  • the feature calculation subunit 703 is configured to calculate a sample variance matrix based on the transposition of the first feature and the second feature.
  • the matrix normalization subunit 704 is configured to perform normalization processing on the sample variance matrix to obtain a covariance matrix.
  • the matrix calculation subunit 705 is configured to calculate a row mean vector based on the depth dimension based on the covariance matrix.
  • the learning subunit 706 is configured to sequentially perform dimensionality reduction learning and dimensionality increase learning on the row mean vector based on the depth dimension to obtain the first weight.
  • the weight normalization subunit 707 is configured to perform normalization processing on the first weight to obtain the second weight.
  • the feature weighting subunit 708 is configured to obtain weighted features based on the first feature and the second weight.
  • the construction subunit 709 is used to construct a second-order channel attention module using the weighted features.
  • the first feature of any layer in the convolutional layer of the convolutional neural network is obtained based on the first processed image as input; the first feature is mapped according to the matrix reorganization method to obtain the second Feature; based on the transposition of the first feature and the second feature, the sample variance matrix is calculated; the sample variance matrix is normalized to obtain the covariance matrix; based on the covariance matrix, the calculation is based on the depth Dimensional row mean vector; performing dimensionality reduction learning and dimensionalization learning on the depth dimension-based row mean vector to obtain a first weight; normalizing the first weight to obtain a second weight; based on the The first feature and the second weight are used to obtain a weighted feature, and the weighted feature is used to construct a second-order channel attention module. Based on the embodiment of the present invention, the second-order channel attention module can be effectively constructed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供一种图像超分辨方法和***，该方案包括：将待处理图像作为卷积神经网络超分模型的输入，卷积神经网络超分模型由四个依次连接的执行模块构成；第一执行模块对待处理图像进行处理，得到第一处理图像；第二执行模块对第一处理图像进行处理，输出包含第二处理图像；第三执行模块对第二处理图像进行处理，输出第三处理图像；第四执行模块对第三处理图像进行处理，输出超分辨率图像。基于本发明，卷积神经网络超分模型为待处理图像设置加权特征，通过对加权特征的学习，确定待处理图像中的重要特征，并依据重要特征进行超分辨处理，从而提高卷积神经网络超分模型的特征表达能力，使得超分辨处理后所得到的超分辨率图像的细节质量大大提高。

Description

一种图像超分辨方法和***
本申请要求于2019年05月24日提交中国专利局、申请号为201910439532.8、发明名称为“一种图像超分辨方法和***”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及人工智能技术领域，尤其涉及一种图像超分辨方法和***。
背景技术
图像超分辨是视频监控、物流、人脸识别等计算机视觉***中一个非常重要的预处理环节。图像超分辨能够从低分辨率图像中重建出超分辨率图像，以满足用户的浏览要求。得益于卷积神经网络(Convolutional Neural Networks,CNN)强大的非线性表达能力，可以利用卷积神经网络超分模型实现对低分辨率图像的超分辨处理，得到超分辨率图像。
目前，现有的卷积神经网络超分模型的超分辨处理性能，往往只与卷积神经网络结构的深度或宽度相关。然而，通过增加卷积神经网络结构的深度或宽度所带来的超分辨处理性能提升效果十分有限，无法满足用户对超分辨率图像的高标准要求。故而，现有的卷积神经网络超分模型生成的超分辨率图像仍然存在较多的不足之处。
发明内容
有鉴于此，本发明实施例提供一种图像超分辨方法和***，以解决经由现有的卷积神经网络超分模型生成的超分辨率图像的图像细节效果不明显的问题。
为实现上述目的，本发明实施例提供如下技术方案：
本发明实施例第一方面公开了一种图像超分辨方法，所述图像超分辨方法包括：
将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成；
经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入，所述第一处理图像的尺寸与所述待处理图像的尺寸相同；
利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型中的第三执行模块的输入；
基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；
经由第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。
可选的，在上述图像超分辨方法中，所述预先构建的卷积神经网络超分模型的过程，包括：
构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；
将所述低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；
基于所述低分辨率图像、与所述低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练所述预设的卷积神经网络模型，直至所述预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型；
其中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
可选的，在上述图像超分辨方法中，所述叠加且嵌入二阶通道注意力模块的残差模块构成第二执行模块的过程，包括：
将预设的二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块；
确定构建第二执行模块所需的具有加权特征的残差模块的个数；
依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。
可选的，在上述图像超分辨方法中，所述将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块，包括：
依次按照顺序连接第一卷积层、激活层、第二卷积层、第一残差单元、二阶通道注意力模块和第二残差单元，得到所述具有加权特征的残差模块。
可选的，在上述图像超分辨方法中，所述预设二阶通道注意力模块的过程，包括：
基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；
依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；
基于所述第一特征和第二特征的转置，计算得到样本方差矩阵；
对所述样本方差矩阵进行归一化处理，得到协方差矩阵；
基于所述协方差矩阵，计算得到基于深度维度的行均值向量；
对所述基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重；
对所述第一权重进行归一化处理，得到第二权重；
基于所述第一特征和第二权重，得到加权特征；
利用所述加权特征构建二阶通道注意力模块。
本发明实施例第二方面公开了一种图像超分辨***，所述图像超分辨***包括：
输入单元，用于将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成；
第一执行单元，用于经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入，所述第一处理图像的尺寸与所述待处理图像的尺寸相同；
第二执行单元，用于利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型中的第三执行模块的输入；
第三执行单元，用于基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；
第四执行单元，用于经由第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。
可选的，在上述图像超分辨***中，所述输入单元包括：
构建子单元，用于构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；
处理子单元，用于将所述低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；
训练子单元，用于基于所述低分辨率图像、与所述低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练所述预设的卷积神经网络模型，直至所述预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型；其中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
可选的，在上述图像超分辨***中，还包括：
第一构建单元，用于将预设的二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块，确定构建第二执行模块所需的具有加权特征的残差模块的个数，依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。
可选的，在上述图像超分辨***中，所述将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块的第一构建单元具体用于依次按照顺序连接第一卷积层、激活层、第二卷积层、二阶通道注意力模块和残差，得到所述具有加权特征的残差模块。
可选的，在上述图像超分辨***中，还包括：第二构建单元，所述第二构建单元包括：
获取子单元，用于基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；
映射子单元，用于依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；
特征计算子单元，用于基于所述第一特征和第二特征的转置，计算得到样本方差矩阵；
矩阵归一化子单元，用于对所述样本方差矩阵进行归一化处理，得到协方差矩阵；
矩阵计算子单元，用于基于所述协方差矩阵，计算得到基于深度维度的行均值向量；
学习子单元，用于对所述基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重；
权重归一化子单元，用于对所述第一权重进行归一化处理，得到第二权重；
特征加权子单元，用于基于所述第一特征和第二权重，得到加权特征；
构建子单元，用于利用所述加权特征构建二阶通道注意力模块。
基于上述本发明实施例提供的一种图像超分辨方法和***，将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由顺序叠加且嵌入二阶通道注意力模块的残差模块构成；经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入；利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型的第三执行模块的输入；基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行上采样处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；经由所述第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。基于本发明实施例，所述卷 积神经网络超分模型为待处理图像设置加权特征，通过对所述加权特征的学习，确定所述待处理图像中的重要特征，并依据重要特征进行超分辨处理，从而提高所述卷积神经网络超分模型的特征表达能力，使得超分辨处理后所得到的超分辨率图像的细节质量大大提高。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本发明实施例提供的一种图像超分辨方法的流程示意图；
图2为本发明实施例提供的一种构建卷积神经网络超分模型的方法的流程示意图；
图3为本发明实施例提供的一种构建第二执行模块的方法的流程示意图；
图4为本发明实施例提供的一种具有加权特征的残差模块的结构示意图；
图5为本发明实施例提供的一种构建二阶通道注意力模块方法的流程示意图；
图6为本发明实施例提供的一种图像超分辨***的结构示意图；
图7为本发明实施例提供的另一种图像超分辨***的结构示意图；
图8为本发明实施例提供的另一种图像超分辨***的结构示意图；
图9为本发明实施例提供的另一种图像超分辨***的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本申请中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排 他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
如图1所示，为本发明实施例提供的一种图像超分辨方法的流程示意图，所述方法包括如下步骤：
S101：将待处理图像作为预先构建的卷积神经网络超分模型的输入。
在S101中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，所述执行模块包括第一执行模块、第二执行模块、第三执行模块和第四执行模块。其中，所述第二执行模块由顺序叠加且嵌入二阶通道注意力模块的残差模块构成。
需要说明的是，所述二阶通道注意力模块利用所述卷积神经网络超分模型中输入图像的特征所具有的二阶统计特性，使得所述卷积神经网络超分模型自适应学习特征的重要性，更好的集中利用有用的特征来提高卷积神经网络的特征表达能力，从而提高图像超分辨的处理效果。
S102：经由卷积神经网络超分模型中的第一执行模块对待处理图像进行处理，得到第一处理图像作为第二执行模块的输入。
在S102中，所述第一处理图像的尺寸与所述待处理图像的尺寸相同。
需要说明的是，所述第一执行模块包括一卷积层，所述第一执行模块基于所述卷积层，对所述待处理图像进行卷积运算，得到所述第一处理图像。
S103：利用第二执行模块对第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为卷积神经网络超分模型中的第三执行模块的输入。
在S103中，所述第二执行模块通过自身所有的多个的具有二阶通道注意力模块的残差模块，对所述第一处理图像所对应的特征进行多次加权，从而确定所述第一处理图像中重要性较高的特征，并得到包含加权特征的第二处理图像。
需要说明的是，因所述第二处理图像包含加权特征，在后续的第三执行模 块和第四执行模块处理过程中，通过对所述加权特征的学习，确定所述待处理图像中的重要特征，并依据重要特征进行超分辨处理，从而提高所述卷积神经网络超分模型的特征表达能力。
需要说明得是，所述第二处理图像的尺寸与所述待处理图像的尺寸相同。
S104：基于第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将第三处理图像作为卷积神经网络超分模型中的第四执行模块的输入。
在S104中，将所述第二处理图像输入到所述第三执行模块中，所述第三执行模块按照预设比例系数对所述第二处理图像进行放大，并向所述第四执行模块输出所述第三处理图像。
需要说明的是，所述第三执行模块包括一卷积层，所述第三执行模块基于所述卷积层，对所述第二处理图像进行特征放大，得到尺寸满足预设比例系数的第三处理图像。
需要说明的是，所述预设比例系数的具体数值可由技术人员根据实际情况进行设置。
S105：经由第四执行模块对第三处理图像进行映射处理，输出对应待处理图像的超分辨率图像。
在S105中，所述超分辨率图像的尺寸与所述待处理图像的尺寸相同。
需要说明的是，所述第四执行模块包括一卷积层，所述第四执行模块基于所述卷积层，对所述第三处理图像进行映射，得到并输出所述超分辨率图像。
在本发明实施例中，将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由顺序叠加且嵌入二阶通道注意力模块的残差模块构成；经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入；利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型的第三执行模块的输入；基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行 处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；经由所述第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。基于本发明实施例，所述卷积神经网络超分模型为待处理图像设置加权特征，通过对所述加权特征的学习，确定所述待处理图像中的重要特征，并依据重要特征进行超分辨处理，从而提高所述卷积神经网络超分模型的特征表达能力，使得超分辨处理后所得到的超分辨率图像的细节质量大大提高。
优选的，上述图1示出的卷积神经网络超分模型，在具体构建过程，参考图2，为本发明实施例提供的一种构建卷积神经网络超分模型的方法的流程示意图，所述方法包括如下步骤：
S201：构建训练集。
在S201中，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像。
S202：将低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像。
在S202中，在所述预设的卷积神经网络模型中，对所述低分辨率图像进行卷积运算；然后，通过卷积神经网络结构对经过卷积运算后的低分辨率图像进行特征提取；其次，通过上采样模型对所述低分辨率图像中的特征进行放大处理；最后，基于与所述低分辨率图像对应的高分辨率图像，对所述低分辨图像中的放大后的特征进行映射处理，得到所述处理后的图像。
S203：基于低分辨率图像、与低分辨率图像对应的高分辨图像、以及处理后的图像，利用预设的损失函数和优化算法训练预设的卷积神经网络模型，直至预设的卷积神经网络模型输出与低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型。
在S203中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，其中，所述第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
需要说明的是，在本发明实施例中，所述损失函数包括但不限于L1-范数，所述优化算法包括但不限于随机梯度下降算法。
在本发明实施例中，通过构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；将低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；基于低分辨率图像、与低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练预设的卷积神经网络模型，直至预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型。基于本发明实施例，能够有效构建卷积神经网络超分模型。
优选的，上述图1示出的，所述第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成，在具体构建过程中，参考图3，为本发明实施例提供的一种构建第二执行模块的方法的流程示意图，所述方法包括如下步骤：
S301：将预设的二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块。
在S301中，所述残差模块包括两个卷积层、一个激活层和两个残差单元。
优选的，依次按照顺序连接所述第一卷积层、激活层、第二卷积层、第一残差单元、二阶通道注意力模块和第二残差单元，得到所述具有加权特征的残差模块。
在具体实现中，所述具有加权特征的残差模块的具体结构可以参考图4。
S302：确定构建第二执行模块所需的具有加权特征的残差模块的个数。
在S302中，所述具有加权特征的残差模块的个数越多，则所述第二执行模块的处理效果越好。
需要说明的是，所述第二执行模块所需的具有加权特征的残差模块的个数可由技术人员根据实际情况进行设置，本发明实施例不做限定。
S303：依次堆叠各个具有加权特征的残差模块，得到第二执行模块。
在S303中，可以通过堆叠多个残差模块构建卷积神经网络结构，换而言之，所述第二执行模块即为所述卷积神经网络结构。因卷积神经网络结构由各个所述具有加权特征的残差模块堆叠而成，故而所述卷积神经网络结构可以为输入图像设置加权特征。由此可知，所述第二执行模块可以基于所述第一处理图像， 生成包含加权特征的第二处理图像。
在本发明实施例中，通过将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块；确定构建第二执行模块所需的具有加权特征的残差模块的个数；依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。基于本发明实施例，能够有效构建得到具有二阶通道注意力机制的第二执行模块。
优选的，上述图3示出的S301中，所述预设的二阶通道注意力模块这一过程在具体实现中，参考图5，为本发明实施例提供的一种构建二阶通道注意力模块方法的流程示意图，所述方法包括如下步骤：
S501：基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征。
在S501中，所述第一特征具体为H×W×C的特征图，并将所述H×W×C的特征图标记为x，其中，H为卷积层的高度，W为卷积层的宽度，C为卷积层的深度。
S502：依据矩阵重组方法对第一特征进行映射处理，得到第二特征。
在S502中，依据所述矩阵重组方法，将所述H×W×C的特征图x映射为(H*W)×C的特征X，并输出X，X即为所述第二特征。
S503：基于第一特征和第二特征的转置，计算得到样本方差矩阵。
在S503中，依据公式(1)计算得到样本方差矩阵Σ。
Figure PCTCN2020088215-appb-000001
其中，
Figure PCTCN2020088215-appb-000002
I指的是(H*W)×(H*W)的单位矩阵，1指的是元素全为1的矩阵。
S504：对样本方差矩阵进行归一化处理，得到协方差矩阵。
在S504中，依据公式(2)，对样本方差矩阵Σ进行归一化处理，得到协方差矩阵Y。
Y＝Σ 0.5＝UΛ 0.5U T        (2)
其中，Λ＝diag(λ 1,...,λ C)，U指的是正交矩阵，Λ指的是元素为特征值λ i的对角矩阵，所述对角矩阵中的各个特征值λ i按照降序的顺序进行排序，λ为正 整数，i指的是所述对角矩阵的列数。
S505：基于协方差矩阵，计算得到基于深度维度的行均值向量。
在S505中，依据公式(3)计算得到协方差矩阵Y的第j行元素均值，并依据所述第j行元素均值确定C维的行均值向量。
Figure PCTCN2020088215-appb-000003
其中，j指的是所述对角矩阵的行数。
S506：对基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重。
在S506中，将C维的行均值向量作为预设的第一全连接网络的输入，对1×1×C/r进行输出，得到输出结果m。将所述输出结果m作为预设的第二全连接网络的输入，对1×1×C进行输出，得到所述第一权重。
S507：对第一权重进行归一化处理，得到第二权重。
在S507中，利用Sigmoid函数对所述第一权重进行归一化计算，得到所述第二权重。
S508：基于第一特征和第二权重，得到加权特征。
在S508中，将所述第一特征和第二权重进行相乘，得到所述加权特征。
S509：利用加权特征构建二阶通道注意力模块。
在S509中，将所述加权特征嵌入到卷积神经网络，生成所述二阶通道注意力模块。
在本发明实施例中，基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；基于所述第一特征和第二特征，计算得到样本方差矩阵；对所述样本方差矩阵进行归一化处理，得到协方差矩阵；基于所述协方差矩阵，计算得到基于深度维度的行均值向量；对所述基于深度维度的行均值向量进行降维学习和升维学习，得到第一权重；对所述第一权重进行归一化处理，得到第二权重；基于所述第一特征和第二权重，得到加权特征，并利用所述加权特征构建二阶通道注意力模块。基于本发明实施例，能够有效构建二阶通道注意力模块。
基于上述本发明实施例提供的一种图像超分辨方法，本发明实施例还对应提供了一种图像超分辨***，如图6所示，为本发明实施例提供的一种图像超分辨***的结构示意图，所述***包括：
输入单元100，用于将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
第一执行单元200，用于经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入，所述第一处理图像的尺寸与所述待处理图像的尺寸相同。
第二执行单元300，用于利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型中的第三执行模块的输入。
第三执行单元400，用于基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入。
第四执行单元500，用于经由第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。
在本发明实施例中，将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由顺序叠加且嵌入二阶通道注意力模块的残差模块构成；经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入；利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型的第三执行模块的输入；基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；经由所述第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图 像。基于本发明实施例，所述卷积神经网络超分模型为待处理图像设置加权特征，通过对所述加权特征的学习，确定所述待处理图像中的重要特征，并依据重要特征进行超分辨处理，从而提高所述卷积神经网络超分模型的特征表达能力，使得超分辨处理后所得到的超分辨率图像的细节质量大大提高。
优选的，结合图6，参考图7，为本发明实施例提供的另一种图像超分辨***的结构示意图，所述输入单元100包括：
构建子单元101，用于构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像。
处理子单元102，用于将所述低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像。
训练子单元103，用于基于所述低分辨率图像、与所述低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练所述预设的卷积神经网络模型，直至所述预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型；其中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
在本发明实施例中，通过构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；将低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；基于低分辨率图像、与低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练预设的卷积神经网络模型，直至预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型。基于本发明实施例，能够有效构建卷积神经网络超分模型。
优选的，结合图6，参考图8，为本发明实施例提供的另一种图像超分辨***的结构示意图，所述***还包括：
第一构建单元600，用于将二阶通道注意力模块嵌入到残差模块中，得到 具有加权特征的残差模块，确定构建第二执行模块所需的具有加权特征的残差模块的个数，依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。
优选的，所述第一构建单元600具体用于依次按照顺序连接第一卷积层、激活层、第二卷积层、二阶通道注意力模块和残差，得到所述具有加权特征的残差模块。
在本发明实施例中，通过将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块；确定构建第二执行模块所需的具有加权特征的残差模块的个数；依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。基于本发明实施例，能够有效构建得到具有二阶通道注意力机制的第二执行模块。
优选的，结合图8，参考图9，为本发明实施例提供的另一种图像超分辨***的结构示意图，所述***还包括：第二构建单元700。
所述第二构建单元700包括：
获取子单元701，用于基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征。
映射子单元702，用于依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征。
特征计算子单元703，用于基于所述第一特征和第二特征的转置，计算得到样本方差矩阵。
矩阵归一化子单元704，用于对所述样本方差矩阵进行归一化处理，得到协方差矩阵。
矩阵计算子单元705，用于基于所述协方差矩阵，计算得到基于深度维度的行均值向量。
学习子单元706，用于对所述基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重。
权重归一化子单元707，用于对所述第一权重进行归一化处理，得到第二权重。
特征加权子单元708，用于基于所述第一特征和第二权重，得到加权特征。
构建子单元709，用于利用所述加权特征构建二阶通道注意力模块。
在本发明实施例中，基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；基于所述第一特征和第二特征的转置，计算得到样本方差矩阵；对所述样本方差矩阵进行归一化处理，得到协方差矩阵；基于所述协方差矩阵，计算得到基于深度维度的行均值向量；对所述基于深度维度的行均值向量进行降维学习和升维学习，得到第一权重；对所述第一权重进行归一化处理，得到第二权重；基于所述第一特征和第二权重，得到加权特征，并利用所述加权特征构建二阶通道注意力模块。基于本发明实施例，能够有效构建二阶通道注意力模块。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于***或***实施例而言，由于其基本相似于方法实施例，所以描述得比较简单，相关之处参见方法实施例的部分说明即可。以上所描述的***及***实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络模型单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
专业人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见 的，本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下，在其它实施例中实现。因此，本发明将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (10)

  1. 一种图像超分辨方法，其特征在于，所述方法包括：
    将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成；
    经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入，所述第一处理图像的尺寸与所述待处理图像的尺寸相同；
    利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型中的第三执行模块的输入；
    基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；
    经由第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。
  2. 根据权利要求1所述的方法，其特征在于，所述预先构建的卷积神经网络超分模型的过程，包括：
    构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；
    将所述低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；
    基于所述低分辨率图像、与所述低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练所述预设的卷积神经网络模型，直至所述预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型；
    其中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
  3. 根据权利要求1所述的方法，其特征在于，所述叠加且嵌入二阶通道注意力模块的残差模块构成第二执行模块的过程，包括：
    将预设的二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块；
    确定构建第二执行模块所需的具有加权特征的残差模块的个数；
    依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。
  4. 根据权利要求3所述的方法，其特征在于，所述将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块，包括：
    依次按照顺序连接第一卷积层、激活层、第二卷积层、第一残差单元、二阶通道注意力模块和第二残差单元，得到所述具有加权特征的残差模块。
  5. 根据权利要求4所述的方法，其特征在于，所述预设二阶通道注意力模块的过程，包括：
    基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；
    依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；
    基于所述第一特征和第二特征的转置，计算得到样本方差矩阵；
    对所述样本方差矩阵进行归一化处理，得到协方差矩阵；
    基于所述协方差矩阵，计算得到基于深度维度的行均值向量；
    对所述基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重；
    对所述第一权重进行归一化处理，得到第二权重；
    基于所述第一特征和第二权重，得到加权特征；
    利用所述加权特征构建二阶通道注意力模块。
  6. 一种图像超分辨***，其特征在于，包括：
    输入单元，用于将待处理图像作为预先构建的卷积神经网络超分模型的输入，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成；
    第一执行单元，用于经由所述卷积神经网络超分模型中的第一执行模块对所述待处理图像进行处理，得到第一处理图像作为所述第二执行模块的输入，所述第一处理图像的尺寸与所述待处理图像的尺寸相同；
    第二执行单元，用于利用所述第二执行模块对所述第一处理图像进行特征提取和特征处理，输出包含加权特征的第二处理图像作为所述卷积神经网络超分模型中的第三执行模块的输入；
    第三执行单元，用于基于所述第三执行模块，以及预先设置的输入图像和输出图像的尺寸预设比例系数，对所述第二处理图像进行处理，得到尺寸满足预设比例系数的第三处理图像，将所述第三处理图像作为所述卷积神经网络超分模型中的第四执行模块的输入；
    第四执行单元，用于经由第四执行模块对所述第三处理图像进行映射处理，输出对应所述待处理图像的超分辨率图像。
  7. 根据权利要求6所述的***，其特征在于，所述输入单元包括：
    构建子单元，用于构建训练集，所述训练集包括低分辨率图像、以及与所述低分辨率图像对应的高分辨率图像；
    处理子单元，用于将所述低分辨率图像输入到预设的卷积神经网络模型中进行特征提取、特征放大以及特征映射，得到处理后的图像；
    训练子单元，用于基于所述低分辨率图像、与所述低分辨率图像对应的高分辨率图像、以及处理后的图像，利用预设的损失函数和优化算法训练所述预设的卷积神经网络模型，直至所述预设的卷积神经网络模型输出与所述低分辨率图像对应的高分辨率图像，确定当前训练得到的卷积神经网络模型为卷积神经网络超分模型；其中，所述卷积神经网络超分模型由四个依次连接的执行模块构成，第二执行模块由叠加且嵌入二阶通道注意力模块的残差模块构成。
  8. 根据权利要求6所述的***，其特征在于，还包括：
    第一构建单元，用于将预设的二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块，确定构建第二执行模块所需的具有加权特征的残差模块的个数，依次堆叠各个所述具有加权特征的残差模块，得到所述第二执行模块。
  9. 根据权利要求8所述的***，其特征在于，所述将二阶通道注意力模块嵌入到残差模块中，得到具有加权特征的残差模块的第一构建单元具体用于依次按照顺序连接第一卷积层、激活层、第二卷积层、二阶通道注意力模块和残差，得到所述具有加权特征的残差模块。
  10. 根据权利要求9所述的***，其特征在于，还包括：第二构建单元，所述第二构建单元包括：
    获取子单元，用于基于作为输入的第一处理图像，获取卷积神经网络卷积层中任一层的第一特征；
    映射子单元，用于依据矩阵重组方法对所述第一特征进行映射处理，得到第二特征；
    特征计算子单元，用于基于所述第一特征和第二特征的转置，计算得到样本方差矩阵；
    矩阵归一化子单元，用于对所述样本方差矩阵进行归一化处理，得到协方差矩阵；
    矩阵计算子单元，用于基于所述协方差矩阵，计算得到基于深度维度的行均值向量；
    学习子单元，用于对所述基于深度维度的行均值向量依次进行降维学习和升维学习，得到第一权重；
    权重归一化子单元，用于对所述第一权重进行归一化处理，得到第二权重；
    特征加权子单元，用于基于所述第一特征和第二权重，得到加权特征；
    构建子单元，用于利用所述加权特征构建二阶通道注意力模块。
PCT/CN2020/088215 2019-05-24 2020-04-30 一种图像超分辨方法和*** WO2020238558A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910439532.8 2019-05-24
CN201910439532.8A CN110175953B (zh) 2019-05-24 2019-05-24 一种图像超分辨方法和***

Publications (1)

Publication Number Publication Date
WO2020238558A1 true WO2020238558A1 (zh) 2020-12-03

Family

ID=67692105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/088215 WO2020238558A1 (zh) 2019-05-24 2020-04-30 一种图像超分辨方法和***

Country Status (2)

Country Link
CN (1) CN110175953B (zh)
WO (1) WO2020238558A1 (zh)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634238A (zh) * 2020-12-25 2021-04-09 武汉大学 一种基于注意力模块的图像质量评价方法
CN112991173A (zh) * 2021-03-12 2021-06-18 西安电子科技大学 基于双通道特征迁移网络的单帧图像超分辨率重建方法
CN113538616A (zh) * 2021-07-09 2021-10-22 浙江理工大学 一种联合PUGAN与改进U-net的磁共振图像重构方法
CN113538231A (zh) * 2021-06-17 2021-10-22 杭州电子科技大学 一种基于像素分布估计的单张图像超分辨重建***及方法
CN113538244A (zh) * 2021-07-23 2021-10-22 西安电子科技大学 一种基于自适应权重学习的轻量化超分辨率重建方法
CN113610706A (zh) * 2021-07-19 2021-11-05 河南大学 基于卷积神经网络的模糊监控图像超分辨率重建方法
CN113706388A (zh) * 2021-09-24 2021-11-26 上海壁仞智能科技有限公司 图像超分辨率重建方法及装置
CN113837941A (zh) * 2021-09-24 2021-12-24 北京奇艺世纪科技有限公司 图像超分模型的训练方法、装置及计算机可读存储介质
CN113947814A (zh) * 2021-10-28 2022-01-18 山东大学 一种基于时空信息增强和多尺度显著性特征提取的跨视角步态识别方法
CN114022348A (zh) * 2021-09-30 2022-02-08 浪潮(北京)电子信息产业有限公司 一种图像的超分辨率方法、***及相关装置
CN114022355A (zh) * 2021-09-26 2022-02-08 陕西师范大学 基于递归注意力机制的图像超分辨率方法
CN114138919A (zh) * 2021-11-30 2022-03-04 河北工业大学 一种基于非局部注意力卷积神经网络的地震数据重建方法
CN114170079A (zh) * 2021-11-19 2022-03-11 天津大学 基于引导注意力机制的深度图超分辨方法
CN115082307A (zh) * 2022-05-14 2022-09-20 西北工业大学深圳研究院 一种基于分数阶微分方程的图像超分辨率方法
CN115100042A (zh) * 2022-07-20 2022-09-23 北京工商大学 一种基于通道注意力滞留网络的病理图像超分辨率方法
CN115564649A (zh) * 2022-09-27 2023-01-03 苏州大学 一种图像超分辨率重建方法、装置及设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110175953B (zh) * 2019-05-24 2023-04-18 鹏城实验室 一种图像超分辨方法和***
CN112767427A (zh) * 2021-01-19 2021-05-07 西安邮电大学 一种补偿边缘信息的低分辨率图像识别算法
CN116775938B (zh) * 2023-08-15 2024-05-17 腾讯科技（深圳）有限公司 解说视频检索方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180075581A1 (en) * 2016-09-15 2018-03-15 Twitter, Inc. Super resolution using a generative adversarial network
CN108765296A (zh) * 2018-06-12 2018-11-06 桂林电子科技大学 一种基于递归残差注意力网络的图像超分辨率重建方法
CN109584161A (zh) * 2018-11-29 2019-04-05 四川大学 基于通道注意力的卷积神经网络的遥感图像超分辨率重建方法
CN110175953A (zh) * 2019-05-24 2019-08-27 鹏城实验室 一种图像超分辨方法和***

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007316161A (ja) * 2006-05-23 2007-12-06 Matsushita Electric Ind Co Ltd 残差補間を用いた超解像処理方法及び装置
CN108734660A (zh) * 2018-05-25 2018-11-02 上海通途半导体科技有限公司 一种基于深度学习的图像超分辨率重建方法及装置
CN108921786B (zh) * 2018-06-14 2022-06-28 天津大学 基于残差卷积神经网络的图像超分辨率重构方法
CN108960261B (zh) * 2018-07-25 2021-09-24 扬州万方电子技术有限责任公司 一种基于注意力机制的显著物体检测方法
CN109741260B (zh) * 2018-12-29 2023-05-12 天津大学 一种基于深度反投影网络的高效超分辨率方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180075581A1 (en) * 2016-09-15 2018-03-15 Twitter, Inc. Super resolution using a generative adversarial network
CN108765296A (zh) * 2018-06-12 2018-11-06 桂林电子科技大学 一种基于递归残差注意力网络的图像超分辨率重建方法
CN109584161A (zh) * 2018-11-29 2019-04-05 四川大学 基于通道注意力的卷积神经网络的遥感图像超分辨率重建方法
CN110175953A (zh) * 2019-05-24 2019-08-27 鹏城实验室 一种图像超分辨方法和***

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LU, YUE ET AL.: "Channel Attention and Multi-level Features Fusion for Single Image Super-Resolution", 2018 IEEE INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (IEEE VCIP), 9 December 2018 (2018-12-09), XP033541890, ISSN: 1018-8770, DOI: 20200717151947Y *

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634238A (zh) * 2020-12-25 2021-04-09 武汉大学 一种基于注意力模块的图像质量评价方法
CN112634238B (zh) * 2020-12-25 2024-03-08 武汉大学 一种基于注意力模块的图像质量评价方法
CN112991173A (zh) * 2021-03-12 2021-06-18 西安电子科技大学 基于双通道特征迁移网络的单帧图像超分辨率重建方法
CN112991173B (zh) * 2021-03-12 2024-04-16 西安电子科技大学 基于双通道特征迁移网络的单帧图像超分辨率重建方法
CN113538231A (zh) * 2021-06-17 2021-10-22 杭州电子科技大学 一种基于像素分布估计的单张图像超分辨重建***及方法
CN113538231B (zh) * 2021-06-17 2024-04-02 杭州电子科技大学 一种基于像素分布估计的单张图像超分辨重建***及方法
CN113538616B (zh) * 2021-07-09 2023-08-18 浙江理工大学 一种联合PUGAN与改进U-net的磁共振图像重构方法
CN113538616A (zh) * 2021-07-09 2021-10-22 浙江理工大学 一种联合PUGAN与改进U-net的磁共振图像重构方法
CN113610706A (zh) * 2021-07-19 2021-11-05 河南大学 基于卷积神经网络的模糊监控图像超分辨率重建方法
CN113538244A (zh) * 2021-07-23 2021-10-22 西安电子科技大学 一种基于自适应权重学习的轻量化超分辨率重建方法
CN113538244B (zh) * 2021-07-23 2023-09-01 西安电子科技大学 一种基于自适应权重学习的轻量化超分辨率重建方法
CN113706388A (zh) * 2021-09-24 2021-11-26 上海壁仞智能科技有限公司 图像超分辨率重建方法及装置
CN113837941A (zh) * 2021-09-24 2021-12-24 北京奇艺世纪科技有限公司 图像超分模型的训练方法、装置及计算机可读存储介质
CN113837941B (zh) * 2021-09-24 2023-09-01 北京奇艺世纪科技有限公司 图像超分模型的训练方法、装置及计算机可读存储介质
CN114022355A (zh) * 2021-09-26 2022-02-08 陕西师范大学 基于递归注意力机制的图像超分辨率方法
CN114022355B (zh) * 2021-09-26 2024-02-20 陕西师范大学 基于递归注意力机制的图像超分辨率方法
CN114022348A (zh) * 2021-09-30 2022-02-08 浪潮(北京)电子信息产业有限公司 一种图像的超分辨率方法、***及相关装置
CN113947814B (zh) * 2021-10-28 2024-05-28 山东大学 一种基于时空信息增强和多尺度显著性特征提取的跨视角步态识别方法
CN113947814A (zh) * 2021-10-28 2022-01-18 山东大学 一种基于时空信息增强和多尺度显著性特征提取的跨视角步态识别方法
CN114170079A (zh) * 2021-11-19 2022-03-11 天津大学 基于引导注意力机制的深度图超分辨方法
CN114138919A (zh) * 2021-11-30 2022-03-04 河北工业大学 一种基于非局部注意力卷积神经网络的地震数据重建方法
CN115082307B (zh) * 2022-05-14 2024-03-19 西北工业大学深圳研究院 一种基于分数阶微分方程的图像超分辨率方法
CN115082307A (zh) * 2022-05-14 2022-09-20 西北工业大学深圳研究院 一种基于分数阶微分方程的图像超分辨率方法
CN115100042A (zh) * 2022-07-20 2022-09-23 北京工商大学 一种基于通道注意力滞留网络的病理图像超分辨率方法
CN115100042B (zh) * 2022-07-20 2024-05-03 北京工商大学 一种基于通道注意力滞留网络的病理图像超分辨率方法
CN115564649A (zh) * 2022-09-27 2023-01-03 苏州大学 一种图像超分辨率重建方法、装置及设备

Also Published As

Publication number Publication date
CN110175953B (zh) 2023-04-18
CN110175953A (zh) 2019-08-27

Similar Documents

Publication Publication Date Title
WO2020238558A1 (zh) 一种图像超分辨方法和***
Qian et al. Deep learning for steganalysis via convolutional neural networks
WO2018205676A1 (zh) 用于卷积神经网络的处理方法和***、和存储介质
CN110782395B (zh) 图像处理方法及装置、电子设备和计算机可读存储介质
US10769757B2 (en) Image processing apparatuses and methods, image processing systems and training methods
CN108763191B (zh) 一种文本摘要生成方法及***
CN106780662B (zh) 人脸图像生成方法、装置及设备
CN108573243A (zh) 一种基于深度卷积神经网络的低质量人脸的比对方法
US11216913B2 (en) Convolutional neural network processor, image processing method and electronic device
JP2019008383A (ja) 画像処理装置、撮像装置、画像処理方法、プログラム、および、記憶媒体
CN113870422B (zh) 一种点云重建方法、装置、设备及介质
CN111340077B (zh) 基于注意力机制的视差图获取方法和装置
CN111325660B (zh) 一种基于文本数据的遥感图像风格转换方法
WO2020253304A1 (zh) 人脸识别装置及图像处理方法、特征提取模型、存储介质
CN115019143A (zh) 一种基于CNN和Transformer混合模型的文本检测方法
Vishwakarma Deterministic learning machine for face recognition with multi-model feature extraction
CN113111970A (zh) 通过构建全局嵌入式注意力残差网络对图像分类的方法
CN111144407A (zh) 一种目标检测方法、***、装置及可读存储介质
CN106909944B (zh) 一种人脸图片聚类的方法
JP2019197445A (ja) 画像認識装置、画像認識方法、およびプログラム
CN115017900B (zh) 一种基于多模态多去偏见的对话情感识别方法
CN112634136B (zh) 一种基于图像特征快速拼接的图像超分辨率方法及其***
US20210142081A1 (en) Image recognition method and device
CN110427892B (zh) 基于深浅层自相关融合的cnn人脸表情特征点定位方法
CN114782779B (zh) 基于特征分布迁移的小样本图像特征学习方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20813740

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20813740

Country of ref document: EP

Kind code of ref document: A1