CN113763248A - Super-resolution image reconstruction method, device, equipment and storage medium - Google Patents
Super-resolution image reconstruction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113763248A CN113763248A CN202111058591.4A CN202111058591A CN113763248A CN 113763248 A CN113763248 A CN 113763248A CN 202111058591 A CN202111058591 A CN 202111058591A CN 113763248 A CN113763248 A CN 113763248A
- Authority
- CN
- China
- Prior art keywords
- resolution
- resolution image
- preset
- training
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 208
- 238000013507 mapping Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 31
- 239000011159 matrix material Substances 0.000 claims description 162
- 238000004590 computer program Methods 0.000 claims description 8
- 239000002699 waste material Substances 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application discloses a super-resolution image reconstruction method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a low-resolution layer to be processed, and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model; performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed; the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer. In the application, only one high-resolution base map corresponding to the corresponding low-resolution map layer is needed, even though the high-resolution visible light image can accurately obtain the preset hyper-resolution image reconstruction model, and resource waste caused by artificial marking is avoided.
Description
Technical Field
The present application relates to the field of artificial intelligence technology of financial technology (Fintech), and in particular, to a super-resolution image reconstruction method, apparatus, device, and storage medium.
Background
With the continuous development of financial science and technology, especially internet science and technology, more and more technologies are applied to the financial field, but the financial industry also puts higher requirements on the technologies, for example, the financial industry also has higher requirements on super-resolution image reconstruction.
At present, a low-resolution image is often mapped to a high-resolution image (i.e., hyper-resolution) based on a deep learning network, and the image is analyzed based on the high-resolution image to further obtain needed sydney. However, the labeling cost of the method is high, and the resource consumption is high.
Disclosure of Invention
The application mainly aims to provide a super-resolution image reconstruction method, a super-resolution image reconstruction device, super-resolution image reconstruction equipment and a storage medium, and aims to solve the technical problem that resources are consumed much when a scheme based on supervised learning is used for carrying out super-resolution reconstruction on a low-resolution image in the prior art.
In order to achieve the above object, the present application provides a super-resolution image reconstruction method applied to a super-resolution image reconstruction apparatus, the super-resolution image reconstruction method including:
acquiring a low-resolution layer to be processed, and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model;
performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
Optionally, before the step of performing pixel-by-pixel hyper-resolution mapping processing on the to-be-processed low-resolution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the to-be-processed low-resolution layer, the method includes:
acquiring unsupervised high-resolution visible light images and training data formed by corresponding low-resolution image layers;
performing iterative training on a preset model to be trained based on the training data, and judging whether the preset model to be trained after iterative training meets a preset training completion condition;
and if the preset training completion condition is met, setting the target model obtained after iterative training as the preset hyper-resolution image reconstruction model.
Optionally, the step of performing iterative training on a preset model to be trained based on the training data, and determining whether the preset model to be trained after the iterative training meets a preset training completion condition includes:
determining an input matrix and a label matrix of the preset model to be trained based on the training data;
and performing iterative training on a preset model to be trained based on the input matrix and the label matrix, and judging whether the preset model to be trained after iterative training meets a preset training completion condition.
Optionally, the step of determining an input matrix and a tag matrix of the preset model to be trained based on the training data includes:
determining a first height and a first width of the high resolution visible light image, and determining a second height and a second width of the respective low resolution image layer, wherein the first height is greater than the second height, and the first width is greater than the second width;
determining a height scaling factor and a width scaling factor for the high resolution visible light image and the corresponding low resolution image layer based on the first height, first width, second height, and second width;
determining image coordinates of the high-resolution visible light image on a corresponding preset channel, and acquiring position coordinates of the high-resolution visible light image;
splicing to obtain high-resolution matrix information of the high-resolution visible light image based on the image coordinate and the position coordinate;
based on the height proportion coefficient and the width proportion coefficient, the height division horizontal and vertical coordinate data are subjected to preset mapping one by one according to elements to obtain an input matrix;
and carrying out preset mapping on the corresponding low-resolution image layers one by one according to elements to obtain a label matrix.
Optionally, the step of performing iterative training on a preset model to be trained based on the input matrix and the tag matrix, and determining whether the preset model to be trained after the iterative training meets a preset training completion condition includes:
performing first convolution training on a preset model to be trained based on an image matrix in the input matrix to obtain a first training matrix;
performing second convolution training on a preset model to be trained based on the coordinate matrix in the input matrix to obtain a second training matrix;
obtaining a merged matrix based on the first training matrix and the second training matrix;
judging whether the trained preset model to be trained meets preset training completion conditions or not based on the merged matrix and the label matrix;
and if the trained preset model to be trained does not meet the preset training completion condition, returning to the step of performing first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition.
Optionally, after the step of performing pixel-by-pixel hyper-resolution mapping processing on the to-be-processed low-resolution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the to-be-processed low-resolution layer, the method includes:
fusing the low-resolution image layer and the high-resolution image to obtain a fused image;
and outputting the fused image.
Optionally, the to-be-processed low-resolution layer includes an earth surface heat distribution layer, and the step of performing pixel-by-pixel hyper-resolution mapping processing on the to-be-processed low-resolution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the to-be-processed low-resolution layer includes:
performing pixel-by-pixel hyper-resolution mapping processing on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the earth surface heat distribution layer;
the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers.
The present application also provides a super-resolution image reconstruction apparatus, including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a low-resolution layer to be processed and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model;
the super-resolution module is used for carrying out pixel-by-pixel super-resolution mapping processing on the low-resolution image layer to be processed based on the preset super-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
Optionally, the super-resolution image reconstruction apparatus further includes:
the second acquisition module is used for acquiring the unsupervised high-resolution visible light image and training data formed by the corresponding low-resolution image layer;
the training module is used for carrying out iterative training on a preset model to be trained based on the training data and judging whether the preset model to be trained after iterative training meets a preset training completion condition or not;
and the setting module is used for setting the target model obtained after iterative training as the preset hyper-resolution image reconstruction model if the preset training completion condition is met.
Optionally, the training module comprises:
the determining unit is used for determining an input matrix and a label matrix of the preset model to be trained based on the training data;
and the judging unit is used for carrying out iterative training on a preset model to be trained based on the input matrix and the label matrix and judging whether the preset model to be trained after the iterative training meets a preset training completion condition.
Optionally, the determining unit includes:
a first determining subunit, configured to determine a first height and a first width of the high-resolution visible light image, and determine a second height and a second width of the corresponding low-resolution image layer, where the first height is greater than the second height, and the first width is greater than the second width;
a second determining subunit, configured to determine a height scaling factor and a width scaling factor of the high-resolution visible light image and the corresponding low-resolution image layer based on the first height, the first width, the second height, and the second width;
the third determining subunit is used for determining the image coordinates of the high-resolution visible light image on the corresponding preset channel and acquiring the position coordinates of the high-resolution visible light image;
the splicing subunit is used for splicing to obtain high-resolution matrix information of the high-resolution visible light image based on the image coordinate and the position coordinate;
the first mapping subunit is used for presetting and mapping the high-resolution horizontal and vertical coordinate data one by one according to elements on the basis of the height proportion coefficient and the width proportion coefficient to obtain an input matrix;
and the second mapping subunit is used for performing preset mapping on the corresponding low-resolution image layers one by one according to the elements to obtain a label matrix.
Optionally, the determining unit includes:
the first obtaining subunit is used for performing first convolution training on a preset model to be trained on the basis of an image matrix in the input matrix to obtain a first training matrix;
the second obtaining subunit is used for performing second convolution training on a preset model to be trained on the basis of the coordinate matrix in the input matrix to obtain a second training matrix;
a third obtaining subunit, configured to obtain a merged matrix based on the first training matrix and the second training matrix;
the first judging subunit is used for judging whether the trained preset model to be trained meets a preset training completion condition or not based on the combined matrix and the label matrix;
and the second judging subunit is configured to, if the trained preset model to be trained does not meet the preset training completion condition, return to the step of performing the first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition.
Optionally, the super-resolution image reconstruction apparatus further includes:
the fusion module is used for fusing the low-resolution image layer and the high-resolution image to obtain a fused image;
and the output module is used for outputting the fused image.
Optionally, the super-divide module includes:
the third obtaining module is configured to perform pixel-by-pixel hyper-resolution mapping processing on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the earth surface heat distribution layer;
the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers.
The present application also provides a super-resolution image reconstruction apparatus, which is an entity node apparatus, the super-resolution image reconstruction apparatus including: a memory, a processor and a program of the super-resolution image reconstruction method stored on the memory and executable on the processor, which when executed by the processor, may implement the steps of the super-resolution image reconstruction method as described above.
The present application also provides a storage medium having stored thereon a program for implementing the above-described super-resolution image reconstruction method, which when executed by a processor, implements the steps of the above-described super-resolution image reconstruction method.
The present application also provides a computer program product, comprising a computer program which, when executed by a processor, realizes the steps of the above-mentioned super-resolution image reconstruction method.
Compared with the prior art that resource consumption is large due to the fact that the super-resolution image reconstruction of a low-resolution image is carried out based on a scheme with supervision learning, the super-resolution image reconstruction method, the super-resolution image reconstruction device, the super-resolution image reconstruction equipment and the storage medium have the advantages that a low-resolution image layer to be processed is obtained and input into a preset super-resolution image reconstruction model; performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed; the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer. That is, in the present application, the actual distribution of the low-resolution data on the high resolution is not needed to be used as a label, but the preset hyper-resolution image reconstruction model is obtained after the iterative training is performed on the preset model to be trained based on the training data formed by the unsupervised high-resolution visible light image and the corresponding low-resolution image layer, and then the low-resolution image layer to be processed is processed to obtain the corresponding high-resolution image.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of a super-resolution image reconstruction method according to a first embodiment of the present application;
FIG. 2 is a schematic flowchart of a refining step of determining input data of the model to be interpreted and determining disturbance data for disturbing the input data in the super-resolution image reconstruction method of the present application;
fig. 3 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present application.
The objectives, features, and advantages of the present application will be further described with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The embodiment of the present application provides a super-resolution image reconstruction method, which is applied to a super-resolution image reconstruction device in a first embodiment of the super-resolution image reconstruction method, and referring to fig. 1, the super-resolution image reconstruction method includes:
step S10, acquiring a low-resolution image layer to be processed, and inputting the low-resolution image layer to be processed into a preset hyper-resolution image reconstruction model;
step S20, performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
The method comprises the following specific steps:
step S10, acquiring a low-resolution image layer to be processed, and inputting the low-resolution image layer to be processed into a preset hyper-resolution image reconstruction model;
in the present embodiment, it should be noted that the super-resolution image reconstruction method can be applied to a super-resolution image reconstruction system subordinate to the super-resolution image reconstruction apparatus. For the super-resolution image reconstruction system, a preset hyper-resolution image reconstruction model is built in the super-resolution image reconstruction system, or the super-resolution image reconstruction system can call preset hyper-resolution image reconstruction models of other components, so that after a low-resolution image layer to be processed is obtained, the low-resolution image layer to be processed is input into the preset hyper-resolution image reconstruction model.
In this embodiment, the specific application scenarios may be:
it is desirable to super-divide surface temperature map data having a spatial resolution (GSD) of 50 meters into image data having a spatial resolution (GSD) of higher resolution, such as a resolution of 17 meters, or to super-divide surface heat map data having a spatial resolution of 30 meters into satellite image data having a higher spatial resolution, such as a resolution of 10 meters, for analysis based on the higher resolution image data, such as oilfield reservoir analysis, or meteorological analysis.
In this embodiment, the super-resolution is to generate or obtain a corresponding high-resolution image based on the low-resolution image.
In the prior art, in order to obtain a high-resolution image corresponding to a low-resolution image, a supervised learning method based on a deep learning convolutional neural network is often used, that is, each low-resolution data image needs a real distribution of the data on the high resolution as a label.
In the prior art, a method for deep learning and cyclic neural deep learning is also based on the deep learning, two generated neural networks are constructed to obtain a high-resolution image corresponding to a low-resolution image, specifically, a first generated neural network takes the low-resolution image as input to obtain the high-resolution image, a second generated neural network takes the high-resolution image as input to generate the low-resolution image, and finally the low-resolution image is reconstructed through the trained first neural network.
In the embodiment, the actual distribution of low-resolution data on high resolution is not required to be acquired, namely, the labeling cost of supervised learning is not required, and only one high-resolution base map is required to perform the weak supervised learning, so that resources are saved.
Specifically, in this embodiment, first, a low-resolution map layer to be processed is obtained, where the low-resolution map layer may be various map layer data, such as heat map layer data, vegetation map layer data, building map layer data, population distribution map layer data, and the like, in this embodiment, the specific description is given by taking the earth surface heat distribution map layer data as an example, specifically, the low-resolution map layer to be processed may be an earth surface heat map layer data with a spatial resolution of 50 meters, and after obtaining the earth surface heat map layer data with a spatial resolution of 50 meters, the earth surface heat map layer data with a spatial resolution of 50 meters is input into a preset hyper-resolution image reconstruction model.
Step S20, performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
In this embodiment, after obtaining a low-resolution layer to be processed, performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution layer to be processed, specifically, inputting the surface thermal image layer data with the spatial resolution of 50 meters into a preset hyper-resolution image reconstruction model, so that the preset hyper-resolution image reconstruction model sufficiently processes the surface thermal image layer data with the spatial resolution of 50 meters to obtain a high-resolution image corresponding to the surface thermal image layer data with the spatial resolution of 50 meters, specifically, the preset hyper-resolution image reconstruction model performs pixel-by-pixel hyper-resolution mapping processing on the surface thermal image layer data with the spatial resolution of 50 meters to obtain a high-resolution image corresponding to the surface thermal image layer data with the spatial resolution of 50 meters, for example, the corresponding high resolution image is an RGB visible light image, which is an image with a spatial resolution of 17 meters.
In this embodiment, the hyper-map is that for a single pixel (xi, yi) of the low resolution layer, there will be a region [ Sh, Sw ] corresponding to it in the high resolution image.
The super-resolution mapping processing is to perform pixel-by-pixel mapping processing on the low-resolution layer to be processed based on the super-resolution mapping logic in the preset super-resolution image reconstruction model, so that for a single pixel (xi, yi) of the low-resolution layer, a region [ Sh, Sw ] corresponds to the single pixel in the high-resolution layer in the high-resolution image.
In this embodiment, the preset hyper-resolution image reconstruction model is obtained after iterative training is performed on a preset model to be trained based on training data composed of an unsupervised high-resolution visible light image and a corresponding low-resolution image layer, and because the model is trained based on the preset model to be trained, the preset hyper-resolution image reconstruction model can accurately obtain a corresponding high-resolution image based on the low-resolution image layer to be processed, for example, accurately obtain a high-resolution image corresponding to surface calorimetric image layer data with a spatial resolution of 50 meters, for example, an image with a spatial resolution of 17 meters.
Referring to fig. 2, before the step of performing pixel-by-pixel hyper-resolution mapping processing on the to-be-processed low-resolution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the to-be-processed low-resolution layer, the method includes:
step S01, acquiring unsupervised high-resolution visible light images and training data formed by corresponding low-resolution image layers;
in this embodiment, the unsupervised high-resolution visible light image and the training data formed by the corresponding low-resolution image layer are obtained first, that is, in this embodiment, only one high-resolution base map may be needed to perform the weakly supervised learning. Specifically, the low-resolution layer may be a Landsat-8 surface thermal layer IMGL including a spatial resolution of 30 meters, and the high-resolution base map may be specifically an RGB visible light image (one), for example, the RGB visible light image is a Sentinel-2 satellite image data IMGH having a spatial resolution of 10 meters.
It should be noted that, in this embodiment, the RGB visible light image and the low-resolution layer in the training data are both images in the same spatial range as that of a certain area (because the RGB visible light image functions as a label).
In the present embodiment, specifically, the RGB visible-light image may be IMGH data whose height and width are HH and WH, respectively, and the low-resolution layer may be IMGL data whose height and width are HL and WL, respectively, where HL < ═ HH and WL < ═ WH, that is, since the resolution of the low-resolution layer is low, a larger same region image is required to correspond thereto, and thus HL < ═ HH and WL < ═ WH.
Step S02, performing iterative training on a preset model to be trained based on the training data, and judging whether the preset model to be trained after iterative training meets a preset training completion condition;
in this embodiment, iterative training is performed on a preset model to be trained based on training data, and it is determined whether the preset model to be trained after the iterative training satisfies a preset training completion condition, specifically, the preset training completion condition refers to that a preset loss function converges or training reaches a preset number of times. In the process of performing iterative training on a preset model to be trained, a prediction result of a low-resolution image layer needs to be compared with an actual high-resolution image based on the model so as to continuously adjust learning parameters or weight parameters of the model until whether the preset model to be trained meets a preset training completion condition.
The step of performing iterative training on a preset model to be trained based on the training data and judging whether the preset model to be trained after the iterative training meets a preset training completion condition includes:
step A1, determining an input matrix and a label matrix of the preset model to be trained based on the training data;
in this embodiment, the data input into the model is in a matrix form, and therefore, it is necessary to determine an input matrix and a tag matrix of the preset model to be trained based on the training data, where the input matrix refers to a matrix obtained based on the low-resolution image layer, and the tag matrix refers to a matrix obtained based on the high-resolution image, and since the high-resolution image functions as a tag, the matrix obtained based on the high-resolution image is the tag matrix.
Wherein, the step of determining the input matrix and the label matrix of the preset model to be trained based on the training data comprises:
step B1, determining a first height and a first width of the high resolution visible light image, and determining a second height and a second width of the respective low resolution image layer, wherein the first height is greater than the second height, and the first width is greater than the second width;
in the present embodiment, specifically, a process of obtaining the input matrix and the tag matrix is performed, specifically, first, a first height and a first width (HH and WH) of the high-resolution visible light image are determined, and a second height and a second width (HL and WL) of the corresponding low-resolution image layer are determined, wherein the first height is greater than the second height, and the first width is greater than the second width (i.e., HL < ═ HH, WL < ═ WH).
Step B2, determining a height scaling factor and a width scaling factor of the high resolution visible light image and the corresponding low resolution image layer based on the first height, the first width, the second height and the second width;
in the present embodiment, based on the first height, the first width, the second height, and the second width, a height scaling factor and a width scaling factor of the high-resolution visible light image and the corresponding low-resolution layer are determined, and specifically, the height scaling factor is SH floor (HH/HL), the width scaling factor is SW floor (WH/WL), and the floor is a rounding-down operation.
Step B3, determining the image coordinates of the high-resolution visible light image on the corresponding preset channel, and acquiring the position coordinates of the high-resolution visible light image;
in the present embodiment, the image coordinates of the high-resolution visible light image on the corresponding preset channel, specifically three channels of R (red), G (green), and B (blue), are determined, specifically, the data RM, GM, BM of the three channels of R (red), G (green), and B (blue) are first extracted from the IMGH image, and then the image coordinates of the high-resolution visible light image on the corresponding preset channel are determined (recorded) based on the RM, GM, BM, and in the present embodiment, the horizontal and vertical coordinates of the IMGH are also gridded: specifically, based on HH and WH, an IMGH ordinate Y and an abscissa X are obtained, after gridding of the ordinate Y and the abscissa X is performed, the ordinate Y and the abscissa X may be [0, 1, 2,. and, HH-1] and [0, 1, 2,. and, WL-1], respectively, in this embodiment, the ordinate Y and the abscissa X may also be normalized to a range of [ -0.5, 0.5], and HH × WH-sized coordinate matrices XM and YM of Y and X, respectively, are obtained.
Assuming that HH and WH are 5, respectively, one can obtain:
the X coordinate matrix of the abscissa may be
[-0.5，-0.25.0.，0.25.0.5]，
[-0.5，-0.25.0.，0.25，0.5]，
[-0.5，-0.25，0.，0.25，0.5]，
[-0.5，-0.25，0.，0.25，0.5]，
[-0.5，-0.25，0.，0.25，0.5]
The ordinate Y-coordinate matrix may be
[-0.5，-0.5，-0.5，-0.5，-0.5]，
[-0.25，-0.25，-0.25，-0.25，-0.25]，
[0.，0.，0.，0.，0.]，
[0.25，0.25，0.25，0.25，0.25]，
[0.5，0.5，0.5，0.5，0.5]：
Step B4, splicing to obtain high resolution matrix information of the high resolution visible light image based on the image coordinates and the position coordinates;
in this embodiment, based on the image coordinates and the position coordinates, high-resolution matrix information of the high-resolution visible light image is obtained by stitching, that is, the RM, GM, BM, XM, and YM matrices obtained in step B3 are stitched to obtain a matrix input (h) of 5 HH WH, where in this embodiment, the matrix input (h) includes image information and coordinate information of IMGH.
B5, presetting and mapping the high-resolution horizontal and vertical coordinate data one by one according to elements based on the height proportion coefficient and the width proportion coefficient to obtain an input matrix;
in this embodiment, a new matrix input (HL) is created based on SH and SW obtained in step B2, and the shape of the new matrix input (HL) is (HL × WL) × 5 × SH × SW, and the matrix input (h) is copied to the matrix input (HL) element by element, so as to obtain the input matrix for model training.
And step B6, performing preset mapping on the corresponding low-resolution layers one by one according to the elements to obtain a label matrix.
In this embodiment, a new matrix INPUT _ img (lh) is created, the shape (h (l) × w (l) × 1), and the matrix IMGL is copied into the matrix INPUT _ img (lh) one by one according to the elements, so as to obtain a label for model training, that is, a label matrix is obtained.
Step A2, performing iterative training on a preset model to be trained based on the input matrix and the label matrix, and judging whether the preset model to be trained after iterative training meets a preset training completion condition.
In this embodiment, a convolution kernel is used to perform convolution operation in the channel by using an input (hl), and after the convolution operation, a loss calculation (the used loss function may be an MSE function) is performed to determine whether the preset model to be trained after iterative training meets a preset training completion condition.
And step S03, if the preset training completion condition is met, setting the target model obtained after iterative training as the preset hyper-resolution image reconstruction model.
In this embodiment, if a preset training completion condition is met, the target model obtained after iterative training is set as the preset hyper-resolution image reconstruction model.
Compared with the prior art that resource consumption is large due to the fact that the super-resolution image reconstruction of a low-resolution image is carried out based on a scheme with supervision learning, the super-resolution image reconstruction method, the super-resolution image reconstruction device, the super-resolution image reconstruction equipment and the storage medium have the advantages that a low-resolution image layer to be processed is obtained and input into a preset super-resolution image reconstruction model; performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed; the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer. That is, in the present application, the actual distribution of the low-resolution data on the high resolution is not needed to be used as a label, but the preset hyper-resolution image reconstruction model is obtained after the iterative training is performed on the preset model to be trained based on the training data formed by the unsupervised high-resolution visible light image and the corresponding low-resolution image layer, and then the low-resolution image layer to be processed is processed to obtain the corresponding high-resolution image.
Further, based on the first embodiment of the present application, another embodiment of the present application is provided, in which the step of performing iterative training on a preset model to be trained based on the input matrix and the tag matrix, and determining whether the preset model to be trained after the iterative training meets a preset training completion condition includes:
step C1, performing first convolution training on a preset model to be trained based on the image matrix in the input matrix to obtain a first training matrix;
step C2, performing second convolution training on a preset model to be trained based on the coordinate matrix in the input matrix to obtain a second training matrix;
step C3, obtaining a merged matrix based on the first training matrix and the second training matrix;
step C4, based on the merged matrix and the label matrix, judging whether the trained preset model to be trained meets a preset training completion condition;
and step C5, if the trained preset model to be trained does not meet the preset training completion condition, returning to the step of performing the first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition.
In this embodiment, the intra-channel convolution operation is performed on input (hl) using convolution kernel, and specifically, the intra-channel convolution operation is performed on input (hl) using convolution kernel with size 1 × 1. In order to keep the size of the original image input unchanged, in this embodiment, it is assumed that a network structure of the model to be trained is preset to have 3 layers of convolution modules, and the convolution kernel size of each layer is 1 × 1, the number of output channels is 8, 64, and 512, respectively.
In this embodiment, input (hl) is further trained in two branches, which are R1 (which performs a first convolution training on a preset model to be trained based on an image matrix in the input matrix to obtain a first training matrix) and R2 (which performs a second convolution training on the preset model to be trained based on a coordinate matrix in the input matrix to obtain a second training matrix), where R1 trains RGB visible light data of an image, which is data input (hl) -RGB of the first three dimensions in the third dimension of input (hl). The R2 trains position information data input (hl) -XY in the image, which is data of the second two dimensions in the third dimension of input (hl), and after convolution, the data obtained by the two branches are combined to obtain a Merge matrix. In this embodiment, the Merge matrix is subjected to convolution twice with CONV4 and CONV5 to obtain a final convolution layer, the final convolution layer is then used for averaging data of the last two dimensions to obtain a Mean matrix, the shape of the Mean matrix is B1 x 1, where B is the number of data batches input into the model, whether the trained preset model to be trained meets a preset training completion condition is judged, and if the trained preset model to be trained does not meet the preset training completion condition, the step of performing the first convolution training on the preset model to be trained based on the image matrix in the input matrix is returned until the trained preset model to be trained meets the preset training completion condition.
After the step of performing pixel-by-pixel hyper-resolution mapping processing on the to-be-processed low-resolution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the to-be-processed low-resolution layer, the method comprises the following steps:
fusing the low-resolution image layer and the high-resolution image to obtain a fused image;
and outputting the fused image.
In this embodiment, the low-resolution image layer and the high-resolution image are further fused to obtain a fused image; the fused image is output, and specifically, the ground surface temperature data of Landsat8 with the spatial resolution (GSD) of 30 meters and the image data of Sentinel-2 with the GSD of 10 meters are fused to obtain and output the ground surface temperature data after the over-resolution.
In this embodiment, a first training matrix is obtained by performing a first convolution training on a preset model to be trained based on an image matrix in the input matrix; performing second convolution training on a preset model to be trained based on the coordinate matrix in the input matrix to obtain a second training matrix; obtaining a merged matrix based on the first training matrix and the second training matrix; judging whether the trained preset model to be trained meets preset training completion conditions or not based on the merged matrix and the label matrix; if the trained preset model to be trained does not meet the preset training completion condition, returning to the step of performing the first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition, specifically implementing training to obtain a preset hyper-resolution image reconstruction model in the embodiment.
Further, based on the first embodiment in the present application, another embodiment of the present application is provided, in which the model to be interpreted is a face recognition model to be interpreted;
the low-resolution image layer to be processed comprises an earth surface heat distribution image layer, and the step of performing pixel-by-pixel hyper-division mapping processing on the low-resolution image layer to be processed based on the preset hyper-division image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed comprises the following steps:
step D1, performing pixel-by-pixel hyper-resolution mapping processing on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the earth surface heat distribution layer;
the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers.
In this embodiment, the method is specifically applied to the field of surface heat data processing, that is, the to-be-processed low-resolution map layer includes a surface heat distribution map layer, and specifically, therefore, a preset hyper-resolution image reconstruction model is first constructed, where the preset hyper-resolution image reconstruction model is obtained after iterative training is performed on a preset to-be-trained model based on training data formed by an unsupervised high-resolution visible light image and a corresponding surface heat distribution map layer, and then, pixel-by-pixel hyper-resolution mapping processing is performed on the surface heat distribution map layer based on the preset hyper-resolution image reconstruction model, so as to obtain a high-resolution image corresponding to the surface heat distribution map layer.
In this embodiment, pixel-by-pixel hyper-resolution mapping processing is performed on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model, so as to obtain a high-resolution image corresponding to the earth surface heat distribution layer; the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers. In this embodiment, the real distribution of the surface heat distribution map layer data on the high resolution is not required to be used as a label, but based on the unsupervised high-resolution visible light image and the training data formed by the corresponding surface heat distribution map layer, the preset model to be trained is iteratively trained to obtain the preset hyper-resolution image reconstruction model, and then the low-resolution map layer to be processed is processed to obtain the corresponding high-resolution image, that is, in the present application, the resource waste caused by artificial labeling is avoided.
Referring to fig. 3, fig. 3 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present application.
As shown in fig. 3, the super-resolution image reconstruction apparatus may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used for realizing connection communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Optionally, the super-resolution image reconstruction device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The rectangular user interface may comprise a Display screen (Display), an input sub-module such as a Keyboard (Keyboard), and the optional rectangular user interface may also comprise a standard wired interface, a wireless interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface).
It will be understood by those skilled in the art that the super-resolution image reconstruction device configuration shown in fig. 3 does not constitute a limitation of the super-resolution image reconstruction device, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 3, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, and a super-resolution image reconstruction program. The operating system is a program that manages and controls hardware and software resources of the super-resolution image reconstruction apparatus, and supports the execution of the super-resolution image reconstruction program as well as other software and/or programs. The network communication module is used for communication among the components inside the memory 1005 and with other hardware and software in the super-resolution image reconstruction system.
In the super-resolution image reconstruction apparatus shown in fig. 3, the processor 1001 is configured to execute a super-resolution image reconstruction program stored in the memory 1005, and implement the steps of the super-resolution image reconstruction method described in any one of the above.
The specific implementation of the super-resolution image reconstruction device of the present application is substantially the same as the embodiments of the super-resolution image reconstruction method described above, and is not described herein again.
The present application also provides a super-resolution image reconstruction apparatus, including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a low-resolution layer to be processed and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model;
the super-resolution module is used for carrying out pixel-by-pixel super-resolution mapping processing on the low-resolution image layer to be processed based on the preset super-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
Optionally, the super-resolution image reconstruction apparatus further includes:
the second acquisition module is used for acquiring the unsupervised high-resolution visible light image and training data formed by the corresponding low-resolution image layer;
the training module is used for carrying out iterative training on a preset model to be trained based on the training data and judging whether the preset model to be trained after iterative training meets a preset training completion condition or not;
and the setting module is used for setting the target model obtained after iterative training as the preset hyper-resolution image reconstruction model if the preset training completion condition is met.
Optionally, the training module comprises:
the determining unit is used for determining an input matrix and a label matrix of the preset model to be trained based on the training data;
and the judging unit is used for carrying out iterative training on a preset model to be trained based on the input matrix and the label matrix and judging whether the preset model to be trained after the iterative training meets a preset training completion condition.
Optionally, the determining unit includes:
a first determining subunit, configured to determine a first height and a first width of the high-resolution visible light image, and determine a second height and a second width of the corresponding low-resolution image layer, where the first height is greater than the second height, and the first width is greater than the second width;
a second determining subunit, configured to determine a height scaling factor and a width scaling factor of the high-resolution visible light image and the corresponding low-resolution image layer based on the first height, the first width, the second height, and the second width;
the third determining subunit is used for determining the image coordinates of the high-resolution visible light image on the corresponding preset channel and acquiring the position coordinates of the high-resolution visible light image;
the splicing subunit is used for splicing to obtain high-resolution matrix information of the high-resolution visible light image based on the image coordinate and the position coordinate;
the first mapping subunit is used for presetting and mapping the high-resolution horizontal and vertical coordinate data one by one according to elements on the basis of the height proportion coefficient and the width proportion coefficient to obtain an input matrix;
and the second mapping subunit is used for performing preset mapping on the corresponding low-resolution image layers one by one according to the elements to obtain a label matrix.
Optionally, the determining unit includes:
the first obtaining subunit is used for performing first convolution training on a preset model to be trained on the basis of an image matrix in the input matrix to obtain a first training matrix;
the second obtaining subunit is used for performing second convolution training on a preset model to be trained on the basis of the coordinate matrix in the input matrix to obtain a second training matrix;
a third obtaining subunit, configured to obtain a merged matrix based on the first training matrix and the second training matrix;
the first judging subunit is used for judging whether the trained preset model to be trained meets a preset training completion condition or not based on the combined matrix and the label matrix;
and the second judging subunit is configured to, if the trained preset model to be trained does not meet the preset training completion condition, return to the step of performing the first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition.
Optionally, the super-resolution image reconstruction apparatus further includes:
the fusion module is used for fusing the low-resolution image layer and the high-resolution image to obtain a fused image;
and the output module is used for outputting the fused image.
Optionally, the super-divide module includes:
the third obtaining module is configured to perform pixel-by-pixel hyper-resolution mapping processing on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the earth surface heat distribution layer;
the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers.
The specific implementation of the super-resolution image reconstruction device of the present application is substantially the same as the embodiments of the super-resolution image reconstruction method described above, and is not described herein again.
The present embodiment provides a storage medium, and the storage medium stores one or more programs, which are further executable by one or more processors for implementing the steps of the super-resolution image reconstruction method according to any one of the above.
The specific implementation of the storage medium of the present application is substantially the same as the embodiments of the super-resolution image reconstruction method, and is not described herein again.
The present application also provides a computer program product, comprising a computer program which, when executed by a processor, realizes the steps of the above-mentioned super-resolution image reconstruction method.
The specific implementation of the computer program product of the present application is substantially the same as the embodiments of the super-resolution image reconstruction method, and is not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (11)
1. A super-resolution image reconstruction method applied to a super-resolution image reconstruction apparatus, the super-resolution image reconstruction method comprising:
acquiring a low-resolution layer to be processed, and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model;
performing pixel-by-pixel hyper-resolution mapping processing on the low-resolution image layer to be processed based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
2. The super-resolution image reconstruction method according to claim 1, wherein before the step of performing pixel-by-pixel super-resolution mapping processing on the to-be-processed low-resolution image layer based on the preset super-resolution image reconstruction model to obtain the high-resolution image corresponding to the to-be-processed low-resolution image layer, the method comprises:
acquiring unsupervised high-resolution visible light images and training data formed by corresponding low-resolution image layers;
performing iterative training on a preset model to be trained based on the training data, and judging whether the preset model to be trained after iterative training meets a preset training completion condition;
and if the preset training completion condition is met, setting the target model obtained after iterative training as the preset hyper-resolution image reconstruction model.
3. The super-resolution image reconstruction method according to claim 2, wherein the step of iteratively training a preset model to be trained based on the training data and determining whether the iteratively trained model satisfies a preset training completion condition comprises:
determining an input matrix and a label matrix of the preset model to be trained based on the training data;
and performing iterative training on a preset model to be trained based on the input matrix and the label matrix, and judging whether the preset model to be trained after iterative training meets a preset training completion condition.
4. The super-resolution image reconstruction method according to any one of claims 1 to 3, wherein the step of determining the input matrix and the label matrix of the preset model to be trained based on the training data comprises:
determining a first height and a first width of the high resolution visible light image, and determining a second height and a second width of the respective low resolution image layer, wherein the first height is greater than the second height, and the first width is greater than the second width;
determining a height scaling factor and a width scaling factor for the high resolution visible light image and the corresponding low resolution image layer based on the first height, first width, second height, and second width;
determining image coordinates of the high-resolution visible light image on a corresponding preset channel, and acquiring position coordinates of the high-resolution visible light image;
splicing to obtain high-resolution matrix information of the high-resolution visible light image based on the image coordinate and the position coordinate;
based on the height proportion coefficient and the width proportion coefficient, the height division horizontal and vertical coordinate data are subjected to preset mapping one by one according to elements to obtain an input matrix;
and carrying out preset mapping on the corresponding low-resolution image layers one by one according to elements to obtain a label matrix.
5. The super-resolution image reconstruction method according to claim 4, wherein the step of iteratively training a preset model to be trained based on the input matrix and the label matrix and determining whether the iteratively trained preset model satisfies a preset training completion condition comprises:
performing first convolution training on a preset model to be trained based on an image matrix in the input matrix to obtain a first training matrix;
performing second convolution training on a preset model to be trained based on the coordinate matrix in the input matrix to obtain a second training matrix;
obtaining a merged matrix based on the first training matrix and the second training matrix;
judging whether the trained preset model to be trained meets preset training completion conditions or not based on the merged matrix and the label matrix;
and if the trained preset model to be trained does not meet the preset training completion condition, returning to the step of performing first convolution training on the preset model to be trained based on the image matrix in the input matrix until the trained preset model to be trained meets the preset training completion condition.
6. The super-resolution image reconstruction method according to claim 1, wherein after the step of performing pixel-by-pixel super-resolution mapping processing on the to-be-processed low-resolution image layer based on the preset super-resolution image reconstruction model to obtain the high-resolution image corresponding to the to-be-processed low-resolution image layer, the method comprises:
fusing the low-resolution image layer and the high-resolution image to obtain a fused image;
and outputting the fused image.
7. The super-resolution image reconstruction method according to claim 1, wherein the to-be-processed low-resolution image layer includes an earth surface heat distribution image layer, and the step of performing pixel-by-pixel hyper-division mapping processing on the to-be-processed low-resolution image layer based on the preset hyper-division image reconstruction model to obtain the high-resolution image corresponding to the to-be-processed low-resolution image layer includes:
performing pixel-by-pixel hyper-resolution mapping processing on the earth surface heat distribution layer based on the preset hyper-resolution image reconstruction model to obtain a high-resolution image corresponding to the earth surface heat distribution layer;
the preset hyper-resolution image reconstruction model is obtained after iterative training is carried out on a preset model to be trained on the basis of training data formed by unsupervised high-resolution visible light images and corresponding earth surface heat distribution image layers.
8. A super-resolution image reconstruction apparatus, characterized by comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a low-resolution layer to be processed and inputting the low-resolution layer to be processed into a preset hyper-resolution image reconstruction model;
the super-resolution module is used for carrying out pixel-by-pixel super-resolution mapping processing on the low-resolution image layer to be processed based on the preset super-resolution image reconstruction model to obtain a high-resolution image corresponding to the low-resolution image layer to be processed;
the preset hyper-resolution image reconstruction model is obtained by performing iterative training on a preset model to be trained based on training data formed by an unsupervised high-resolution visible light image and a corresponding low-resolution image layer.
9. A super-resolution image reconstruction apparatus characterized by comprising: a memory, a processor, and a program stored on the memory for implementing the super-resolution image reconstruction method,
the memory is used for storing a program for realizing the super-resolution image reconstruction method;
the processor is configured to execute a program for implementing the super-resolution image reconstruction method to implement the steps of the super-resolution image reconstruction method according to any one of claims 1 to 7.
10. A storage medium, characterized in that the storage medium has stored thereon a program for implementing a super-resolution image reconstruction method, which is executed by a processor to implement the steps of the super-resolution image reconstruction method according to any one of claims 1 to 7.
11. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111058591.4A CN113763248A (en) | 2021-09-08 | 2021-09-08 | Super-resolution image reconstruction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111058591.4A CN113763248A (en) | 2021-09-08 | 2021-09-08 | Super-resolution image reconstruction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113763248A true CN113763248A (en) | 2021-12-07 |
Family
ID=78794645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111058591.4A Pending CN113763248A (en) | 2021-09-08 | 2021-09-08 | Super-resolution image reconstruction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113763248A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023231138A1 (en) * | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
-
2021
- 2021-09-08 CN CN202111058591.4A patent/CN113763248A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023231138A1 (en) * | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11758366B2 (en) | Methods and devices for displaying a heat map and providing heat data | |
| CN112734641B (en) | Training method and device for target detection model, computer equipment and medium | |
| CN109117760B (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN109934792B (en) | Electronic device and control method thereof | |
| CN107911753A (en) | Method and apparatus for adding digital watermarking in video | |
| US11461999B2 (en) | Image object detection method, device, electronic device and computer readable medium | |
| JP2024526065A (en) | Method and apparatus for recognizing text - Patents.com | |
| CN109118456B (en) | Image processing method and device | |
| CN111311480B (en) | Image fusion method and device | |
| CN110826632A (en) | Image change detection method, device, equipment and computer readable storage medium | |
| CN110852980A (en) | Interactive image filling method and system, server, device and medium | |
| CN111598084B (en) | Defect segmentation network training method, device, equipment and readable storage medium | |
| CN113569852A (en) | Training method and device of semantic segmentation model, electronic equipment and storage medium | |
| CN114241326A (en) | Progressive intelligent production method and system for ground feature elements of remote sensing images | |
| CN113763248A (en) | Super-resolution image reconstruction method, device, equipment and storage medium | |
| CN113421217A (en) | Method and device for detecting travelable area | |
| CN108961268B (en) | Saliency map calculation method and related device | |
| CN112330671A (en) | Method and device for analyzing cell distribution state, computer equipment and storage medium | |
| CN113033715A (en) | Target detection model training method and target vehicle detection information generation method | |
| CN112686939A (en) | Depth image rendering method, device and equipment and computer readable storage medium | |
| CN115656189B (en) | Defect detection method and device based on luminosity stereo and deep learning algorithm | |
| CN116229130A (en) | Type identification method and device for blurred image, computer equipment and storage medium | |
| CN115756461A (en) | Annotation template generation method, image identification method and device and electronic equipment | |
| CN116310308A (en) | Image segmentation method, device, computer equipment and storage medium | |
| CN116823700A (en) | Image quality determining method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |