CN116563174B - Image reconstruction method, device and computer storage medium - Google Patents

Image reconstruction method, device and computer storage medium Download PDF

Info

Publication number
CN116563174B
CN116563174B CN202310846523.7A CN202310846523A CN116563174B CN 116563174 B CN116563174 B CN 116563174B CN 202310846523 A CN202310846523 A CN 202310846523A CN 116563174 B CN116563174 B CN 116563174B
Authority
CN
China
Prior art keywords
image
denoising
information
space information
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310846523.7A
Other languages
Chinese (zh)
Other versions
CN116563174A (en
Inventor
陈勇
桂鑫锋
袁飞望
罗俊
吴斌
周阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi Normal University
Original Assignee
Jiangxi Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangxi Normal University filed Critical Jiangxi Normal University
Priority to CN202310846523.7A priority Critical patent/CN116563174B/en
Publication of CN116563174A publication Critical patent/CN116563174A/en
Application granted granted Critical
Publication of CN116563174B publication Critical patent/CN116563174B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/20Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing
    • G06T2207/10036Multispectral image; Hyperspectral image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Architecture (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present application relates to the field of computer technologies, and in particular, to a method and apparatus for reconstructing an image, and a computer storage medium; the reconstruction method comprises the following steps: acquiring a compressed image; performing image decomposition based on the compressed image to obtain three-dimensional space information and image spectrum information; denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image; reconstructing an image based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image; the three-dimensional space information corresponding to the compressed image is denoised in a complementary mode by adopting the image prior processing model and the image denoising model, so that the denoising precision in the image reconstruction process is improved, and the image reconstruction precision of the target image is improved.

Description

Image reconstruction method, device and computer storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for reconstructing an image, and a computer storage medium.
Background
The hyperspectral image is three-dimensional data comprising a plurality of spectrum bands, and has been widely applied to the fields of remote sensing, medical imaging, military and the like because of the abundant spectrum information; in the prior art, a hyperspectral image is usually compressed, stored and reconstructed based on a snapshot compression imaging technology of a compressed sensing theory, but the reconstruction result of the image is often greatly different from that of an original image, so that the image precision of the reconstructed image is not high.
Disclosure of Invention
Aiming at the problems in the prior art, the application aims to improve the denoising precision in the image reconstruction process and further improve the image reconstruction precision of a target image by denoising three-dimensional space information corresponding to a compressed image in a complementary mode by adopting an image prior processing model and an image denoising model.
In order to solve the above problems, the present application provides a method for reconstructing an image, comprising:
acquiring a compressed image;
performing image decomposition based on the compressed image to obtain three-dimensional space information and image spectrum information corresponding to the compressed image;
Denoising the three-dimensional space information based on an image prior processing model and an image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on the three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image;
and carrying out image reconstruction based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image.
In the embodiment of the present application, denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information includes:
determining the three-dimensional space information as current space information;
denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information;
determining the current denoising space information as the current space information under the condition that the current denoising space information and the last denoising space information do not meet a preset error condition;
repeating the steps: denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information, and determining the current denoising space information as current space information until the current denoising space information and the last denoising space information meet preset error conditions under the condition that the current denoising space information and the last denoising space information do not meet preset error conditions;
And determining the current denoising space information as the target space information.
In the embodiment of the present application, denoising the current spatial information based on the image prior processing model and the image denoising model, to obtain current denoising spatial information includes:
inputting the current spatial information into the image prior processing model for denoising processing to obtain first spatial information;
inputting the first spatial information into the image denoising model to perform denoising processing to obtain second spatial information;
updating the current circulation times;
determining the second spatial information as the current spatial information under the condition that the current cycle number is smaller than a preset cycle number;
repeating the steps: the current space information is input into the image prior processing model to be subjected to denoising processing, first space information is obtained, and the second space information is determined to be the current space information until the current cycle number is greater than or equal to the preset cycle number under the condition that the current cycle number is smaller than the preset cycle number;
and determining the second spatial information as the current denoising spatial information.
In the embodiment of the present application, the image decomposition based on the compressed image to obtain three-dimensional spatial information and image spectrum information corresponding to the compressed image includes:
performing low-rank decomposition on matrix information corresponding to the compressed image to obtain the three-dimensional space information and the image spectrum information; the three-dimensional spatial information includes multi-layer two-dimensional plane information.
In the embodiment of the application, the image denoising model comprises a first scaling layer, a denoising layer and a second scaling layer; inputting the first spatial information into the image denoising model for denoising processing, and obtaining second spatial information comprises:
based on the first scaling layer, carrying out normalization processing on matrix information corresponding to the multi-layer two-dimensional plane information to obtain normalized plane information corresponding to each of the multi-layer two-dimensional plane information;
based on the denoising layer, denoising the normalized plane information corresponding to each of the plurality of layers of two-dimensional plane information to obtain a plurality of layers of two-dimensional denoising information;
and performing inverse normalization processing on matrix information corresponding to the multi-layer two-dimensional denoising information based on the second scaling layer to obtain the second spatial information.
In the embodiment of the present application, the denoising layer includes a convolution layer and a residual layer, and the denoising processing is performed on the normalized plane information corresponding to each of the multiple layers of two-dimensional plane information based on the denoising layer, so as to obtain multiple layers of two-dimensional denoising information, where the denoising processing includes:
based on the convolution layer, carrying out convolution processing on matrix information corresponding to the normalized plane information to obtain a convolution matrix;
and carrying out residual connection based on the residual layer, matrix information corresponding to the normalized plane information and the convolution matrix to obtain two-dimensional denoising information corresponding to the normalized plane information.
In the embodiment of the present application, the image prior processing model includes a stride convolution layer, a transpose convolution layer, and an output layer, the output layer includes a parametric activation function, and the step of inputting the current spatial information into the image prior processing model to perform denoising processing includes:
based on the stride convolution layer, carrying out convolution processing on matrix information corresponding to the current space information to obtain a first sampling matrix;
based on the transpose convolution layer, carrying out convolution processing on the sampling matrix to obtain a second sampling matrix;
And carrying out priori learning on the second sampling matrix based on the band parameter activation function of the output layer to obtain the first spatial information.
On the other hand, the application also provides an image reconstruction device, which comprises:
the acquisition module is used for acquiring the compressed image;
the decomposition module is used for carrying out image decomposition based on the compressed image to obtain three-dimensional space information and image spectrum information corresponding to the compressed image;
the denoising module is used for denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on the three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image;
and the reconstruction module is used for carrying out image reconstruction based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image.
In another aspect, the present application further provides an electronic device, where the device includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, where the at least one instruction or the at least one program is loaded and executed by the processor to implement a method for reconstructing an image as described above.
In another aspect, the present application further provides a computer storage medium, where at least one instruction or at least one program is stored, where the at least one instruction or the at least one program is loaded and executed by a processor to implement a method for reconstructing an image as described above.
Due to the technical scheme, the image reconstruction method has the following beneficial effects:
the image prior processing model is adopted to perform prior learning based on self space information corresponding to the three-dimensional space information, and denoising is performed; denoising the three-dimensional space information based on training priori information of the sample image by adopting an image denoising model; the method and the device realize denoising based on the prior knowledge corresponding to the spatial information of the three-dimensional spatial information and the prior knowledge learned by the sample image, and realize denoising prior complementation, thereby improving the denoising precision of the three-dimensional spatial information and further improving the image reconstruction precision of the reconstructed target image.
Drawings
In order to more clearly illustrate the technical solution of the present application, the following description will make a brief introduction to the drawings used in the description of the embodiments or the prior art. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a schematic flow chart of an image reconstruction method according to an embodiment of the present application;
fig. 2 is a schematic diagram of a target space information determining process in an image reconstruction method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a current denoising spatial information determination flow in an image reconstruction method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a second spatial information determining flow in an image reconstruction method according to an embodiment of the present application;
fig. 5 is a schematic diagram of a two-dimensional denoising information determination flow in an image reconstruction method according to an embodiment of the present application;
fig. 6 is a schematic diagram of a first spatial information determining flow in an image reconstruction method according to an embodiment of the present application;
FIG. 7 is a schematic structural diagram of an image reconstruction device according to an embodiment of the present application;
fig. 8 is a hardware block diagram of an image reconstruction method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic may be included in at least one implementation of the application. In the description of the present application, it should be understood that the directions or positional relationships indicated by the terms "upper", "lower", "left", "right", "top", "bottom", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present application. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may include one or more of the feature, either explicitly or implicitly. Moreover, the terms "first," "second," and the like, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein.
Referring to fig. 1, a method for reconstructing an image according to an embodiment of the present application is described, where the method includes:
s1001, obtaining a compressed image; compressed images refer to image information that is convenient for transmission and storage; specifically, the compressed image may be a two-dimensional compressed image corresponding to the hyperspectral image.
S1002, performing image decomposition based on a compressed image to obtain three-dimensional space information and image spectrum information corresponding to the compressed image; image decomposition refers to a process of decomposing a plurality of dimensional information of a compressed image; specifically, the information in the compressed image is decomposed into three-dimensional space information and image spectrum information; the three-dimensional space information refers to space information in a compressed image, specifically comprises information of three dimensions of length, width and depth, and can be (x, y, z) used for representing the space information corresponding to any point in a three-dimensional space; the image spectrum information refers to the distribution of intensity or reflectivity of the color distribution of each pixel in the compressed image in different wavelength ranges, and specifically, the image spectrum information is hyperspectral information.
In the embodiment of the present application, S1002 includes:
s10021, decompressing and imaging the two-dimensional compressed image based on the image mask to obtain an initial hyperspectral image; the image mask is a template of the image filter; the initial hyperspectral image is an image obtained by initially decompressing the two-dimensional compressed image.
S10022, performing image decomposition on the initial hyperspectral image to obtain three-dimensional space information and image spectrum information.
In the embodiment of the application, the image mask is adopted to decompress and image the two-dimensional compressed image, and then the image is decomposed, so that the richness of the image information is improved, and the image precision in the reconstruction process is further improved.
In a specific embodiment of the present application, the three-dimensional spatial information may represent a spatial representation coefficient Z of the compressed image, and the image spectral information may represent a spectral base E of the compressed image.
S1003, denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image; it can be understood that the image prior processing model is a model for denoising learning based on a spatial prior contained in the three-dimensional spatial information, and the image denoising model is a model for denoising based on a spatial prior learned by the sample image; the image prior processing model is used for denoising the three-dimensional space information based on the space prior contained in the three-dimensional space information, namely, the image internal space prior, and the image denoising model is used for denoising the three-dimensional space information based on the space prior learned by the sample image, namely, the external space prior, so that complementation of the denoising prior is realized.
In a specific embodiment of the present application, the Image Prior processing model may refer to an unsupervised depth Image Prior model (DIP), and the Image denoising model may refer to a training depth denoising Prior model (Deep Denoising Prior, DDP).
S1004, reconstructing an image based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image; the image reconstruction process can be regarded as an information fusion process and also can be regarded as an inverse process of image decomposition; the information precision corresponding to the target image is greater than the information precision corresponding to the compressed image.
In a specific embodiment of the present application, determining the target image as the initial hyperspectral image is repeatedly performed: S1002-S1004 to further optimize the target image.
In the embodiment of the application, the image prior processing model is adopted to perform prior learning based on the self space information corresponding to the three-dimensional space information, and denoising is performed; denoising the three-dimensional space information based on training priori information of the sample image by adopting an image denoising model; the method and the device realize denoising based on the prior knowledge corresponding to the spatial information of the three-dimensional spatial information and the prior knowledge learned by the sample image, and realize denoising prior complementation, thereby improving the denoising precision of the three-dimensional spatial information and further improving the image reconstruction precision of the reconstructed target image.
In a specific embodiment of the present application, the process for image reconstruction can be regarded as the solving process of the following formula (1):
(1)
wherein,,the value of x under the condition that the solving function obtains the minimum value is shown as the reference value, Y is a captured compression measurement diagram, x is an initial hyperspectral image, and +.>Refers to a compression map determined based on the initial hyperspectral image,/i>Refers to the fidelity term, i.e. the F-norm between the compressed measurement map and the compressed map,/->Refers to the normal rule parameter, ">Refers to the positive rule prior.
In a specific embodiment of the present application, the decomposition process for a compressed image can be regarded as the decomposition process of the following formula (2):
(2)
wherein x refers to the initial hyperspectral image,refers to the third dimension multiplication of tensors, +.>Refers to three-dimensional spatial information, and E refers to image spectral information.
In the specific embodiment of the application, based on the image spectrum information in the reconstruction process, transformation is not needed, so that the above-mentioned solving process can be converted into a solving process for Z, the prior for learning an image prior processing model and the regular prior for an image denoising model are introduced, and the following solving formula (3) is formed by combining a semi-Quadratic splitting algorithm (pseudoquadric):
(3)
wherein,,means +. >Is a value of->Refers to network parameters in the image prior processing model, < ->Refers to the augmentation variable introduced in the image denoising model,/->For parameters corresponding to penalty function->Refers to an image prior processing model,/->And processing random noise in the model learning process for the image prior, wherein the dimension of the random noise is consistent with the three-dimensional space information.
In a specific embodiment of the application, the image prior processing model is a pairIs to solve the problem of image denoising modelIs a solution to the problem.
Referring to fig. 2, in an embodiment of the present application, S1003 includes:
s2001, three-dimensional space information is determined as current space information.
S2002, denoising the current space information based on the image prior processing model and the image denoising model to obtain the current denoising space information.
S2003, determining the current denoising space information as the current space information under the condition that the current denoising space information and the last denoising space information do not meet the preset error condition; in the process of repeatedly executing S2002-S2003, current denoising space information corresponding to the current cycle exists, and the last denoising space information is the current denoising space information in the last cycle; the current denoising spatial information and the last denoising spatial information do not meet the preset error condition, so that a large difference still exists between the current denoising spatial information and the last denoising spatial information, and the current denoising spatial information can still be continuously denoised, therefore, the current denoising spatial information is determined to be the current spatial information, the current denoising spatial information is further denoised, and the denoising precision of the three-dimensional spatial information is improved.
The steps are repeatedly executed: S2002-S2003, executing S2004 until the current denoising spatial information and the last denoising spatial information meet the preset error condition; the current denoising space information and the last denoising space information meet the preset error condition, so that the difference between the current denoising space information and the last denoising space information is smaller, denoising cannot be continuously performed through circulation, and therefore skipping can be performed.
In a specific embodiment of the present application, the preset error condition may be the following error formula (4):
(4)
wherein,,refers to the current denoising spatial information, +.>Refers to the previous denoising spatial information, +.>The F norm corresponding to the difference value between the current denoising space information and the last denoising space information is referred; />The F norm corresponding to the previous denoising space information is referred; />Refers to a preset upper limit value.
In another embodiment of the present application, the preset error condition may be that the euclidean distance between the current denoising spatial information and the previous denoising spatial information is smaller than a preset distance value, or that the cosine similarity between the current denoising spatial information and the previous denoising spatial information is smaller than a preset cosine value.
In another embodiment of the present application, the executing step S2002-S2003 is regarded as an external cycle, the number of external cycles corresponding to the external cycle is obtained, and when the number of external cycles is greater than the preset number of external cycles, warning information is sent; under the condition that the external circulation times are larger than the preset external circulation times, the characterization circulation times are too much, and the condition of overfitting possibly exists, so that warning information needs to be sent out to avoid the occurrence of the condition of overfitting, the denoising precision of the three-dimensional space information is further improved, and the image reconstruction precision of the reconstructed target image is improved.
And S2004, determining the current denoising spatial information as target spatial information.
In the embodiment of the application, the similarity between the current denoising space information and the last denoising space information is evaluated by adopting the preset error condition, so that the cycle is ended under the condition that the similarity between the current denoising space information and the last denoising space information is large, invalid cycle is avoided, the confirmation rate of the target space information is improved, and the reconstruction rate of image reconstruction is improved; in the process of each external circulation, denoising the three-dimensional space information based on training priori information of the sample image by adopting an image denoising model; the method and the device realize denoising based on the prior knowledge corresponding to the spatial information of the three-dimensional spatial information and the prior knowledge learned by the sample image, and realize denoising prior complementation, thereby improving the denoising precision of the three-dimensional spatial information and further improving the image reconstruction precision of the reconstructed target image.
Referring to fig. 3, in an embodiment of the present application, S2002 includes:
s3001, inputting the current spatial information into an image prior processing model for denoising processing, and obtaining first spatial information.
S3002, inputting the first spatial information into an image denoising model for denoising processing, and obtaining second spatial information.
S3003, updating the current cycle times; the execution of steps S3001 to S3004 is regarded as one internal cycle, and the current cycle number refers to the cycle number of the internal cycle.
S3004, determining the second space information as current space information when the current cycle number is smaller than the preset cycle number; the preset number of cycles is based on experience and a preset value for debugging.
The steps are repeatedly executed: S3001-S3004, until the current number of cycles is greater than or equal to the preset number of cycles, then S3005 is executed.
S3005, determining the second spatial information as current denoising spatial information.
In the embodiment of the application, the current denoising spatial information is acquired by repeating the cycle for a plurality of times, so that the current denoising spatial information is more accurate, the denoising precision of the three-dimensional spatial information is improved, and the image reconstruction precision of the reconstructed target image is improved.
In the embodiment of the present application, S1002 includes:
performing low-rank decomposition on matrix information corresponding to the compressed image to obtain three-dimensional space information and image spectrum information; the three-dimensional space information comprises multi-layer two-dimensional plane information; the low-rank decomposition refers to a process of decomposing a matrix into two low-rank matrices, that is, decomposing matrix information corresponding to a compressed image into matrix information corresponding to three-dimensional space information and matrix information corresponding to image spectrum information; wherein the matrix information corresponding to the three-dimensional space information is a three-dimensional matrix, which is also called a cube matrix; the matrix information corresponding to the multi-layer two-dimensional plane information is a plurality of two-dimensional matrixes obtained by taking a certain dimension in the three-dimensional matrix as a reference.
In the embodiment of the application, the matrix information corresponding to the compressed image is subjected to low-rank decomposition, and the image information is subjected to split analysis, so that the interference of redundant information on the reconstruction process is reduced, and the image reconstruction precision of the reconstructed target image is improved.
Referring to fig. 4, in an embodiment of the present application, an image denoising model includes a first scaling layer, a denoising layer, and a second scaling layer; the first scaling layer is used for carrying out normalization processing on the data, the denoising layer is used for carrying out denoising processing on the data, and the second scaling layer is used for carrying out inverse normalization processing on the data; s3002 includes:
s5001, based on a first scaling layer, carrying out normalization processing on matrix information corresponding to the multi-layer two-dimensional plane information to obtain normalized plane information corresponding to each of the multi-layer two-dimensional plane information; each element in the matrix information corresponding to the normalized plane information is between (0, 1).
S5002, based on the denoising layer, denoising the normalization plane information corresponding to each of the plurality of layers of two-dimensional plane information to obtain a plurality of layers of two-dimensional denoising information; the denoising process is a process of denoising based on a priori knowledge of sample image learning.
S5003, based on the second scaling layer, performing inverse normalization processing on matrix information corresponding to the multi-layer two-dimensional denoising information to obtain second spatial information; the data dimension corresponding to the second spatial information is consistent with the data dimension corresponding to the two-dimensional plane information.
In the embodiment of the application, the data complexity in the data processing process is reduced by adopting normalization processing and inverse normalization processing, so that the data processing speed is improved, and the image reconstruction speed is improved.
In a specific embodiment of the present application, the image denoising model may use a Deep Residual U-Net (DRUNet), so the solution for U can be regarded as the following equation (5):
(5)
wherein,,refers to the nth of the two-dimensional plane information of (x, y) obtained by taking the dimension of n as a reference,refers to an image denoising model, < >>Refers to the n-th corresponding two-dimensional plane information in the obtained (x, y) two-dimensional plane information by taking the dimension of n as a reference, < ->Refers to the corresponding augmentation variable of the nth in the obtained (x, y) two-dimensional plane information by taking the dimension of n as a reference,/for the two-dimensional plane information>Refers to the nth corresponding noise level in the obtained (x, y) two-dimensional plane information by taking the dimension of n as a reference.
In the embodiment of the present application, the normalization process may use the following formulas (6) - (7):
(6)
(7)
wherein the method comprises the steps of,Means multiplying power during normalization processing, +.>Refers to parameters during normalization processing, +. >Refers to the n-th corresponding two-dimensional plane information in the obtained (x, y) two-dimensional plane information by taking the dimension of n as a reference after normalization, wherein ∈n is a dimension of n>The n-th corresponding noise level in the obtained (x, y) two-dimensional plane information by taking the dimension where n is as a reference after normalization.
In the specific embodiment of the present application,and +.>The calculation formula of (a) can refer to formulas (8) to (9):
(8)
(9)
wherein,,refers to->Maximum value of (2), min->Refers to->Is the minimum value of (a).
Referring to fig. 5, in an embodiment of the present application, the denoising layer includes a convolution layer for performing feature extraction on image information and a residual layer for performing residual connection on output data and input data; s5002 includes:
s6001, carrying out convolution processing on matrix information corresponding to normalized plane information based on a convolution layer to obtain a convolution matrix; the convolution matrix characterizes the image features corresponding to the normalized plane information.
S6002, residual connection is carried out based on the residual layer, matrix information corresponding to the normalized plane information and the convolution matrix, and two-dimensional denoising information corresponding to the normalized plane information is obtained.
In the embodiment of the application, the matrix information corresponding to the normalized plane information and the convolution matrix are subjected to residual connection, so that the distortion of data is reduced, the denoising precision of the three-dimensional space information is further improved, and the image reconstruction precision of the reconstructed target image is improved.
Referring to fig. 6, in the embodiment of the present application, the image prior processing model includes a stride convolution layer, a transposed convolution layer, and an output layer, where the output layer includes a parameter activation function, the stride convolution layer is used for extracting features of image information, the transposed convolution is used for performing deconvolution operation on the extracted image features, and the parameter activation function is used for performing parameter adjustment based on corresponding prior knowledge of spatial information of three-dimensional spatial information; s3001 includes:
s7001, carrying out convolution processing on matrix information corresponding to current space information based on a stride convolution layer to obtain a first sampling matrix; the first sampling matrix represents a feature matrix corresponding to the current spatial information; the stride convolution layer can comprise 3×3 stride convolutions, and a repeated convolution module is adopted to more accurately capture information in a local window so as to more effectively extract features.
S7002, carrying out convolution processing on the first sampling matrix based on the transpose convolution layer to obtain a second sampling matrix; the matrix size of the second sampling matrix is consistent with the matrix size of the matrix information corresponding to the current space information; the transpose convolution layer may include a 3×3 transpose convolution, and employ a repeated convolution module to more accurately capture information within the partial window, thereby more effectively performing feature extraction.
S7003, performing priori learning on the second sampling matrix based on the parametric activation function of the output layer to obtain first space information; specifically, function parameters in the band parameter activation function are adjusted based on the second sampling matrix.
In the embodiment of the application, the output of the image prior processing model is adjusted by adopting the parametric activation function, so that the robustness and the generalization capability of the image prior processing model are improved, the denoising precision of the image prior processing model is improved, the risk of over-fitting is reduced, and the image reconstruction precision of the reconstructed target image is further improved.
Referring to fig. 7, an embodiment of the present application further provides an apparatus for reconstructing an image, including:
an acquisition module 101 for acquiring a compressed image;
the decomposition module 102 is configured to perform image decomposition based on the compressed image, so as to obtain three-dimensional spatial information and image spectrum information corresponding to the compressed image;
the denoising module 103 is used for denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image;
And the reconstruction module 104 is configured to perform image reconstruction based on the target spatial information and the image spectrum information, so as to obtain a target image corresponding to the compressed image.
The denoising module comprises:
a first information determining unit configured to determine three-dimensional space information as current space information;
the denoising unit is used for denoising the current space information based on the image prior processing model and the image denoising model to obtain the current denoising space information;
the second information determining unit is used for determining the current denoising space information as the current space information under the condition that the current denoising space information and the last denoising space information do not meet the preset error condition;
a first loop unit configured to repeatedly perform: denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information, wherein the current denoising space information is determined to be the current space information until the current denoising space information and the last denoising space information meet the preset error condition under the condition that the current denoising space information and the last denoising space information do not meet the preset error condition;
and a third information determining unit for determining the current denoising spatial information as target spatial information.
The denoising unit includes:
the first denoising processing unit is used for inputting the current spatial information into the image prior processing model to perform denoising processing to obtain first spatial information;
the second denoising processing unit is used for inputting the first spatial information into the image denoising model to perform denoising processing to obtain second spatial information;
an updating unit for updating the current cycle number;
a fourth information determining unit configured to determine the second spatial information as current spatial information in a case where the current cycle number is smaller than a preset cycle number;
a second loop unit for repeatedly performing: the current space information is input into an image prior processing model to be subjected to denoising processing, so that first space information is obtained, and the second space information is determined as the current space information until the current cycle number is greater than or equal to the preset cycle number under the condition that the current cycle number is smaller than the preset cycle number;
and a fifth information determining unit configured to determine the second spatial information as current denoising spatial information.
The decomposition module comprises:
the low-rank decomposition unit is used for carrying out low-rank decomposition on matrix information corresponding to the compressed image to obtain three-dimensional space information and image spectrum information; the three-dimensional spatial information includes multi-layered two-dimensional plane information.
The image denoising model comprises a first scaling layer, a denoising layer and a second scaling layer; the second denoising processing unit includes:
the normalization unit is used for performing normalization processing on matrix information corresponding to the multi-layer two-dimensional plane information based on the first scaling layer to obtain normalization plane information corresponding to each of the multi-layer two-dimensional plane information;
the normalization denoising unit is used for denoising the normalization plane information corresponding to each of the plurality of layers of two-dimensional plane information based on the denoising layer to obtain a plurality of layers of two-dimensional denoising information;
and the inverse normalization unit is used for performing inverse normalization processing on the matrix information corresponding to the multi-layer two-dimensional denoising information based on the second scaling layer to obtain second spatial information.
The denoising layer includes convolution layer and residual error layer, and the normalization denoising unit includes:
the convolution unit is used for carrying out convolution processing on matrix information corresponding to the normalized plane information based on the convolution layer to obtain a convolution matrix;
and the residual unit is used for carrying out residual connection based on the residual layer, matrix information corresponding to the normalized plane information and the convolution matrix, and obtaining two-dimensional denoising information corresponding to the normalized plane information.
The image prior processing model comprises a stride convolution layer, a transpose convolution layer and an output layer, and the first denoising processing unit comprises:
The first sampling unit is used for carrying out convolution processing on matrix information corresponding to the current space information based on the stride convolution layer to obtain a first sampling matrix;
the second sampling unit is used for carrying out convolution processing on the first sampling matrix based on the transpose convolution layer to obtain a second sampling matrix;
and the output unit is used for carrying out priori learning on the second sampling matrix based on the band parameter activation function of the output layer to obtain the first spatial information.
The embodiment of the application also provides electronic equipment, which comprises a processor and a memory, wherein at least one instruction or at least one section of program is stored in the memory, and the at least one instruction or the at least one section of program is loaded and executed by the processor to realize the image reconstruction method.
The memory may be used to store software programs and modules that the processor executes to perform various functional applications and data processing by executing the software programs and modules stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for functions, and the like; the storage data area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one hard disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide access to the memory by the processor.
The method embodiment provided by the embodiment of the application can be executed in electronic equipment such as a mobile terminal, a computer terminal, a server or similar computing devices. Fig. 8 is an electronic device provided in an embodiment of the present application. As shown in fig. 8, the electronic device 900 may vary considerably in configuration or performance, and may include one or more central processors 910 (the central processor 910 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA), a memory 930 for storing data, one or more storage media 920 (e.g., one or more mass storage devices) for storing applications 923 or data 922. Wherein memory 930 and storage medium 920 may be transitory or persistent storage. The program stored on the storage medium 920 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Still further, the central processor 910 may be configured to communicate with a storage medium 920 and execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input/output interfaces 940, and/or one or more operating systems 921, such as Windows ServerTM, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.
The input-output interface 940 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communications provider of the electronic device 900. In one example, the input-output interface 940 includes a network adapter (Network Interface Controller, NIC) that may be connected to other network devices through a base station to communicate with the internet. In one example, the input/output interface 940 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.
It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 8 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, electronic device 900 may also include more or fewer components than shown in FIG. 8, or have a different configuration than shown in FIG. 8.
Embodiments of the present application also provide a storage medium having at least one instruction or at least one program stored therein, the at least one instruction or the at least one program loaded and executed by a processor to implement the image reconstruction method as described above.
The foregoing description has fully disclosed specific embodiments of this application. It should be noted that any modifications to the specific embodiments of the application may be made by those skilled in the art without departing from the scope of the application as defined in the appended claims. Accordingly, the scope of the claims of the present application is not limited to the foregoing detailed description.

Claims (9)

1. A method of reconstructing an image, comprising:
acquiring a compressed image;
performing image decomposition based on the compressed image to obtain three-dimensional space information and image spectrum information corresponding to the compressed image;
denoising the three-dimensional space information based on an image prior processing model and an image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on the three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image;
performing image reconstruction based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image; the image reconstruction process characterizes an information fusion process or an inverse process of the image decomposition;
the denoising processing is carried out on the three-dimensional space information based on the image prior processing model and the image denoising model, and the obtaining of the target space information comprises the following steps:
determining the three-dimensional space information as current space information;
denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information;
Determining the current denoising space information as the current space information under the condition that the current denoising space information and the last denoising space information do not meet a preset error condition;
repeating the steps: denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information, and determining the current denoising space information as current space information until the current denoising space information and the last denoising space information meet preset error conditions under the condition that the current denoising space information and the last denoising space information do not meet preset error conditions;
and determining the current denoising space information as the target space information.
2. The method for reconstructing an image according to claim 1, wherein said denoising the current spatial information based on the image prior processing model and the image denoising model comprises:
inputting the current spatial information into the image prior processing model for denoising processing to obtain first spatial information;
Inputting the first spatial information into the image denoising model to perform denoising processing to obtain second spatial information;
updating the current circulation times;
determining the second spatial information as the current spatial information under the condition that the current cycle number is smaller than a preset cycle number;
repeating the steps: the current space information is input into the image prior processing model to be subjected to denoising processing, first space information is obtained, and the second space information is determined to be the current space information until the current cycle number is greater than or equal to the preset cycle number under the condition that the current cycle number is smaller than the preset cycle number;
and determining the second spatial information as the current denoising spatial information.
3. The method for reconstructing an image according to claim 2, wherein said performing image decomposition based on said compressed image to obtain three-dimensional spatial information and image spectrum information corresponding to said compressed image comprises:
performing low-rank decomposition on matrix information corresponding to the compressed image to obtain the three-dimensional space information and the image spectrum information; the three-dimensional spatial information includes multi-layer two-dimensional plane information.
4. A method of reconstructing an image according to claim 3, wherein said image denoising model comprises a first scaling layer, a denoising layer and a second scaling layer; inputting the first spatial information into the image denoising model for denoising processing, and obtaining second spatial information comprises:
based on the first scaling layer, carrying out normalization processing on matrix information corresponding to the multi-layer two-dimensional plane information to obtain normalized plane information corresponding to each of the multi-layer two-dimensional plane information;
based on the denoising layer, denoising the normalized plane information corresponding to each of the plurality of layers of two-dimensional plane information to obtain a plurality of layers of two-dimensional denoising information;
and performing inverse normalization processing on matrix information corresponding to the multi-layer two-dimensional denoising information based on the second scaling layer to obtain the second spatial information.
5. The method for reconstructing an image according to claim 4, wherein the denoising layer comprises a convolution layer and a residual layer, the denoising processing is performed on the normalized plane information corresponding to each of the plurality of layers of two-dimensional plane information based on the denoising layer, and obtaining the plurality of layers of two-dimensional denoising information comprises:
Based on the convolution layer, carrying out convolution processing on matrix information corresponding to the normalized plane information to obtain a convolution matrix;
and carrying out residual connection based on the residual layer, matrix information corresponding to the normalized plane information and the convolution matrix to obtain two-dimensional denoising information corresponding to the normalized plane information.
6. The method of claim 2, wherein the image prior processing model includes a stride convolution layer, a transpose convolution layer, and an output layer, the output layer includes a band-parameter activation function, the inputting the current spatial information into the image prior processing model for denoising, and obtaining the first spatial information includes:
based on the stride convolution layer, carrying out convolution processing on matrix information corresponding to the current space information to obtain a first sampling matrix;
based on the transpose convolution layer, carrying out convolution processing on the first sampling matrix to obtain a second sampling matrix;
and carrying out priori learning on the second sampling matrix based on the band parameter activation function of the output layer to obtain the first spatial information.
7. An image reconstruction apparatus, comprising:
The acquisition module is used for acquiring the compressed image;
the decomposition module is used for carrying out image decomposition based on the compressed image to obtain three-dimensional space information and image spectrum information corresponding to the compressed image;
the denoising module is used for denoising the three-dimensional space information based on the image prior processing model and the image denoising model to obtain target space information; the image prior processing model is a model for prior learning based on the three-dimensional space information; the image denoising model is obtained by training based on a sample image and a sample noise image corresponding to the sample image;
the reconstruction module is used for carrying out image reconstruction based on the target space information and the image spectrum information to obtain a target image corresponding to the compressed image; the image reconstruction process characterizes an information fusion process or an inverse process of the image decomposition;
the denoising processing is carried out on the three-dimensional space information based on the image prior processing model and the image denoising model, and the obtaining of the target space information comprises the following steps:
determining the three-dimensional space information as current space information;
denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information;
Determining the current denoising space information as the current space information under the condition that the current denoising space information and the last denoising space information do not meet a preset error condition;
repeating the steps: denoising the current space information based on the image prior processing model and the image denoising model to obtain current denoising space information, and determining the current denoising space information as current space information until the current denoising space information and the last denoising space information meet preset error conditions under the condition that the current denoising space information and the last denoising space information do not meet preset error conditions;
and determining the current denoising space information as the target space information.
8. A computer storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, loaded and executed by a processor to implement the method of reconstructing an image according to any one of claims 1-6.
9. An electronic device, characterized in that it comprises a processor and a memory in which at least one instruction or at least one program is stored, which is loaded and executed by the processor to implement the method of reconstructing an image according to any of claims 1-6.
CN202310846523.7A 2023-07-11 2023-07-11 Image reconstruction method, device and computer storage medium Active CN116563174B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310846523.7A CN116563174B (en) 2023-07-11 2023-07-11 Image reconstruction method, device and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310846523.7A CN116563174B (en) 2023-07-11 2023-07-11 Image reconstruction method, device and computer storage medium

Publications (2)

Publication Number Publication Date
CN116563174A CN116563174A (en) 2023-08-08
CN116563174B true CN116563174B (en) 2023-09-29

Family

ID=87498626

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310846523.7A Active CN116563174B (en) 2023-07-11 2023-07-11 Image reconstruction method, device and computer storage medium

Country Status (1)

Country Link
CN (1) CN116563174B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110501072A (en) * 2019-08-26 2019-11-26 北京理工大学 A kind of reconstructing method of the fast illuminated spectrum imaging system based on the constraint of tensor low-rank
CN113508417A (en) * 2020-06-08 2021-10-15 广州超视计生物科技有限公司 Image processing system and method
CN114612297A (en) * 2022-01-24 2022-06-10 北京工业大学 Hyperspectral image super-resolution reconstruction method and device
CN114677300A (en) * 2022-03-25 2022-06-28 西安交通大学 Hyperspectral image depth noise reduction method and system based on two-stage learning framework
CN114842308A (en) * 2022-03-16 2022-08-02 电子科技大学 Method for establishing target prejudgment optimization model based on full-feature fusion
CN115908180A (en) * 2022-11-18 2023-04-04 大连海事大学 Hyperspectral remote sensing image denoising method coupling depth prior and low-rank tensor decomposition
CN115950837A (en) * 2023-03-10 2023-04-11 湖南师范大学 Snapshot type spectral imaging method, system and medium based on plug-and-play prior
WO2023114923A1 (en) * 2021-12-17 2023-06-22 The Regents Of The University Of California Fusion of deep-learning based image reconstruction with noisy image measurements

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100738075B1 (en) * 2005-09-09 2007-07-12 삼성전자주식회사 Apparatus and method for encoding and decoding image
US10861143B2 (en) * 2017-09-27 2020-12-08 Korea Advanced Institute Of Science And Technology Method and apparatus for reconstructing hyperspectral image using artificial intelligence
US11125844B2 (en) * 2018-07-06 2021-09-21 The Board Of Trustees Of The Leland Stanford Junior University Deep learning based methods to accelerate multi-spectral imaging
US11035919B2 (en) * 2019-03-01 2021-06-15 Siemens Healthcare Gmbh Image reconstruction using a colored noise model with magnetic resonance compressed sensing
CN112001856A (en) * 2020-07-29 2020-11-27 东软医疗***股份有限公司 Training method of denoising model, image noise removing method and related device
CN114840831A (en) * 2021-02-01 2022-08-02 腾讯科技(深圳)有限公司 Face image validity verification method and device, electronic equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110501072A (en) * 2019-08-26 2019-11-26 北京理工大学 A kind of reconstructing method of the fast illuminated spectrum imaging system based on the constraint of tensor low-rank
CN113508417A (en) * 2020-06-08 2021-10-15 广州超视计生物科技有限公司 Image processing system and method
WO2023114923A1 (en) * 2021-12-17 2023-06-22 The Regents Of The University Of California Fusion of deep-learning based image reconstruction with noisy image measurements
CN114612297A (en) * 2022-01-24 2022-06-10 北京工业大学 Hyperspectral image super-resolution reconstruction method and device
CN114842308A (en) * 2022-03-16 2022-08-02 电子科技大学 Method for establishing target prejudgment optimization model based on full-feature fusion
CN114677300A (en) * 2022-03-25 2022-06-28 西安交通大学 Hyperspectral image depth noise reduction method and system based on two-stage learning framework
CN115908180A (en) * 2022-11-18 2023-04-04 大连海事大学 Hyperspectral remote sensing image denoising method coupling depth prior and low-rank tensor decomposition
CN115950837A (en) * 2023-03-10 2023-04-11 湖南师范大学 Snapshot type spectral imaging method, system and medium based on plug-and-play prior

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging;Ziyi Meng;2021 IEEE/CVF International Conference on Computer Vision (ICCV);全文 *
基于卷积神经网络的红外图像去噪方法研究;朱宜生;孙成;;环境技术(第06期);全文 *
基于非局部相似联合低秩表示的高光谱图像去噪;张显;叶军;;计算机科学(第01期);全文 *
空间光谱联合稀疏表示的高光谱图像超分辨率方法;许蒙恩;谢宝陵;徐国明;;激光与光电子学进展(07);全文 *
计算成像***中的模糊图像重建研究;张志刚;李涛;张玉霞;刘艳辉;;激光杂志(07);全文 *

Also Published As

Publication number Publication date
CN116563174A (en) 2023-08-08

Similar Documents

Publication Publication Date Title
Huang et al. Deep gaussian scale mixture prior for spectral compressive imaging
Zhuang et al. Fast hyperspectral image denoising and inpainting based on low-rank and sparse representations
Xiong et al. MAC-Net: Model-aided nonlocal neural network for hyperspectral image denoising
Chen et al. Hyperspectral image compressive sensing reconstruction using subspace-based nonlocal tensor ring decomposition
US9349072B2 (en) Local feature based image compression
Chen et al. Sparse hyperspectral unmixing based on constrained lp-l 2 optimization
EP2511680A2 (en) Optimized orthonormal system and method for reducing dimensionality of hyperspectral Images
US9607362B2 (en) Compressive imaging using approximate message passing with denoising
CN111579506B (en) Multi-camera hyperspectral imaging method, system and medium based on deep learning
Garcia et al. Multi-resolution compressive spectral imaging reconstruction from single pixel measurements
CN110378975B (en) Compressed coding aperture imaging method and system based on deep neural network
CN115950837B (en) Snapshot type spectrum imaging method, system and medium based on plug-and-play priori
Zhao et al. Hyperspectral image denoising using spectral-spatial transform-based sparse and low-rank representations
CN114419392A (en) Hyperspectral snapshot image recovery method, device, equipment and medium
Gelvez et al. Nonlocal low-rank abundance prior for compressive spectral image fusion
CN116579966B (en) Image reconstruction method and device, electronic equipment and storage medium
Ahmed et al. PIQI: perceptual image quality index based on ensemble of Gaussian process regression
Wei et al. Effects of lossy compression on remote sensing image classification based on convolutional sparse coding
CN117333409A (en) Big data analysis method based on image
Rajaei An analysis and improvement of the BLS-GSM denoising method
Freitas et al. Image quality assessment using BSIF, CLBP, LCP, and LPQ operators
George Reconstruction of cloud-contaminated satellite remote sensing images using kernel PCA-based image modelling
CN116563174B (en) Image reconstruction method, device and computer storage medium
Cai et al. Towards a blind image quality evaluator using multi-scale second-order statistics
Aetesam et al. Proximal approach to denoising hyperspectral images under mixed‐noise model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant