CN114663544A - Electrical impedance image reconstruction method based on depth image prior - Google Patents
Electrical impedance image reconstruction method based on depth image prior Download PDFInfo
- Publication number
- CN114663544A CN114663544A CN202210578270.5A CN202210578270A CN114663544A CN 114663544 A CN114663544 A CN 114663544A CN 202210578270 A CN202210578270 A CN 202210578270A CN 114663544 A CN114663544 A CN 114663544A
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- convolutional neural
- deep convolutional
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 103
- 238000013507 mapping Methods 0.000 claims abstract description 39
- 230000009466 transformation Effects 0.000 claims abstract description 39
- 238000005457 optimization Methods 0.000 claims abstract description 37
- 238000013528 artificial neural network Methods 0.000 claims abstract description 27
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 239000002131 composite material Substances 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 88
- 238000009826 distribution Methods 0.000 claims description 31
- 238000003860 storage Methods 0.000 claims description 17
- 238000005259 measurement Methods 0.000 claims description 9
- 230000003190 augmentative effect Effects 0.000 claims description 7
- 238000013329 compounding Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 description 22
- 238000002593 electrical impedance tomography Methods 0.000 description 16
- 238000012549 training Methods 0.000 description 10
- 230000015654 memory Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012804 iterative process Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000002591 computed tomography Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 238000002600 positron emission tomography Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an electrical impedance image reconstruction method based on depth image prior, which comprises the following steps: acquiring mapping transformation between an output image of the deep convolutional neural network and finite element grid data; performing composite operation with the calculated voltage function and the regularization function respectively by using mapping transformation to obtain a composite calculated voltage function and a composite regularization function; inputting a random noise image or a conductivity prior reference image into a depth convolution neural network to obtain an unknown image; respectively inputting the unknown image into the compounded calculation voltage function and the compounded regularization function to obtain a calculation voltage and a regularization value; inputting the measured data, the calculated voltage and the regularization value into a loss function to obtain a loss value; according to the loss value, parameters of the deep convolutional neural network are iteratively optimized by using an optimization algorithm to obtain a parameter-optimized deep convolutional neural network; and outputting a reconstructed image of the electrical impedance image by using the mapping transformation and the parameter optimized deep convolutional neural network.
Description
Technical Field
The invention relates to the technical field of electrical impedance tomography, in particular to an electrical impedance image reconstruction method based on depth image prior, electronic equipment and a storage medium.
Background
Electrical Impedance Tomography (EIT) is a functional imaging technique that reconstructs two-dimensional or three-dimensional Electrical impedance distribution images in the human body by applying a tiny safe excitation current (voltage) to the body surface and measuring the corresponding response voltage (current) with electrodes placed on the body surface. Due to the advantages of no damage, no radiation, low cost, portability and the like, the technology is widely concerned and applied in the fields of medicine, industry, geophysical and the like.
However, due to the nonlinear and highly ill-posed nature of EIT inversion reconstruction, EIT images suffer from low spatial resolution, poor contrast, and the like. Therefore, the research and development of the EIT system and the inversion algorithm with high precision and stable performance, the improvement of the imaging quality and the exploration of the application of the EIT system and the inversion algorithm in the clinical medicine and non-medical fields are the current hot and difficult problems.
With the rapid development of computer technology and the continuous improvement of computational power level in recent years, machine learning algorithms are widely applied to the fields of computer vision, image and voice recognition and the like, and a series of research results are obtained. In the field of medical Imaging, a great deal of research work has been performed in recent years on processing and reconstructing Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET) images by using a machine learning algorithm. Meanwhile, methods for reconstructing images by using convolutional neural networks, LeNet networks, U-net networks and the like appear in the EIT field, and the methods fit data to images or nonlinear relations between the images through training data. Compared with the traditional EIT method, the EIT method based on machine learning has the advantages that the prior information is flexibly introduced through training data, the imaging speed is high after model training is finished, and the like, and meanwhile, the resolution of a reconstructed image is also improved. However, accurate images of the distribution of the electrical impedance inside the human body are usually difficult to obtain, and the number of training sets is limited, which has a great influence on the accuracy and generalization capability of the training model.
Disclosure of Invention
In view of the above problems, the present invention provides an electrical impedance image reconstruction method based on depth image prior, an electronic device, and a storage medium, which are intended to solve one of the above problems.
According to a first aspect of the invention, there is provided an electrical impedance image reconstruction method based on depth image priors, comprising:
acquiring mapping transformation between an output image of the deep convolutional neural network and finite element grid data, wherein the output image of the deep convolutional neural network is used for representing a conductivity distribution image;
performing composite operation with the calculated voltage function and the regularization function respectively by using mapping transformation to obtain a composite calculated voltage function and a composite regularization function;
inputting the random noise image or the conductivity prior reference image into a depth convolution neural network to obtain an unknown image;
respectively inputting the unknown image into the compounded calculation voltage function and the compounded regularization function to obtain a calculation voltage and a regularization value;
inputting the measured data, the calculated voltage and the regularization value into a loss function to obtain a loss value;
according to the loss value, parameters of the deep convolutional neural network are iteratively optimized by using an optimization algorithm to obtain a parameter-optimized deep convolutional neural network;
and outputting a reconstructed image of the electrical impedance image by using the mapping transformation and the parameter optimized deep convolutional neural network.
According to an embodiment of the present invention, the iteratively optimizing the parameters of the deep convolutional neural network by using an optimization algorithm according to the loss value to obtain the parameter-optimized deep convolutional neural network includes:
taking the parameters of the deep convolutional neural network as initial values of an optimization algorithm to obtain an output image of the deep convolutional neural network;
mapping the output image of the deep convolutional neural network to finite element grid data by using mapping transformation to obtain a conductivity distribution map, a calculated voltage value and a Jacobian matrix;
calculating the gradient of an output image of the deep convolutional neural network relative to the parameters of the deep convolutional neural network by using a reverse transmission method, and calculating the gradient and the gradient of the loss function relative to the conductivity to obtain the gradient of the loss function relative to the deep convolutional neural network;
calculating an iteration direction by utilizing an optimization algorithm according to the gradient of the loss function relative to the deep convolutional neural network;
updating parameters of the deep convolutional neural network according to the iteration direction and the preset learning rate, and obtaining new conductivity distribution and a calculated voltage value by using the updated deep convolutional neural network;
and when the iteration times meet a preset condition, obtaining a parameter optimized deep convolutional neural network.
According to the embodiment of the invention, the optimization algorithm comprises an alternating direction multiplier method and a random optimization method of adaptive momentum.
According to an embodiment of the present invention, the above-mentioned loss function is determined by equation (1):
wherein,
which represents the measurement data, is,
representing the calculated voltage function after being compounded with the mapping transformation,
representing the regularization function after composition with the mapping transformation,
a deep convolutional neural network is represented that,
parameters that represent a deep convolutional neural network,
representing a random noise image or a conductivity prior image,
show about
Is measured.
According to an embodiment of the present invention, the loss function is solved by an augmented lagrange function determined by equation (2):
wherein,
is a function of the lagrange multiplier(s),
in order to be a lagrange multiplier,
indicating that the conductivity is transformed from the form of grid data to the form of image data of the same size as the deep convolutional neural network output,
representing a loss function with lagrange multipliers,
show about
Is measured.
According to the embodiment of the invention, each parameter can be iteratively solved through formulas (3) - (5) by the augmented lagrangian function:
wherein,
is the number of iterations.
According to an embodiment of the present invention, the regularization function adopts a full-variation regularization mode or a regularization mode based on a support set.
According to a second aspect of the invention, there is provided an electrical impedance image reconstruction system based on a depth image prior, comprising:
the mapping transformation acquisition module is used for acquiring mapping transformation between an output image of the deep convolutional neural network and finite element grid data, wherein the output image of the deep convolutional neural network is used for representing a conductivity distribution image;
the compounding module is used for respectively carrying out compounding operation with the calculated voltage function and the regularization function by utilizing mapping transformation to obtain a compounded calculated voltage function and a compounded regularization function;
the unknown image obtaining module is used for inputting the random noise image or the conductivity prior reference image into the depth convolution neural network to obtain an unknown image;
the acquisition module is used for respectively inputting the unknown image into the compounded calculation voltage function and the compounded regularization function to obtain a calculation voltage and a regularization value;
the loss value obtaining module is used for inputting the measured data, the calculated voltage and the regularization value into a loss function to obtain a loss value;
the parameter optimization module is used for iteratively optimizing parameters of the deep convolutional neural network by utilizing an optimization algorithm according to the loss value to obtain the parameter optimized deep convolutional neural network;
and the image reconstruction module is used for outputting a reconstructed image of the electrical impedance image by using the mapping transformation and the parameter optimized deep convolutional neural network.
According to a third aspect of the present invention, there is provided an electronic apparatus comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described depth image-a priori-based electrical impedance image reconstruction method.
According to a fourth aspect of the invention, there is provided a computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the above-described method of electrical impedance image reconstruction based on depth image priors.
The invention realizes the reconstruction of the conductivity distribution image in the iterative process by taking the output of the deep convolutional neural network as the conductivity distribution image, carrying out parametric expression on the conductivity distribution and optimizing the parameters of the deep convolutional neural network. The electrical impedance image reconstruction method based on depth image prior provided by the invention can improve the electrical impedance image reconstruction precision and solve a series of problems of low spatial resolution, poor imaging stability and the like of electrical impedance reconstruction images caused by uncertain factors such as noise, errors and the like to a certain extent. Meanwhile, the electrical impedance imaging method based on depth image prior provided by the invention not only has high resolution image reconstruction capability, but also has good robustness on model errors, measurement noises and the like commonly existing in medical electrical impedance imaging.
Drawings
FIG. 1 shows a flow chart of a method of electrical impedance image reconstruction based on depth image priors according to an embodiment of the invention;
FIG. 2 is a flow diagram of obtaining a parameter optimized deep convolutional neural network according to an embodiment of the present invention;
FIG. 3 is a block diagram of an electrical impedance image reconstruction method according to an embodiment of the invention;
FIG. 4 is a schematic diagram of a depth image prior based electrical impedance image reconstruction system according to an embodiment of the present invention;
fig. 5 schematically shows a block diagram of an electronic device adapted to implement a depth image prior based electrical impedance image reconstruction method according to an embodiment of the invention.
Detailed Description
In order that the objects, technical solutions and advantages of the present invention will become more apparent, the present invention will be further described in detail with reference to the accompanying drawings in conjunction with the following specific embodiments.
The invention aims to realize an electrical impedance image reconstruction technology without pre-training based on depth image prior, and aims to improve electrical impedance image reconstruction precision and solve series problems of low spatial resolution, poor imaging stability and the like of electrical impedance reconstruction images caused by uncertain factors such as noise, errors and the like to a certain extent.
The invention realizes the solution of the conductivity distribution image in the iterative process by taking the output of the neural network as the conductivity distribution image, carrying out parametric expression on the conductivity distribution and optimizing the parameters of the neural network. The neural network parameters can be updated by optimization methods such as Adam and ADMM, and derivatives of the loss function related to the neural network parameters in the solving process can be obtained through a chain rule. And finally realizing the reconstruction of the conductivity distribution image after iterating the minimized loss function. The invention provides an electrical impedance imaging technology based on depth image prior, which not only has high resolution image reconstruction capability, but also has good robustness to model errors, measurement noises and the like commonly existing in medical electrical impedance imaging.
Deep convolutional neural networks have become a mainstream tool for image generation and image restoration. Research work based on DIP (Deep Image Prior) shows that the network structure has a certain implicit regularization capacity. In the DIP method, good results can be obtained in standard inverse problems, such as image denoising, super-resolution solution, and image inpainting problems, by inputting a random noise image. Based on the research work, the DIP technology is applied to electrical impedance imaging, and the purpose is to realize a depth electrical impedance image reconstruction method without pre-training.
The image reconstruction method provided by the invention mainly improves the EIT inverse problem, does not directly optimize the conductivity, and optimizes the parameters of the deep convolutional neural network after the conductivity is parameterized by the deep convolutional neural network, so as to reconstruct and obtain the conductivity distribution image.
Fig. 1 shows a flow chart of a method of electrical impedance image reconstruction based on depth image priors according to an embodiment of the invention.
As shown in FIG. 1, the electrical impedance image reconstruction method based on depth image prior comprises operation S110-operation S170.
In operation S110, a mapping transformation between an output image of the deep convolutional neural network and finite element mesh data is acquired, wherein the output image of the deep convolutional neural network is used to represent the conductivity distribution image.
For the area needing image reconstruction, analyzing according to a Finite Element Method (FEM), and establishing an FEM model to solve corresponding calculated voltage and a Jacobian matrix. Since the conductivity is generally defined on a triangular mesh, and the output of the deep convolutional neural network is a rectangular mesh, a transformation operation, i.e., a mapping transformation described in operation S110, is required between the output of the deep convolutional neural network and the conductivity data.
The output of the deep convolutional neural network is used as a conductivity distribution image, so that the conductivity distribution is parameterized.
In operation S120, a composite operation is performed with the calculated voltage function and the regularization function by using mapping transformation, so as to obtain a composite calculated voltage function and a composite regularization function.
In operation S130, the random noise image or the conductivity prior reference image is input to the deep convolutional neural network, resulting in an unknown image.
Because the input data of the deep convolutional neural network has higher flexibility, a random noise image can be used as the input of the deep convolutional neural network, and the conductivity prior reference image can be used as a support set to construct a support prior, so that multi-mode fusion is realized, and the imaging precision and the imaging stability are improved; the unknown image represents an output image of the deep convolutional neural network when the input is a random noise image or a conductivity prior reference image.
The random noise image or the conductivity prior reference image is used as the input of the neural network, and the conductivity distribution image can be changed by changing the parameters of the deep convolution neural network.
In operation S140, the unknown image is respectively input to the compounded calculated voltage function and the compounded regularization function, so as to obtain a calculated voltage and a regularization value.
In operation S150, the measurement data, the calculated voltage, and the regularization value are input into a loss function, resulting in a loss value.
In operation S160, parameters of the deep convolutional neural network are iteratively optimized by using an optimization algorithm according to the loss value, so as to obtain a parameter-optimized deep convolutional neural network.
In operation S170, a reconstructed image of the electrical impedance image is output using the mapping transformation and the parameter-optimized deep convolutional neural network.
The invention realizes the reconstruction of the conductivity distribution image in the iterative process by taking the output of the deep convolution neural network as the conductivity distribution image, carrying out parametric expression on the conductivity distribution and optimizing the parameters of the deep convolution neural network. The electrical impedance image reconstruction method based on depth image prior provided by the invention can improve the electrical impedance image reconstruction precision and solve a series of problems of low spatial resolution, poor imaging stability and the like of electrical impedance reconstruction images caused by uncertain factors such as noise, errors and the like to a certain extent. Meanwhile, the electrical impedance imaging method based on depth image prior provided by the invention not only has high resolution image reconstruction capability, but also has good robustness on model errors, measurement noise and the like commonly existing in medical electrical impedance imaging.
FIG. 2 is a flow diagram of obtaining a parameter optimized deep convolutional neural network according to an embodiment of the present invention.
As shown in fig. 2, iteratively optimizing parameters of the deep convolutional neural network by using an optimization algorithm according to the loss value to obtain the parameter-optimized deep convolutional neural network includes operations S210 to S260.
In operation S210, an output image of the deep convolutional neural network is obtained by using the parameters of the deep convolutional neural network as initial values of the optimization algorithm.
In operation S220, the output image of the deep convolutional neural network is mapped to finite element mesh data using a mapping transformation, so as to obtain a conductivity distribution map, a calculated voltage value, and a jacobian matrix.
Taking the initialized neural network parameters as the initial value Of an optimization iterative Algorithm (ADMM), mapping the neural network output to FEM grid data after obtaining the neural network output, and solving the initial conductivity distribution, the calculation voltage value and the Jacobian matrix.
In operation S230, a gradient of an output image of the deep convolutional neural network with respect to a parameter of the deep convolutional neural network is calculated using an inverse transmission method, and the gradient and a gradient of the loss function with respect to the conductivity are calculated to obtain a gradient of the loss function with respect to the deep convolutional neural network.
In operation S240, an iteration direction is calculated using an optimization algorithm according to a gradient of the loss function with respect to the deep convolutional neural network.
In operation S250, parameters of the deep convolutional neural network are updated according to the iteration direction and the preset learning rate, and a new conductivity distribution and a calculated voltage value are obtained using the updated deep convolutional neural network.
The operation utilizes a back propagation method to calculate the gradient of the neural network output relative to the neural network parameters, and the gradient of the loss function relative to the neural network parameters is obtained by combining the gradient of the loss function relative to the conductivity, so that the method is used for calculating the iteration direction by the Adam/ADMM method; at the same time, the parameters of the neural network are updated according to the iteration direction and the predetermined learning rate, a new conductivity distribution is calculated, and a voltage value is calculated.
In operation S260, when the number of iterations satisfies a preset condition, a parameter-optimized deep convolutional neural network is obtained.
Determining whether a termination condition (which may be set here to a number of iterations greater than some given value) is satisfied: if yes, terminating, and reconstructing the image; if not, continuing the iteration.
According to the method, the parameters of the deep convolutional neural network are optimized by utilizing an optimization algorithm according to the loss value, so that the parameters with higher precision can be obtained, and the deep convolutional neural network is ensured to obtain the electrical impedance reconstruction image with high imaging precision and spatial resolution in the subsequent image reconstruction process.
According to the embodiment of the invention, the optimization algorithm comprises an alternating direction multiplier method and a random optimization method of adaptive momentum.
The Alternating Direction Multiplier Method (ADMM) can avoid the problem of gradient disappearance in the parameter optimization process of the deep convolutional neural network, and has strong expansibility; the adaptive momentum random optimization method (Adam) can more efficiently utilize gradient information and realize efficient optimization of parameters of the deep convolutional neural network.
According to an embodiment of the present invention, the above-mentioned loss function is determined by equation (1):
wherein,
which represents the measurement data, is,
representing the calculated voltage function after being compounded with the mapping transformation,
representing the regularization function after composition with the mapping transformation,
a deep convolutional neural network is represented that,
parameters that represent a deep convolutional neural network,
representing a random noise image or a conductivity prior image,
show about
Is measured.
According to an embodiment of the present invention, the loss function is solved by an augmented lagrange function determined by equation (2):
wherein,
is a function of the lagrange multiplier(s),
in order to be a lagrange multiplier,
indicating that the conductivity is transformed from the form of grid data to the form of image data of the same size as the deep convolutional neural network output,
representing a loss function with lagrange multipliers,
show about
Is measured.
According to the embodiment of the present invention, the augmented lagrangian function can iteratively solve each parameter through formulas (3) - (5):
wherein,
is the number of iterations.
According to an embodiment of the present invention, the regularization function adopts a full-variation regularization mode or a regularization mode based on a support set.
Fig. 3 is a schematic block diagram of an electrical impedance image reconstruction method according to an embodiment of the present invention.
The electrical impedance image reconstruction method provided by the present invention is further described in detail below with reference to fig. 3.
First, mathematically, the EIT technique is a positive and negative problem of solving an elliptic partial differential equation, and an observation model thereof can be expressed by the following formula (6):
wherein,
the measured data is shown to be the data of,
for the conductivity to be solved for,
is of electrical conductivity of
When the voltage of the corresponding calculation is measured,
is additive noise.
In EIT static imaging, the conductivity can be solved by optimizing the following objective function
As shown in equation (7):
wherein the regularization function
According to the practical application, for example, a fully-variant regularization mode or a regularization mode based on a support set can be selected.
As shown in fig. 3, in the reconstruction method based on the DIP frame, the output of the network can be regarded as a conductivity image, as shown in equation (8):
wherein,
representing the output image of the neural network,
a neural network is represented that is a network of nerves,
representing parameters in a deep convolutional neural network,
representing a random noise image or a conductivity prior image, is used as an input to the neural network.
Conductivity when solving EIT positive problem using finite element method
Typically defined on a triangular mesh. Therefore, electrical conductivity
And neural network output image
There is a transformation operation, as shown in equation (9):
due to the change
Is fixed, will
And with
Is compounded with
Record as
， 
And
is compounded with
Record as
(it can also be directed to
A regularization term is designed) when the objective function (7) becomes the objective function shown in equation (1):
the conductivity distribution can thus be obtained by maximum a posteriori estimation
As shown in equation (10) and equation (11):
wherein,
is an optimum parameter for representing
And (3) taking the corresponding parameter when the minimum value is obtained, and because the deep convolutional neural network and the EIT positive problem are fused in the optimization process, the direct solution of the equation (11) is relatively difficult. Therefore, it can be changed to the optimization problem with constraints as shown in equation (12):
wherein,
indicating that the conductivity is transformed from the form of grid data to the form of image data of the same size as the deep convolutional neural network output. Thus, the problem (12) can be solved by employing an optimization algorithm, such as Alternating Direction Multiplier Method (ADMM). To this end, an augmented Lagrangian function of the optimization problem (12) is obtained, as shown in equation (2):
wherein,
is a function of the lagrange multiplier and,
is a lagrange multiplier. Then, the parameters can be solved by adopting the formula (3) to the formula (5) alternatively and iteratively:
wherein,
is the number of iterations.
Fig. 3 is a process of inputting a random noise image or a known reference image into a deep convolutional neural network, and finally obtaining an EIT reconstructed image by continuously updating parameters of the neural network, wherein the EIT reconstructed image is divided into three parts, namely mapping transformation acquisition, parameter optimization solution of the deep convolutional neural network, and image output.
FIG. 4 is a schematic diagram of an electrical impedance image reconstruction system based on depth image priors, in accordance with an embodiment of the present invention.
As shown in fig. 4, the depth image prior-based electrical impedance image reconstruction system 400 includes a mapping transformation acquisition module 410, a composition module 420, an unknown image acquisition module 430, an acquisition module 440, a loss value acquisition module 450, a parameter optimization module 460, and an image reconstruction module 470.
A mapping transformation obtaining module 410, configured to obtain a mapping transformation between an output image of the deep convolutional neural network and the finite element grid data, where the output image of the deep convolutional neural network is used to represent the conductivity distribution image;
a compounding module 420, configured to perform a compounding operation with the calculated voltage function and the regularization function respectively by using mapping transformation, so as to obtain a compounded calculated voltage function and a compounded regularization function;
an unknown image obtaining module 430, configured to input the random noise image or the conductivity prior reference image to a deep convolutional neural network to obtain an unknown image;
an obtaining module 440, configured to input the unknown image into the combined calculated voltage function and the combined regularization function, respectively, to obtain a calculated voltage and a regularization value;
a loss value obtaining module 450, configured to input the measured data, the calculated voltage, and the regularization value into a loss function to obtain a loss value;
the parameter optimization module 460 is configured to iteratively optimize parameters of the deep convolutional neural network by using an optimization algorithm according to the loss value to obtain a parameter-optimized deep convolutional neural network;
and an image reconstruction module 470 for outputting a reconstructed image of the electrical impedance image using the mapping transformation and the parameter optimized deep convolutional neural network.
The electrical impedance image reconstruction system based on the DIP can solve the problems of denoising, super-resolution, image restoration and the like by using the DIP, and has the advantages in many aspects compared with the prior art: the image reconstruction system does not need pre-training and does not depend on a training data set, so that potential deviation caused by limited number of the training data sets is avoided; meanwhile, the system has higher flexibility, and not only can input the conductivity prior reference image, but also can input a random noise image; in addition, the system has the technical characteristics of multi-modal imaging, and regularization functions are designed
The reference image structure prior information can be directly fused into a loss function, for example, wavelet analysis operation is respectively carried out on the reference image and an output image of a neural network, a part of the reference image with large amplitude in a wavelet domain is taken as a support set to construct support prior, multi-mode fusion is further realized, and imaging precision and imaging stability are improved; the reconstructed electrical impedance image obtained by the system has high imaging precision and spatial resolution, and has good robustness to model errors, measurement noise and the like commonly existing in medical electrical impedance imaging.
Fig. 5 schematically shows a block diagram of an electronic device adapted to implement a depth image a priori based electrical impedance image reconstruction method according to an embodiment of the invention.
As shown in fig. 5, an electronic device 500 according to an embodiment of the present invention includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may comprise, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 501 may also include onboard memory for caching purposes. Processor 501 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the present invention.
In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present invention by executing programs in the ROM 502 and/or the RAM 503. Note that the programs may also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of method flows according to embodiments of the present invention by executing programs stored in the one or more memories.
According to an embodiment of the present invention, electronic device 500 may also include an input/output (I/O) interface 505, input/output (I/O) interface 505 also being connected to bus 504. The electronic device 500 may also include one or more of the following components connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. A drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
The present invention also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the present invention.
According to embodiments of the present invention, the computer readable storage medium may be a non-volatile computer readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the invention, a computer-readable storage medium may include ROM 502 and/or RAM 503 and/or one or more memories other than ROM 502 and RAM 503 as described above.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An electrical impedance image reconstruction method based on depth image prior comprises the following steps:
acquiring mapping transformation between an output image of a deep convolutional neural network and finite element grid data, wherein the output image of the deep convolutional neural network is used for representing a conductivity distribution image;
performing composite operation with the calculated voltage function and the regularization function respectively by using the mapping transformation to obtain a composite calculated voltage function and a composite regularization function;
inputting a random noise image or a conductivity priori reference image into the depth convolution neural network to obtain an unknown image;
respectively inputting the unknown image into the compounded calculation voltage function and the compounded regularization function to obtain a calculation voltage and a regularization value;
inputting the measured data, the calculated voltage and the regularization value into a loss function to obtain a loss value;
according to the loss value, parameters of the deep convolutional neural network are iteratively optimized by using an optimization algorithm to obtain a parameter-optimized deep convolutional neural network;
and outputting a reconstructed image of the electrical impedance image by using the mapping transformation and the parameter-optimized deep convolution neural network.
2. The method of claim 1, wherein iteratively optimizing parameters of the deep convolutional neural network using an optimization algorithm based on the loss values to obtain a parameter-optimized deep convolutional neural network comprises:
taking the parameters of the deep convolutional neural network as initial values of the optimization algorithm to obtain an output image of the deep convolutional neural network;
mapping the output image of the deep convolutional neural network to finite element grid data by using the mapping transformation to obtain a conductivity distribution map, a calculated voltage value and a Jacobian matrix;
calculating the gradient of an output image of the deep convolutional neural network relative to the parameters of the deep convolutional neural network by using a reverse transmission method, and calculating the gradient and the gradient of the loss function relative to the conductivity to obtain the gradient of the loss function relative to the deep convolutional neural network;
calculating an iteration direction by utilizing an optimization algorithm according to the gradient of the loss function relative to the deep convolutional neural network;
updating parameters of the deep convolutional neural network according to the iteration direction and a preset learning rate, and obtaining new conductivity distribution and a calculated voltage value by using the updated deep convolutional neural network;
and when the iteration times meet a preset condition, obtaining a parameter optimized deep convolutional neural network.
3. The method according to claim 1 or 2, wherein the optimization algorithm comprises an alternating direction multiplier method and an adaptive momentum stochastic optimization method.
4. The method of claim 1, wherein the loss function is determined by equation (1):
wherein,
which represents the measurement data, is,
representing a calculated voltage function compounded with the mapping transformation,
representing the regularization function as composited with the mapping transformation,
representing the deep convolutional neural network in a manner that,
a parameter representing the deep convolutional neural network,
representing the random noise image or the conductivity prior image,
show about
Is measured.
5. The method of claim 4, wherein the loss function is solved by an augmented Lagrangian function determined by equation (2):
wherein,
is a function of the lagrange multiplier and,
in order to be a lagrange multiplier,
representing the transformation of the conductivity from the form of grid data to the form of image data of the same size as the deep convolutional neural network output,
representing a loss function with lagrange multipliers,
show about
Is measured.
6. The method of claim 5, wherein the augmented Lagrangian function can solve each parameter iteratively through equations (3) - (5):
wherein,
is the number of iterations.
7. The method of claim 1, wherein the regularization function employs a fully variant regularization approach or a support set based regularization approach.
8. An electrical impedance image reconstruction system based on depth image priors, comprising:
the mapping transformation acquisition module is used for acquiring mapping transformation between an output image of the deep convolutional neural network and finite element grid data, wherein the output image of the deep convolutional neural network is used for representing a conductivity distribution image;
the compounding module is used for respectively carrying out compounding operation with the calculated voltage function and the regularization function by utilizing the mapping transformation to obtain a compounded calculated voltage function and a compounded regularization function;
an unknown image obtaining module, configured to input a random noise image or a conductivity prior reference image to the deep convolutional neural network to obtain an unknown image;
an obtaining module, configured to input the unknown image into the compounded computed voltage function and the compounded regularized function respectively to obtain a computed voltage and a regularized value;
the loss value obtaining module is used for inputting the measurement data, the calculation voltage and the regularization value into a loss function to obtain a loss value;
the parameter optimization module is used for iteratively optimizing the parameters of the deep convolutional neural network by utilizing an optimization algorithm according to the loss value to obtain a parameter-optimized deep convolutional neural network;
and the image reconstruction module is used for outputting a reconstructed image of the electrical impedance image by utilizing the mapping transformation and the parameter optimized depth convolution neural network.
9. An electronic device, comprising:
one or more processors;
a storage device to store one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.
10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210578270.5A CN114663544A (en) | 2022-05-26 | 2022-05-26 | Electrical impedance image reconstruction method based on depth image prior |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210578270.5A CN114663544A (en) | 2022-05-26 | 2022-05-26 | Electrical impedance image reconstruction method based on depth image prior |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114663544A true CN114663544A (en) | 2022-06-24 |
Family
ID=82038295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210578270.5A Pending CN114663544A (en) | 2022-05-26 | 2022-05-26 | Electrical impedance image reconstruction method based on depth image prior |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663544A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115512172A (en) * | 2022-09-30 | 2022-12-23 | 赵营鸽 | Uncertainty quantification method for multi-dimensional parameters in electrical impedance imaging technology |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447891A (en) * | 2019-01-09 | 2019-03-08 | 北京理工大学 | A kind of high quality imaging method of the spectrum imaging system based on convolutional neural networks |
| CN109598768A (en) * | 2018-10-11 | 2019-04-09 | 天津大学 | Electricity tomographic image method for reconstructing based on convolutional neural networks |
| CN109674471A (en) * | 2018-12-18 | 2019-04-26 | 中国科学技术大学 | A kind of electrical impedance imaging method and system based on generation confrontation network |
| CN110251130A (en) * | 2019-05-24 | 2019-09-20 | 中国科学技术大学 | Electrical impedance imaging method, apparatus and system based on removable deformed component |
| CN110490832A (en) * | 2019-08-23 | 2019-11-22 | 哈尔滨工业大学 | A kind of MR image reconstruction method based on regularization depth image transcendental method |
| CN110766769A (en) * | 2019-10-23 | 2020-02-07 | 深圳先进技术研究院 | Magnetic resonance image reconstruction method, device, equipment and medium |
| CN112401865A (en) * | 2020-11-11 | 2021-02-26 | 中国科学技术大学 | Electrical impedance imaging method based on super-shape |
| CN114511666A (en) * | 2020-11-16 | 2022-05-17 | 中国科学院深圳先进技术研究院 | Model generation method, image reconstruction method, device, equipment and medium |
-
2022
- 2022-05-26 CN CN202210578270.5A patent/CN114663544A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598768A (en) * | 2018-10-11 | 2019-04-09 | 天津大学 | Electricity tomographic image method for reconstructing based on convolutional neural networks |
| CN109674471A (en) * | 2018-12-18 | 2019-04-26 | 中国科学技术大学 | A kind of electrical impedance imaging method and system based on generation confrontation network |
| CN109447891A (en) * | 2019-01-09 | 2019-03-08 | 北京理工大学 | A kind of high quality imaging method of the spectrum imaging system based on convolutional neural networks |
| CN110251130A (en) * | 2019-05-24 | 2019-09-20 | 中国科学技术大学 | Electrical impedance imaging method, apparatus and system based on removable deformed component |
| CN110490832A (en) * | 2019-08-23 | 2019-11-22 | 哈尔滨工业大学 | A kind of MR image reconstruction method based on regularization depth image transcendental method |
| CN110766769A (en) * | 2019-10-23 | 2020-02-07 | 深圳先进技术研究院 | Magnetic resonance image reconstruction method, device, equipment and medium |
| CN112401865A (en) * | 2020-11-11 | 2021-02-26 | 中国科学技术大学 | Electrical impedance imaging method based on super-shape |
| CN114511666A (en) * | 2020-11-16 | 2022-05-17 | 中国科学院深圳先进技术研究院 | Model generation method, image reconstruction method, device, equipment and medium |
Non-Patent Citations (3)
| Title |
|---|
| FENG LI ET AL: "Landweber Iterative Image Reconstruction Method Incorporated Deep Learning for Electrical Resistance Tomography", 《 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT》 * |
| RONG FU ET AL: "A Regularization-Guided Deep Imaging Method for Electrical Impedance Tomography", 《IEEE SENSORS JOURNAL》 * |
| 赵地等: "基于支撑先验与深度图像先验的无预训练磁共振图像重建方法", 《物理学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115512172A (en) * | 2022-09-30 | 2022-12-23 | 赵营鸽 | Uncertainty quantification method for multi-dimensional parameters in electrical impedance imaging technology |
| CN115512172B (en) * | 2022-09-30 | 2023-09-15 | 赵营鸽 | Uncertainty quantification method for multidimensional parameters in electrical impedance imaging technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Huang et al. | Full-wave iterative image reconstruction in photoacoustic tomography with acoustically inhomogeneous media | |
| US11257191B2 (en) | Systems and methods for deblurring medical images using deep neural network | |
| Oh et al. | A general framework for nonlinear multigrid inversion | |
| Duits et al. | Morphological and linear scale spaces for fiber enhancement in DW-MRI | |
| WO2020114329A1 (en) | Fast magnetic resonance parametric imaging and device | |
| CN109035352B (en) | Regularization reconstruction method for L1-L2 space self-adaptive electrical tomography | |
| US20140015852A1 (en) | Method Of Processing Image Data From An Electromagnetic Tomography Machine | |
| Quan et al. | Compressed sensing reconstruction of dynamic contrast enhanced MRI using GPU-accelerated convolutional sparse coding | |
| CN112798654B (en) | Rapid gradient method and adaptive jacobian matrix reconstruction method for electrical impedance tomography | |
| CN113924503B (en) | Parameter map determination for time domain magnetic resonance | |
| Kobler et al. | Total deep variation: A stable regularization method for inverse problems | |
| CN117011673B (en) | Electrical impedance tomography image reconstruction method and device based on noise diffusion learning | |
| Brás et al. | An alternating direction algorithm for total variation reconstruction of distributed parameters | |
| CN114663544A (en) | Electrical impedance image reconstruction method based on depth image prior | |
| CN112330565A (en) | Image denoising method in low-dose CT projection domain based on improved U-net | |
| Burger et al. | Inverse problems in imaging | |
| Creusen et al. | Numerical schemes for linear and non-linear enhancement of DW-MRI | |
| WO2024109762A1 (en) | Pet parameter determination method and apparatus, and device and storage medium | |
| Dela Haije et al. | Sharpening fibers in diffusion weighted MRI via erosion | |
| Hu et al. | High-order fractional partial differential equation transform for molecular surface construction | |
| Giacomini | An equilibrated fluxes approach to the certified descent algorithm for shape optimization using conforming finite element and discontinuous Galerkin discretizations | |
| Balima et al. | New developments in frequency domain optical tomography. Part II. application with a l-bfgs associated to an inexact line search | |
| Zhou et al. | Discretization error analysis and adaptive meshing algorithms for fluorescence diffuse optical tomography in the presence of measurement noise | |
| WO2022160899A1 (en) | Three-dimensional ventilation image generation method, and controller and apparatus | |
| Guven et al. | Discretization error analysis and adaptive meshing algorithms for fluorescence diffuse optical tomography: Part I |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220624 |