CN113096208B - Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution - Google Patents
Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution Download PDFInfo
- Publication number
- CN113096208B CN113096208B CN202110278963.8A CN202110278963A CN113096208B CN 113096208 B CN113096208 B CN 113096208B CN 202110278963 A CN202110278963 A CN 202110278963A CN 113096208 B CN113096208 B CN 113096208B
- Authority
- CN
- China
- Prior art keywords
- domain
- convolution
- image
- neural network
- magnetic resonance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 31
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000003062 neural network model Methods 0.000 claims abstract description 7
- 238000010586 diagram Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 230000009977 dual effect Effects 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 238000002595 magnetic resonance imaging Methods 0.000 abstract description 14
- 238000005070 sampling Methods 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 21
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 210000004872 soft tissue Anatomy 0.000 description 2
- 238000012307 MRI technique Methods 0.000 description 1
- 241000218680 Pinus banksiana Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention relates to a reconstruction method of a neural network magnetic resonance image based on double-domain alternating convolution, which is characterized by comprising the following steps of: the method comprises the following steps: step 1, acquiring a K space data set; step 2, generating undersampled K space data; step 3, establishing a coding and decoding neural network structure of the double-domain alternating convolution; step 4, training a coding and decoding neural network model of double-domain alternating convolution by using the undersampled K space data generated in the step 2 and the image domain data obtained by performing inverse Fourier transform on the K domain information in the step 1; and 5, reconstructing undersampled magnetic resonance data by using the trained dual-domain alternating encoding and decoding neural network to obtain a magnetic resonance reconstructed image with higher definition. The invention realizes the accelerated reconstruction of the magnetic resonance imaging by using the small-kernel convolutional neural network on the K domain, can reconstruct and obtain clear magnetic resonance imaging while eliminating the artifact caused by breaking through the Nyquist sampling limit, and improves the reconstruction precision.
Description
Technical Field
The invention belongs to the technical field of image processing, relates to a reconstruction method of an under-sampled magnetic resonance image, and particularly relates to a reconstruction method of a neural network magnetic resonance image based on double-domain alternating convolution.
Background
At present, magnetic resonance imaging can achieve non-invasive visual inspection of human soft tissue, but due to its lengthy scan time and susceptibility to motion artifacts, patients experience severe discomfort during use. The scan time is directly related to the sample size of the measurement data and an undersampled scan can be performed in order to speed up the scan speed, but subject to the nyquist sampling theorem, artifacts occur in the case of sampling below the nyquist sampling rate.
Lustig M proposes a technique Compressed Sensing MRI [ Lustig M, donoho D L, santos J M, et al. Compressed Sensing MRI [ J ]. IEEE Signal Processing Magazine,2008,25 (2): 72-82 ] that recovers a high resolution image from fewer measurements using a priori information about the underlying Signal to achieve fast magnetic resonance imaging. Compressed sensing algorithms can reconstruct high resolution images from undersampled k-space data by exploiting the sparsity of the signals in certain transform domains (e.g., signal transform domains). Wavelet transform, etc. Although the CS method exhibits high performance, the computational complexity is very high, and in practice, the hyper-parameter adjustment is a difficult task. M.a. griswold designed a general auto-calibrating partially parallel acquisition (GRAPPA) parallel Magnetic resonance imaging technique [ m.a. griswold, p.m.jakob, r.m.heidemann, m.nittka, v.jellus, j.wang, b.kiefer, and a.haase, "Generalized auto-calibrating partial parallel acquisition (GRAPPA)," Magnetic resonance in media, vol.47, no.6, pp.1202-1210,2002 ]. It interpolates missing k-space data by exploiting the diversity of the coil sensitivity maps. The method is based on the brightness component of the image in the suspicious region and the median comparison of the image block, but the final reconstruction result of the algorithm is poor in structural similarity and peak signal-to-noise ratio. Jinming Duan proposes a deep learning method for parallel magnetic resonance imaging reconstruction, called variable split network (VS-Net), for efficient, high quality reconstruction of undersampled multi-coil MR data. [ Duan J, schlemper J, qin C, et al.VS-Net: variable spacing network for acceptable parallel MRI recovery [ J ].2019 ]. Variable split network (VS-Net) is a deep learning method for parallel Magnetic Resonance Imaging (MRI) reconstruction for efficient, high quality reconstruction of undersampled multi-coil magnetic resonance data. The method formulates generalized parallel compressed sensing reconstruction as an energy minimization problem, and derives a variable splitting optimization method aiming at the problem. However, the algorithm relies on the sensitivity maps generated by the multi-coil data during the reconstruction process, and is not strictly end-to-end reconstruction. HanY proposes a fully data-driven K-Space interpolation depth Learning algorithm [ HanY, sunwoo L, ye J C.k-Space Deep Learning for accessed MRI [ J ].2018 ]. The method is characterized in that the K-space deep learning and the image domain loss function are used, the interpolation of undersampled K-space data is realized through the K-space interpolation driven by complete data, and then the reconstruction of magnetic resonance imaging is realized.
In summary, in the known reconstruction algorithm based on K-space magnetic resonance imaging, the effective convolution of the K-domain convolutional neural network is not well solved, and the existing methods have the defects of large calculation amount, incapability of really realizing end-to-end reconstruction, artifact in the reconstruction result and the like.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a reconstruction method of a neural network magnetic resonance image based on double-domain alternating convolution, can realize the characteristic extraction of a K space through small-kernel convolution, and constructs a coding and decoding neural network of the double-domain convolution so as to achieve the aim of fully utilizing double-domain information to improve the reconstruction precision.
The invention solves the practical problem by adopting the following technical scheme:
a reconstruction method of a neural network magnetic resonance image based on double-domain alternating convolution comprises the following steps:
step 1, acquiring a K space data set;
step 2, shielding the K space data obtained in the step 1 by using a mask, and performing zero filling on a shielded part to further generate undersampled K space data;
step 3, establishing a coding and decoding neural network structure of the double-domain alternating convolution;
and 4, training a coding and decoding neural network model of the double-domain alternating convolution by using the undersampled K space data generated in the step 2 and the image domain data obtained by performing inverse Fourier transform on the K domain information in the step 1.
And 5, obtaining a magnetic resonance reconstruction image with higher definition.
Moreover, the specific method of step 1 is: and scanning the sample by using the existing magnetic resonance scanning equipment to acquire K space information, exporting and preserving without loss, and preserving the data in a h5 file in a complex form.
Moreover, the specific method of the step 2 is as follows: and (3) performing mask covering on the K space data acquired in the step (1) by using a mask, adaptively adjusting the mask to ensure that the covered K space information contains enough middle and low frequency parts, and filling the covered parts with zeros to generate the undersampled K space data.
The specific method of step 3 is:
constructing a dual-domain depth convolution neural network based on an encoder and a decoder, wherein an encoder part of the dual-domain depth convolution neural network can extract image characteristics and K space characteristics through dual-domain alternating convolution, a decoder part recovers an image domain and a K domain characteristic diagram extracted by the encoder to the original size of an image, the dual-domain characteristics are fused at the last output end of the decoder, the characteristics of each layer of encoding and decoding of the network are connected through cross connection, and the fusion of scale information can be realized;
the encoder part of the dual-domain deep convolutional neural network comprises four convolutional layers, two K-space radial long-kernel convolutional kernels and two image domain 3x3 convolutional kernels are arranged in each convolutional layer, the convolutional layers are alternately arranged according to the sequence of the K-domain image domains, and after each convolutional layer, maxpooling is adopted for downsampling;
the encoder and the decoder of the double-domain deep convolutional neural network are connected by a convolutional block; the convolution block is composed of a double-domain convolution layer, the image domain convolution uses a 3x3 convolution kernel with the step length of 1, the K domain convolution uses a 1x1 small kernel convolution with the step length of 1, the convolution is carried out alternately according to the K domain and the image domain, and the cross connection between the double domains realizes full connection through inverse Fourier transform and Fourier transform;
the decoder part of the dual-domain deep convolutional neural network is formed by alternately performing convolution on an image domain and a K domain, wherein the image domain adopts 3x3 convolution with the step size of 1, the K domain adopts 1x1 convolution with the step size of 1, the conversion between the dual domains is realized by using Fourier transform and inverse Fourier transform according to the alternate convolution of the image domain of the K domain, and the InstanceNorm normalization and LeakyReLu activation functions are used after each layer of convolution.
Moreover, the specific method of the step 4 is as follows:
and (3) sending the image domain information obtained by inverse Fourier transform of the double-domain information, namely undersampled K space data and the fully sampled K domain information into the coding and decoding neural network of the double-domain alternating convolution by using the coding and decoding network constructed in the step (3), using the fully sampled K domain information and the image obtained by inverse Fourier transform as supervision, using an L1 norm as the difference between loss function calculation and a true value in the K domain, using structural similarity loss as the difference between the loss function calculation and the image domain true value in the image domain, adding the loss function and the true value according to a certain weight to obtain a final loss function, then performing back propagation, updating network parameters by using random gradient descent, continuously training and optimizing, and storing the model according to the minimum loss of a verification result.
Moreover, the specific method of the step 5 is as follows: and (5) storing the trained dual-domain alternating convolution coding and decoding neural network model in the step (4) in an embedded device, acquiring undersampled K space data by the magnetic resonance device according to an accelerating factor in a clinical application stage, and reconstructing the undersampled magnetic resonance data by using the trained dual-domain alternating coding and decoding neural network to finally obtain a magnetic resonance reconstructed image with higher definition.
The invention has the advantages and beneficial effects that:
1. the invention realizes the accelerated reconstruction of the magnetic resonance imaging by using the small-kernel convolutional neural network on the K domain, can reconstruct and obtain clear magnetic resonance imaging while eliminating the artifact caused by breaking through the Nyquist sampling limit, improves the reconstruction precision, and solves the problems of larger calculation amount, lower acceleration multiplying power and the like of the traditional algorithms such as compressed sensing and the like.
2. The invention realizes the conversion between the two domains by utilizing the Fourier transform and the inverse Fourier transform, realizes the full connection between the two domains by the domain conversion, and the reception field of the convolution network is all K domains, thereby the reconstruction result has better performance on the similarity and the peak signal-to-noise ratio.
3. The invention provides a method for reconstructing an undersampled magnetic resonance image by two domains based on two-domain convolution, which realizes full connection between the two domains and finally end-to-end magnetic resonance imaging reconstruction by introducing frequency domain convolution and conversion between the domains into a coding and decoding network.
Drawings
FIG. 1 is a flow chart of the dual domain reconstruction of the present invention;
FIG. 2 is a diagram of a dual-domain alternating convolutional encoding and decoding neural network of the present invention;
FIG. 3 is a diagram of a dual-domain alternating convolution block of the present invention;
fig. 4 (a) is a magnetic resonance image (K-domain convolution kernel is 3 × 3) reconstructed by a two-domain codec network according to the present invention, fig. 4 (b) is a true value image corresponding thereto, and fig. 4 (c) is an error difference diagram between a reconstruction result corresponding thereto and a true value;
fig. 5 (a) is a magnetic resonance image reconstructed by a two-domain codec network according to the present invention (K-domain convolution kernel is 1 × 1), fig. 5 (b) is a true value image thereof, and fig. 5 (c) is an error difference diagram between a reconstruction result and a true value thereof.
Detailed Description
The embodiments of the invention are further described in the following with reference to the drawings:
a reconstruction method of a neural network magnetic resonance image based on two-domain alternating convolution, as shown in fig. 1, comprising the following steps:
step 1, scanning by using a magnetic common scanning device to obtain a K space data set;
the specific method of the step 1 comprises the following steps: and scanning the sample by using the existing magnetic resonance scanning equipment to acquire K space information, exporting and preserving without loss, and preserving the data in a h5 file in a complex form.
Step 2, shielding the K space data obtained in the step 1 by using a mask, and performing zero filling on a shielded part to further generate undersampled K space data;
the specific method of the step 2 comprises the following steps: and (3) performing mask covering on the K space data acquired in the step (1) by using a mask, adaptively adjusting the mask to ensure that the covered K space information contains enough middle and low frequency parts, and filling the covered parts with zeros to generate the undersampled K space data.
In this embodiment, the mask needs to be selected by considering a required acceleration factor, a low frequency part to be reserved, and a mask type finally determined according to a K-domain filling method (a cartesian filling method, a concentric filling method, or a radial filling method).
In this embodiment, the original data set is K-domain information of full sampling, so in the off-line training phase of the algorithm, the original K-domain information needs to be shielded according to a certain acceleration multiple, and the shielded part of the K space uses zero padding.
Step 3, establishing a coding and decoding neural network structure of the double-domain alternating convolution;
the specific method of the step 3 comprises the following steps:
the method comprises the steps of constructing a dual-domain depth convolution neural network based on an encoder and a decoder, wherein an encoder part of the dual-domain depth convolution neural network can extract image features and K space features at the same time through dual-domain alternating convolution, a decoder part can restore the image domain and the K domain feature map extracted by the encoder to the original size of an image, the dual-domain features are fused at the last output end of the decoder, the features of the encoding and decoding layers of the network are connected through cross connection, and the fusion of scale information can be realized so as to enhance reconstruction quality.
As shown in fig. 2, the dual-domain convolutional codec network is composed of an encoder and a decoder, and the encoder includes four layers of convolutional blocks. The convolution block is shown in fig. 3, and is composed of two convolution layers, the convolution of the first layer is performed on the K domain information, and due to the fact that the K domain information is discontinuous under undersampling, the convolution kernel adopted in the method is a small kernel convolution with 1x1, and the step length is 1. The convolution of a K image domain uses a 3x3 convolution kernel with the step length of 1, after the convolution of the K domain uses 1K domain information with the step length of 1 for processing, 2-dimensional inverse Fourier transform is used for converting the K domain information into the image domain information, the convolution of the image domain uses a 3x3 convolution kernel with the convolution step length of 1, after the convolution of the image domain, 2-dimensional Fourier transform is used for converting the image domain information into the K domain information again, the convolution of the K domain information is performed again, and thus the K domain image domain is alternately convolved for 2 times respectively. The convolutional layer at the encoding end of the two-domain convolutional coding and decoding network is then downsampled by Max boosting, and the encoding end contains four layers of convolutions and is accompanied by four downsampling. The decoder is composed of convolution layer and deconvolution up-sampling, after each up-sampling, the corresponding encoding end convolution layer and decoding end convolution layer are connected in a cross-connection mode, the corresponding layer of the encoding end and the corresponding layer of the decoding end are added, and the decoding end also contains four layers of convolution layers.
The encoder part of the double-domain deep convolutional neural network comprises four convolutional layers, two K-space radial long-kernel convolutional kernels and two image domain 3x3 convolutional kernels are arranged in each convolutional layer in an alternating mode according to the sequence of the K-domain image domains, and Maxpooling is adopted for downsampling after each convolutional layer.
The encoder, the decoder and the decoder of the double-domain deep convolutional neural network are connected by a convolutional block; the convolution block is composed of double-domain convolution layers, image domain convolution uses a 3x3 convolution kernel with the step length of 1, K domain convolution uses a 1x1 small kernel convolution with the step length of 1, convolution is carried out alternately according to a K domain and an image domain, and cross connection between the double domains is achieved through inverse Fourier transform and Fourier transform.
As shown in fig. 3, the convolution block is composed of two-domain convolution layers, the first layer of convolution processes K-domain information, and because the K-domain information is discontinuous under undersampling, the convolution kernel adopted in the method is a small kernel convolution with 1 × 1, and the step size is 1. The method comprises the steps of performing convolution on a K image domain by using a 3x3 convolution kernel with the step length of 1, performing convolution on the K image domain by using 1K domain information with the step length of 1, converting the K domain information into image domain information by using 2-dimensional inverse Fourier transform, performing convolution on the image domain by using a 3x3 convolution kernel with the convolution step length of 1, performing convolution on the image domain by using 2-dimensional Fourier transform to convert the image domain information into the K domain information again, and performing convolution on the K domain information again, so that the K domain image domain is alternately convolved for 2 times.
The decoding part of the dual-domain deep convolutional neural network is similar to the encoding part, image domains and K domains are alternately convoluted, the image domains are convoluted by 3x3 with the step size of 1, the K domains are convoluted by 1x1 with the step size of 1, the image domains are alternately convoluted according to the K domains, the conversion between the dual domains is realized by using Fourier transform and inverse Fourier transform, and the InstanceNorm normalization and LeakyReLu activation functions are used after each layer of convolution.
In this embodiment, a dual-domain alternating convolution codec network is constructed, where the network encoder includes four convolution layers, each convolution layer includes a K-domain convolution block and an image-domain convolution block that are alternately connected, the K-domain convolution block and the image-domain convolution block are connected by inverse fourier transform and fourier transform, and are alternately connected according to the order of the K-domain image domain, and each K-domain convolution block has a convolution kernel of 1x1 or 3x3, a step size of 1 convolution layer, a leaky relu activation function, and an Instance Norm normalization layer. Each image domain convolution block has convolution kernel of 3x3, convolution layer with step size of 1, leakyReLu activation function and Instance Norm normalization layer. The encoder downsampling uses Maxpooling downsampling. The decoder is similar to the encoder and comprises four decoding convolution layers, wherein K domain convolution blocks and image domain convolution blocks in each convolution layer are alternately carried out, in addition, the encoder and the decoder realize residual error connection in a characteristic layer through cross connection, and upsampling is realized through deconvolution.
And 4, training a coding and decoding neural network model of double-domain alternating convolution by using the undersampled K space data generated in the step 2 and the image domain data obtained by performing inverse Fourier transform on the K domain information in the step 1.
The specific method of the step 4 comprises the following steps:
and 3, sending image domain information obtained by inverse Fourier transform of double-domain information, namely undersampled K space data and fully-sampled K domain information into the coding and decoding neural network of the double-domain alternating convolution by using the coding and decoding network constructed in the step 3, using the fully-sampled K domain information and an image obtained by inverse Fourier transform as supervision, using an L1 norm as a difference between loss function calculation and a true value in the K domain, using structural similarity loss as a difference between the loss function calculation and an image domain true value in the image domain, adding the loss function and the loss function according to a certain weight to obtain a final loss function, then performing back propagation, updating network parameters by using random gradient descent, continuously training and optimizing, and storing a model according to the minimum loss of a verification result.
In this embodiment, fully sampled K-domain information and an image obtained through inverse fourier transform are used as supervision, the K-domain uses an L1 norm as a loss function to calculate a difference with a true value, the image domain uses a structural similarity loss as a loss function to calculate a difference with the true value of the image domain, and the two are added according to a certain weight to obtain a final loss function. And then carrying out back propagation, updating network parameters by using random gradient descent, continuously training and optimizing, and storing the model according to the minimum loss of the verification result.
And 5, storing the double-domain alternating convolution coding and decoding neural network model trained in the step 4 in an embedded device, acquiring undersampled K space data by the magnetic resonance device according to an accelerating factor in a clinical application stage, and reconstructing the undersampled magnetic resonance data by using the trained double-domain alternating coding and decoding neural network to finally obtain a magnetic resonance reconstructed image with higher definition.
The test results show that:
in this embodiment, in order to objectively represent the reconstruction effect of the undersampled K space, the reconstruction index is evaluated by using the structural similarity, the peak signal-to-noise ratio, and the standard mean square error as evaluation indexes.
Table 1 shows the test set image reconstruction results SSIM, PSNR, NMSE.
Table 1 test set test results SSIM, PSNR, NMSE
The method of the invention is adopted to reconstruct the undersampled magnetic resonance data, the reconstruction effect is as shown in fig. 4 (a) -4 (c) and fig. 5 (a) -5 (c), the texture of the reconstruction result in the figure is clear, the contrast of the light and the shade of the soft tissue of the human body is obvious, and the reconstruction artifact is not generated.
The experimental result obtained in this embodiment can show that the method of the present invention can solve the problems of poor reconstruction result of K-domain convolution, insufficient utilization of K-domain information, etc., and can improve the problem of poor reconstruction accuracy of K-domain encoding and decoding networks through two-domain convolution, thereby realizing the magnetic resonance imaging reconstruction of the under-sampled K space.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Claims (4)
1. A reconstruction method of a neural network magnetic resonance image based on double-domain alternating convolution is characterized in that: the method comprises the following steps:
step 1, acquiring a K space data set;
step 2, shielding the K space data obtained in the step 1 by using a mask, and performing zero filling on a shielded part to further generate undersampled K space data;
step 3, establishing a coding and decoding neural network structure of the double-domain alternating convolution;
step 4, training a coding and decoding neural network model of double-domain alternating convolution by using the undersampled K space data generated in the step 2 and the image domain data obtained by performing inverse Fourier transform on the K domain information in the step 1;
step 5, obtaining a magnetic resonance reconstruction image with higher definition;
the specific method of the step 3 comprises the following steps:
constructing a dual-domain depth convolution neural network based on an encoder and a decoder, wherein an encoder part of the dual-domain depth convolution neural network can extract image characteristics and K space characteristics through dual-domain alternating convolution, a decoder part recovers an image domain and a K domain characteristic diagram extracted by the encoder to the original size of an image, the dual-domain characteristics are fused at the last output end of the decoder, the characteristics of each layer of encoding and decoding of the network are connected through cross connection, and the fusion of scale information can be realized;
the encoder part of the dual-domain deep convolutional neural network comprises four convolutional layers, two K-space radial long-kernel convolutional kernels and two image domain 3x3 convolutional kernels are arranged in each convolutional layer, the convolutional layers are alternately arranged according to the sequence of the K-domain image domains, and after each convolutional layer, maxpooling is adopted for downsampling;
a convolution block is connected between an encoder and a decoder of the double-domain deep convolution neural network; the convolution block is composed of a double-domain convolution layer, image domain convolution uses a 3x3 convolution kernel with the step length of 1, K domain convolution uses a 1x1 small kernel convolution with the step length of 1, convolution is carried out alternately according to a K domain and an image domain, and cross connection between the double domains is realized through inverse Fourier transform and Fourier transform;
the decoder part of the dual-domain deep convolutional neural network is alternately convolved by an image domain and a K domain, the image domain adopts 3x3 convolution with the step length of 1, the K domain adopts 1x1 convolution with the step length of 1, the conversion between the dual domains is realized by using Fourier transform and inverse Fourier transform according to the alternate convolution of the image domain of the K domain, and the InstanceNorm normalization and the LeakyReLu activation function are used after each layer of convolution;
the specific method of the step 4 comprises the following steps:
and 3, sending image domain information obtained by inverse Fourier transform of double-domain information, namely undersampled K space data and fully-sampled K domain information into the coding and decoding neural network of the double-domain alternating convolution by using the coding and decoding network constructed in the step 3, using the fully-sampled K domain information and an image obtained by inverse Fourier transform as supervision, using an L1 norm as a difference between loss function calculation and a true value in the K domain, using structural similarity loss as a difference between the loss function calculation and an image domain true value in the image domain, adding the loss function and the loss function according to a certain weight to obtain a final loss function, then performing back propagation, updating network parameters by using random gradient descent, continuously training and optimizing, and storing a model according to the minimum loss of a verification result.
2. The reconstruction method of the neural network magnetic resonance image based on the double-domain alternating convolution as claimed in claim 1, characterized in that: the specific method of the step 1 comprises the following steps: and scanning the sample by using the existing magnetic resonance scanning equipment to acquire K space information, exporting and preserving without loss, and preserving the data in a h5 file in a complex form.
3. The reconstruction method of the neural network magnetic resonance image based on the double-domain alternating convolution as claimed in claim 1, characterized in that: the specific method of the step 2 comprises the following steps: and (3) performing mask covering on the K space data acquired in the step (1) by using a mask, adaptively adjusting the mask to ensure that the covered K space information contains enough middle and low frequency parts, and filling the covered parts with zeros to generate the undersampled K space data.
4. The reconstruction method of the neural network magnetic resonance image based on the double-domain alternating convolution as claimed in claim 1, characterized in that: the specific method of the step 5 comprises the following steps: and (5) storing the trained dual-domain alternating convolution coding and decoding neural network model in the step (4) in an embedded device, acquiring undersampled K space data by the magnetic resonance device according to an accelerating factor in a clinical application stage, and reconstructing the undersampled magnetic resonance data by using the trained dual-domain alternating coding and decoding neural network to finally obtain a magnetic resonance reconstructed image with higher definition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110278963.8A CN113096208B (en) | 2021-03-16 | 2021-03-16 | Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110278963.8A CN113096208B (en) | 2021-03-16 | 2021-03-16 | Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113096208A CN113096208A (en) | 2021-07-09 |
| CN113096208B true CN113096208B (en) | 2022-11-18 |
Family
ID=76667415
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110278963.8A Active CN113096208B (en) | 2021-03-16 | 2021-03-16 | Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113096208B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113592973A (en) * | 2021-07-30 | 2021-11-02 | 哈尔滨工业大学（深圳） | Magnetic resonance image reconstruction method and device based on multi-frequency complex convolution |
| CN113920212B (en) * | 2021-09-27 | 2022-07-05 | 深圳技术大学 | Magnetic resonance reconstruction model training method, computer device and storage medium |
| CN114010180B (en) * | 2021-11-05 | 2024-04-26 | 清华大学 | Magnetic resonance rapid imaging method and device based on convolutional neural network |
| CN114581550B (en) * | 2021-12-31 | 2023-04-07 | 浙江大学 | Magnetic resonance imaging down-sampling and reconstruction method based on cross-domain network |
| CN114114116B (en) * | 2022-01-27 | 2022-08-23 | 南昌大学 | Magnetic resonance imaging generation method, system, storage medium and computer equipment |
| CN114693823B (en) * | 2022-03-09 | 2024-06-04 | 天津大学 | Magnetic resonance image reconstruction method based on space-frequency double-domain parallel reconstruction |
| CN115272510B (en) * | 2022-08-08 | 2023-09-22 | 中国科学院精密测量科学与技术创新研究院 | Pulmonary gas MRI reconstruction method based on coding enhancement complex value network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109350061A (en) * | 2018-11-21 | 2019-02-19 | 成都信息工程大学 | MR imaging method based on depth convolutional neural networks |
| CN111047551A (en) * | 2019-11-06 | 2020-04-21 | 北京科技大学 | Remote sensing image change detection method and system based on U-net improved algorithm |
| CN111487573A (en) * | 2020-05-18 | 2020-08-04 | 厦门大学 | Enhanced residual error cascade network model for magnetic resonance undersampling imaging |
| CN111899165A (en) * | 2020-06-16 | 2020-11-06 | 厦门大学 | Multi-task image reconstruction convolution network model based on functional module |
| CN111951344A (en) * | 2020-08-09 | 2020-11-17 | 昆明理工大学 | Magnetic resonance image reconstruction method based on cascade parallel convolution network |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109557489B (en) * | 2019-01-08 | 2021-06-18 | 上海东软医疗科技有限公司 | Magnetic resonance imaging method and device |
| CN110570486B (en) * | 2019-08-23 | 2023-04-07 | 清华大学深圳研究生院 | Under-sampling nuclear magnetic resonance image reconstruction method based on deep learning |
| CN111784792A (en) * | 2020-06-30 | 2020-10-16 | 四川大学 | Rapid magnetic resonance reconstruction system based on double-domain convolution neural network and training method and application thereof |
-
2021
- 2021-03-16 CN CN202110278963.8A patent/CN113096208B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109350061A (en) * | 2018-11-21 | 2019-02-19 | 成都信息工程大学 | MR imaging method based on depth convolutional neural networks |
| CN111047551A (en) * | 2019-11-06 | 2020-04-21 | 北京科技大学 | Remote sensing image change detection method and system based on U-net improved algorithm |
| CN111487573A (en) * | 2020-05-18 | 2020-08-04 | 厦门大学 | Enhanced residual error cascade network model for magnetic resonance undersampling imaging |
| CN111899165A (en) * | 2020-06-16 | 2020-11-06 | 厦门大学 | Multi-task image reconstruction convolution network model based on functional module |
| CN111951344A (en) * | 2020-08-09 | 2020-11-17 | 昆明理工大学 | Magnetic resonance image reconstruction method based on cascade parallel convolution network |
Non-Patent Citations (1)
| Title |
|---|
| 黄敏等.基于K空间数据的深度核磁共振图像重建.《生物医学工程研究》.2020,(第02期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113096208A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113096208B (en) | Reconstruction method of neural network magnetic resonance image based on double-domain alternating convolution | |
| Otazo et al. | Low‐rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components | |
| Souza et al. | A hybrid, dual domain, cascade of convolutional neural networks for magnetic resonance image reconstruction | |
| US10671939B2 (en) | System, method and computer-accessible medium for learning an optimized variational network for medical image reconstruction | |
| US20190064296A1 (en) | System, method and computer-accessible medium for highly-accelerated dynamic magnetic resonance imaging using golden-angle radial sampling and compressed sensing | |
| Cheng et al. | Highly scalable image reconstruction using deep neural networks with bandpass filtering | |
| US9709650B2 (en) | Method for calibration-free locally low-rank encouraging reconstruction of magnetic resonance images | |
| US20200341094A1 (en) | Multi-contrast mri image reconstruction using machine learning | |
| Dar et al. | Synergistic reconstruction and synthesis via generative adversarial networks for accelerated multi-contrast MRI | |
| WO2022183988A1 (en) | Systems and methods for magnetic resonance image reconstruction with denoising | |
| CN113971706A (en) | Rapid magnetic resonance intelligent imaging method | |
| Pour Yazdanpanah et al. | Deep plug-and-play prior for parallel MRI reconstruction | |
| Lv et al. | Parallel imaging with a combination of sensitivity encoding and generative adversarial networks | |
| Cui et al. | Motion artifact reduction for magnetic resonance imaging with deep learning and k-space analysis | |
| Yiasemis et al. | On retrospective k-space subsampling schemes for deep MRI reconstruction | |
| Virtue et al. | Learning contrast synthesis from MR fingerprinting | |
| CN107942271B (en) | SPEED rapid magnetic resonance imaging method based on iteration | |
| CN109920017A (en) | The parallel MR imaging reconstructing method of the full variation Lp pseudonorm of joint from consistency based on feature vector | |
| CN116725515B (en) | Magnetic resonance rapid imaging method | |
| US20230380714A1 (en) | Method and system for low-field mri denoising with a deep complex-valued convolutional neural network | |
| US11125846B2 (en) | Method for correction of phase-contrast magnetic resonance imaging data using a neural network | |
| KR102163337B1 (en) | Method for accelerating multiple-acquisition magnetic resonance imaging by varying undersampling-dimension and device for the same | |
| Yaman et al. | Improved supervised training of physics-guided deep learning image reconstruction with multi-masking | |
| CN112634385B (en) | Rapid magnetic resonance imaging method based on deep Laplace network | |
| CN113192150B (en) | Magnetic resonance interventional image reconstruction method based on cyclic neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |