CN112634385B - Rapid magnetic resonance imaging method based on deep Laplace network - Google Patents

Abstract

A fast magnetic resonance imaging method based on a depth Laplace network comprises the following steps: step 1 inputting a u-fold undersampled MR imageStep2 by deep Laplace networkObtaining a reconstructed MR imageFIG. 1 shows an example of a deep Laplace network f _Lap (.; Θ). The invention combines the sampling principle of the MR image with the deep learning technology, and performs layer-by-layer reconstruction on the high-power undersampled MR image based on the Laplacian pyramid idea, thereby effectively reducing the influence of a large amount of artifacts in input data on the reconstruction effect and effectively improving the reconstruction speed and accuracy of the MR image.

Description

Rapid magnetic resonance imaging method based on deep Laplace network

Technical Field

The invention relates to the field of medical image processing, in particular to the technical field of medical images of magnetic resonance imaging, which is mainly used for realizing the accelerated reconstruction of medical magnetic resonance imaging.

Background

Magnetic resonance imaging (Magnetic Resonance Imaging, MRI) is widely used as a medical in vivo imaging technique due to its advantages of non-invasiveness, high resolution, biosafety, and excellent soft tissue contrast. Although MRI has little radiation hazard to the human body, its acquisition time is long (typically over 30 minutes), which can lead to three problems: (1) higher inspection costs are incurred; (2) Can greatly affect patient comfort and compliance, resulting in localized artifacts that can cause MR (Magnetic Resonance ) images due to involuntary patient movement; (3) It is difficult to apply to disease screening, such as stroke, where diagnostic time is a high requirement. Since the advent of MRI in the 1970 s, improving imaging speed has been a major ongoing research goal, and is one of the recent research hotspots.

Conventional MR rapid imaging techniques are mainly based on compressive sensing (Compressed Sensing, CS). The compression sensing realizes the multiple shortening of the magnetic resonance scanning time by directly collecting the compressed image. However, compressive sensing imaging has not been widely used in the magnetic resonance field for more than a decade due to factors such as data acquisition, spatial conversion, reconstruction algorithms, and processing systems.

In the last three years, with the rapid development of deep learning in the field of natural image processing, research and application thereof in the field of medical images have also received a great deal of attention. Compared with the traditional MR rapid imaging method based on compressive sensing, the MR rapid imaging method based on deep learning has higher accuracy and faster speed, which indicates a new direction for rapid MR imaging. Existing deep learning algorithms are basically direct learning of the mapping from high-power undersampled MR data to fully sampled MR data, which often requires a large amount of zero padding of the high-power undersampled MR data to meet the needs of network training. However, a large amount of zero padding not only adds additional computational overhead, but also introduces significant artifacts in the reconstructed MR image, resulting in a lower signal-to-noise ratio of the reconstructed MR image.

Disclosure of Invention

In order to overcome the defects of the prior art, in order to eliminate a large amount of artifacts existing in the high-power undersampled MR image and efficiently learn complex end-to-end mapping by utilizing limited network capacity, the invention combines a depth cascade network structure and a Laplacian pyramid idea, provides a rapid magnetic resonance imaging method based on a depth Laplacian network, combines a sampling principle of the MR image with a depth learning technology, and performs layer-by-layer reconstruction on the high-power undersampled MR image based on the Laplacian pyramid idea, so that the influence of a large amount of artifacts in input data on reconstruction effects can be effectively reduced, and the reconstruction speed and accuracy of the MR image can be effectively improved.

The technical scheme adopted for solving the technical problems is as follows:

A method of fast magnetic resonance imaging based on a deep laplace network (DEEP LAPLACIAN network, deepLap), comprising the steps of:

step 1 inputting a u-fold undersampled MR image For an MR image undersampled by a factor u in k-space,/>Representing k-space in which the MR image is located,/>Representing the dimension of y _u, N is the dimension of the fully sampled MR image; the undersampled MR image y _u is obtained by:

y_u＝M_uy＝M_uFx，

Wherein, A fully sampled image representing y _u in k-space,/>Representing a fully sampled image of y _u in pixel space,/>Representing the pixel space in which the MR image is located,/>Is a fourier transform matrix for transforming the MR image from pixel space to k-space; m _u∈{0,1}^M×N denotes a mask matrix that undersamples y by a factor of u;

step2 by deep Laplace network Obtaining a reconstructed MR image

The deep Laplace network f _Lap (.; Θ) is defined as follows:

Where Θ= { Θ ₀,Θ₁,...,Θ_D } is a parameter set of the network, H represents Hermitian transpose operation, D is the number of sub-networks of f _Lap (·; Θ)), and f _d(·;Θ_d) is a sub-network of the deep laplace network, defined as follows:

Here, D e {1,2,..d }, For the upsampling operator, we mean that the u _d-1 times undersampled MR image in k-space is mapped to the u _d times undersampled MR image in pixel space, u _d＜u_d-1,/>Is a parameter thereof; /(I)A reconstruction module, which represents a network module that performs reconstruction computation on the MR image in the pixel domain, and θ _d is a parameter thereof; /(I)Representing data consistency operators, let/> Element/>Calculated by the following method:

Wherein, Ω represents the index subset used to construct the elements in y of y _u.

Further, in the step 2, the parameter set Θ in the deep laplace network f _Lap (·; Θ) is trained by the following steps:

step 2.1 construction of MR image set for training depth Laplace network f _Lap (.; Θ)

Here Γ= {1,2,.., For the MR image sampled at the gamma Zhang Quan th,

Wherein,A mask matrix representing u _d times undersampling of MR images in k-space;

Step 2.2 build a validation set of deep Laplace network f _Lap (.; Θ):

Wherein, Λ= {1, 2..k } is the index set of verification samples, K is the number of verification samples;

step 2.3 let the iteration period number (Epoch) e=0, the iteration number t=0, and the parameter set Θ of the random initialization f _Lap (; Θ) be Θ ⁽⁰⁾:

step 2.4 let e=e+1, training batch b=0, randomly partitioning the index set Γ of training samples into B disjoint subsets:

Step 2.5 let t=t+1, b=b+1, D=1, 2, D, calculating

Wherein,

Step 2.6 minimizes the following loss function by gradient descent method:

Wherein l (·, ·) represents a distance metric operation between the two images;

Step 2.7 evaluate the quality of the deep laplace network model f _Lap(·;Θ^(t)) on the validation set:

wherein Q (·, ·) is a quality assessment function;

Step 2.8, iteratively executing the steps 2.5 to 2.7 until b=b;

step 2.9, iteratively executing the steps 2.4 to 2.8 until e=e, where E is a preset maximum training period number;

Step 2.10 selects the optimal parameter set Θ=Θ ^(t＊) for the deep laplace network f _Lap (·; Θ), where,

The technical conception of the invention is as follows: the existing fast magnetic resonance imaging based on deep learning mainly directly learns the mapping relation from undersampled data to full sampled data. This requires a large amount of zero padding of the incoming undersampled data to conform the data to the network training requirements. However, a large amount of zero padding can cause serious artifacts in the generated magnetic resonance image at the beginning, so that subsequent end-to-end learning of the network is greatly interfered, and the reconstruction effect of the network is seriously affected. In order to reduce the difficulty of network learning, the invention provides a deep cascade network formed by a plurality of subnets based on the idea of the Laplacian pyramid, and zero filling is gradually carried out on an input high-power undersampled image, so that each subnet is used for centrally learning the mapping relation of different parts, and the final reconstruction effect is improved.

The beneficial effects of the invention are mainly shown in the following steps: (1) The high-power undersampled magnetic resonance image is gradually reconstructed from low frequency to high frequency based on the depth Laplace network, so that the reconstruction effect of the high-power undersampled magnetic resonance image can be effectively improved; (2) Since only a portion (but not all) of the undersampled data is reconstructed per subnet, the proposed method has a faster reconstruction speed.

Drawings

Fig. 1 is a block diagram of a deep laplace network designed according to the present invention, where y _u is the u-fold undersampled MR image in input k-space,For the MR image in the reconstructed pixel space of the output,/> For the intermediate output result of the d-th subnet, U _d is the upsampling operator of the d-th subnet, R _d is the reconstruction module of the d-th subnet, conv _d,j represents the j-th convolution layer of R _d, F _DC is the data consistency operator, PZF represents the partial zero padding, and F ^H represents the inverse fourier transform.

Fig. 2 is an example of an MR image and a fully sampled MR image, wherein (a) and (b) are examples of an 8 x undersampled MR image and a fully sampled MR image in k-space, respectively, and (c) and (d) are examples of a reconstructed MR image and a fully sampled MR image in pixel space, respectively.

Fig. 3 is an example of a mask matrix, where (a) is an example of a center undersampled mask matrix and (b) is an example of a gaussian undersampled mask matrix.

Fig. 4 is two examples of upsampling operators used in fig. 1, where (a) is a bicubic interpolation based upsampling operator, where,Representing a bicubic interpolation operator; (b) Is based on a partial zero-padding up-sampling operator, where PZF represents the partial zero-padding and F ^H represents the inverse fourier transform.

FIG. 5 is an effect presentation of the data consistency operator f _DC, where the first, second, and third rows represent the MR image before f _DC, the MR image after f _DC, the fully sampled MR image, respectively; the first, second, and third columns represent MR images output by three subnets of the depth laplace network, respectively.

Fig. 6 is an example of 5 MR training samples for training a deep laplace network, one for each row, each consisting of 5 MR images: u times undersampled MR images in k-space (column 1), u ₁ = 6 times, u ₂ = 4 times, u ₃ = 2 times undersampled MR images in pixel space (columns 2 to 4), fully sampled MR images (last column).

Fig. 7 is a quantized comparison of PSNR/SSIM values of reconstruction results of the deep laplace network (DeepLap), the conventional padding method (ZF), and the 4-preamble method DEEPCASCADE, REFINEGAN, UNET, DENSEUNET proposed by the present invention. Using MR images of 8 subjects as training samples, MR images of the remaining 2 subjects as test samples, columns 2 through 6 of fig. 7 report PSNR/SSIM values reconstructed from 16-fold undersampled MR images of subjects numbered 2 and 3, 0 and 1, 4 and 5, 6 and 7, 8 and 9, respectively.

Fig. 8 is a comparison of the reconstruction effect, from left to right, of the MR image of full sampling and the reconstructed image of ZF, deepLap, deepCascade, refineGAN, UNet, denseUNet methods, respectively.

Detailed Description

The invention is further described below with reference to the accompanying drawings.

Referring to fig. 1-8, a fast magnetic resonance imaging method based on a deep laplace network (DEEP LAPLACIAN network, deepLap) includes the steps of:

step 1 inputting a u-fold undersampled MR image

Step2 by deep Laplace networkObtaining a reconstructed MR imageFIG. 1 shows an example of a deep Laplace network f _Lap (.; Θ).

Further, in the step2,For an MR image undersampled by a factor u in k-space, κ represents the k-space in which the MR image is located,/>Representing the dimension of y _u, N is the dimension of the fully sampled MR image; the undersampled MR image has the following relationship with the fully sampled MR image:

y_u＝M_uy＝M_uFx，

Wherein, A fully sampled image representing y _u in k-space,/>Representing a fully sampled image of y _u in pixel space,/>Representing the pixel space in which the MR image is located,/>Is a fourier transform matrix for transforming the MR image from pixel space to k-space; m _u∈{0,1}^M×N denotes a mask matrix that undersamples y by a factor of u. An example of u=8 times undersampled y _u is given in fig. 2 (a), and examples of fully sampled (n=336) MR images y and x in k-space and pixel space, respectively, are given in fig. 2 (b) and (d); mask matrix M _u may use a center undersampling matrix or a Gaussian undersampling matrix, as shown in FIG. 3, where (a) in FIG. 2 is an MR image in k-space resulting from center undersampling of (b) in FIG. 2.

Further, in the step 2, the deep Laplace network f _Lap (.; Θ) is defined as follows:

Where Θ= { Θ ₀,Θ₁,...,Θ_D } is a parameter set of the network, H represents Hermitian transpose operation, D is the number of sub-networks of f _Lap (·; Θ)), and f _d(·;Θ_d) is a sub-network of the deep laplace network, defined as follows:

Here, D e {1,2,..d }, For the upsampling operator, we mean that the u _d-1 times undersampled MR image in k-space is mapped to the u _d times undersampled MR image in pixel space, u _d＜u_d-1,/>Is a parameter thereof; /(I)A reconstruction module, which represents a network module that performs reconstruction computation on the MR image in the pixel domain, and θ _d is a parameter thereof; /(I)Representing data consistency operators, let/> Element/>Calculated by the following method:

Wherein, Ω represents the index subset used to construct the elements in y of y _u. Fig. 1 shows an example of a deep laplace network f _Lap (·; Θ): the depth D can be 3 or 5; the upsampling operator U _d of each subnet f _d(·;Θ_d) of f _Lap (·; Θ) is a core operator, and bicubic interpolation or partial zero padding may be used, examples of bicubic interpolation and partial zero padding are given in (a) and (b) of FIG. 4, respectively, and U _d of FIG. 1 uses the partial zero padding; the reconstruction module R _d may use a residual convolution module or a self-coding module, and in fig. 1, a residual convolution module (taking R ₁ as an example) is used: the main body part of the device consists of j (j can take 5 or 7) cascaded convolution layers, the size of each convolution kernel is a multiplied by a (a can take 3 or 5), and the output of the last convolution layer is added with the input of the convolution module; the data consistency operator f _DC further processes the output of Rd, and the first and second rows of FIG. 5 compare the changes in MR images before and after the three sub-networks of f _Lap (.; Θ) use f _DC.

Further, FIG. 2 (c) shows an MR image reconstructed from the 8 times undersampled y _u shown in FIG. 2 (a) using the depth Laplace network f _Lap (. Theta.; Θ) described aboveIs an example of (a).

Further, in the step 2, the parameter set Θ in the deep laplace network f _Lap (·; Θ) is trained by the following steps:

step 2.1 construction of MR image set for training depth Laplace network f _Lap (.; Θ)

Here Γ= {1,2,..l } is the index set of training samples, L is the number of training samples, u=u ₀＞u₁＞u₂＞...＞u_D is the different undersampling multiple of the MR image, For the MR image sampled at the gamma Zhang Quan th,

Wherein,A mask matrix representing u _d times undersampling of MR images in k-space; FIG. 6 shows the method for construction/>Is an example of 5 training samples;

Step 2.2 build a validation set of deep Laplace network f _Lap (.; Θ):

Wherein, Λ= {1, 2..k } is the index set of verification samples, K is the number of verification samples;

step 2.3 let the iteration period number (Epoch) e=0, the iteration number t=0, and the parameter set Θ of the random initialization f _Lap (; Θ) be Θ ⁽⁰⁾:

step 2.4 let e=e+1, training batch b=0, randomly partitioning the index set Γ of training samples into B disjoint subsets: Wherein the Batch Size (Batch Size), i.e. >, is > 6 Or 8;

Step 2.5 let t=t+1, b=b+1, Calculation of

Wherein,

Step 2.6 minimizes the following loss function by gradient descent method:

Where l (·, ·) represents the distance metric operation between the two images, either the l ₁ norm, the l ₂ norm, or the Charbonnier penalty function may be employed, where Charbonnier penalty function is defined as:

Step 2.7 evaluate the quality of the deep laplace network model f _Lap(·;Θ^(t)) on the validation set:

Wherein, Q (·, ·) is a quality assessment function, which may be a peak signal-to-noise ratio (PSNR) or a Structured Similarity (SSIM);

Step 2.8, iteratively executing the steps 2.5 to 2.7 until b=b;

Step 2.9, iteratively executing the steps 2.4 to 2.8 until e=e, where E is a preset maximum training period number, and e=100 may be set;

step 2.10 selecting an optimal parameter set of the deep Laplace network f _Lap (; Θ) Wherein,

The beneficial effects of the deep Laplace network provided by the invention are verified through experiments. 10 subjects were scanned using a philips Ingenia3T scanner, MR images of the T2WI modality were acquired, and each MR slice was cut to a size of 336 x 261. The U-Net network was trained using the Adam optimization algorithm, with momentum set to 0.9, batch size set to 6, initial learning rate set to 0.001, and 10-fold drop after 50 epochs per iteration, 100-Epoch iterations were performed.

In order to quantitatively evaluate the reconstruction performance of the network, the reconstructed MR images are quality assessed using peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM). The proposed deep laplace network (DeepLap) is compared with the padding method (Zero padding, ZF) and the 4-preamble method DEEPCASCADE, REFINEGAN, UNET, DENSEUNET. The MR images of 8 subjects were used as training samples, leaving the MR images of 2 subjects as test samples. The stability of the compared methods was tested using a cross-validation method, and 16-fold undersampled MR images of subjects numbered 2 and 3, 0 and 1, 4 and 5, 6 and 7, 8 and 9 were reconstructed, respectively. Comparing the reconstructed result with the fully sampled image, the PSNR/SSIM values are shown in FIG. 7, and it can be seen that the proposed method exceeds other methods.

Fig. 8 further shows a graph of the re-effects of the compared methods, from left to right, for the fully sampled MR image and for the reconstructed images of the ZF, deepLap, deepCascade, refineGAN, UNet, denseUNet methods, respectively. It can be seen that the DeepLap method proposed by the present invention can better reconstruct the detailed information of the high-power undersampled MR image.

The embodiments described in this specification are merely illustrative of the manner in which the inventive concepts may be implemented. The scope of the present invention should not be construed as being limited to the specific forms set forth in the embodiments, but the scope of the present invention and the equivalents thereof as would occur to one skilled in the art based on the inventive concept.

Claims (1)

1. A depth laplace network-based rapid magnetic resonance imaging method, comprising the steps of:

step 1 inputting a u-fold undersampled MR image For an MR image undersampled by a factor u in k-space,/>Representing k-space in which the MR image is located,/>Representing the dimension of y _u, N is the dimension of the fully sampled MR image; the undersampled MR image y _u is obtained by:

y_u＝M_uy＝M_uFx，

Wherein, A fully sampled image representing y _u in k-space,/>

Representing a fully sampled image of y _u in pixel space,Representing the pixel space in which the MR image is located,/>Is a fourier transform matrix for transforming the MR image from pixel space to k-space; m _u∈{0,1}^M×N denotes a mask matrix that undersamples y by a factor of u;

step 2 consists of a deep laplace network f _Lap (.; Θ): Obtaining a reconstructed MR image

The deep Laplace network f _Lap (.; Θ) is defined as follows:

f_Lap(y_u;Θ)＝F^Hf_D(f_D-1((…f₂(f₁(y_u;Θ₁);Θ₂));Θ_D-1);Θ_D),

Where Θ= { Θ ₀,Θ₁,…,Θ_D } is a parameter set of the network, H represents Hermitian transpose operation, D is the number of sub-networks of f _Lap (·; Θ)), and f _d(·;Θ_d) is a sub-network of the deep laplace network, defined as follows:

Here, D ε {1,2, …, D }, For the upsampling operator, we mean that the u _d-1 times undersampled MR image in k-space is mapped to the u _d times undersampled MR image in pixel space, u _d＜u_d-1,/>Is a parameter thereof; r _d(·;θ_d)/>A reconstruction module, which represents a network module that performs reconstruction computation on the MR image in the pixel domain, and θ _d is a parameter thereof; /(I)Representing data consistency operators, let/> Element/>Calculated by the following method:

Wherein, Ω represents the index subset used to construct the elements in y of y _u;

In the step 2, the parameter set Θ in the deep Laplace network f _Lap (.; Θ) is trained by the following steps:

step 2.1 construction of MR image set for training depth Laplace network f _Lap (.; Θ)

Here Γ= {1,2, …, L } is the index set of training samples, L is the number of training samples, u=u ₀＞u₁＞u₂＞…＞u_D is a different undersampling multiple of the MR image, For the MR image sampled at the gamma Zhang Quan th,

Wherein,Mask matrix representing u _d times undersampling of MR images in k-space,/>

Step 2.2 build a validation set of deep Laplace network f _Lap (.; Θ):

Wherein, Λ= {1,2, …, K } is the index set of verification samples, K is the number of verification samples;

Step 2.3 let the iteration period number e=0, the iteration number t=0, and the parameter set Θ of the random initialization f _Lap (·; Θ) is Θ ⁽⁰⁾:

step 2.4 let e=e+1, training batch b=0, randomly partitioning the index set Γ of training samples into B disjoint subsets:

Step 2.5 let t=t+1, b=b+1, Calculation of

Wherein,

Step 2.6 minimizes the following loss function by gradient descent method:

Wherein l (·, ·) represents a distance metric operation between the two images;

Step 2.7 evaluate the quality of the deep laplace network model f _Lap(·;Θ^(t)) on the validation set:

wherein Q (·, ·) is a quality assessment function;

Step 2.8, iteratively executing the steps 2.5 to 2.7 until b=b;

step 2.9, iteratively executing the steps 2.4 to 2.8 until e=e, where E is a preset maximum training period number;

step 2.10 selecting an optimal parameter set of the deep Laplace network f _Lap (; Θ) Wherein,