WO2022206021A1 - Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium - Google Patents
Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium Download PDFInfo
- Publication number
- WO2022206021A1 WO2022206021A1 PCT/CN2021/137623 CN2021137623W WO2022206021A1 WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1 CN 2021137623 W CN2021137623 W CN 2021137623W WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- resolution
- magnetic resonance
- reconstruction
- preset
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000012545 processing Methods 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 59
- 230000006870 function Effects 0.000 claims abstract description 38
- 230000008569 process Effects 0.000 claims description 18
- 238000006731 degradation reaction Methods 0.000 claims description 17
- 230000015556 catabolic process Effects 0.000 claims description 16
- 230000009467 reduction Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 10
- 238000004904 shortening Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 15
- 238000002595 magnetic resonance imaging Methods 0.000 description 10
- 230000000747 cardiac effect Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000593 degrading effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 230000004217 heart function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Definitions
- Embodiments of the present invention relate to the technical field of medical image processing, and in particular, to an image reconstruction model generation and image reconstruction method, apparatus, device, and medium.
- Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and imaging at any level.
- cardiac magnetic resonance cine has been regarded as the imaging gold standard for assessing cardiac function.
- high-resolution image acquisition is performed clinically, a long acquisition time is often required.
- it is often difficult to obtain high-resolution cardiac magnetic resonance images in the actual clinical process due to the influence of patient tolerance and respiratory movement.
- the methods for obtaining high-resolution magnetic resonance cardiac cine images mainly include image reconstruction methods based on interpolation, image reconstruction methods based on traditional machine learning, and image reconstruction methods based on deep learning.
- the method based on the deep learning network has good processing ability for linear and nonlinear methods, and can reconstruct the magnetic resonance image with higher image quality.
- the embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, device and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, and at the same time improve the reconstructed image resolution. better image effects.
- an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a preset ratio to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image, including:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- performing image interpolation processing on the low-resolution image includes:
- the method before training the high-resolution image reconstruction model, the method further includes:
- the image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
- a residual learning method is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
- the loss function between the described label data is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
- an embodiment of the present invention further provides an image reconstruction method, the method comprising:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- an embodiment of the present invention further provides a device for generating an image reconstruction model, the device comprising:
- an image degradation module configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image
- an image interpolation module configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
- the model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model.
- the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
- the image degradation module is specifically used for:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- the image interpolation module is specifically used for:
- the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- the model training module is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to combine the residual of the input data after passing through the convolution layer with the input. After the data itself is added, a loss function between the data and the label data is calculated.
- an embodiment of the present invention further provides an image reconstruction device, the device comprising:
- an image preprocessing module configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;
- the image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset resolution enhancement factor obtained by the image reconstruction model generation method described in any one of the embodiments.
- the target image reconstruction model of the target magnetic resonance image reconstruction image is obtained.
- an embodiment of the present invention further provides a computer device, the computer device comprising:
- processors one or more processors
- memory for storing one or more programs
- the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method provided by any embodiment of the present invention.
- an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method or image provided by any embodiment of the present invention rebuild method.
- a preliminary magnetic resonance reconstruction image is acquired, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a multiple according to a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image, so as to obtain a corresponding low-resolution image;
- Perform image interpolation processing wherein the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used.
- a high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention
- FIG. 2 is a schematic structural diagram of a generative adversarial network according to Embodiment 1 of the present invention.
- FIG. 3 is a schematic diagram of a network structure of a discriminator in a generative adversarial network provided by Embodiment 1 of the present invention.
- FIG. 4 is a schematic diagram of a generator network structure in a generative adversarial network according to Embodiment 1 of the present invention.
- FIG. 5 is a schematic diagram of a network training process of an image reconstruction model according to Embodiment 1 of the present invention.
- FIG. 6 is a flowchart of an image reconstruction method according to Embodiment 2 of the present invention.
- FIG. 7 is a schematic diagram of an image reconstruction process according to Embodiment 2 of the present invention.
- FIG. 9 is a schematic structural diagram of an apparatus for generating an image reconstruction model according to Embodiment 3 of the present invention.
- FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention.
- FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
- FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
- the method may be executed by an image reconstruction model generating apparatus, which may be implemented in software and/or hardware, and integrated into an electronic device with an application development function.
- the image reconstruction model generation method includes the following steps:
- the preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by a current magnetic resonance imaging device, that is, an image with relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction).
- the preset image resolution enhancement multiple is a multiple that is expected to be able to improve the resolution of the preliminary magnetic resonance reconstruction image. Exemplarily, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution
- the enhancement factor can be obtained by dividing (512*512) by (128*128).
- the preset image resolution enhancement factor is 16.
- the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution increase multiple to obtain a corresponding low-resolution image, and the purpose is to use the preliminary magnetic resonance reconstruction image itself as the training parameter of the image reconstruction network, Instead of matching the initial MR reconstruction image with the corresponding high-resolution high-quality MR reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for image reconstruction models.
- the way to reduce the image resolution can be a down-sampling method to perform down-sampling processing on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.
- a generative adversarial network can also be used to degrade the preliminary magnetic resonance image reconstruction image to obtain a low-resolution image whose resolution is reduced by a preset image resolution enhancement factor.
- a kernel function can be determined for performing the same image degradation process on one or more preliminary MRI reconstruction images.
- the preliminary magnetic resonance reconstruction image is input into the preset generative adversarial network as shown in FIG. 2 .
- the generator G performs convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image, and obtains a low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
- the obtained image can be regarded as the original magnetic resonance reconstruction image.
- Fake version image F further, cut out image blocks with the same size and matching position as image F from the preliminary magnetic resonance reconstruction image, as real sample image T.
- the discriminator analyzes the possibility that the image F is a real image pixel by pixel.
- the goal of the generator is to make the output image F fool the discriminator as much as possible.
- the two networks, the generator and the discriminator constantly adjust the parameters in the process of confrontation with each other. Finally, when the discriminator cannot judge whether the output result F of the generator is real, the network training is completed.
- the image F and the image T are compared pixel by pixel, and every two pixels compared with each other are the probability value of the same pixel point, when the probability value of the year in the heat map is satisfied.
- the probability value of the judgment is affirmative, it can be determined that the discriminator cannot judge whether the output result F of the generator is true.
- all convolutional layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be respectively combined with the magnetic resonance image.
- the image degradation kernel function performs convolution to obtain the corresponding low-resolution image.
- the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted.
- the generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1.
- the structure of the discriminator in the preset Generative Adversarial Network may be the structure shown in FIG. 3 , which consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are: 7 ⁇ 7 and 1 ⁇ 1 respectively. , 1x1, 1x1, 1x1, 1x1 and 1x1.
- the network structure of the generator is shown in Figure 4, which consists of 6 convolution layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1.
- the value of the convolution step size of the last convolutional layer of the generator is the preset image resolution enhancement multiple, so as to achieve the effect of downsampling the input image.
- the acquired preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance cine image, which includes multiple frames of magnetic resonance images.
- any frame of the cardiac magnetic resonance cine image can be taken as the input image.
- image interpolation processing is performed on the low-resolution image, and the multiple of image interpolation is the preset image resolution enhancement multiple, so as to keep the size of the input image of the image reconstruction model consistent with the size of the output image, which can be shortened.
- Model training time optimizing the model training process.
- the interpolation method may adopt a bicubic interpolation (Bicubic) algorithm. This is because the bicubic interpolation can preserve more image details during the image enlargement process, and the enlarged image has the function of anti-aliasing. At the same time, the enlarged image has a more realistic effect than the source image.
- Biubic bicubic interpolation
- the image that has undergone image interpolation processing and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation can be used as a new high-resolution image.
- Rate image reconstruction model training sample data For example, rotate the image pair at 0, 90, 180, and 270 degrees, and then perform mirror symmetry operations in the horizontal and vertical directions, respectively, to obtain 8 sets of training data, so that the training data can be obtained. enhanced.
- the phases of the data are consistent, and no registration is required, thereby eliminating hidden dangers such as poor learning effects caused by data deviations.
- a sufficient amount of model training sample data is constructed, and then the model training process can begin.
- the image that has undergone image interpolation processing is used as the input data for model training, and the preliminary MRI reconstruction image corresponding to the low-resolution image is used as the label data.
- the to-be-obtained preliminary magnetic resonance image is used as the label data, and the low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
- a high-resolution image that increases the resolution of the preliminary magnetic resonance reconstruction image by a preset image resolution can be obtained from the output of the image reconstruction model.
- the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers.
- the convolution kernel size of the first layer is 3x3, the number of channels is f, and the second to seventh layers are
- the size of the convolution kernel is 3x3, the number of channels is 64, and the size of the convolution kernel of the last layer is 3x3, and the number of channels is f.
- f represents the number of image frames simultaneously input to the image reconstruction network.
- the f value is 1.
- the value of f is the number of image frames in the cardiac cine image.
- the number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the size of the convolution kernel is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.
- the model training process in Fig. 5 adopts a preferred deep learning method, that is, the residual learning method, which compares the residual of the image data of the input model after passing through the convolution layer with the input image data itself. After adding, calculate the loss function between and the label data.
- the loss function may not only use the L1 loss function shown in FIG. 5 , but also use loss functions such as L2 loss and perceptual loss.
- optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad can also be used to optimize the network learning process.
- conv represents the convolution kernel in the convolutional neural network
- “.mat” is the abbreviation of the file format. All the parameters of the convolutional layers of the device are convolved layer by layer to obtain the degraded kernel function of the magnetic resonance image.
- a corresponding low-resolution image is obtained by reducing the image resolution of the obtained preliminary magnetic resonance reconstruction image, and the resolution reduction factor is controllable; then, image interpolation processing is performed on the low-resolution image , where the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data,
- the high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- Fig. 6 is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention, and this embodiment can be applied to the situation of reconstructing a collected low-resolution medical image to obtain a high-resolution image.
- the method may be performed by an image reconstruction apparatus, and the apparatus may be implemented in software and/or hardware, and integrated into a computer device with an application development function.
- the image reconstruction method includes the following steps:
- the preliminary magnetic resonance reconstruction image is a low-resolution image whose resolution needs to be improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image.
- the multiple of image interpolation is the preset resolution enhancement multiple.
- the target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above-mentioned embodiment according to the requirement of increasing the preset resolution. Specifically, for the process of image reconstruction, reference may be made to the schematic diagram shown in FIG. 7 .
- Fig. 8 shows the reconstructed image obtained by performing image reconstruction on the same preliminary magnetic resonance reconstructed image by different graphic reconstruction methods, wherein (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the double The result obtained by the cubic interpolation algorithm, (c) zero-padding is the result of transforming the image back to the image domain after the image is transformed to the Fourier domain, and then inversely transforming back to the image domain after zero-filling the surrounding area, (d) SR (Super Resolution) is the image of this example The results obtained by the reconstruction method. Intuitively, from the results, the image reconstruction results obtained by the image reconstruction method in this embodiment are clearer.
- FIG. 9 is a schematic structural diagram of an image reconstruction model generating apparatus according to Embodiment 3 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
- the image reconstruction model generating apparatus includes an image degradation module 310 , an image interpolation module 320 and a model training module 330 .
- the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; the image interpolation module 320, for performing image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; model training module 330, for using the image subjected to image interpolation processing as model training
- the input data of the high-resolution image reconstruction model is trained by using the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as the label data. When the loss function of the high-resolution image reconstruction model converges to the preset value , to generate the target high-resolution image reconstruction model.
- the technical solution of this embodiment by acquiring a preliminary magnetic resonance reconstruction image, and increasing the resolution of the preliminary magnetic resonance image by a multiple, the resolution of the preliminary magnetic resonance reconstruction image is reduced, and a corresponding low-resolution image is obtained; Perform image interpolation processing on the low-resolution images, wherein the multiple of image interpolation is the preset image resolution improvement multiple; the image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance imaging corresponding to the low-resolution image is used.
- the reconstructed image is used as label data to train a high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- the image degradation module 310 is specifically used for:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- the image interpolation module 320 is specifically used for:
- the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to compare the residual of the input data after passing through the convolutional layer with the residual. After the input data itself is added, a loss function between the input data and the label data is calculated.
- the image reconstruction model generation apparatus provided by the embodiment of the present invention can execute the image reconstruction model generation method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
- FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention. This embodiment can be applied to a situation in which a low-resolution medical image obtained by acquisition is reconstructed to obtain a high-resolution image.
- the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420 .
- the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple; the image reconstruction module 420 is used for image interpolation processing.
- the preliminary magnetic resonance reconstruction image is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor, and the target magnetic resonance image is obtained. Rebuild the image.
- the image reconstruction apparatus provided by the embodiment of the present invention can execute the image reconstruction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
- FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
- Figure 11 shows a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention.
- the computer device 12 shown in FIG. 11 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.
- the computer device 12 may be any terminal device with computing capability connected to the magnetic resonance scanning imaging device, such as an intelligent controller, a server, a mobile phone and other terminal devices.
- computer device 12 takes the form of a general-purpose computing device.
- Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .
- Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.
- these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.
- Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including both volatile and nonvolatile media, removable and non-removable media.
- System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 .
- Computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
- storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 11, commonly referred to as a "hard drive”).
- a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives.
- each drive may be connected to bus 18 through one or more data media interfaces.
- System memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.
- a program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment.
- Program modules 42 generally perform the functions and/or methods of the described embodiments of the present invention.
- Computer device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with computer device 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, computer device 12 may communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet, through network adapters 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be understood that, although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems, etc.
- the processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, to realize the steps of an image reconstruction model generation method provided by the embodiment of the present invention, and the method includes:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- the sixth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method provided by any embodiment of the present invention, including:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- the computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums.
- the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
- the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above.
- a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
- a computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
- a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
- Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
- Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional A procedural programming language, such as the "C" language or similar programming language.
- the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, through the Internet using an Internet service provider) connect).
- LAN local area network
- WAN wide area network
- Internet service provider an external computer
- modules or steps of the present invention can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices.
- they may be implemented in program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated separately into individual integrated circuit modules, or a plurality of modules of them Or the steps are made into a single integrated circuit module to realize.
- the present invention is not limited to any specific combination of hardware and software.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
Disclosed in the embodiments of the present invention are an image reconstruction model generation method and apparatus, an image reconstruction method and apparatus, and a device and a medium. The image reconstruction model generation method comprises: acquiring a preliminary reconstructed magnetic resonance image, and reducing the resolution of the preliminary reconstructed magnetic resonance image according to a preset image resolution increase multiple, so as to obtain a low-resolution image; performing image interpolation processing on the low-resolution image, a multiple for image interpolation being the preset image resolution increase multiple; and training a high-resolution image reconstruction model by using the image having been subjected to interpolation processing as input data for model training and using the preliminary reconstructed magnetic resonance image as label data, and when a loss function of the high-resolution image reconstruction model converges to a preset value, generating a target high-resolution image reconstruction model. By means of the technical solution of the present embodiment, hidden issues such as a poor learning effect caused by data deviation are eliminated, and the resolution of a reconstructed image can be increased while shortening the imaging time, thereby achieving a better image effect.
Description
本发明实施例涉及医学图像处理技术领域,尤其涉及一种图像重建模型生成及图像重建方法、装置、设备和介质。Embodiments of the present invention relate to the technical field of medical image processing, and in particular, to an image reconstruction model generation and image reconstruction method, apparatus, device, and medium.
磁共振成像(Magnetic Resonance Imaging,MRI)技术以其无创,无辐射性,良好的软组织对比度以及任意层面成像等特点,被广泛应用于临床诊断和医学研究。现阶段,心脏磁共振电影已经被认为是评估心脏功能的影像学金标准。然而,临床上进行高分辨率图像采集时,往往需要较长的采集时间。同时,受到病人耐受性以及呼吸运动等影响,实际临床过程中,往往很难得到高分辨率心脏磁共振图像。Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and imaging at any level. At this stage, cardiac magnetic resonance cine has been regarded as the imaging gold standard for assessing cardiac function. However, when high-resolution image acquisition is performed clinically, a long acquisition time is often required. At the same time, it is often difficult to obtain high-resolution cardiac magnetic resonance images in the actual clinical process due to the influence of patient tolerance and respiratory movement.
目前,得到高分辨率的磁共振心脏电影图像的方法主要有基于插值的图像重建方法、基于传统机器学习的图像重建方法和基于深度学习的图像重建方法。其中,基于深度学习网络的方法,对于线性及非线性方法均有良好的处理能力,能够重建得到图像质量更高的磁共振图像。At present, the methods for obtaining high-resolution magnetic resonance cardiac cine images mainly include image reconstruction methods based on interpolation, image reconstruction methods based on traditional machine learning, and image reconstruction methods based on deep learning. Among them, the method based on the deep learning network has good processing ability for linear and nonlinear methods, and can reconstruct the magnetic resonance image with higher image quality.
但是,现有的基于深度学习进行超分辨率图像重建的相关方法并没有解决其对数据的依赖问题,仍需要大量配对的低分辨率图像和高分辨率图像进行学习。然而,从实际操作和病人伦理的角度,以及医学图像的特殊性,配对的高分辨率和低分辨率电影图像很难获得。同时,对于不同分辨率、不同场强及不同相位的电影成像,需要进行标准化配准,而这一过程受到机器及几何形变的影响往往并不准确。这使得现有的基于深度学习得超分辨方法容易受到数据偏差的影响,增大了其大范围临床推广的难度。However, the existing deep learning-based methods for super-resolution image reconstruction have not solved their dependence on data, and still require a large number of paired low-resolution images and high-resolution images for learning. However, paired high-resolution and low-resolution cine images are difficult to obtain from practical and patient ethics perspectives, as well as the specificity of medical images. At the same time, for film imaging with different resolutions, different field strengths and different phases, standardized registration is required, and this process is often inaccurate due to the influence of machine and geometric deformation. This makes the existing deep learning-based super-resolution methods susceptible to data bias, which increases the difficulty of large-scale clinical promotion.
发明内容SUMMARY OF THE INVENTION
本发明实施例提供了一种图像重建模型生成及图像重建方法、装置、设备 和介质,以实现在缩短成像的时间,且降低图像重建网络的复杂度的同时,能够提高重建图像分辨率,有更好的图像效果。The embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, device and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, and at the same time improve the reconstructed image resolution. better image effects.
第一方面,本发明实施例提供了一种图像重建模型生成方法,该方法包括:In a first aspect, an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:
获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;
对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
可选的,获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像,包括:Optionally, a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a preset ratio to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image, including:
将所述初步磁共振重建图像输入至预设生成对抗网络的生成器,由所述生成器对所述初步磁共振重建图像进行卷积和降采样处理,得到分辨率降低所述预设图像分辨率提升倍数的构造低分辨率图像;Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;
在所述初步磁共振重建图像提取出与所述构造低分辨率图像相匹配的判别图像,并将所述判别图像与所述构造低分辨率图像输入至所述预设生成对抗网络的判别器,以对所述预设生成对抗网络进行训练;A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
将训练完成的预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到磁共振图像降质核函数;Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;
将所述初步磁共振重建图像与所述磁共振图像降质核函数进行卷积,得到对应的低分辨率图像。Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.
优选的,所述预设生成对抗网络的生成器包括六层卷积层,且所述六层卷积层的最后一层的卷积步长为所述预设图像分辨率提升倍数;所述预设生成对抗网络的判别器包括七层卷积层。Preferably, the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
可选的,所述对所述低分辨率图像进行图像插值处理,包括:Optionally, performing image interpolation processing on the low-resolution image includes:
对所述低分辨率图像进行所述预设图像分辨率提升倍数的双立方插值处理。Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.
可选的,在对高分辨率图像重建模型进行训练之前,所述方法还包括:Optionally, before training the high-resolution image reconstruction model, the method further includes:
将所述经过图像插值处理的图像和所述初步磁共振重建图像,同步进行旋转或镜像操作,并将旋转或镜像操作后得到的图像对作为新的高分辨率图像重建模型训练样本数据。The image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
可选的,在所述高分辨率图像重建模型的训练过程中,采用残差学习方式,将所述输入数据通过卷积层后的残差与所述输入数据本身相加后,计算和所述标签数据之间的损失函数。Optionally, in the training process of the high-resolution image reconstruction model, a residual learning method is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data. The loss function between the described label data.
第二方面,本发明实施例还提供了一种图像重建方法,该方法包括:In a second aspect, an embodiment of the present invention further provides an image reconstruction method, the method comprising:
获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;
将经过图像插值处理的初步磁共振重建图像,输入至由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。Inputting the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
第三方面,本发明实施例还提供了一种图像重建模型生成装置,该装置包括:In a third aspect, an embodiment of the present invention further provides a device for generating an image reconstruction model, the device comprising:
图像降质模块,用于获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;an image degradation module, configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image;
图像插值模块,用于对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;an image interpolation module, configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
模型训练模块,用于将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
可选的,图像降质模块具体用于:Optionally, the image degradation module is specifically used for:
将所述初步磁共振重建图像输入至预设生成对抗网络的生成器,由所述生成器对所述初步磁共振重建图像进行卷积和降采样处理,得到分辨率降低所述预设图像分辨率提升倍数的构造低分辨率图像;Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;
在所述初步磁共振重建图像提取出与所述构造低分辨率图像相匹配的判别图像,并将所述判别图像与所述构造低分辨率图像输入至所述预设生成对抗网络的判别器,以对所述预设生成对抗网络进行训练;A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
将训练完成的预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到磁共振图像降质核函数;Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;
将所述初步磁共振重建图像与所述磁共振图像降质核函数进行卷积,得到对应的低分辨率图像。Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.
优选的,所述预设生成对抗网络的生成器包括六层卷积层,且所述六层卷积层的最后一层的卷积步长为所述预设图像分辨率提升倍数;所述预设生成对抗网络的判别器包括七层卷积层。Preferably, the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
可选的,所述图像插值模块具体用于:Optionally, the image interpolation module is specifically used for:
对所述低分辨率图像进行所述预设图像分辨率提升倍数的双立方插值处理。Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.
可选的,所述图像重建模型生成装置还包括训练样本增强模块,用于在对高分辨率图像重建模型进行训练之前,将所述经过图像插值处理的图像和所述初步磁共振重建图像,同步进行旋转或镜像操作,并将旋转或镜像操作后得到的图像对作为新的高分辨率图像重建模型训练样本数据。Optionally, the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
可选的,所述模型训练模块还用于,在所述高分辨率图像重建模型的训练过程中,采用残差学习方式,将所述输入数据通过卷积层后的残差与所述输入数据本身相加后,计算和所述标签数据之间的损失函数。Optionally, the model training module is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to combine the residual of the input data after passing through the convolution layer with the input. After the data itself is added, a loss function between the data and the label data is calculated.
第四方面,本发明实施例还提供了一种图像重建装置,该装置包括:In a fourth aspect, an embodiment of the present invention further provides an image reconstruction device, the device comprising:
图像预处理模块,用于获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;an image preprocessing module, configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;
图像重建模块,用于将经过图像插值处理的初步磁共振重建图像,输入至 由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。The image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset resolution enhancement factor obtained by the image reconstruction model generation method described in any one of the embodiments. In the target image reconstruction model of , the target magnetic resonance image reconstruction image is obtained.
第五方面,本发明实施例还提供了一种计算机设备,所述计算机设备包括:In a fifth aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:
一个或多个处理器;one or more processors;
存储器,用于存储一个或多个程序;memory for storing one or more programs;
当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如本发明任意实施例所提供的图像重建模型生成方法或图像重建方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method provided by any embodiment of the present invention.
第六方面,本发明实施例还提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现如本发明任意实施例所提供的图像重建模型生成方法或图像重建方法。In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method or image provided by any embodiment of the present invention rebuild method.
上述发明中的实施例具有如下优点或有益效果:The embodiments in the above invention have the following advantages or beneficial effects:
本发明实施例,通过获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。解决了现有技术中对磁共振图像重建模型训练对数据的依赖问题,利用目前获取到的磁共振图像本身进行降质得到训练数据和标签数据,可完成图像重建模型的网络训练,实现了无需额外收集大量配对好的低分辨率-高分辨率磁共振参数定量图像数据训练神经网络,可利用自身图像内部进行学习,无需大量配对图像数据,是一种无监督的学习方法,成像速度快,同时,消除了数据偏差带来的学习效果不佳等隐患。In this embodiment of the present invention, a preliminary magnetic resonance reconstruction image is acquired, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a multiple according to a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image, so as to obtain a corresponding low-resolution image; Perform image interpolation processing, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used. As the label data, a high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
图1是本发明实施例一提供的一种图像重建模型生成方法的流程图;1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention;
图2是本发明实施例一提供的一种生成对抗网络结构示意图;2 is a schematic structural diagram of a generative adversarial network according to Embodiment 1 of the present invention;
图3是本发明实施例一提供的一种生成对抗网络中判别器的网络结构示意图;3 is a schematic diagram of a network structure of a discriminator in a generative adversarial network provided by Embodiment 1 of the present invention;
图4是本发明实施例一提供的一种生成对抗网络中生成器网络结构示意图;4 is a schematic diagram of a generator network structure in a generative adversarial network according to Embodiment 1 of the present invention;
图5是本发明实施例一提供的一种图像重建模型网络训练过程示意图;5 is a schematic diagram of a network training process of an image reconstruction model according to Embodiment 1 of the present invention;
图6是本发明实施例二提供的一种图像重建方法的流程图;6 is a flowchart of an image reconstruction method according to Embodiment 2 of the present invention;
图7是本发明实施例二提供的一种图像重建过程示意图;7 is a schematic diagram of an image reconstruction process according to Embodiment 2 of the present invention;
图8是本发明实施例二提供的一种不同图像重建方法进行磁共振图像重建效果对比图;8 is a comparison diagram of the effect of magnetic resonance image reconstruction performed by different image reconstruction methods according to Embodiment 2 of the present invention;
图9是本发明实施例三提供的一种图像重建模型生成装置的结构示意图;9 is a schematic structural diagram of an apparatus for generating an image reconstruction model according to Embodiment 3 of the present invention;
图10是本发明实施例四提供的一种图像重建装置的结构示意图;10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention;
图11是本发明实施例五提供的一种计算机设备的结构示意图。FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
下面结合附图和实施例对本发明作进一步的详细说明。可以理解的是,此处所描述的具体实施例仅仅用于解释本发明,而非对本发明的限定。另外还需要说明的是,为了便于描述,附图中仅示出了与本发明相关的部分而非全部结构。The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.
实施例一Example 1
图1为本发明实施例一提供的一种图像重建模型生成方法的流程图,本实施例可适用于利用低分辨率的磁共振图像本身进行图像重建模型训练的情况。该方法可以由图像重建模型生成装置执行,该装置可以由软件和/或硬件的方式来实现,集成于具有应用开发功能的电子设备中。FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training. The method may be executed by an image reconstruction model generating apparatus, which may be implemented in software and/or hardware, and integrated into an electronic device with an application development function.
如图1所示,图像重建模型生成方法包括以下步骤:As shown in Figure 1, the image reconstruction model generation method includes the following steps:
S110、获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像。S110. Acquire a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image.
其中,初步磁共振重建图像是目前磁共振成像设备能够扫描获得的磁共振图像,即相对分辨率较低,且还未经过提高分辨率重建(超分辨重构)的图像。而预设图像分辨率提升倍数则是希望能够对初步磁共振重建图像进行分辨率提升的倍数。示例性的,若初步磁共振重建图像的分辨率是128*128,希望能够对初步磁共振重建图像进行图像重建,得到分辨率为512*512的高分辨率图像;那么,预设图像分辨率提升倍数即可通过(512*512)除以(128*128)得到,预设图像分辨率提升倍数为16。The preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by a current magnetic resonance imaging device, that is, an image with relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction). The preset image resolution enhancement multiple is a multiple that is expected to be able to improve the resolution of the preliminary magnetic resonance reconstruction image. Exemplarily, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution The enhancement factor can be obtained by dividing (512*512) by (128*128). The preset image resolution enhancement factor is 16.
在本实施例中,按照预设图像分辨率提升倍数,降低初步磁共振重建图像的分辨率,得到对应的低分辨率图像,目的是以初步磁共振重建图像本身作为图像重建网络的训练参数,而不是去给初步磁共振重建图像匹配对应的高分辨率的高质量磁共振重建图像。从而无需额外收集大量配对好的低分辨率-高分辨率磁共振参数定量图像数据训练神经网络,可以降低获取图像重建模型训练样本的难度。In this embodiment, the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution increase multiple to obtain a corresponding low-resolution image, and the purpose is to use the preliminary magnetic resonance reconstruction image itself as the training parameter of the image reconstruction network, Instead of matching the initial MR reconstruction image with the corresponding high-resolution high-quality MR reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for image reconstruction models.
具体的,降低图像分辨率的方式可以采用降采样方法,对初步磁共振重建图像进行降采样处理,得到对应的低分辨率图像。Specifically, the way to reduce the image resolution can be a down-sampling method to perform down-sampling processing on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.
在一种优选的实施方式中,还可以利用生成对抗网络,对初步磁共振重建图像进行降质处理,得到分辨率降低预设图像分辨率提升倍数的低分辨率图像。在训练生成对抗网络过程中,可以确定一个核函数,用于对一个或多个初步磁共振重建图像进行同样的图像降质处理。具体的,首先,将初步磁共振重建图像输入至如图2所示的预设生成对抗网络中。先由生成器G对初步磁共振重建图像进行卷积与降采样处理,得到分辨率降低预设图像分辨率提升倍数的构造低分辨率图像,可将得到的图像视为初步磁共振重建图像的伪造版本图像F;进一步的,从初步磁共振重建图像中截取出和图像F大小一样且位置匹配的图像块,作为真实样本图像T。图像T和图像F一起输入至判别器D后,根据 图像T,判别器逐像素点分析图像F是真实图像的可能性大小。生成器的目标是尽可能地使输出的图像F欺骗判别器。生成器和判别器两个网络在相互对抗过程中,不断调整参数,最终判别器无法判断生成器的输出结果F是否真实时,网络训练完成。由判别器输出的热点图(heat map)中,是图像F和图像T逐像素点比较,相互比较的每两个像素点是同一像素点的概率值,当热点图中年的概率值均满足肯定判定的概率值时,即可确定判别器无法判断生成器的输出结果F是否真实。进一步的,将训练完成的预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到磁共振图像降质核函数;从而,可以将各初步磁共振重建图像分别与磁共振图像降质核函数进行卷积,得到对应的低分辨率图像。在本实施例中,预设生成对抗网络中的生成器中的卷积核大小和个数可以变更调整的,生成器包括六个卷积核,各卷积核的大小为:5x5、3x3、1x1、1x1、1x1和1x1。在一个优选的实施例中,预设生成对抗网络中的判别器结构可以是图3所示的结构,由7层卷积层组成,各卷积层的卷积核大小分别为:7x7、1x1、1x1、1x1、1x1、1x1和1x1。生成器的网络结构如图4所示,由6层卷积层组成,卷积核大小分别为:7x7、5x5、3x3、1x1、1x1、1x1。其中,生成器最后一层卷积层的卷积步长数值为预设图像分辨率提升倍数,以达到对输入图像进行降采样的效果。In a preferred embodiment, a generative adversarial network can also be used to degrade the preliminary magnetic resonance image reconstruction image to obtain a low-resolution image whose resolution is reduced by a preset image resolution enhancement factor. During training of the generative adversarial network, a kernel function can be determined for performing the same image degradation process on one or more preliminary MRI reconstruction images. Specifically, first, the preliminary magnetic resonance reconstruction image is input into the preset generative adversarial network as shown in FIG. 2 . First, the generator G performs convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image, and obtains a low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple. The obtained image can be regarded as the original magnetic resonance reconstruction image. Fake version image F; further, cut out image blocks with the same size and matching position as image F from the preliminary magnetic resonance reconstruction image, as real sample image T. After the image T and the image F are input to the discriminator D together, according to the image T, the discriminator analyzes the possibility that the image F is a real image pixel by pixel. The goal of the generator is to make the output image F fool the discriminator as much as possible. The two networks, the generator and the discriminator, constantly adjust the parameters in the process of confrontation with each other. Finally, when the discriminator cannot judge whether the output result F of the generator is real, the network training is completed. In the heat map output by the discriminator, the image F and the image T are compared pixel by pixel, and every two pixels compared with each other are the probability value of the same pixel point, when the probability value of the year in the heat map is satisfied. When the probability value of the judgment is affirmative, it can be determined that the discriminator cannot judge whether the output result F of the generator is true. Further, all convolutional layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be respectively combined with the magnetic resonance image. The image degradation kernel function performs convolution to obtain the corresponding low-resolution image. In this embodiment, the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted. The generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1. In a preferred embodiment, the structure of the discriminator in the preset Generative Adversarial Network may be the structure shown in FIG. 3 , which consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are: 7×7 and 1×1 respectively. , 1x1, 1x1, 1x1, 1x1 and 1x1. The network structure of the generator is shown in Figure 4, which consists of 6 convolution layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1. Among them, the value of the convolution step size of the last convolutional layer of the generator is the preset image resolution enhancement multiple, so as to achieve the effect of downsampling the input image.
这里需要说明的是,若磁共振成像对象为心脏,那么获取到的初步磁共振重建图像为心脏磁共振电影图像,其中,包括多帧磁共振图像。在进行生成对抗网络训练时,可取心脏磁共振电影图像中任意一帧作为输入图像。It should be noted here that, if the magnetic resonance imaging object is the heart, the acquired preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance cine image, which includes multiple frames of magnetic resonance images. During the training of the generative adversarial network, any frame of the cardiac magnetic resonance cine image can be taken as the input image.
S120、对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数。S120. Perform image interpolation processing on the low-resolution image, where the multiple of image interpolation is the preset image resolution enhancement multiple.
在本步骤中,将低分辨率图像进行图像插值处理,且图像插值的倍数为预设图像分辨率提升倍数,是为了使图像重建模型的输入图像的大小与输出图像的大小保持一致,可以缩短模型训练的时间,优化模型训练的过程。In this step, image interpolation processing is performed on the low-resolution image, and the multiple of image interpolation is the preset image resolution enhancement multiple, so as to keep the size of the input image of the image reconstruction model consistent with the size of the output image, which can be shortened. Model training time, optimizing the model training process.
具体的,在本实施例中,插值方法可以采用双立方插值(Bicubic)算法。 这是由于双立方插值在图像放大过程可以保留更多的图像细节,放大以后的图像带有反锯齿的功能,同时,放大图像和源图像相比效果更加真实。Specifically, in this embodiment, the interpolation method may adopt a bicubic interpolation (Bicubic) algorithm. This is because the bicubic interpolation can preserve more image details during the image enlargement process, and the enlarged image has the function of anti-aliasing. At the same time, the enlarged image has a more realistic effect than the source image.
在一种优选的实施方式中,还可以将经过图像插值处理的图像和对应的初步磁共振重建图像,同步进行旋转或镜像操作,并将旋转或镜像操作后得到的图像对作为新的高分辨率图像重建模型训练样本数据。例如,将图像对进行0度、90度、180度及270度的旋转操作,在此基础上进行分别水平和垂直方向的镜面对称操作,便可以由此得到8组训练数据,使训练数据得到增强。而且,经过同步处理的图像对,数据之间相位一致,无需进行配准,从而消除了数据偏差带来的学习效果不佳等隐患。In a preferred embodiment, the image that has undergone image interpolation processing and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation can be used as a new high-resolution image. Rate image reconstruction model training sample data. For example, rotate the image pair at 0, 90, 180, and 270 degrees, and then perform mirror symmetry operations in the horizontal and vertical directions, respectively, to obtain 8 sets of training data, so that the training data can be obtained. enhanced. Moreover, for the synchronized image pairs, the phases of the data are consistent, and no registration is required, thereby eliminating hidden dangers such as poor learning effects caused by data deviations.
S130、将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。S130. Use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
在经过上述步骤之后,构建了足够数量的模型训练样本数据,便可以开始模型训练的过程。将经过图像插值处理的图像作为模型训练的输入数据,将低分辨率图像对应的初步磁共振重建图像作为标签数据。即将获取到的初步磁共振图像作为标签数据,将经过分辨率降低了预设图像分辨率提升倍数的低分辨图像。那么,在将初步磁共振重建图像输入到训练好的图像重建模型中,便可以由图像重建模型输出得到将初步磁共振重建图像分辨率提升预设图像分辨率提升倍数的高分辨率图像。After the above steps, a sufficient amount of model training sample data is constructed, and then the model training process can begin. The image that has undergone image interpolation processing is used as the input data for model training, and the preliminary MRI reconstruction image corresponding to the low-resolution image is used as the label data. The to-be-obtained preliminary magnetic resonance image is used as the label data, and the low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple. Then, after the preliminary magnetic resonance reconstruction image is input into the trained image reconstruction model, a high-resolution image that increases the resolution of the preliminary magnetic resonance reconstruction image by a preset image resolution can be obtained from the output of the image reconstruction model.
具体的,目标图像重建模型的训练过程,可以参考图5所示的过程。在图5所示的过程中,图像重建网络为全卷积网络,一共有8层卷积层,其中,第一层的卷积核大小为3x3,通道数为f,第二至第七层的卷积核大小为3x3,通道数为64,最后一层卷积核大小为3x3,通道数为f。其中f代表同时输入至图像重建网络中的图像帧数。对应一张图像来说,f数值即为1。当采集到的初步 磁共振重建图像为心脏电影图像时,f的数值即为心脏电影图像中图像帧数。图像重建网络的卷积层个数也可变更为6及以上,同时卷积核大小不限于3x3,可以根据计算需求进行参数调整。Specifically, for the training process of the target image reconstruction model, reference may be made to the process shown in FIG. 5 . In the process shown in Figure 5, the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers. The convolution kernel size of the first layer is 3x3, the number of channels is f, and the second to seventh layers are The size of the convolution kernel is 3x3, the number of channels is 64, and the size of the convolution kernel of the last layer is 3x3, and the number of channels is f. where f represents the number of image frames simultaneously input to the image reconstruction network. For an image, the f value is 1. When the acquired preliminary MRI reconstructed image is a cardiac cine image, the value of f is the number of image frames in the cardiac cine image. The number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the size of the convolution kernel is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.
进一步的,图5中的模型训练过程,采用了一种优选的深度学习方式,即残差学习方式,将输入模型的图像数据通过卷积层后的残差与所述输入的图像数据本身相加后,计算和所述标签数据之间的损失函数。这里需要说明的是,损失函数不仅可以采用图5中所示的L1损失函数,还可以采用L2损失、感知损失等损失函数。进一步的,还可以采用Adam优化算法、随机梯度下降算法或AdaGrad等优化算法对网络学习过程进行优化。Further, the model training process in Fig. 5 adopts a preferred deep learning method, that is, the residual learning method, which compares the residual of the image data of the input model after passing through the convolution layer with the input image data itself. After adding, calculate the loss function between and the label data. It should be noted here that the loss function may not only use the L1 loss function shown in FIG. 5 , but also use loss functions such as L2 loss and perceptual loss. Further, optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad can also be used to optimize the network learning process.
这里要说明的是,在各网络的结构图中,conv代表卷积神经网络中的卷积核,“.mat”是文件格式的缩写,在本实施例中表示由预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到的磁共振图像降质核函数。It should be noted here that in the structure diagram of each network, conv represents the convolution kernel in the convolutional neural network, and ".mat" is the abbreviation of the file format. All the parameters of the convolutional layers of the device are convolved layer by layer to obtain the degraded kernel function of the magnetic resonance image.
本实施例的技术方案,通过将获取到的初步磁共振重建图像进行降低图像分辨率处理得到对应的低分辨率图像,并且可控分辨率降低倍数;然后,对低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。解决了现有技术中对磁共振图像重建模型训练对数据的依赖问题,利用目前获取到的磁共振图像本身进行降质得到训练数据和标签数据,可完成图像重建模型的网络训练,实现了无需额外收集大量配对好的低分辨率-高分辨率磁共振参数定量图像数据训练神经网络,可利用自身图像内部进行学习,无需大量配对图像数据,是一种无监督的学习方法,成像速度快,同时,消除了数据偏差带来的学习效果不佳等隐患。In the technical solution of this embodiment, a corresponding low-resolution image is obtained by reducing the image resolution of the obtained preliminary magnetic resonance reconstruction image, and the resolution reduction factor is controllable; then, image interpolation processing is performed on the low-resolution image , where the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, The high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
实施例二Embodiment 2
图6为本发明实施例二提供的一种图像重建方法的流程图,本实施例可适 用于对采集得到的低分辨率医学图像重建,得到高分辨率图像的情况。该方法可以由图像重建装置执行,该装置可以由软件和/或硬件的方式来实现,集成于具有应用开发功能的计算机设备中。Fig. 6 is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention, and this embodiment can be applied to the situation of reconstructing a collected low-resolution medical image to obtain a high-resolution image. The method may be performed by an image reconstruction apparatus, and the apparatus may be implemented in software and/or hardware, and integrated into a computer device with an application development function.
如图6所示,图像重建方法包括以下步骤:As shown in Figure 6, the image reconstruction method includes the following steps:
S210、获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理。S210. Acquire a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image by a preset resolution increase multiple.
初步磁共振重建图像即为需要提高分辨率的低分辨率图像,可跟据分辨率提升需求,对初步磁共振重建图像进行插值重建,得到预处理后的图像。图像插值的倍数即为预设分辨率提升倍数。The preliminary magnetic resonance reconstruction image is a low-resolution image whose resolution needs to be improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image. The multiple of image interpolation is the preset resolution enhancement multiple.
S220、将经过图像插值处理的初步磁共振重建图像,输入至由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。S220. Input the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction obtained by the image reconstruction model generation method according to any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor In the model, the target magnetic resonance reconstruction image is obtained.
其中,目标图像重建模型,是根据预设分辨率提升倍数的需求,通过上述实施例中的图像重建模型生成方法,训练的到的一个图像重建模型。具体的,图像重建的过程,可参考图7所示的示意图。The target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above-mentioned embodiment according to the requirement of increasing the preset resolution. Specifically, for the process of image reconstruction, reference may be made to the schematic diagram shown in FIG. 7 .
进一步的,图8中示出了不同图形重建方法对同一张初步磁共振重建图像进行图像重建得到的重建图像,其中,(a)NN为最近邻插值算法得到的结果,(b)Bicubic为双立方插值算法得到的结果,(c)zero-padding为图像变换到傅里叶域后,对四周进行补零后反变换回图像域的结果,(d)SR(Super Resolution)为本实施例图像重建方法得到的结果。直观的从结果中看,本实施例图像重建方法得到的图像重建结果更为清晰。Further, Fig. 8 shows the reconstructed image obtained by performing image reconstruction on the same preliminary magnetic resonance reconstructed image by different graphic reconstruction methods, wherein (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the double The result obtained by the cubic interpolation algorithm, (c) zero-padding is the result of transforming the image back to the image domain after the image is transformed to the Fourier domain, and then inversely transforming back to the image domain after zero-filling the surrounding area, (d) SR (Super Resolution) is the image of this example The results obtained by the reconstruction method. Intuitively, from the results, the image reconstruction results obtained by the image reconstruction method in this embodiment are clearer.
本实施例的技术方案,通过将低分辨率图像输入至预先训练好了,可将图像分辨率提升预设倍数的图像重建模型中,得到重建后的高分辨率图像;解决了现有技术中图像重建时间长,成像速度慢的问题,实现了通过结构较为简单的图像重建网络,无需大量计算,便可在较短时间内得到边缘清晰的高分辨图像。In the technical solution of this embodiment, by inputting a low-resolution image into a pre-trained image reconstruction model that can increase the image resolution by a preset multiple, a reconstructed high-resolution image is obtained; Due to the problems of long image reconstruction time and slow imaging speed, it is possible to obtain high-resolution images with clear edges in a relatively short period of time without a lot of computation through a relatively simple image reconstruction network.
实施例三Embodiment 3
图9为本发明实施例三提供的一种图像重建模型生成装置的结构示意图,本实施例可适用于利用低分辨率的磁共振图像本身进行图像重建模型训练的情况。FIG. 9 is a schematic structural diagram of an image reconstruction model generating apparatus according to Embodiment 3 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
如图9所示,图像重建模型生成装置包括图像降质模块310、图像插值模块320和模型训练模块330。As shown in FIG. 9 , the image reconstruction model generating apparatus includes an image degradation module 310 , an image interpolation module 320 and a model training module 330 .
其中,图像降质模块310,用于获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;图像插值模块320,用于对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;模型训练模块330,用于将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。Wherein, the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; the image interpolation module 320, for performing image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; model training module 330, for using the image subjected to image interpolation processing as model training The input data of the high-resolution image reconstruction model is trained by using the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as the label data. When the loss function of the high-resolution image reconstruction model converges to the preset value , to generate the target high-resolution image reconstruction model.
本实施例的技术方案,通过获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。解决了现有技术中对磁共振图像重建模型训练对数据的依赖问题,利用目前获取到的磁共振图像本身进行降质得到训练数据和标签数据,可完成图像重建模型的网络训练,实现了无需额外收集大量配对好的低分辨率-高分辨率磁共振参数定量图像数据训练神经网络,可利用自身图像内部进行学习,无需大量配对图像数据,是一种无监督的学习方法,成像速度快,同时,消除了数据偏差带来的学习效果不佳等隐患。In the technical solution of this embodiment, by acquiring a preliminary magnetic resonance reconstruction image, and increasing the resolution of the preliminary magnetic resonance image by a multiple, the resolution of the preliminary magnetic resonance reconstruction image is reduced, and a corresponding low-resolution image is obtained; Perform image interpolation processing on the low-resolution images, wherein the multiple of image interpolation is the preset image resolution improvement multiple; the image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance imaging corresponding to the low-resolution image is used. The reconstructed image is used as label data to train a high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for A large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
可选的,图像降质模块310具体用于:Optionally, the image degradation module 310 is specifically used for:
将所述初步磁共振重建图像输入至预设生成对抗网络的生成器,由所述生成器对所述初步磁共振重建图像进行卷积和降采样处理,得到分辨率降低所述预设图像分辨率提升倍数的构造低分辨率图像;Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;
在所述初步磁共振重建图像提取出与所述构造低分辨率图像相匹配的判别图像,并将所述判别图像与所述构造低分辨率图像输入至所述预设生成对抗网络的判别器,以对所述预设生成对抗网络进行训练;A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
将训练完成的预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到磁共振图像降质核函数;Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;
将所述初步磁共振重建图像与所述磁共振图像降质核函数进行卷积,得到对应的低分辨率图像。Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.
优选的,所述预设生成对抗网络的生成器包括六层卷积层,且所述六层卷积层的最后一层的卷积步长为所述预设图像分辨率提升倍数;所述预设生成对抗网络的判别器包括七层卷积层。Preferably, the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
可选的,所述图像插值模块320具体用于:Optionally, the image interpolation module 320 is specifically used for:
对所述低分辨率图像进行所述预设图像分辨率提升倍数的双立方插值处理。Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.
可选的,所述图像重建模型生成装置还包括训练样本增强模块,用于在对高分辨率图像重建模型进行训练之前,将所述经过图像插值处理的图像和所述初步磁共振重建图像,同步进行旋转或镜像操作,并将旋转或镜像操作后得到的图像对作为新的高分辨率图像重建模型训练样本数据。Optionally, the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
可选的,所述模型训练模块330还用于,在所述高分辨率图像重建模型的训练过程中,采用残差学习方式,将所述输入数据通过卷积层后的残差与所述输入数据本身相加后,计算和所述标签数据之间的损失函数。Optionally, the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to compare the residual of the input data after passing through the convolutional layer with the residual. After the input data itself is added, a loss function between the input data and the label data is calculated.
本发明实施例所提供的图像重建模型生成装置可执行本发明任意实施例所提供的图像重建模型生成方法,具备执行方法相应的功能模块和有益效果。The image reconstruction model generation apparatus provided by the embodiment of the present invention can execute the image reconstruction model generation method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
实施例四Embodiment 4
图10为本发明实施例四提供的一种图像重建装置的结构示意图,本实施例可适用于对采集得到的低分辨率医学图像重建,得到高分辨率图像的情况。FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention. This embodiment can be applied to a situation in which a low-resolution medical image obtained by acquisition is reconstructed to obtain a high-resolution image.
如图10所示,图像重建装置包括图像预处理模块410和图像重建模块420。As shown in FIG. 10 , the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420 .
其中,图像预处理模块410,用于获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;图像重建模块420,用于将经过图像插值处理的初步磁共振重建图像,输入至由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。Among them, the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple; the image reconstruction module 420 is used for image interpolation processing. The preliminary magnetic resonance reconstruction image is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor, and the target magnetic resonance image is obtained. Rebuild the image.
本实施例的技术方案,通过将低分辨率图像输入至预先训练好了,可将图像分辨率提升预设倍数的图像重建模型中,得到重建后的高分辨率图像;解决了现有技术中图像重建时间长,成像速度慢的问题,实现了通过结构较为简单的图像重建网络,无需大量计算,便可在较短时间内得到边缘清晰的高分辨图像。In the technical solution of this embodiment, by inputting a low-resolution image into a pre-trained image reconstruction model that can increase the image resolution by a preset multiple, a reconstructed high-resolution image is obtained; Due to the problems of long image reconstruction time and slow imaging speed, it is possible to obtain high-resolution images with clear edges in a relatively short period of time without a lot of computation through a relatively simple image reconstruction network.
本发明实施例所提供的图像重建装置可执行本发明任意实施例所提供的图像重建方法,具备执行方法相应的功能模块和有益效果。The image reconstruction apparatus provided by the embodiment of the present invention can execute the image reconstruction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
实施例五Embodiment 5
图11为本发明实施例五提供的一种计算机设备的结构示意图。图11示出了适于用来实现本发明实施方式的示例性计算机设备12的框图。图11显示的计算机设备12仅仅是一个示例,不应对本发明实施例的功能和使用范围带来任何限制。计算机设备12可以是与磁共振扫描成像设备相连接的,任意具有计算能力的终端设备,如智能控制器及服务器、手机等终端设备。FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention. Figure 11 shows a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 11 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention. The computer device 12 may be any terminal device with computing capability connected to the magnetic resonance scanning imaging device, such as an intelligent controller, a server, a mobile phone and other terminal devices.
如图11所示,计算机设备12以通用计算设备的形式表现。计算机设备12的组件可以包括但不限于:一个或者多个处理器或者处理单元16,***存储器28,连接不同***组件(包括***存储器28和处理单元16)的总线18。As shown in FIG. 11, computer device 12 takes the form of a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .
总线18表示几类总线结构中的一种或多种,包括存储器总线或者存储器控制器,***总线,图形加速端口,处理器或者使用多种总线结构中的任意总线 结构的局域总线。举例来说,这些体系结构包括但不限于工业标准体系结构(ISA)总线,微通道体系结构(MAC)总线,增强型ISA总线、视频电子标准协会(VESA)局域总线以及***组件互连(PCI)总线。 Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.
计算机设备12典型地包括多种计算机***可读介质。这些介质可以是任何能够被计算机设备12访问的可用介质,包括易失性和非易失性介质,可移动的和不可移动的介质。 Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including both volatile and nonvolatile media, removable and non-removable media.
***存储器28可以包括易失性存储器形式的计算机***可读介质,例如随机存取存储器(RAM)30和/或高速缓存存储器32。计算机设备12可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机***存储介质。仅作为举例,存储***34可以用于读写不可移动的、非易失性磁介质(图11未显示,通常称为“硬盘驱动器”)。尽管图11中未示出,可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器,以及对可移动非易失性光盘(例如CD-ROM,DVD-ROM或者其它光介质)读写的光盘驱动器。在这些情况下,每个驱动器可以通过一个或者多个数据介质接口与总线18相连。***存储器28可以包括至少一个程序产品,该程序产品具有一组(例如至少一个)程序模块,这些程序模块被配置以执行本发明各实施例的功能。 System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 . Computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. For example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 11, commonly referred to as a "hard drive"). Although not shown in Figure 11, a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives. In these cases, each drive may be connected to bus 18 through one or more data media interfaces. System memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.
具有一组(至少一个)程序模块42的程序/实用工具40,可以存储在例如***存储器28中,这样的程序模块42包括但不限于操作***、一个或者多个应用程序、其它程序模块以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本发明所描述的实施例中的功能和/或方法。A program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the described embodiments of the present invention.
计算机设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信,还可与一个或者多个使得用户能与该计算机设备12交互的设备通信,和/或与使得该计算机设备12能与一个或多个其它计算设备进行通信的任何设备(例如网卡,调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且,计算机设备12还可以通过网络适配器20与 一个或者多个网络(例如局域网(LAN),广域网(WAN)和/或公共网络,例如因特网)通信。如图所示,网络适配器20通过总线18与计算机设备12的其它模块通信。应当明白,尽管图11中未示出,可以结合计算机设备12使用其它硬件和/或软件模块,包括但不限于:微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID***、磁带驱动器以及数据备份存储***等。 Computer device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with computer device 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, computer device 12 may communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet, through network adapters 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be understood that, although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems, etc.
处理单元16通过运行存储在***存储器28中的程序,从而执行各种功能应用以及数据处理,例如实现本发实施例所提供的一种图像重建模型生成方法步骤,该方法包括:The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, to realize the steps of an image reconstruction model generation method provided by the embodiment of the present invention, and the method includes:
获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;
对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
或者,还可以实现本发实施例所提供的一种图像重建方法步骤,该方法包括:Alternatively, the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;
将经过图像插值处理的初步磁共振重建图像,输入至由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。Inputting the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
实施例六Embodiment 6
本实施例六提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现如本发明任意实施例所提供的图像重建模型生成方法, 包括:The sixth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method provided by any embodiment of the present invention, including:
获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;
对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
或者,还可以实现本发实施例所提供的一种图像重建方法步骤,该方法包括:Alternatively, the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;
将经过图像插值处理的初步磁共振重建图像,输入至由任一实施例所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。Inputting the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
本发明实施例的计算机存储介质,可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是但不限于:电、磁、光、电磁、红外线、或半导体的***、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行***、装置或者器件使用或者与其结合使用。The computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读介质可以发送、传播或者传输用于由指令执行***、装置或者器件使用或者与其结合使用的程序。A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:无线、电线、光缆、RF等等,或者上述的任意合适的组合。Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码,所述程序设计语言包括面向对象的程序设计语言,诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言,诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络,包括局域网(LAN)或广域网(WAN),连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional A procedural programming language, such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, through the Internet using an Internet service provider) connect).
本领域普通技术人员应该明白,上述的本发明的各模块或各步骤可以用通用的计算装置来实现,它们可以集中在单个计算装置上,或者分布在多个计算装置所组成的网络上,可选地,他们可以用计算机装置可执行的程序代码来实现,从而可以将它们存储在存储装置中由计算装置来执行,或者将它们分别制作成各个集成电路模块,或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样,本发明不限制于任何特定的硬件和软件的结合。Those of ordinary skill in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices. Optionally, they may be implemented in program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated separately into individual integrated circuit modules, or a plurality of modules of them Or the steps are made into a single integrated circuit module to realize. As such, the present invention is not limited to any specific combination of hardware and software.
注意,上述仅为本发明的较佳实施例及所运用技术原理。本领域技术人员会理解,本发明不限于这里所述的特定实施例,对本领域技术人员来说能够进 行各种明显的变化、重新调整和替代而不会脱离本发明的保护范围。因此,虽然通过以上实施例对本发明进行了较为详细的说明,但是本发明不仅仅限于以上实施例,在不脱离本发明构思的情况下,还可以包括更多其他等效实施例,而本发明的范围由所附的权利要求范围决定。Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.
Claims (11)
- 一种图像重建模型生成方法,其特征在于,包括:A method for generating an image reconstruction model, comprising:获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;Acquiring a preliminary magnetic resonance reconstruction image, increasing the resolution of the preliminary magnetic resonance image by a multiple, and reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image;对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;Perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained. When the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- 根据权利要求1所述的方法,其特征在于,获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像,包括:The method according to claim 1, wherein a preliminary magnetic resonance reconstruction image is obtained, and a resolution of the preliminary magnetic resonance reconstruction image is reduced according to a preset image resolution increase multiple to obtain a corresponding low-resolution image, include:将所述初步磁共振重建图像输入至预设生成对抗网络的生成器,由所述生成器对所述初步磁共振重建图像进行卷积和降采样处理,得到分辨率降低所述预设图像分辨率提升倍数的构造低分辨率图像;Input the preliminary magnetic resonance reconstruction image to a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain the resolution reduction of the predetermined image resolution. Construct low-resolution images with increased rate;在所述初步磁共振重建图像提取出与所述构造低分辨率图像相匹配的判别图像,并将所述判别图像与所述构造低分辨率图像输入至所述预设生成对抗网络的判别器,以对所述预设生成对抗网络进行训练;A discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;将训练完成的预设生成对抗网络中生成器的所有卷积层参数进行逐层卷积,得到磁共振图像降质核函数;Perform layer-by-layer convolution on all convolutional layer parameters of the generator in the trained preset generative adversarial network to obtain a magnetic resonance image degradation kernel function;将所述初步磁共振重建图像与所述磁共振图像降质核函数进行卷积,得到对应的低分辨率图像。Convolving the preliminary magnetic resonance reconstruction image and the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.
- 根据权利要求2所述的方法,其特征在于,所述预设生成对抗网络的生成器包括六层卷积层,且所述六层卷积层的最后一层的卷积步长为所述预设图像分辨率提升倍数;所述预设生成对抗网络的判别器包括七层卷积层。The method according to claim 2, wherein the generator of the preset generative adversarial network comprises six convolution layers, and the convolution step size of the last layer of the six convolution layers is the The preset image resolution is increased by a multiple; the discriminator of the preset generative adversarial network includes seven convolution layers.
- 根据权利要求1-3中任一所述的方法,其特征在于,所述对所述低分辨 率图像进行图像插值处理,包括:The method according to any one of claims 1-3, wherein, performing image interpolation processing on the low-resolution image, comprising:对所述低分辨率图像进行所述预设图像分辨率提升倍数的双立方插值处理。Perform bicubic interpolation processing on the low-resolution image by the preset image resolution enhancement factor.
- 根据权利要求1-3中任一所述的方法,其特征在于,在对高分辨率图像重建模型进行训练之前,所述方法还包括:The method according to any one of claims 1-3, wherein, before the high-resolution image reconstruction model is trained, the method further comprises:将所述经过图像插值处理的图像和所述初步磁共振重建图像,同步进行旋转或镜像操作,并将旋转或镜像操作后得到的图像对作为新的高分辨率图像重建模型训练样本数据。The image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
- 根据权利要求1-3中任一所述的方法,其特征在于,在所述高分辨率图像重建模型的训练过程中,采用残差学习方式,将所述输入数据通过卷积层后的残差与所述输入数据本身相加后,计算和所述标签数据之间的损失函数。The method according to any one of claims 1-3, wherein, in the training process of the high-resolution image reconstruction model, a residual learning method is used to pass the input data through the residual after the convolution layer. After the difference is added to the input data itself, a loss function between the difference and the label data is calculated.
- 一种图像重建方法,其特征在于,包括:An image reconstruction method, comprising:获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase;将经过图像插值处理的初步磁共振重建图像,输入至由权利要求1-6中任一所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。Inputting the preliminary magnetic resonance reconstructed image that has undergone image interpolation processing into the target of increasing the resolution of the magnetic resonance image by the preset resolution enhancement multiple obtained by the image reconstruction model generation method according to any one of claims 1-6 In the image reconstruction model, the target magnetic resonance reconstruction image is obtained.
- 一种图像重建模型生成装置,其特征在于,包括:A device for generating an image reconstruction model, comprising:图像降质模块,用于获取初步磁共振重建图像,并按照预设图像分辨率提升倍数,降低所述初步磁共振重建图像的分辨率,得到对应的低分辨率图像;an image degradation module, configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image;图像插值模块,用于对所述低分辨率图像进行图像插值处理,其中,图像插值的倍数为所述预设图像分辨率提升倍数;an image interpolation module, configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;模型训练模块,用于将经过图像插值处理的图像作为模型训练的输入数据,将所述低分辨率图像对应的初步磁共振重建图像作为标签数据,对高分辨率图像重建模型进行训练,当所述高分辨率图像重建模型的损失函数收敛于预设数值时,生成目标高分辨率图像重建模型。The model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
- 一种图像重建装置,其特征在于,包括:An image reconstruction device, comprising:图像预处理模块,用于获取初步磁共振重建图像,并对所述初步磁共振重建图像进行预设分辨率提升倍数的图像插值处理;an image preprocessing module, configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;图像重建模块,用于将经过图像插值处理的初步磁共振重建图像,输入至由权利要求1-6中任一所述的图像重建模型生成方法得到的将磁共振图像分辨率提升所述预设分辨率提升倍数的目标图像重建模型中,得到目标磁共振重建图像。The image reconstruction module is used to input the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset magnetic resonance image resolution enhancement obtained by the image reconstruction model generation method according to any one of claims 1-6. In the target image reconstruction model of which the resolution is increased, the target magnetic resonance reconstruction image is obtained.
- 一种计算机设备,其特征在于,所述计算机设备包括:A computer device, characterized in that the computer device comprises:一个或多个处理器;one or more processors;存储器,用于存储一个或多个程序;memory for storing one or more programs;当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-7中任一所述的图像重建模型生成方法或图像重建方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method according to any one of claims 1-7 .
- 一种计算机可读存储介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现如权利要求1-7中任一所述的图像重建模型生成方法或图像重建方法。A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the image reconstruction model generation method or the image reconstruction method according to any one of claims 1-7 is implemented.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110340196.9 | 2021-03-30 | ||
CN202110340196.9A CN115147502A (en) | 2021-03-30 | 2021-03-30 | Image reconstruction model generation and image reconstruction method, device, equipment and medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022206021A1 true WO2022206021A1 (en) | 2022-10-06 |
Family
ID=83403739
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/137623 WO2022206021A1 (en) | 2021-03-30 | 2021-12-13 | Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN115147502A (en) |
WO (1) | WO2022206021A1 (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610194A (en) * | 2017-08-14 | 2018-01-19 | 成都大学 | MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN |
CN110599401A (en) * | 2019-08-19 | 2019-12-20 | 中国科学院电子学研究所 | Remote sensing image super-resolution reconstruction method, processing device and readable storage medium |
CN111353935A (en) * | 2020-01-03 | 2020-06-30 | 首都医科大学附属北京友谊医院 | Magnetic resonance imaging optimization method and device based on deep learning |
CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
US20200311926A1 (en) * | 2019-03-27 | 2020-10-01 | The General Hospital Corporation | Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation |
-
2021
- 2021-03-30 CN CN202110340196.9A patent/CN115147502A/en active Pending
- 2021-12-13 WO PCT/CN2021/137623 patent/WO2022206021A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610194A (en) * | 2017-08-14 | 2018-01-19 | 成都大学 | MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN |
US20200311926A1 (en) * | 2019-03-27 | 2020-10-01 | The General Hospital Corporation | Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation |
CN110599401A (en) * | 2019-08-19 | 2019-12-20 | 中国科学院电子学研究所 | Remote sensing image super-resolution reconstruction method, processing device and readable storage medium |
CN111353935A (en) * | 2020-01-03 | 2020-06-30 | 首都医科大学附属北京友谊医院 | Magnetic resonance imaging optimization method and device based on deep learning |
CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
Also Published As
Publication number | Publication date |
---|---|
CN115147502A (en) | 2022-10-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10387765B2 (en) | Image correction using a deep generative machine-learning model | |
CN110809782A (en) | Attenuation correction system and method | |
JP7340107B2 (en) | Image reconstruction method, apparatus, device, system and computer-readable storage medium | |
de Leeuw Den Bouter et al. | Deep learning-based single image super-resolution for low-field MR brain images | |
CN115953494B (en) | Multi-task high-quality CT image reconstruction method based on low dose and super resolution | |
CN113870104A (en) | Super-resolution image reconstruction | |
Zhu et al. | Residual dense network for medical magnetic resonance images super-resolution | |
WO2021102644A1 (en) | Image enhancement method and apparatus, and terminal device | |
US11967066B2 (en) | Method and apparatus for processing image | |
CN105654425A (en) | Single-image super-resolution reconstruction method applied to medical X-ray image | |
WO2024051018A1 (en) | Pet parameter image enhancement method and apparatus, device, and storage medium | |
Zhao et al. | Gibbs-ringing artifact suppression with knowledge transfer from natural images to MR images | |
CN114494022B (en) | Model training method, super-resolution reconstruction method, device, equipment and medium | |
Yang et al. | Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance Imaging--Mini Review, Comparison and Perspectives | |
Lu et al. | A novel 3D medical image super-resolution method based on densely connected network | |
Li et al. | Multi-scale residual denoising GAN model for producing super-resolution CTA images | |
CN111243052A (en) | Image reconstruction method and device, computer equipment and storage medium | |
US20230386067A1 (en) | Systems and methods for segmenting 3d images | |
US20230079353A1 (en) | Image correction using an invertable network | |
CN111047512A (en) | Image enhancement method and device and terminal equipment | |
CN116630634B (en) | Image processing method, device, equipment and storage medium | |
WO2024021796A1 (en) | Image processing method and apparatus, electronic device, storage medium, and program product | |
WO2022206021A1 (en) | Image reconstruction model generation method and apparatus, image reconstruction method and apparatus, and device and medium | |
Yang et al. | Quasi-supervised learning for super-resolution PET | |
WO2022127318A1 (en) | Scanning positioning method and apparatus, storage medium and electronic device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21934657 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21934657 Country of ref document: EP Kind code of ref document: A1 |