WO2022206021A1 - Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support - Google Patents
Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support Download PDFInfo
- Publication number
- WO2022206021A1 WO2022206021A1 PCT/CN2021/137623 CN2021137623W WO2022206021A1 WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1 CN 2021137623 W CN2021137623 W CN 2021137623W WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- resolution
- magnetic resonance
- reconstruction
- preset
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000012545 processing Methods 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 59
- 230000006870 function Effects 0.000 claims abstract description 38
- 230000008569 process Effects 0.000 claims description 18
- 238000006731 degradation reaction Methods 0.000 claims description 17
- 230000015556 catabolic process Effects 0.000 claims description 16
- 230000009467 reduction Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 10
- 238000004904 shortening Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 15
- 238000002595 magnetic resonance imaging Methods 0.000 description 10
- 230000000747 cardiac effect Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000593 degrading effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 230000004217 heart function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Definitions
- Embodiments of the present invention relate to the technical field of medical image processing, and in particular, to an image reconstruction model generation and image reconstruction method, apparatus, device, and medium.
- Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and imaging at any level.
- cardiac magnetic resonance cine has been regarded as the imaging gold standard for assessing cardiac function.
- high-resolution image acquisition is performed clinically, a long acquisition time is often required.
- it is often difficult to obtain high-resolution cardiac magnetic resonance images in the actual clinical process due to the influence of patient tolerance and respiratory movement.
- the methods for obtaining high-resolution magnetic resonance cardiac cine images mainly include image reconstruction methods based on interpolation, image reconstruction methods based on traditional machine learning, and image reconstruction methods based on deep learning.
- the method based on the deep learning network has good processing ability for linear and nonlinear methods, and can reconstruct the magnetic resonance image with higher image quality.
- the embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, device and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, and at the same time improve the reconstructed image resolution. better image effects.
- an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a preset ratio to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image, including:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- performing image interpolation processing on the low-resolution image includes:
- the method before training the high-resolution image reconstruction model, the method further includes:
- the image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
- a residual learning method is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
- the loss function between the described label data is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
- an embodiment of the present invention further provides an image reconstruction method, the method comprising:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- an embodiment of the present invention further provides a device for generating an image reconstruction model, the device comprising:
- an image degradation module configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image
- an image interpolation module configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
- the model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model.
- the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
- the image degradation module is specifically used for:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- the image interpolation module is specifically used for:
- the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- the model training module is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to combine the residual of the input data after passing through the convolution layer with the input. After the data itself is added, a loss function between the data and the label data is calculated.
- an embodiment of the present invention further provides an image reconstruction device, the device comprising:
- an image preprocessing module configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;
- the image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset resolution enhancement factor obtained by the image reconstruction model generation method described in any one of the embodiments.
- the target image reconstruction model of the target magnetic resonance image reconstruction image is obtained.
- an embodiment of the present invention further provides a computer device, the computer device comprising:
- processors one or more processors
- memory for storing one or more programs
- the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method provided by any embodiment of the present invention.
- an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method or image provided by any embodiment of the present invention rebuild method.
- a preliminary magnetic resonance reconstruction image is acquired, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a multiple according to a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image, so as to obtain a corresponding low-resolution image;
- Perform image interpolation processing wherein the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used.
- a high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention
- FIG. 2 is a schematic structural diagram of a generative adversarial network according to Embodiment 1 of the present invention.
- FIG. 3 is a schematic diagram of a network structure of a discriminator in a generative adversarial network provided by Embodiment 1 of the present invention.
- FIG. 4 is a schematic diagram of a generator network structure in a generative adversarial network according to Embodiment 1 of the present invention.
- FIG. 5 is a schematic diagram of a network training process of an image reconstruction model according to Embodiment 1 of the present invention.
- FIG. 6 is a flowchart of an image reconstruction method according to Embodiment 2 of the present invention.
- FIG. 7 is a schematic diagram of an image reconstruction process according to Embodiment 2 of the present invention.
- FIG. 9 is a schematic structural diagram of an apparatus for generating an image reconstruction model according to Embodiment 3 of the present invention.
- FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention.
- FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
- FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
- the method may be executed by an image reconstruction model generating apparatus, which may be implemented in software and/or hardware, and integrated into an electronic device with an application development function.
- the image reconstruction model generation method includes the following steps:
- the preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by a current magnetic resonance imaging device, that is, an image with relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction).
- the preset image resolution enhancement multiple is a multiple that is expected to be able to improve the resolution of the preliminary magnetic resonance reconstruction image. Exemplarily, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution
- the enhancement factor can be obtained by dividing (512*512) by (128*128).
- the preset image resolution enhancement factor is 16.
- the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution increase multiple to obtain a corresponding low-resolution image, and the purpose is to use the preliminary magnetic resonance reconstruction image itself as the training parameter of the image reconstruction network, Instead of matching the initial MR reconstruction image with the corresponding high-resolution high-quality MR reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for image reconstruction models.
- the way to reduce the image resolution can be a down-sampling method to perform down-sampling processing on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.
- a generative adversarial network can also be used to degrade the preliminary magnetic resonance image reconstruction image to obtain a low-resolution image whose resolution is reduced by a preset image resolution enhancement factor.
- a kernel function can be determined for performing the same image degradation process on one or more preliminary MRI reconstruction images.
- the preliminary magnetic resonance reconstruction image is input into the preset generative adversarial network as shown in FIG. 2 .
- the generator G performs convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image, and obtains a low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
- the obtained image can be regarded as the original magnetic resonance reconstruction image.
- Fake version image F further, cut out image blocks with the same size and matching position as image F from the preliminary magnetic resonance reconstruction image, as real sample image T.
- the discriminator analyzes the possibility that the image F is a real image pixel by pixel.
- the goal of the generator is to make the output image F fool the discriminator as much as possible.
- the two networks, the generator and the discriminator constantly adjust the parameters in the process of confrontation with each other. Finally, when the discriminator cannot judge whether the output result F of the generator is real, the network training is completed.
- the image F and the image T are compared pixel by pixel, and every two pixels compared with each other are the probability value of the same pixel point, when the probability value of the year in the heat map is satisfied.
- the probability value of the judgment is affirmative, it can be determined that the discriminator cannot judge whether the output result F of the generator is true.
- all convolutional layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be respectively combined with the magnetic resonance image.
- the image degradation kernel function performs convolution to obtain the corresponding low-resolution image.
- the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted.
- the generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1.
- the structure of the discriminator in the preset Generative Adversarial Network may be the structure shown in FIG. 3 , which consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are: 7 ⁇ 7 and 1 ⁇ 1 respectively. , 1x1, 1x1, 1x1, 1x1 and 1x1.
- the network structure of the generator is shown in Figure 4, which consists of 6 convolution layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1.
- the value of the convolution step size of the last convolutional layer of the generator is the preset image resolution enhancement multiple, so as to achieve the effect of downsampling the input image.
- the acquired preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance cine image, which includes multiple frames of magnetic resonance images.
- any frame of the cardiac magnetic resonance cine image can be taken as the input image.
- image interpolation processing is performed on the low-resolution image, and the multiple of image interpolation is the preset image resolution enhancement multiple, so as to keep the size of the input image of the image reconstruction model consistent with the size of the output image, which can be shortened.
- Model training time optimizing the model training process.
- the interpolation method may adopt a bicubic interpolation (Bicubic) algorithm. This is because the bicubic interpolation can preserve more image details during the image enlargement process, and the enlarged image has the function of anti-aliasing. At the same time, the enlarged image has a more realistic effect than the source image.
- Biubic bicubic interpolation
- the image that has undergone image interpolation processing and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation can be used as a new high-resolution image.
- Rate image reconstruction model training sample data For example, rotate the image pair at 0, 90, 180, and 270 degrees, and then perform mirror symmetry operations in the horizontal and vertical directions, respectively, to obtain 8 sets of training data, so that the training data can be obtained. enhanced.
- the phases of the data are consistent, and no registration is required, thereby eliminating hidden dangers such as poor learning effects caused by data deviations.
- a sufficient amount of model training sample data is constructed, and then the model training process can begin.
- the image that has undergone image interpolation processing is used as the input data for model training, and the preliminary MRI reconstruction image corresponding to the low-resolution image is used as the label data.
- the to-be-obtained preliminary magnetic resonance image is used as the label data, and the low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
- a high-resolution image that increases the resolution of the preliminary magnetic resonance reconstruction image by a preset image resolution can be obtained from the output of the image reconstruction model.
- the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers.
- the convolution kernel size of the first layer is 3x3, the number of channels is f, and the second to seventh layers are
- the size of the convolution kernel is 3x3, the number of channels is 64, and the size of the convolution kernel of the last layer is 3x3, and the number of channels is f.
- f represents the number of image frames simultaneously input to the image reconstruction network.
- the f value is 1.
- the value of f is the number of image frames in the cardiac cine image.
- the number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the size of the convolution kernel is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.
- the model training process in Fig. 5 adopts a preferred deep learning method, that is, the residual learning method, which compares the residual of the image data of the input model after passing through the convolution layer with the input image data itself. After adding, calculate the loss function between and the label data.
- the loss function may not only use the L1 loss function shown in FIG. 5 , but also use loss functions such as L2 loss and perceptual loss.
- optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad can also be used to optimize the network learning process.
- conv represents the convolution kernel in the convolutional neural network
- “.mat” is the abbreviation of the file format. All the parameters of the convolutional layers of the device are convolved layer by layer to obtain the degraded kernel function of the magnetic resonance image.
- a corresponding low-resolution image is obtained by reducing the image resolution of the obtained preliminary magnetic resonance reconstruction image, and the resolution reduction factor is controllable; then, image interpolation processing is performed on the low-resolution image , where the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data,
- the high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- Fig. 6 is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention, and this embodiment can be applied to the situation of reconstructing a collected low-resolution medical image to obtain a high-resolution image.
- the method may be performed by an image reconstruction apparatus, and the apparatus may be implemented in software and/or hardware, and integrated into a computer device with an application development function.
- the image reconstruction method includes the following steps:
- the preliminary magnetic resonance reconstruction image is a low-resolution image whose resolution needs to be improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image.
- the multiple of image interpolation is the preset resolution enhancement multiple.
- the target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above-mentioned embodiment according to the requirement of increasing the preset resolution. Specifically, for the process of image reconstruction, reference may be made to the schematic diagram shown in FIG. 7 .
- Fig. 8 shows the reconstructed image obtained by performing image reconstruction on the same preliminary magnetic resonance reconstructed image by different graphic reconstruction methods, wherein (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the double The result obtained by the cubic interpolation algorithm, (c) zero-padding is the result of transforming the image back to the image domain after the image is transformed to the Fourier domain, and then inversely transforming back to the image domain after zero-filling the surrounding area, (d) SR (Super Resolution) is the image of this example The results obtained by the reconstruction method. Intuitively, from the results, the image reconstruction results obtained by the image reconstruction method in this embodiment are clearer.
- FIG. 9 is a schematic structural diagram of an image reconstruction model generating apparatus according to Embodiment 3 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
- the image reconstruction model generating apparatus includes an image degradation module 310 , an image interpolation module 320 and a model training module 330 .
- the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; the image interpolation module 320, for performing image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; model training module 330, for using the image subjected to image interpolation processing as model training
- the input data of the high-resolution image reconstruction model is trained by using the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as the label data. When the loss function of the high-resolution image reconstruction model converges to the preset value , to generate the target high-resolution image reconstruction model.
- the technical solution of this embodiment by acquiring a preliminary magnetic resonance reconstruction image, and increasing the resolution of the preliminary magnetic resonance image by a multiple, the resolution of the preliminary magnetic resonance reconstruction image is reduced, and a corresponding low-resolution image is obtained; Perform image interpolation processing on the low-resolution images, wherein the multiple of image interpolation is the preset image resolution improvement multiple; the image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance imaging corresponding to the low-resolution image is used.
- the reconstructed image is used as label data to train a high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
- a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
- the image degradation module 310 is specifically used for:
- a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
- the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
- the image interpolation module 320 is specifically used for:
- the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
- the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to compare the residual of the input data after passing through the convolutional layer with the residual. After the input data itself is added, a loss function between the input data and the label data is calculated.
- the image reconstruction model generation apparatus provided by the embodiment of the present invention can execute the image reconstruction model generation method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
- FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention. This embodiment can be applied to a situation in which a low-resolution medical image obtained by acquisition is reconstructed to obtain a high-resolution image.
- the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420 .
- the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple; the image reconstruction module 420 is used for image interpolation processing.
- the preliminary magnetic resonance reconstruction image is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor, and the target magnetic resonance image is obtained. Rebuild the image.
- the image reconstruction apparatus provided by the embodiment of the present invention can execute the image reconstruction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
- FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
- Figure 11 shows a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention.
- the computer device 12 shown in FIG. 11 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.
- the computer device 12 may be any terminal device with computing capability connected to the magnetic resonance scanning imaging device, such as an intelligent controller, a server, a mobile phone and other terminal devices.
- computer device 12 takes the form of a general-purpose computing device.
- Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .
- Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.
- these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.
- Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including both volatile and nonvolatile media, removable and non-removable media.
- System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 .
- Computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
- storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 11, commonly referred to as a "hard drive”).
- a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives.
- each drive may be connected to bus 18 through one or more data media interfaces.
- System memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.
- a program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment.
- Program modules 42 generally perform the functions and/or methods of the described embodiments of the present invention.
- Computer device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with computer device 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, computer device 12 may communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet, through network adapters 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be understood that, although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems, etc.
- the processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, to realize the steps of an image reconstruction model generation method provided by the embodiment of the present invention, and the method includes:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- the sixth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method provided by any embodiment of the present invention, including:
- the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
- the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
- the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
- the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
- the computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums.
- the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
- the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above.
- a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
- a computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
- a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
- Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
- Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional A procedural programming language, such as the "C" language or similar programming language.
- the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, through the Internet using an Internet service provider) connect).
- LAN local area network
- WAN wide area network
- Internet service provider an external computer
- modules or steps of the present invention can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices.
- they may be implemented in program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated separately into individual integrated circuit modules, or a plurality of modules of them Or the steps are made into a single integrated circuit module to realize.
- the present invention is not limited to any specific combination of hardware and software.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
Sont divulgués dans les modes de réalisation de la présente invention un procédé et un appareil de génération de modèle de reconstruction d'image, un procédé et un appareil de reconstruction d'image, ainsi qu'un dispositif et un support. Le procédé de génération de modèle de reconstruction d'image comprend : l'acquisition d'une image de résonance magnétique reconstruite préliminaire, et la réduction de la résolution de l'image de résonance magnétique reconstruite préliminaire selon un multiple d'augmentation de résolution d'image prédéfinie, de façon à obtenir une image à faible résolution ; la réalisation d'un traitement d'interpolation d'image sur l'image à faible résolution, un multiple pour l'interpolation d'image étant le multiple d'augmentation de résolution d'image prédéfinie ; et l'entraînement d'un modèle de reconstruction d'image à haute résolution en utilisant l'image ayant été soumise au traitement d'interpolation en tant que données d'entrée pour un entraînement de modèle et en utilisant l'image de résonance magnétique reconstruite préliminaire en tant que données d'étiquette, et lorsqu'une fonction de perte du modèle de reconstruction d'image à haute résolution converge vers une valeur prédéfinie, la génération d'un modèle de reconstruction d'image à haute résolution cible. Au moyen de la solution technique du présent mode de réalisation, des problèmes cachés tels qu'un mauvais effet d'apprentissage provoqué par un écart de données sont éliminés, et la résolution d'une image reconstruite peut être augmentée tout en raccourcissant le temps d'imagerie, ce qui permet d'obtenir un meilleur effet d'image.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110340196.9 | 2021-03-30 | ||
CN202110340196.9A CN115147502A (zh) | 2021-03-30 | 2021-03-30 | 图像重建模型生成及图像重建方法、装置、设备和介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022206021A1 true WO2022206021A1 (fr) | 2022-10-06 |
Family
ID=83403739
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/137623 WO2022206021A1 (fr) | 2021-03-30 | 2021-12-13 | Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN115147502A (fr) |
WO (1) | WO2022206021A1 (fr) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610194A (zh) * | 2017-08-14 | 2018-01-19 | 成都大学 | 基于多尺度融合cnn的磁共振图像超分辨率重建方法 |
CN110599401A (zh) * | 2019-08-19 | 2019-12-20 | 中国科学院电子学研究所 | 遥感图像超分辨率重建方法、处理装置及可读存储介质 |
CN111353935A (zh) * | 2020-01-03 | 2020-06-30 | 首都医科大学附属北京友谊医院 | 基于深度学习的磁共振成像优化方法及其设备 |
CN111583109A (zh) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | 基于生成对抗网络的图像超分辨率方法 |
US20200311926A1 (en) * | 2019-03-27 | 2020-10-01 | The General Hospital Corporation | Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation |
-
2021
- 2021-03-30 CN CN202110340196.9A patent/CN115147502A/zh active Pending
- 2021-12-13 WO PCT/CN2021/137623 patent/WO2022206021A1/fr active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107610194A (zh) * | 2017-08-14 | 2018-01-19 | 成都大学 | 基于多尺度融合cnn的磁共振图像超分辨率重建方法 |
US20200311926A1 (en) * | 2019-03-27 | 2020-10-01 | The General Hospital Corporation | Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation |
CN110599401A (zh) * | 2019-08-19 | 2019-12-20 | 中国科学院电子学研究所 | 遥感图像超分辨率重建方法、处理装置及可读存储介质 |
CN111353935A (zh) * | 2020-01-03 | 2020-06-30 | 首都医科大学附属北京友谊医院 | 基于深度学习的磁共振成像优化方法及其设备 |
CN111583109A (zh) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | 基于生成对抗网络的图像超分辨率方法 |
Also Published As
Publication number | Publication date |
---|---|
CN115147502A (zh) | 2022-10-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10387765B2 (en) | Image correction using a deep generative machine-learning model | |
CN110809782A (zh) | 衰减校正***和方法 | |
JP7340107B2 (ja) | 画像再構成方法、装置、デバイス、システムおよびコンピューター可読記憶媒体 | |
de Leeuw Den Bouter et al. | Deep learning-based single image super-resolution for low-field MR brain images | |
CN115953494B (zh) | 基于低剂量和超分辨率的多任务高质量ct图像重建方法 | |
CN113870104A (zh) | 超分辨率图像重建 | |
Zhu et al. | Residual dense network for medical magnetic resonance images super-resolution | |
WO2021102644A1 (fr) | Procédé et appareil d'amélioration d'image, et dispositif terminal | |
US11967066B2 (en) | Method and apparatus for processing image | |
CN105654425A (zh) | 一种应用于医学x光图像的单幅图像超分辨率重建方法 | |
WO2024051018A1 (fr) | Procédé et appareil d'amélioration d'image de paramètre tep, dispositif et support d'enregistrement | |
Zhao et al. | Gibbs-ringing artifact suppression with knowledge transfer from natural images to MR images | |
CN114494022B (zh) | 模型训练方法、超分辨率重建方法、装置、设备及介质 | |
Yang et al. | Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance Imaging--Mini Review, Comparison and Perspectives | |
Lu et al. | A novel 3D medical image super-resolution method based on densely connected network | |
Li et al. | Multi-scale residual denoising GAN model for producing super-resolution CTA images | |
CN111243052A (zh) | 图像重建方法、装置、计算机设备和存储介质 | |
US20230386067A1 (en) | Systems and methods for segmenting 3d images | |
US20230079353A1 (en) | Image correction using an invertable network | |
CN111047512A (zh) | 图像增强方法、装置及终端设备 | |
CN116630634B (zh) | 一种图像处理方法、装置、设备及存储介质 | |
WO2024021796A1 (fr) | Procédé et appareil de traitement d'image, dispositif électronique, support de stockage et produit de programme | |
WO2022206021A1 (fr) | Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support | |
Yang et al. | Quasi-supervised learning for super-resolution PET | |
WO2022127318A1 (fr) | Procédé et appareil de positionnement de balayage, support de stockage et dispositif électronique |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21934657 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21934657 Country of ref document: EP Kind code of ref document: A1 |