WO2021189383A1 - 生成高能ct图像模型的训练及生成方法、设备、存储介质 - Google Patents
生成高能ct图像模型的训练及生成方法、设备、存储介质 Download PDFInfo
- Publication number
- WO2021189383A1 WO2021189383A1 PCT/CN2020/081504 CN2020081504W WO2021189383A1 WO 2021189383 A1 WO2021189383 A1 WO 2021189383A1 CN 2020081504 W CN2020081504 W CN 2020081504W WO 2021189383 A1 WO2021189383 A1 WO 2021189383A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- energy
- image
- low
- training
- loss
- Prior art date
Links
- 238000012549 training Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 79
- 230000000306 recurrent effect Effects 0.000 claims abstract description 56
- 238000005070 sampling Methods 0.000 claims description 58
- 230000006870 function Effects 0.000 claims description 43
- 238000000605 extraction Methods 0.000 claims description 39
- 230000007246 mechanism Effects 0.000 claims description 39
- 125000004122 cyclic group Chemical group 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 230000004927 fusion Effects 0.000 claims description 13
- 230000001131 transforming effect Effects 0.000 claims description 12
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000002591 computed tomography Methods 0.000 description 334
- 239000002184 metal Substances 0.000 description 24
- 238000004364 calculation method Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 12
- 210000001519 tissue Anatomy 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 239000000284 extract Substances 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000005855 radiation Effects 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 230000006378 damage Effects 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001575 pathological effect Effects 0.000 description 2
- 238000013441 quality evaluation Methods 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000002603 single-photon emission computed tomography Methods 0.000 description 1
- 239000003826 tablet Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
Definitions
- This application relates to the field of computer tomography technology, in particular to a training method, generation method, equipment and storage medium for generating a high-energy CT image model.
- Computer tomography can provide the main anatomical and pathological information of the human body, greatly improving the level of medical diagnosis, and has been widely used in the field of clinical medicine.
- CT Computer tomography
- the use of high-dose X-rays in CT scans can obtain clear CT images.
- X-rays can cause damage to the patient’s body, and may even cause cancer.
- the main technical problem addressed by this application is to provide a training method for generating high-energy CT image models, a method for generating high-energy CT images, computer equipment and storage media, aiming to generate high-energy CT images based on low-energy CT images, so as not to increase the number of patients Improve the quality of CT image while receiving radiation dose.
- the first technical solution adopted by this application is to provide a training method for generating high-energy CT image models, including: inputting low-energy CT images for training into a recurrent network to transform the first low-energy CT images into the first High-energy CT image, and then from the first high-energy CT image to the second low-energy CT image training, to obtain the loss function;
- the loss function is used to construct the first high-energy CT image generation model.
- the second technical solution adopted in this application is to provide a method for generating high-energy CT images, including: inputting the low-energy CT images to be processed into the first high-energy CT image generation model;
- the first high-energy CT image generation model is the first high-energy CT image generation model trained by the above-mentioned training method for generating high-energy CT image models.
- the third technical solution adopted in this application is to provide a computer device including a memory and a processor
- the memory is used to store computer programs
- the processor is used to execute a computer program and implement the above-mentioned training method for generating a high-energy CT image model or realize the above-mentioned method for generating a high-energy CT image when the computer program is executed.
- the fourth technical solution adopted in this application is to provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor realizes the above-mentioned generation
- this application constructs a recurrent network to transform the first low-energy CT image into the first high-energy CT image, and then into the second low-energy CT image to obtain the loss function.
- the loss function is used to construct the first high-energy CT image generation model. Since the recurrent network continuously reduces the value of the loss function during the training process to keep it within the preset error range, the high-energy CT image generation network and the low-energy CT image generation network of the recurrent network are continuously optimized during the training process, and at the same time Due to the cyclic structure of the network, the two optimize each other and improve the performance of the two networks. Finally, the generated high-energy CT image generation network model can more accurately extract the deep feature information of the image, while retaining the details of the tissue structure around the metal object. Information to generate high-quality high-energy CT images.
- FIG. 1 is a schematic flowchart of a first embodiment of a training method for generating a high-energy CT image model according to this application;
- Figure 2 is a schematic diagram of the structure of the high-energy CT image discriminator network of the application.
- FIG. 3 is a schematic flowchart of a second embodiment of a training method for generating a high-energy CT image model according to this application;
- FIG. 4 is a schematic diagram of a flow chart of a method for generating high-energy CT images according to this application;
- FIG. 5 is a schematic flowchart of a specific application scenario of a method for generating high-energy CT images according to this application;
- FIG. 6 is a schematic diagram of the structure of the feature extraction layer in FIG. 5;
- FIG. 7 is a schematic diagram of the structure of the first attention mechanism and the second attention mechanism in FIG. 5 and FIG. 6;
- Figure 8 is a low-energy CT image to be processed obtained by dual-energy CT scanning
- Fig. 9 is a high-energy CT image corresponding to Fig. 5 obtained by dual-energy CT scanning
- FIG. 10 is a high-energy CT image generated by using the first high-energy CT image generation model of the present application.
- FIG. 11 is a schematic block diagram of the structure of the first implementation manner of a computer device according to this application.
- FIG. 12 is a schematic structural block diagram of a second implementation manner of a computer device according to this application.
- FIG. 13 is a schematic block diagram of the structure of a computer-readable storage medium of this application.
- CT computer tomography
- X-rays are used in CT imaging to obtain main anatomical and pathological information of the human body.
- the use of high-energy CT can obtain clear CT images, but the high radiation dose of high-energy CT causes serious radiation damage to the human body. Therefore, low-energy CT is tended to be used in clinical practice to reduce CT radiation, but when there is metal in human tissues
- the reconstructed images obtained by low-energy CT scans will produce metal artifacts, resulting in poor CT image quality, making it difficult to judge the structure of human tissues, and may lead to misdiagnosed results.
- this application proposes at least the following embodiments.
- FIG. 1 is a schematic flowchart of a first embodiment of a training method for generating a high-energy CT image model according to this application.
- the first embodiment of the training method for generating a high-energy CT image model of the present application includes:
- Step S10 Input the low-energy CT image for training into the recurrent network to perform training of transforming the first low-energy CT image into the first high-energy CT image, and then from the first high-energy CT image to the second low-energy CT image, to obtain the loss function.
- the recurrent network may include a pair of generator networks.
- one generator network is a high-energy CT image generator network, which is used to convert input low-energy CT images into high-energy CT images.
- the other generator network is a low-energy CT image generator network, which is used to convert input high-energy CT images into low-energy CT images.
- the high-energy CT image generator network and the low-energy CT image generator network form a cyclic network to convert the input low-energy CT image into a high-energy CT image and then into a new low-energy CT image.
- the recurrent network may also include a pair of discriminator networks.
- one of the discriminator networks is a high-energy CT image discriminator network, which corresponds to the high-energy CT image generator network, and is used to discriminate the results of the high-energy CT image generator network and conduct counter-training with the high-energy CT image generator network.
- the high-energy CT image generated by the high-energy CT image generator network is more close to the real image.
- the other discriminator network is the low-energy CT image discriminator network, which corresponds to the low-energy CT image generator network. It is used to discriminate the generation results of the low-energy CT image generator network and conduct counter-training with the low-energy CT image generator network.
- the low-energy CT image generated by the low-energy CT image generator network is more close to the real image.
- the high-energy CT image discriminator network of the present application includes multiple convolutional layers, for example, five convolutional layers.
- the high-energy CT image discriminator network extracts the image feature information input to the high-energy CT image discriminator network through multi-layer convolutional layers to discriminate the input images and measure the high-energy CT images generated by the high-energy CT image generator network and the real high-energy CT images
- the gap is fed back to the high-energy CT image generator network of the recurrent network to optimize the parameters of each layer of the high-energy CT image generator network, and finally generate high-quality high-energy CT images.
- the low-energy CT image for training may be a low-energy CT image from a dual-energy CT image group.
- the dual-energy CT image group may be a pair of low-energy CT images and high-energy CT images obtained by dual-energy CT scanning.
- the generator network and the corresponding discriminator network obtain an adversarial loss function during adversarial training, and the adversarial loss function includes a generator loss function and a discriminator loss function. Furthermore, every time the recurrent network loops, a new input graph will be regenerated.
- the low-energy CT image for training is input into the recurrent network, after one cycle, a first high-energy CT image is generated, and then a second low-energy CT image is generated from the first high-energy CT image, that is, a new low-energy CT image is generated. Therefore, the loss function can also include reconstruction loss and cyclic loss.
- the reconstruction loss is the image loss between the first high-energy CT image and the real high-energy CT image.
- the cycle loss is the image loss of the second low-energy CT image and the low-energy CT image for training.
- the processing flow of low-energy CT images for training is as follows: first, the low-energy CT images for training are input into the recurrent network, and the high-energy CT image generator network of the recurrent network inputs the low-energy CT images for training, that is, the first low-energy CT image. The image is processed and the first high-energy CT image is output. Then the low-energy CT image generator network of the cyclic network processes the input first high-energy CT image, outputs the second low-energy CT image, and completes a cyclic action. In this process, the generator network of the cyclic network and the corresponding discriminator network generate the confrontation loss, and the cyclic network generates the reconstruction loss and the cyclic loss after completing a cyclic action, thereby obtaining the loss function.
- the confrontation loss includes generator network loss and discriminator network loss.
- the calculation formula for the network loss of the high-energy CT image generator is:
- G lh is a high-energy CT image generator
- D h is a high-energy CT image discriminator
- x is an image from a low-energy CT image data set P data (x)
- y is an image from a high-energy CT image data set P data (y) Image
- 1 is the label of the real image.
- 0 is the label of the generated image.
- the high-energy CT image generator network of the recurrent network and the high-energy CT image discriminator network confront each other, and improve the performance of the two in the process of mutual confrontation, so that the first high-energy CT image generated by the high-energy CT image generator network is more realistic High-energy CT image.
- x cyc is a low-energy CT image generated cyclically.
- step S10 includes:
- Step S101 down-sampling the low-energy CT image for training to obtain a first feature map
- the low-energy CT image for training is input into the high-energy CT image generator network of the recurrent network.
- the high-energy CT image generator network includes a down-sampling layer, and the low-energy CT image for training is down-sampled in the down-sampling layer to obtain the first feature map.
- the down-sampling layer includes multiple convolutional layers.
- the first convolutional layer of the down-sampling layer first performs shallow feature extraction on the low-energy CT image for training, and extracts the low-energy CT image for training.
- the low-level simple features of the CT image are obtained from the feature map after the shallow feature extraction.
- the other convolution layers of the down-sampling layer respectively convolve the feature map after the convolution of the previous layer to further reduce the size of the feature map, and further extract the image feature information of the feature map.
- the first feature map is output through the last convolutional layer of the down-sampling layer.
- the recurrent network also includes two input layers, which are a low-energy CT image input layer and a high-energy CT image input layer.
- the low-energy CT image used for training is input into the high-energy CT image generator network of the recurrent network, for example, through the low-energy CT image generator network.
- the CT image input layer receives low-energy CT images for training.
- the input layer can also standardize the input images, and then input them into the down-sampling layer to improve the learning efficiency and performance of the recurrent network.
- Step S102 Perform multiple feature extraction on the first feature map to obtain a second feature map
- the high-energy CT image generator network of the recurrent network includes multiple feature extraction layers, and each feature extraction layer includes multiple convolutional layers to perform deep feature extraction on the first feature map. Accumulate more feature information on the basis of the map, and map the image data information of the first feature map to other distribution spaces, isolate the data distribution of metal artifacts and remove them, and obtain a second feature that eliminates metal artifacts more thoroughly. picture.
- multi-level image feature information extraction can be achieved, forming a richer description of image features, and obtaining a second feature map with better metal artifact removal effect.
- step S102 includes:
- Step S1021 Input the first feature map into multiple residual networks, and introduce an attention mechanism to the multiple residual networks;
- the feature extraction layer includes multiple residual networks, that is, every two convolutional layers of the feature extraction layer add a shortcut to form a residual block, and multiple residual blocks are connected to form a residual network.
- the residual network can highlight small changes. In the process of generating the second feature map, it may cause loss of image information. For example, for the input low-energy CT image, it may be possible in feature extraction, especially in the processing of metal artifacts. Some tissue information of the scanned object in the area near the metal object will be lost, mainly bone information. Through the residual network, the original features of the image can be re-introduced to prevent the loss of information, for example, to prevent the loss of some organizational information near the metal object of the scanned object.
- the attention mechanism can help the recurrent network assign different weights to each part of the input, pick out the more critical and important information in the image, so that the recurrent network can more accurately judge the importance of different parts of the image, and the introduction of the attention mechanism does not It will bring additional burden to the storage and calculation of the cyclic network.
- each residual network introduces two attention mechanisms, namely the first attention mechanism and the second attention mechanism.
- the first attention mechanism includes an average pooling layer and two convolutional layers
- the second attention mechanism includes two convolutional layers.
- the function of introducing the attention mechanism is to enhance the expressive ability of the cyclic network through the interdependence between the channels of the convolutional layer, and assign a weight to the channels of each convolutional layer of the residual network, and apply the weight to the corresponding image
- the features are weighted to highlight the more concerned part of the feature map and allocate more information processing resources to this part.
- the more critical image parts in the feature map are highlighted to focus on the key features of the feature map in subsequent image generation. Part of the image.
- Step S1022 Use a convolutional layer to cascade the output images after multiple feature extractions
- the output results of multiple feature extraction layers are jump-connected to the convolutional layer, and the convolutional layer is used to cascade the output images after the multiple feature extraction layers to avoid the loss of image feature information, and at the same time to reduce the dimensionality of the image data , Simplify the complexity of network calculation.
- a convolutional layer with a convolution kernel size of 3*3 is used to cascade the output results of multiple feature extraction layers, that is, the output results of multiple feature extraction layers are jump-connected to one A 3*3 convolutional layer, through which the output results of multiple feature extraction layers are merged, and dimensionality reduction processing is performed to simplify the network complexity.
- Step S1023 Introduce an attention mechanism to the new output map after cascading to obtain a second feature map.
- the attention mechanism includes the first attention mechanism and the second attention mechanism.
- the first attention mechanism includes an average pooling layer and two convolutional layers.
- the second attention mechanism includes two convolutional layers.
- the cascaded new output image first passes through the first attention mechanism, and then passes through the second attention mechanism. Its purpose is to highlight the more concerned image features in the new output image. Information to get the second feature map.
- Step S103 Combine the first feature map and the second feature map and then input the up-sampling layer for up-sampling to obtain a first high-energy CT image.
- the first feature map and the second feature map are combined and then input to the up-sampling layer for up-sampling, so as to reduce the information loss of the image in the feature extraction, and avoid the information loss during the deep feature extraction of the image. Missing results in a large deviation between the generated image and the target image.
- the feature map of the input up-sampling layer is up-sampled in the up-sampling layer.
- the feature map is up-sampled by bilinear interpolation.
- the deconvolution method can also be used. Or one or any combination of the de-pooling method to obtain the first high-energy CT image.
- step S103 includes:
- Step S1031 cascade the feature maps of the same size in the up-sampling layer and down-sampling;
- the up-sampling layer includes multiple convolutional layers, and the convolution kernel size of the multiple convolutional layers is all 3*3, so as to extract richer image feature information.
- the shallow features of the feature map obtained by the down-sampling layer can be fused with the high-level features of the feature map extracted in the up-sampling layer, thereby preserving detailed information such as the organization information of the image.
- the up-sampling layer and the down-sampling layer are cascaded Fusion is more conducive to network training.
- Step S1032 Up-sampling the new feature map after cascading
- the new cascaded feature maps are up-sampled again.
- the feature map size of the input up-sampling layer is first enlarged by two times using the bilinear interpolation method, and then the twice-enlarged feature map and the feature map of the same size in the down-sampling layer are staged. Then use bilinear interpolation to enlarge the size of the new cascaded feature map twice, and then upsample the new feature map with twice the size to have the same size as the down-sampling layer.
- Figures are cascaded.
- the cyclic network includes a cascade fusion of multiple up-sampling layers and down-sampling layers, so that the shallow features and high-level features of the feature map can be more fused to avoid the lack of important information of the feature map.
- Step S1033 Perform feature fusion on the output result of the up-sampling layer to obtain the first high-energy CT image.
- the recurrent network high-energy CT image generator network contains a fusion output layer, which is used to perform feature fusion on the output result of the up-sampling layer, and output the first high-energy CT image fused with all feature map image information.
- the fusion output layer is a convolutional layer, and the convolutional layer performs feature fusion on the feature map output by the up-sampling layer, so as to obtain the first high-energy CT image.
- step S10 also includes:
- Step S104 The network structure for converting the first high-energy CT image into the second low-energy CT image is the same as the network structure for converting the first low-energy CT image into the first high-energy CT image.
- the recurrent network includes a high-energy CT image generator network and a low-energy CT image generator network.
- the structure of the high-energy CT image generator network is the same as that of the low-energy CT generation network, but the function is opposite.
- the high-energy CT image generator network is used to convert input low-energy CT images into high-energy CT images; the low-energy CT image generator network is used to convert input high-energy CT images into low-energy CT images.
- the network structure for transforming the first high-energy CT image into the second low-energy CT image is the same as the network structure for transforming the first low-energy CT image into the first high-energy CT image. Therefore, the two generator networks of the recurrent network can be trained at the same time to optimize the network parameters and network structure, and build a high-energy CT image generation model with better image generation effect.
- Step S11 Use the loss function to construct the first high-energy CT image generation model.
- the recurrent network when the value of the loss function is outside the preset error range, the recurrent network continues to perform iterative training until the value of the loss function is within the preset error range, then the recurrent network stops training and obtains the first high energy CT image generation model. In addition, after the recurrent network is trained, the first high-energy CT image discrimination model, low-energy CT image generation model, and low-energy CT image discrimination model are also obtained.
- the purpose of iterative training of the recurrent network is to reduce the value of the loss function, that is, to reduce the confrontation loss, reconstruction loss and recurring loss, so that the images generated by the two generator networks are more close to the real image.
- the value of the loss function includes high-energy CT image generation loss, high-energy CT image discrimination loss, low-energy CT image generation loss, low-energy CT image discrimination loss, high-energy CT image reconstruction loss, and low-energy CT image cycle loss.
- step S11 includes:
- Step S110 Use Adam algorithm to iteratively optimize the parameters of each layer of the recurrent network
- the Adam algorithm is used to adjust and optimize the parameters of each layer of the recurrent network, and its effect is to make the value of the loss function within the preset error range.
- the Adam algorithm is used to adjust and optimize the parameters of the recurrent network model, including adjusting the down-sampling layer, feature extraction layer, up-sampling layer, and fusion output layer of the convolution kernel feature values and weights of the recurrent network, as well as the convolution of other parts Kernel feature values and weights, etc., until the value of the loss function is within the preset error range, the model converges, and the first high-energy CT image generation model is obtained.
- Step S111 Use the sample set to train the recurrent network to obtain the first high-energy CT image generation model.
- the sample set includes a dual-energy CT image group
- the dual-energy CT image group may be data obtained by dual-energy CT scanning, including a pair of low-energy CT images and high-energy CT images.
- the dual-energy CT image group is used to train the cyclic network
- the low-energy CT image is used for training, that is, the low-energy CT image from the dual-energy CT image group, which is the first low-energy CT image input loop
- the high-energy CT image generator network of the network distinguishes the first high-energy CT image generated from the high-energy CT image in the dual-energy CT image group, and performs the second low-energy CT image and the low-energy CT image of the dual-energy CT image group.
- the first low-energy CT image is transformed into the first high-energy CT image, and then into the second low-energy CT image.
- the loss function is obtained
- the first high-energy CT image generation model is constructed using the loss function.
- the counter loss, reconstruction loss and cyclic loss are obtained.
- the high-energy CT image generation network and the low-energy CT image generation network are continuously optimized during the training process, so that the two interact with each other. Promote and improve the network performance of the two at the same time, so that the generated high-energy CT image generation network model can extract the deep feature information of the image more accurately, while retaining the detailed information of the tissue structure around the metal object to generate high-quality high-energy CT images.
- the recurrent network of this embodiment has an "end-to-end” structure.
- the low-energy CT image for training is directly operated “end-to-end", without involving projection data calculations and other complex calculations, which simplifies the cycle The calculation process and operation process when using the network.
- FIG. 3 is a schematic flowchart of a second embodiment of a training method for generating a high-energy CT image model according to the present application.
- the second embodiment of the training method for generating a high-energy CT image model of the present application is further explained on the basis of the first embodiment of the training method for generating a high-energy CT image model of the present application. Therefore, this embodiment and the present application for generating high-energy CT images The same steps in the first implementation of the model training method will not be repeated here.
- This embodiment includes:
- Step S20 Input the low-energy CT image for training into the recurrent network to perform the training of transforming the first low-energy CT image into the first high-energy CT image, and then from the first high-energy CT image to the second low-energy CT image, and the high-energy CT image for training
- the image input recurrent network performs training to transform the second high-energy CT image into the third low-energy CT image, and then from the third low-energy CT image to the third high-energy CT image, to obtain the loss function.
- the cyclic network includes a high-energy CT image generator network and a low-energy CT image generator network.
- the two networks have the same structure and opposite functions.
- the two generator networks are trained at the same time, that is, when the low-energy CT image for training is input into the high-energy CT image generator network, the second high-energy CT image is input into the low-energy CT image generator network.
- Two networks are trained at the same time, which can further optimize the parameters of each layer of the network and make the generated image effect better.
- the low-energy CT image discriminator network has the same structure as the high-energy CT image discriminator network, and is used for confrontation training with the low-energy CT image generator network to optimize the performance of the two, and finally play a role in optimizing the overall performance of the recurrent network.
- the high-energy CT image generator network and the low-energy CT image generator network are trained at the same time, that is, the low-energy CT image for training and the second high-energy CT image are input at the same time, and the loss function is obtained after the training is completed.
- the loss function includes high-energy CT image generation loss, high-energy CT image discrimination loss, low-energy CT image generation loss, low-energy CT image discrimination loss, high-energy CT image reconstruction loss, high-energy CT image cycle loss, low-energy CT image reconstruction loss, and low-energy CT image Circulation loss.
- G hl is a low-energy CT image generator network.
- y cyc is a high-energy CT image generated cyclically.
- Step S21 Use the loss function to construct the first high-energy CT image generation model.
- the training method for generating a high-energy CT image model in this embodiment is to simultaneously train the two generator networks of the recurrent network by simultaneously inputting a low-energy CT image and a high-energy CT image.
- the two generator networks are trained at the same time, which further improves the network performance of the recurrent network.
- the high-energy CT image is input at the same time to train the recurrent network, it is more accurate to adjust and optimize the parameters of the two generator networks according to the value of the loss function obtained, so that the performance of the first high-energy CT image generation model is better.
- the removal of metal artifacts in the first high-energy CT image is more thorough, and the tissue information around the metal object is better preserved, the structure is clearer, and it is closer to the real high-energy CT image.
- the first high-energy CT image generation model training method can adjust the network structure of the recurrent network according to the processing requirements of different images, such as adjusting the number of feature extraction layers, upsampling methods, etc., or increasing Or reduce the network structure of the recurrent network, such as removing the attention mechanism or changing the number or position of the attention mechanism, and then use the corresponding sample set to train the recurrent network after the structure adjustment, so that the recurrent network after the training is generated
- the model can meet the corresponding image processing needs, for example, it is used to improve the image quality of positron emission computed tomography (PET), single photon emission computed tomography (SPECT), etc.
- PET positron emission computed tomography
- SPECT single photon emission computed tomography
- FIG. 4 is a schematic flowchart of a method for generating a high-energy CT image according to the present application.
- the method for generating high-energy CT images in this application includes:
- Step S31 Input the low-energy CT image to be processed into the first high-energy CT image generation model
- the first high-energy CT image is the first high-energy CT image generation model trained by the training method of the first high-energy CT image generation model provided by any of the foregoing embodiments of this application.
- step S31 it may further include:
- Step S30 Obtain a low-energy CT image to be processed
- the device on which the first high-energy CT image generation model runs can directly receive the low-energy CT image to be processed sent by the CT scanning device, or it can directly send the acquisition command to the CT image database server.
- the acquisition order includes patient information, examination time, and so on.
- the CT image database server After the CT image database server receives the acquisition command, it searches based on the patient information, examination time and other information to obtain the corresponding low-energy CT images to be processed, and sends the retrieved low-energy CT images to be processed to the first high-energy CT image generation model institute.
- the operating equipment enables the first high-energy CT image generation model to obtain the low-energy CT image to be processed and run the generation program.
- the low-energy CT images to be processed can also be input into the recurrent network manually or in other ways.
- FIG. 5 is a schematic flow diagram of a specific application scenario of the method for generating high-energy CT images in this application
- Fig. 6 is a schematic diagram of the structure of the feature extraction layer in Fig. 5
- Fig. 7 is Fig. 5
- the device where the first high-energy CT image generation model is located receives the low-energy CT image to be processed from the CT scanning device, and instructs the first high-energy CT image generation model to execute the generation program.
- the down-sampling layer of the first high-energy CT image generation model includes three convolutional layers, which are convolutional layer 1, convolutional layer 2, and convolutional layer 3.
- the convolutional layer 1 is a 7*7 convolutional layer, which is used to extract the shallow features of the input image
- the convolutional layer 2 and convolutional layer 3 are both 3*3 convolutional layers.
- the size of the feature map after the convolution of a layer of convolutional layer is reduced by 1/2, and the image feature information is further extracted at the same time, and the first feature map is output.
- the first feature map is input to the feature extraction layer of the first high-energy CT image generation model.
- the high-energy CT image generation model includes three feature extraction layers, and each feature extraction layer includes three residual networks and a 3*3 The convolutional layer. Among them, each residual network includes two 3*3 convolutional layers and two attention mechanisms. Further, the above two 3*3 convolutional layers are connected by skipping to form a residual block.
- the two attention mechanisms include the first attention mechanism and the second attention mechanism. Among them, the first attention mechanism includes an average pooling layer and two layers of 3*3 convolutional layers, and the second attention mechanism includes two layers of 3*3 convolutional layers. The above residual block is connected with the first attention mechanism and then connected with the second attention mechanism.
- the above two attention mechanisms can both be cascaded jump connections to avoid the loss of characteristic information such as image organization structure.
- the input image first passes through three residual networks, and then connects to a 3*3 convolutional layer to reduce the dimensionality of the image.
- three feature extraction layers are jump-connected to a 3*3 convolutional layer to reduce the dimensionality of the feature maps passing through the feature extraction layer.
- the feature map after the dimensionality reduction process passes through the first attention mechanism and then through the second attention mechanism to highlight the feature information of the focused part of the feature map to obtain the second feature map.
- the second feature map is combined with the first feature map and then input to the up-sampling layer.
- the up-sampling layer includes 4 layers of 3*3 convolutional layers, which are convolutional layer 1, convolutional layer 2, convolutional layer 3, and convolutional layer 4, respectively.
- convolutional layer 1 and convolutional layer 3 before convolving the feature map of the input upsampling layer first use bilinear interpolation to enlarge the size of the feature map by two times, that is, use bilinear interpolation to respectively After layers 1 and 3 are convolved, the feature map size before convolution is doubled, and then convolution is carried out through convolution layer 1, convolution layer 3.
- Convolution layer 2 and convolution layer 4 are convolved with the convolution in the downsampling layer. Layer 2 and convolutional layer 1 are cascaded to fuse the feature map information of the down-sampling layer.
- the output image of the up-sampling layer outputs the first high-energy CT image through the fusion output layer.
- the fusion output layer includes a layer of 7*7 convolutional layer, which is used to fuse the feature information of all feature maps and output the first high-energy CT image.
- Figure 8 is the low-energy CT image to be processed obtained by dual-energy CT scan
- Figure 9 is the high-energy CT image corresponding to Figure 5 obtained by dual-energy CT scan
- Figure 10 is the use of this Apply for the high-energy CT image generated by the first high-energy CT image generation model. It can be seen that using the high-energy CT image generated by the first high-energy CT image generation model of the present application, metal artifacts are removed more thoroughly, and the tissue structure near the metal object is retained better.
- Table 1 is the image quality evaluation parameter table of the low-energy CT image to be processed and the high-energy CT image (reconstructed high-energy CT image) generated by this application.
- Table 1 compared with the low-energy CT image to be processed,
- the first high-energy CT image obtained by the method of generating high-energy CT images of the present application has significantly improved peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and has a smaller normalized mean square error (NMSE). It is proved that the image obtained by the method of generating high-energy CT images of the present application is more close to the real high-energy CT image, thereby verifying the effectiveness of the method of generating high-energy CT images of the present application.
- PSNR peak signal-to-noise ratio
- SSIM structural similarity
- NMSE normalized mean square error
- Table 1 The image quality evaluation parameter table of the low-energy CT image to be processed and the reconstructed high-energy CT image
- the first high-energy CT image generation model includes multiple feature extraction layers and multiple jump connections, which can extract image feature information in a deep level.
- multiple feature extraction layers include multiple residual networks, which can remove metal artifacts while retaining the organization information near the metal objects, and improve the quality of processed images.
- this embodiment reduces the complexity of the network by reducing the number of convolutional layers in the residual network, and avoids the problem of image distortion after multiple feature extractions through cascading jump connections, which can remove metal artifacts while improving Good retention of the details of the structure near the metal object.
- FIG. 11 is a schematic structural block diagram of a first embodiment of a computer device according to this application.
- the computer device 40 of this embodiment may be a server or a terminal.
- the server can be an independent server or a server cluster.
- Terminals can be electronic devices such as mobile phones, tablet computers, notebook computers, desktop computers, personal digital assistants, and wearable devices.
- the computer device 40 of this embodiment includes a memory 41 and a processor 42 connected by a system bus.
- the memory 41 can store a computer program.
- the computer program includes program instructions.
- the processor 42 can execute any training method for generating a high-energy CT image model of the present application, or execute the method for generating a high-energy CT image of the present application.
- the processor 42 is used to provide calculation and control capabilities, and support the operation of the entire computer equipment.
- the processor 42 may be a central processing unit (CPU).
- the processor 42 may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components, etc.
- DSP digital signal processors
- ASIC application specific integrated circuits
- FPGA field programmable gate arrays
- the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.
- FIG. 12 is a schematic structural block diagram of a second implementation manner of a computer device according to this application.
- the computer device 50 of this embodiment includes a down-sampling module 51, a feature extraction module 52, an image dimensionality reduction module 53, a first attention module 54, a second attention module 55, an up-sampling module 56 and fusion Output module 57.
- the down-sampling module 51 is used to down-sample the input image to extract image feature information and reduce the size of the image to generate a first feature map.
- the feature extraction module 52 is configured to perform deep feature extraction on the first feature map to obtain a feature map with key information.
- the image dimensionality reduction module 53 is used to perform dimensionality reduction processing on the above-mentioned feature map to simplify calculation complexity.
- the first attention module 54 and the second attention module 55 are used to highlight important feature information of the image.
- the first feature map generates a second feature map through a feature extraction module, an image dimensionality reduction module, a first attention module, and a second attention module.
- the first feature map and the second feature map are combined and input to the up-sampling module 56.
- the up-sampling module 56 is used for up-sampling the combined feature map to enlarge the image size.
- the fusion output module 57 is used to fuse the image information of all the feature maps and output the target image.
- modules described as separate components may or may not be physically separated.
- the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of this embodiment.
- the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
- the integrated unit can be implemented in the form of hardware or software functional unit.
- modules of the computer device 50 or more functions of each module in this embodiment please refer to any of the above-mentioned training methods for generating high-energy CT image models in this application, or refer to the above-mentioned method for generating high-energy CT images in this application.
- FIG. 13 is a schematic block diagram of the structure of a computer-readable storage medium of this application.
- the computer-readable storage medium 60 provided by the present application stores program data 61.
- the program data 61 can be stored in a computer program, and the program data 61 can be executed by a processor to implement any training method for generating a high-energy CT image model in this application, or refer to the above-mentioned method for generating a high-energy CT image in this application.
- the computer-readable storage medium 60 may be an internal storage unit of any computer device in the foregoing embodiments, for example, a hard disk or memory of any computer device in the foregoing embodiment. It may also be an external storage device of any computer device in the foregoing embodiments, for example, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. of the device on the computer device.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
Description
评估参数 | PSNR | SSIM | NMSE |
低能CT图像 | 24.71 | 0.6052 | 0.1007 |
重建高能CT图像 | 27.22 | 0.6452 | 0.0552 |
Claims (11)
- 一种生成高能CT图像模型的训练方法,其特征在于,所述训练方法包括以下步骤:将训练用低能CT图像输入循环网络进行第一低能CT图像转变为第一高能CT图像、再从所述第一高能CT图像转变为第二低能CT图像的训练,得到损失函数;利用所述损失函数构建第一高能CT图像生成模型。
- 根据权利要求1所述的方法,其特征在于,所述将训练用低能CT图像输入循环网络进行第一低能CT图像转变为第一高能CT图像、再从所述第一高能CT图像转变为第二低能CT图像的训练,得到损失函数的同时,包括:将训练用高能CT图像输入所述循环网络进行第二高能CT图像转变为第三低能CT图像、再从所述第三低能CT图像转变为第三高能CT图像的训练,得到所述损失函数。
- 根据权利要求2所述的方法,其特征在于,所述将训练用低能CT图像输入循环网络进行第一低能CT图像转变为第一高能CT图像,包括:对所述训练用低能CT图像进行下采样,以得到第一特征图;对所述第一特征图进行多次特征提取,以得到第二特征图;将所述第一特征图与所述第二特征图结合后输入上采样层进行上采样,以得到所述第一高能CT图像。
- 根据权利要求3所述的方法,其特征在于,对所述第一特征图进行多次特征提取,以得到第二特征图,包括:将所述第一特征图输入多个残差网络,并对所述多个残差网络引入注意力机制;利用卷积层对所述多次特征提取后的输出图进行级联;对级联后的新的所述输出图引入注意力机制,以得到所述第二特征图。
- 根据权利要求3所述的方法,其特征在于,将所述第一特征图与所述第二特征图结合后输入上采样层进行上采样,以得到所述第一高能CT图像,包括:将所述上采样层与所述下采样中相同尺寸的特征图进行级联;对级联后的新的所述特征图进行上采样;对所述上采样层的输出结果进行特征融合,以得到所述第一高能CT图像。
- 根据权利要求2所述的方法,其特征在于,所述再从所述第一高能CT图像转变为第二低能CT图像,包括:所述第一高能CT图像转变为所述第二低能CT图像的网络结构与所述第一低能CT图像转变为第一高能CT图像的网络结构相同。
- 根据权利要求2所述的方法,其特征在于,所述利用所述损失函数构建第一高能CT图像生成模型,包括:利用所述第一高能CT图像的生成损失和所述第二低能CT图像的生成损失、所述第一高能CT图像的判别损失和所述第二低能CT图像的判别损失、所述第一高能CT图像的重建损失和所示第二低能CT图像的重建损失、所述第一低能CT图像的循环损失构建损失函数;和/或,利用所述第三低能CT图像的生成损失和所述第三高能CT图像的生成损失、所述第三低能CT图像的判别损失和所述第三高能CT图像的判别损失、所述第三低能CT图像的重建损失和所示第三高能CT图像的重建损失、所述第三高能CT图像的循环损失构建损失函数。
- 根据权利要求2所述的方法,其特征在于,所述利用所述损失函数构建第一高能CT图像生成模型,包括:利用Adam算法对所述循环网络的各层参数进行迭代优化;利用样本集对所述循环网络进行训练,以得到所述第一高能CT图像生成模型。
- 一种生成高能CT图像的方法,其特征在于,所述方法包括:将待处理低能CT图像输入第一高能CT图像生成模型;其中,所述第一高能CT图像生成模型为采用权利要求1-8任一所述的生成高能CT图像模型的训练方法训练得到的所述第一高能CT图像生成模型。
- 一种计算机设备,其特征在于,所述计算机设备包括存储器和处理器;所述存储器用于存储计算机程序;所述处理器用于执行所述计算机程序并在执行所述计算机程序时实现如权利要求1-8中任一项所述的生成高能CT图像模型的训练方法,或者如权利要求9所述的生成高能CT图像的方法。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质存 储有计算机程序,所述计算机程序被处理器执行时使所述处理器实现如权利要求1-8中任一所述的生成高能CT图像模型的训练方法,或者如权利要求9所述的生成高能CT图像的方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/081504 WO2021189383A1 (zh) | 2020-03-26 | 2020-03-26 | 生成高能ct图像模型的训练及生成方法、设备、存储介质 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/081504 WO2021189383A1 (zh) | 2020-03-26 | 2020-03-26 | 生成高能ct图像模型的训练及生成方法、设备、存储介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021189383A1 true WO2021189383A1 (zh) | 2021-09-30 |
Family
ID=77889889
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/081504 WO2021189383A1 (zh) | 2020-03-26 | 2020-03-26 | 生成高能ct图像模型的训练及生成方法、设备、存储介质 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2021189383A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117274418A (zh) * | 2023-10-08 | 2023-12-22 | 北京长木谷医疗科技股份有限公司 | 基于正侧位x线图像的ct图像生成方法、装置及设备 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090208084A1 (en) * | 2008-02-15 | 2009-08-20 | Xin Liu | System and method for quantitative imaging of chemical composition to decompose more than two materials |
CN109299342A (zh) * | 2018-11-30 | 2019-02-01 | 武汉大学 | 一种基于循环生成式对抗网络的跨模态检索方法 |
WO2019027641A1 (en) * | 2017-08-01 | 2019-02-07 | Varex Imaging Corporation | DOUBLE LAYER DETECTOR FOR MOVING MOUSE TISSUE MOTION |
CN109949215A (zh) * | 2019-03-29 | 2019-06-28 | 浙江明峰智能医疗科技有限公司 | 一种低剂量ct图像模拟方法 |
CN110559009A (zh) * | 2019-09-04 | 2019-12-13 | 中山大学 | 基于gan的多模态低剂量ct转换高剂量ct的方法、***及介质 |
CN110728727A (zh) * | 2019-09-03 | 2020-01-24 | 天津大学 | 一种低剂量能谱ct投影数据的恢复方法 |
-
2020
- 2020-03-26 WO PCT/CN2020/081504 patent/WO2021189383A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090208084A1 (en) * | 2008-02-15 | 2009-08-20 | Xin Liu | System and method for quantitative imaging of chemical composition to decompose more than two materials |
WO2019027641A1 (en) * | 2017-08-01 | 2019-02-07 | Varex Imaging Corporation | DOUBLE LAYER DETECTOR FOR MOVING MOUSE TISSUE MOTION |
CN109299342A (zh) * | 2018-11-30 | 2019-02-01 | 武汉大学 | 一种基于循环生成式对抗网络的跨模态检索方法 |
CN109949215A (zh) * | 2019-03-29 | 2019-06-28 | 浙江明峰智能医疗科技有限公司 | 一种低剂量ct图像模拟方法 |
CN110728727A (zh) * | 2019-09-03 | 2020-01-24 | 天津大学 | 一种低剂量能谱ct投影数据的恢复方法 |
CN110559009A (zh) * | 2019-09-04 | 2019-12-13 | 中山大学 | 基于gan的多模态低剂量ct转换高剂量ct的方法、***及介质 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117274418A (zh) * | 2023-10-08 | 2023-12-22 | 北京长木谷医疗科技股份有限公司 | 基于正侧位x线图像的ct图像生成方法、装置及设备 |
CN117274418B (zh) * | 2023-10-08 | 2024-04-02 | 北京长木谷医疗科技股份有限公司 | 基于正侧位x线图像的ct图像生成方法、装置及设备 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11854160B2 (en) | CT super-resolution GAN constrained by the identical, residual and cycle learning ensemble (GAN-circle) | |
US10970829B2 (en) | Synthesizing and segmenting cross-domain medical images | |
Bera et al. | Noise conscious training of non local neural network powered by self attentive spectral normalized Markovian patch GAN for low dose CT denoising | |
Tang et al. | Unpaired low‐dose CT denoising network based on cycle‐consistent generative adversarial network with prior image information | |
JP2023025056A (ja) | ディープ畳み込みニューラルネットワークを使用した医用画像化のための線量低減 | |
CN110648337A (zh) | 髋关节分割方法、装置、电子设备和存储介质 | |
EP3980972A1 (en) | Sct image generation using cyclegan with deformable layers | |
CN111368849B (zh) | 图像处理方法、装置、电子设备及存储介质 | |
CN111489406B (zh) | 生成高能ct图像模型的训练及生成方法、设备、存储介质 | |
Zhou et al. | DuDoUFNet: dual-domain under-to-fully-complete progressive restoration network for simultaneous metal artifact reduction and low-dose CT reconstruction | |
Bai et al. | Probabilistic self‐learning framework for low‐dose CT denoising | |
CN110223255A (zh) | 一种用于低剂量ct图像去噪的浅层残差编解码递归网络 | |
WO2023142781A1 (zh) | 图像三维重建方法、装置、电子设备及存储介质 | |
CN114863225A (zh) | 图像处理模型训练方法、生成方法、装置、设备及介质 | |
Kim et al. | Convolutional neural network–based metal and streak artifacts reduction in dental CT images with sparse‐view sampling scheme | |
WO2021189383A1 (zh) | 生成高能ct图像模型的训练及生成方法、设备、存储介质 | |
CN117813055A (zh) | 用于从快速spect扫描和ct图像合成spect图像的多模态和多尺度特征聚合 | |
Xie et al. | Artificial intelligence–based computed tomography processing framework for surgical telementoring of congenital heart disease | |
CN114511497A (zh) | 应用于锥束ct稀疏采样的成像方法及装置 | |
Liu et al. | A modality conversion approach to MV‐DRs and KV‐DRRs registration using information bottlenecked conditional generative adversarial network | |
Shang et al. | Short‐Axis PET Image Quality Improvement by Attention CycleGAN Using Total‐Body PET | |
Kim et al. | CNN-based CT denoising with an accurate image domain noise insertion technique | |
WO2021159236A1 (zh) | 基于非衰减校正pet图像生成合成pet-ct图像的方法和*** | |
CN111445406B (zh) | 一种低剂量ct图片质量改善方法、***及设备 | |
Gu et al. | Contrast-enhanced to noncontrast CT transformation via an adjacency content-transfer-based deep subtraction residual neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20926856 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20926856 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20926856 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10.07.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20926856 Country of ref document: EP Kind code of ref document: A1 |