CN117635479A - Magnetic particle image denoising method, system and equipment based on double-stage diffusion model - Google Patents
Magnetic particle image denoising method, system and equipment based on double-stage diffusion model Download PDFInfo
- Publication number
- CN117635479A CN117635479A CN202410102203.5A CN202410102203A CN117635479A CN 117635479 A CN117635479 A CN 117635479A CN 202410102203 A CN202410102203 A CN 202410102203A CN 117635479 A CN117635479 A CN 117635479A
- Authority
- CN
- China
- Prior art keywords
- image
- denoising
- module
- mpi
- noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009792 diffusion process Methods 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 56
- 239000006249 magnetic particle Substances 0.000 title claims abstract description 38
- 238000007781 pre-processing Methods 0.000 claims abstract description 32
- 238000004088 simulation Methods 0.000 claims description 44
- 238000012545 processing Methods 0.000 claims description 21
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 20
- 230000004927 fusion Effects 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 13
- 230000009191 jumping Effects 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims 1
- 238000003384 imaging method Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000002792 vascular Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of magnetic particle imaging, in particular relates to a magnetic particle image denoising method, a system and equipment based on a dual-stage diffusion model, and aims to solve the problem that the existing MPI image denoising method cannot remove streak artifacts and retain detail information. The method comprises the following steps: acquiring a two-dimensional MPI image to be denoised as an input image; extracting a target region of interest of an input image and preprocessing to obtain a preprocessed target image; inputting a trained MPI image denoising model into the preprocessing target image to obtain a clean noiseless MPI image; the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module. The invention removes streak artifact noise in the magnetic particle image, simultaneously retains details of effective information in the image, and realizes accurate denoising.
Description
Technical Field
The invention belongs to the field of magnetic particle imaging, and particularly relates to a magnetic particle image denoising method, system and equipment based on a dual-stage diffusion model.
Background
Magnetic Particle Imaging (MPI) is a novel medical imaging technology and is widely applied to preclinical researches such as cell tracing, vascular imaging, tumor positioning and the like. However, when the imaging target signal is low, the MPI image quality may significantly decrease. A large number of streak artifacts appear in the image, which typically have relatively prominent signal strengths, which can severely interfere with the localization and diagnosis of useful information. In addition, streak artifact noise in low signal images is non-uniform and difficult to model. How to improve the denoising algorithm, and the detail information is not lost while removing streak artifacts is a current problem to be solved urgently.
The denoising method based on medical image post-processing has wide development prospect due to development of big data and computing power. However, the existing denoising method based on deep learning is difficult to accurately identify noise background and effective signal when performing denoising task, so that the denoised image has the problems of blurring, smoothing, distortion and the like. Aiming at the problem, the invention provides a magnetic particle image denoising method based on a double-stage diffusion model, which is used for precisely removing streak artifact noise in an MPI image and keeping details of effective information in the image.
Disclosure of Invention
In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem that the existing MPI image denoising method cannot remove streak artifacts and retain detail information, the first aspect of the present invention provides a magnetic particle image denoising method based on a dual-stage diffusion model, which includes:
s100, acquiring a two-dimensional MPI image to be denoised as an input image;
s200, extracting a target region of interest of the input image and preprocessing to obtain a preprocessed target image;
s300, inputting a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; forward diffusion and reverse denoising are carried out on the stripe noise prior feature by combining the noise condition, so that the sampled stripe noise prior feature is obtained;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
In some preferred embodiments, the implicit spatial coding module comprises 3 sequentially connected3 convolutional layer, residual block, 3->A convolution layer, an average pooling layer and three linear connection layers;
the residual block comprises a concatenation 33 convolution layer, lrerlu activation layer and 3 +.>3, a convolution layer;
the input of the residual block and the second of the residual blocks3 pieces ofAnd 3, connecting output residual errors of the convolution layers as the output of the residual error block.
In some preferred embodiments, the multi-scale denoising network is an encoder-decoder structure;
the encoder part comprises three feature processing modules on the feature scale; the characteristic processing modules on three characteristic scales of the encoder part are respectively used as a first module, a second module and a third module; the first module and the second module comprise a priori fusion layer and a characteristic embedding block which are sequentially connected; the third module comprises a priori fusion layer, a convolution layer, 6 attention blocks and a deconvolution lamination layer which are connected in sequence; the feature embedding block comprises a33 convolution layers and three residual blocks;
the decoder part comprises two feature processing modules on the feature scale; the feature processing modules on two feature scales of the decoder part are respectively used as a fourth module and a fifth module;
the fourth module comprises a priori fusion layer, a convolution layer, 3 attention blocks and a deconvolution layer which are connected in sequence; the fifth module comprises a priori fusion layer, a convolution layer, two residual blocks and the convolution layer which are sequentially connected;
the output of the second module is connected with the output residual error of the third module and is used as the input of the fourth module; the output of the first module is connected with the output residual error of the fourth module and is used as the input of the fifth module;
the prior fusion layer processes the input characteristics through a normalization layer and a linear layer to be used as Q; processing the stripe noise priori features through normalization layers and linear layers with different parameters respectively to obtain K, V; calculating cross-attention in combination with the K, the V and the Q; adding the cross attention to the features input by the prior fusion layer after being processed by a linear layer to be used as the output of the prior fusion layer;
the attention block divides the input features into vertical component features after normalization layer processingX V And horizontal component characteristicsX H And generating multi-head Q, K, V by linear projection respectively to obtain multi-head characteristics in vertical directionT V And horizontal multi-headed featuresT H The method comprises the steps of carrying out a first treatment on the surface of the The saidT V The saidT H Performing residual connection with the input addition of the attention block after splicing in the channel dimension to obtain a first residual characteristic; and processing the first residual features sequentially through a normalization layer and a multi-layer perceptron, then adding the processed first residual features with the first residual features again to carry out residual connection, and taking the features obtained after the residual connection again as the output of the attention block.
In some preferred embodiments, cross-attention is calculated in combination with the K, V and Q by:
；
where T represents the transpose and C represents the number of channels.
In some preferred embodiments, the MPI image denoising model is trained by the following steps:
a100, generating a matched noisy simulation MPI image and a noiseless simulation MPI image, and constructing a simulation data set;
a200, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into the hidden space encoder to generate stripe noise priori features; downsampling the stripe noise prior feature by different scales to obtain a multi-scale stripe noise prior feature;
inputting the multi-scale stripe noise prior characteristic and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image serving as a first image;
calculating a loss value through a pre-constructed loss function at a first stage based on the first image and the corresponding noiseless simulation MPI image, and updating network parameters of the hidden space encoder and the multi-scale denoising module;
a300, judging whether the set cycle times or the set precision are reached, if not, jumping to A200, otherwise, completing the first-stage training, and jumping to A400;
a400, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into a first hidden space encoder to generate stripe noise priori features; inputting the noisy simulated MPI image into a second hidden space encoder to obtain a noise condition; the first hidden space encoder and the second hidden space encoder have the same structure and share parameters; the first hidden space encoder is a hidden space encoder in the MPI image denoising model;
forward diffusion is carried out on the stripe noise priori features for a set time; based on the forward diffusion characteristic, carrying out reverse denoising for a set time by combining the noise condition to obtain a stripe noise priori characteristic of sampling;
downsampling the sampled stripe noise prior characteristics in different scales to obtain multi-scale sampled stripe noise prior characteristics; inputting the stripe noise prior characteristics of the multi-scale sampling and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image as a second image;
calculating a loss value through a pre-constructed loss function of a second stage based on the second image and the corresponding noiseless simulation MPI image, and updating network parameters of the MPI image denoising model;
a500, judging whether the set cycle times or the set precision are reached, if not, jumping to A400, otherwise, completing the second-stage training, and jumping to A600;
a600, saving the optimal parameters of the trained hidden space encoder and the multi-scale denoising network, and combining noise condition extraction and inverse denoising to obtain the trained MPI image denoising model.
In some preferred embodiments, forward diffusion and inverse denoising are performed on the stripe noise prior feature to obtain a sampled stripe noise prior feature, and the method comprises the following steps:
forward diffusion:
；
wherein y is 0 Let y be the stripe noise a priori characteristic obtained by the hidden space encoder, y t For the prior characteristic of stripe noise generated at the moment T when forward diffusion is carried out, t=1, 2, …, T and I are identity matrixes,is a super parameter for controlling noise variation, < ->Representing a gaussian distribution->I.e. y t ，t=1, 2,…, T；
Reverse denoising:
；
wherein,，/>，/>representing the spread noise-canceling network, c representing the noise condition generated by the noisy image after it has passed through the hidden space encoder,/for the noise-canceling network>Representing variance->。
In some preferred embodiments, the first stage loss function is:
wherein,representing pixel loss, +.>MPI image representing clean noiseless, +.>A noise-free simulated MPI image is represented.
In some preferred embodiments, the loss function of the second stage is:
wherein,indicating loss of association->Indicating diffusion loss->Stripe noise a priori features representing samples, +.>Representing the streak noise a priori features.
In a second aspect of the present invention, a magnetic particle image denoising system based on a dual-stage diffusion model is provided, the system comprising:
the image acquisition module is configured to acquire a two-dimensional MPI image to be denoised as an input image;
the preprocessing module is configured to extract a target region of interest of the input image and preprocess the target region of interest to obtain a preprocessed target image;
the image denoising module is configured to input a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
In a third aspect of the present invention, a magnetic particle image denoising apparatus based on a dual-stage diffusion model is provided, comprising; at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the magnetic particle image denoising method based on the dual-stage diffusion model.
The invention has the beneficial effects that:
the image denoising method of the invention uses the hidden encoder to generate noise priori characteristics, and guides the denoising network to denoise, so that the network can effectively identify noise background, remove streak artifact noise in the magnetic particle image, and simultaneously retain details of effective information in the image, thereby realizing accurate denoising;
according to the image denoising method, the diffusion model is adopted to sample on the noise priori characteristics, so that the image denoising efficiency is effectively improved, and the network has greater generalization capability to cope with the noise of a real environment;
the image denoising method can denoise the acquired real image to be identified, and the real image denoising task has better quantization index and visual effect.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings.
FIG. 1 is a flow chart of a magnetic particle image denoising method based on a dual-stage diffusion model according to an embodiment of the present invention;
FIG. 2 is a schematic illustration of a first stage training process of a two-stage diffusion model according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a process of a second stage training of a two-stage diffusion model according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of the structure of a hidden encoder according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the architecture of a multi-scale denoising network according to one embodiment of the present invention;
FIG. 6 is a schematic diagram of the structure of an a priori fusion layer of an embodiment of the invention;
fig. 7 is a schematic view of the structure of an attention block according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
The magnetic particle image denoising method based on the double-stage diffusion model, as shown in fig. 1, comprises the following steps:
s100, acquiring a two-dimensional MPI image to be denoised as an input image;
s200, extracting a target region of interest of the input image and preprocessing to obtain a preprocessed target image;
s300, inputting a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
In order to more clearly describe the magnetic particle image denoising method based on the dual-stage diffusion model, each step in one embodiment of the method of the present invention is described in detail below with reference to the accompanying drawings.
In the following embodiments, a training method of an MPI image denoising model is described first, and then a procedure of acquiring a clean noiseless MPI image by a magnetic particle image denoising method based on a dual-stage diffusion model is described.
1. Training method of MPI image denoising model as shown in FIG. 2 and FIG. 3
A100, generating a matched noisy simulation MPI image and a noiseless simulation MPI image, and constructing a simulation data set;
in the present embodiment, the noise-free simulated MPI imageAnd generating through an X space reconstruction simulation platform. The X space reconstruction simulation platform is used for simulating magnetic particle signal generation and image reconstruction in a real situation. The simulated image is a black-and-white binary image, white represents the position of the magnetic particle, and black represents the background.
Noisy simulated MPI imageThe method is obtained by collecting the real MPI equipment, performing cutting, rotation, contrast adjustment and other processing, and then overlapping with the noiseless simulation MPI image.
A200, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into the hidden space encoder to generate stripe noise priori features; downsampling the stripe noise prior feature by different scales to obtain a multi-scale stripe noise prior feature;
inputting the multi-scale stripe noise prior characteristic and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image serving as a first image;
calculating a loss value through a pre-constructed loss function at a first stage based on the first image and the corresponding noiseless simulation MPI image, and updating network parameters of the hidden space encoder and the multi-scale denoising module;
in the present embodiment, it willAnd->Splicing in the channel dimension to obtain a spliced imageAnd then inputting the stripe noise prior characteristic y into a hidden space encoder.
The hidden space coding module comprises 3 which are connected in sequence3 convolutional layer, residual block, 3->A convolution layer, an average pooling layer and three linear connection layers; the residual block comprises 3 +.>3 convolution layer, lrerlu activation layer and 3 +.>3, a convolution layer; input of residual block and second 3 +_ in residual block>The outputs of the 3 convolutional layers are connected as residual blocks, as shown in fig. 4.
Then downsampling y to obtain multi-scale stripe noise priori features {y 1 ,y 2 ,y 3 (in the present invention, y) 1 =y. For y 1 Double downsampling in the first dimensiony 2 For a pair ofy 2 Double downsampling in the first dimension to give y 3 ) Will beAnd three-scale multi-scale stripe noise prior characteristics are input into a multi-scale denoising network, and the three-scale multi-scale stripe noise prior characteristics are input into a multi-scale denoising networkTraining is performed as a tag.
In the first stage, the hidden space encoder and the multi-scale denoising network are jointly trained. During training, the noisy simulation MPI image is used as input, the noiseless simulation MPI image is used as a label, and two modules are trained simultaneously. And saving optimal parameters of the hidden space encoder and the multi-scale denoising network. The optimal parameters include: linear weights of the hidden space encoder, learning rate in the multi-scale denoising module, batch size, and the like.
The loss function of the first stage is:
（1）
wherein,representing pixel loss, +.>MPI image representing clean noiseless, +.>A noise-free simulated MPI image is represented.
In addition, the multi-scale denoising network is an encoder-decoder structure, as shown in fig. 5;
the encoder part comprises three feature processing modules on the feature scale; the characteristic processing modules on three characteristic scales of the encoder part are respectively used as a first module, a second module and a third module; the first module and the second module comprise a priori fusion layer and a characteristic embedding block which are sequentially connected; the third module comprises a priori fusion layer, a convolution layer, 6 attention blocks and a deconvolution layer which are connected in sequence;
the feature embedding block comprises a33 convolution layers and three residual blocks (namely, a convolution layer, a first residual block, a second residual block and a third residual block in sequence);
the decoder part comprises two feature processing modules on the feature scale; the feature processing modules on two feature scales of the decoder part are respectively used as a fourth module and a fifth module;
the fourth module comprises a priori fusion layer, a convolution layer, 3 attention blocks and a deconvolution layer which are connected in sequence; the fifth module comprises a priori fusion layer, a convolution layer, two residual blocks and the convolution layer which are sequentially connected;
the output of the second module is connected with the output residual error of the third module and is used as the input of the fourth module; the output of the first module is connected with the output residual error of the fourth module and is used as the input of the fifth module;
the prior fusion layer processes the input characteristics through a normalization layer and a linear layer to be used as Q (query); the stripe noise priori features are respectively processed by a normalization layer and a linear layer with different parameters and then used as K (key) and V (value); combining K, V and Q, calculating cross-attention;
combining K, V and Q, cross-attention was calculated by:
（2）
wherein T represents a transpose and C represents the number of channels, as shown in fig. 6;
the attention block divides the input features into vertical component features after normalization layer processingX V And horizontal component characteristicsX H Dividing the two characteristic components into non-overlapping tokens according to columns and rows respectivelyAnd->. Multi-heads Q, K and V (vertical direction +.>,/>And->The horizontal direction is +.>,/>And->). The multi-head attention calculation is according to formula (2), where C is +.>And->. Where D is the product of the second dimension (width) and the third dimension (number of channels) of the feature and m is the number of heads. After attention calculation, the calculated attention and the calculated attention are combinedX V、 X H Multiplying to obtain multi-head characteristic T in vertical and horizontal directions V And T H The method comprises the steps of carrying out a first treatment on the surface of the The saidT V The saidT H Performing residual connection with the input addition of the attention block after splicing in the channel dimension to obtain a first residual characteristic; and processing the first residual features sequentially through a normalization layer and a multi-layer perceptron, then adding the processed first residual features with the first residual features again to perform residual connection, and taking the features obtained after the residual connection again as the output of the attention block, as shown in fig. 7.
A300, judging whether the set cycle times or the set precision are reached, if not, jumping to A200, otherwise, completing the first-stage training, and jumping to A400;
in this embodiment, the hidden space encoder and the multi-scale denoising network are trained in cycles until a set number of cycles or a set accuracy is reached.
A400, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into a first hidden space encoder to generate stripe noise priori features; inputting the noisy simulated MPI image into a second hidden space encoder to obtain a noise condition; the first hidden space encoder and the second hidden space encoder have the same structure and share parameters; the first hidden space encoder is a hidden space encoder in the MPI image denoising model;
forward diffusion is carried out on the stripe noise priori features for a set time; based on the forward diffusion characteristic, carrying out reverse denoising for a set time by combining the noise condition to obtain a stripe noise priori characteristic of sampling;
downsampling the sampled stripe noise prior characteristics in different scales to obtain multi-scale sampled stripe noise prior characteristics; inputting the stripe noise prior characteristics of the multi-scale sampling and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image as a second image;
calculating a loss value through a pre-constructed loss function of a second stage based on the second image and the corresponding noiseless simulation MPI image, and updating network parameters of the MPI image denoising model;
in this embodiment, the first implicit space encoder (i.e., the implicit space encoder in the first stage training process) and the second implicit space encoder have the same structure and share parameters. And obtaining stripe noise priori characteristics and noise conditions through the first hidden space coder and the second hidden space coder.
Then forward diffusion is carried out for the stripe noise prior characteristic for a set time; based on the forward diffusion characteristic, carrying out reverse denoising for a set time by combining with noise conditions to obtain the prior characteristic of the sampled stripe noise。
Forward diffusion:
（3）
wherein y is 0 The =y is the implicit space encoder resultStripe noise a priori features, y t For the prior characteristic of stripe noise generated at the moment T when forward diffusion is carried out, t=1, 2, …, T and I are identity matrixes,is a super parameter for controlling noise variation, < ->Representing a gaussian distribution->I.e. y t ，t=1, 2,…, T；
Reverse denoising:
（4）
wherein,，/>，/>representing a spread-spectrum denoising network (the invention is preferably a U-net network), c represents the noise condition generated by the noisy image after passing through the hidden space encoder, < +.>Representing variance->。
Will beDownsampling to obtain multi-scale sampled stripe noise prior feature ++>Will->And the three-scale sampled stripe noise prior characteristic is input into a multi-scale denoising network, and is subjected to +.>Training is performed as a label, and the second stage training is completed.
In the second stage training process, the loss function is:
（5）
（6）
wherein,indicating loss of association->Indicating diffusion loss->Stripe noise a priori features representing samples, +.>Representing the streak noise a priori features.
A500, judging whether the set cycle times or the set precision are reached, if not, jumping to A400, otherwise, completing the second-stage training, and jumping to A600;
in this embodiment, the hidden space encoder, the conditional diffusion module, and the multi-scale denoising network are trained in cycles until a set number of cycles or a set accuracy is reached.
A600, saving the optimal parameters of the trained hidden space encoder and the multi-scale denoising network, and combining noise condition extraction and inverse denoising to obtain the trained MPI image denoising model.
2. Magnetic particle image denoising method based on double-stage diffusion model
S100, acquiring a two-dimensional MPI image to be denoised as an input image;
in this embodiment, a two-dimensional MPI image (preferably a low-density, low-signal image) to be denoised is acquired first.
S200, extracting a target region of interest of the input image and preprocessing to obtain a preprocessed target image;
in this embodiment, preprocessing is performed on the region of interest of the two-dimensional MPI image, including gray-scale conversion, image cropping.
S300, inputting the trained MPI image denoising model into the preprocessing target image to obtain a clean noiseless MPI image.
In this embodiment, the preprocessing target image is input into the trained MPI image denoising model, so as to obtain a clean noiseless MPI image.
In the embodiment, the hidden space coding module performs feature extraction on the preprocessing target image to obtain stripe noise priori features; the condition diffusion module is used for extracting noise conditions (namely, inputting an input image into the hidden space coding module to obtain the noise conditions); reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature; and the multi-scale denoising network is used for denoising the preprocessing target image by combining the sampled stripe noise prior features of a plurality of scales (namely, sampling (preferably downsampling) the sampled stripe noise prior features to obtain the sampled stripe noise prior features of a plurality of scales) so as to obtain a clean noiseless MPI image.
The magnetic particle image denoising system based on a dual-stage diffusion model according to a second embodiment of the present invention comprises:
the image acquisition module is configured to acquire a two-dimensional MPI image to be denoised as an input image;
the preprocessing module is configured to extract a target region of interest of the input image and preprocess the target region of interest to obtain a preprocessed target image;
the image denoising module is configured to input a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working processes and related descriptions of the above-described system may refer to corresponding processes in the foregoing method embodiments, which are not repeated herein.
It should be noted that, in the magnetic particle image denoising system based on the dual-stage diffusion model provided in the above embodiment, only the division of the above functional modules is illustrated, in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are decomposed or combined again, for example, the modules in the embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
The magnetic particle image denoising equipment based on a double-stage diffusion model of the third embodiment of the invention comprises; at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the magnetic particle image denoising method based on the dual-stage diffusion model.
A computer-readable storage medium of a fourth embodiment of the present invention stores computer instructions for execution by the computer to implement the above-described magnetic particle image denoising method based on a dual-stage diffusion model.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the magnetic particle image denoising apparatus and the computer readable storage medium based on the dual-stage diffusion model described above and the related description may refer to the corresponding process in the foregoing method example, and will not be repeated here.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the program(s) corresponding to the software modules, method steps, may be embodied in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not intended to be limiting.
The terms "first," "second," "third," and the like, are used for distinguishing between similar objects and not for describing a particular sequential or chronological order.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will be within the scope of the present invention.
Claims (10)
1. The magnetic particle image denoising method based on the double-stage diffusion model is characterized by comprising the following steps of:
s100, acquiring a two-dimensional MPI image to be denoised as an input image;
s200, extracting a target region of interest of the input image and preprocessing to obtain a preprocessed target image;
s300, inputting a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
2. The method for denoising a magnetic particle image based on a dual-stage diffusion model according to claim 1, wherein the hidden space coding module comprises 3 sequentially connected3 convolutional layer, residual block, 3->A convolution layer, an average pooling layer and three linear connection layers;
the residual block comprises a concatenation 33 convolution layer, lrerlu activation layer and 3 +.>3, a convolution layer;
the input of the residual block and the second 3 of the residual blocksAnd 3, connecting output residual errors of the convolution layers as the output of the residual error block.
3. The method for denoising a magnetic particle image based on a two-stage diffusion model according to claim 2, wherein the multi-scale denoising network is an encoder-decoder structure;
the encoder part comprises three feature processing modules on the feature scale; the characteristic processing modules on three characteristic scales of the encoder part are respectively used as a first module, a second module and a third module; the first module and the second module comprise a priori fusion layer and a characteristic embedding block which are sequentially connected; the third module comprises a priori fusion layer, a convolution layer and 6 attention blocks which are sequentially connected; the feature embedding block comprises a33 convolution layers and three residual blocks;
the decoder part comprises two feature processing modules on the feature scale; the feature processing modules on two feature scales of the decoder part are respectively used as a fourth module and a fifth module;
the fourth module comprises a priori fusion layer, a convolution layer and 3 attention blocks which are sequentially connected; the fifth module comprises a priori fusion layer, a convolution layer, two residual blocks and the convolution layer which are sequentially connected;
the output of the second module is connected with the output residual error of the third module and is used as the input of the fourth module; the output of the first module is connected with the output residual error of the fourth module and is used as the input of the fifth module;
the prior fusion layer processes the input characteristics through a normalization layer and a linear layer to be used as Q; processing the stripe noise priori features through normalization layers and linear layers with different parameters respectively to obtain K, V; calculating cross-attention in combination with the K, the V and the Q; adding the cross attention to the features input by the prior fusion layer after being processed by a linear layer to be used as the output of the prior fusion layer;
the attention block divides the input features into vertical component features after normalization layer processingX V And horizontal component characteristicsX H And generating multi-head Q, K, V by linear projection respectively to obtain multi-head characteristics in vertical directionT V And horizontal multi-headed featuresT H The method comprises the steps of carrying out a first treatment on the surface of the The saidT V The saidT H Performing residual connection with the input addition of the attention block after splicing in the channel dimension to obtain a first residual characteristic; and processing the first residual features sequentially through a normalization layer and a multi-layer perceptron, then adding the processed first residual features with the first residual features again to carry out residual connection, and taking the features obtained after the residual connection again as the output of the attention block.
4. A magnetic particle image denoising method based on a two-stage diffusion model according to claim 3, wherein the cross attention is calculated by combining the K, V and Q by:
；
where T represents the transpose and C represents the number of channels.
5. A magnetic particle image denoising method based on a dual-stage diffusion model according to claim 3, wherein the MPI image denoising model is trained by the following steps:
a100, generating a matched noisy simulation MPI image and a noiseless simulation MPI image, and constructing a simulation data set;
a200, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into the hidden space encoder to generate stripe noise priori features; downsampling the stripe noise prior feature by different scales to obtain a multi-scale stripe noise prior feature;
inputting the multi-scale stripe noise prior characteristic and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image serving as a first image;
calculating a loss value through a pre-constructed loss function at a first stage based on the first image and the corresponding noiseless simulation MPI image, and updating network parameters of the hidden space encoder and the multi-scale denoising module;
a300, judging whether the set cycle times or the set precision are reached, if not, jumping to A200, otherwise, completing the first-stage training, and jumping to A400;
a400, splicing the paired noiseless simulation MPI image and the noisy simulation MPI image in the simulation data set in the channel dimension to obtain a spliced image; inputting the spliced image into a first hidden space encoder to generate stripe noise priori features; inputting the noisy simulated MPI image into a second hidden space encoder to obtain a noise condition; the first hidden space encoder and the second hidden space encoder have the same structure and share parameters; the first hidden space encoder is a hidden space encoder in the MPI image denoising model;
forward diffusion is carried out on the stripe noise priori features for a set time; based on the forward diffusion characteristic, carrying out reverse denoising for a set time by combining the noise condition to obtain a stripe noise priori characteristic of sampling;
downsampling the sampled stripe noise prior characteristics in different scales to obtain multi-scale sampled stripe noise prior characteristics; inputting the stripe noise prior characteristics of the multi-scale sampling and the noisy simulation MPI image into the multi-scale denoising network to obtain a clean noiseless MPI image as a second image;
calculating a loss value through a pre-constructed loss function of a second stage based on the second image and the corresponding noiseless simulation MPI image, and updating network parameters of the MPI image denoising model;
a500, judging whether the set cycle times or the set precision are reached, if not, jumping to A400, otherwise, completing the second-stage training, and jumping to A600;
a600, saving the optimal parameters of the trained hidden space encoder and the multi-scale denoising network, and combining noise condition extraction and inverse denoising to obtain the trained MPI image denoising model.
6. The method for denoising a magnetic particle image based on a dual-stage diffusion model according to claim 5, wherein the stripe noise prior feature is subjected to forward diffusion and reverse denoising to obtain a sampled stripe noise prior feature, and the method comprises the following steps:
forward diffusion:
；
wherein y is 0 Let y be the stripe noise a priori characteristic obtained by the hidden space encoder, y t For the prior characteristic of stripe noise generated at the moment T in forward diffusion, t=1, 2, …, T, I is an identity matrix,is a utilityIn controlling the super-parameters of the noise variation, +.>Representing a gaussian distribution->I.e. y t ，t=1, 2,…, T；
Reverse denoising:
；
wherein, ， />，/>representing the spread noise-canceling network, c representing the noise condition generated by the noisy image after it has passed through the hidden space encoder,/for the noise-canceling network>Representing variance->。
7. The method for denoising a magnetic particle image based on a two-stage diffusion model according to claim 5, wherein the loss function of the first stage is:
；
wherein,representing pixel loss, +.>MPI image representing clean noiseless, +.>A noise-free simulated MPI image is represented.
8. The method for denoising a magnetic particle image based on a two-stage diffusion model according to claim 7, wherein the loss function of the second stage is:
；
；
wherein,indicating loss of association->Indicating diffusion loss->Stripe noise a priori features representing samples, +.>Representing the streak noise a priori features.
9. A magnetic particle image denoising system based on a dual-stage diffusion model, comprising:
the image acquisition module is configured to acquire a two-dimensional MPI image to be denoised as an input image;
the preprocessing module is configured to extract a target region of interest of the input image and preprocess the target region of interest to obtain a preprocessed target image;
the image denoising module is configured to input a trained MPI image denoising model to the preprocessing target image to obtain a clean noiseless MPI image;
the MPI image denoising model is a denoising network based on a double-stage diffusion model; the MPI image denoising model comprises a hidden space coding module, a conditional diffusion module and a multi-scale denoising module;
the hidden space coding module is used for extracting the characteristics of the preprocessing target image to obtain stripe noise priori characteristics;
the condition diffusion module is used for extracting noise conditions; reversely denoising the stripe noise prior feature by combining the noise condition to obtain a sampled stripe noise prior feature;
the multi-scale denoising network is used for denoising the preprocessing target image by combining stripe noise prior characteristics of sampling of a plurality of scales to obtain a clean noiseless MPI image.
10. Magnetic particle image denoising equipment based on double-stage diffusion model, characterized by comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for execution by the processor to implement the dual-stage diffusion model-based magnetic particle image denoising method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410102203.5A CN117635479B (en) | 2024-01-25 | 2024-01-25 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410102203.5A CN117635479B (en) | 2024-01-25 | 2024-01-25 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117635479A true CN117635479A (en) | 2024-03-01 |
| CN117635479B CN117635479B (en) | 2024-04-23 |
Family
ID=90032446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410102203.5A Active CN117635479B (en) | 2024-01-25 | 2024-01-25 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117635479B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114677300A (en) * | 2022-03-25 | 2022-06-28 | 西安交通大学 | Hyperspectral image depth noise reduction method and system based on two-stage learning framework |
| CN114723631A (en) * | 2022-04-01 | 2022-07-08 | 西安交通大学 | Image denoising method, system and device based on depth context prior and multi-scale reconstruction sub-network |
| CN115526946A (en) * | 2022-10-13 | 2022-12-27 | 中国科学院自动化研究所 | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion |
| CN115844365A (en) * | 2023-02-07 | 2023-03-28 | 北京航空航天大学 | Small animal magnetic particle imaging and fluorescent molecular tomography multi-mode imaging system |
| CN116503507A (en) * | 2023-06-26 | 2023-07-28 | 中国科学院自动化研究所 | Magnetic particle image reconstruction method based on pre-training model |
| CN116777764A (en) * | 2023-05-23 | 2023-09-19 | 武汉理工大学 | Diffusion model-based cloud and mist removing method and system for optical remote sensing image |
| US11816767B1 (en) * | 2022-11-18 | 2023-11-14 | Institute Of Automation, Chinese Academy Of Sciences | Method and system for reconstructing magnetic particle distribution model based on time-frequency spectrum enhancement |
-
2024
- 2024-01-25 CN CN202410102203.5A patent/CN117635479B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114677300A (en) * | 2022-03-25 | 2022-06-28 | 西安交通大学 | Hyperspectral image depth noise reduction method and system based on two-stage learning framework |
| CN114723631A (en) * | 2022-04-01 | 2022-07-08 | 西安交通大学 | Image denoising method, system and device based on depth context prior and multi-scale reconstruction sub-network |
| CN115526946A (en) * | 2022-10-13 | 2022-12-27 | 中国科学院自动化研究所 | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion |
| US11816767B1 (en) * | 2022-11-18 | 2023-11-14 | Institute Of Automation, Chinese Academy Of Sciences | Method and system for reconstructing magnetic particle distribution model based on time-frequency spectrum enhancement |
| CN115844365A (en) * | 2023-02-07 | 2023-03-28 | 北京航空航天大学 | Small animal magnetic particle imaging and fluorescent molecular tomography multi-mode imaging system |
| CN116777764A (en) * | 2023-05-23 | 2023-09-19 | 武汉理工大学 | Diffusion model-based cloud and mist removing method and system for optical remote sensing image |
| CN116503507A (en) * | 2023-06-26 | 2023-07-28 | 中国科学院自动化研究所 | Magnetic particle image reconstruction method based on pre-training model |
Non-Patent Citations (3)
| Title |
|---|
| JIAXIN ZHANG: "Magnetic particle imaging deblurring with dual contrastive learning and adversarial framework", 《COMPUTERS IN BIOLOGY AND MEDICINE》, 9 September 2023 (2023-09-09), pages 1 - 11 * |
| YU AN: "Progressive Pretraining Network for 3D System Matrix Calibration in Magnetic Particle Imaging", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》, 30 November 2023 (2023-11-30), pages 1 - 13 * |
| 陈作钧: "微光环境下的视频图像降噪算法研究", 《中国优秀硕士学位论文全文数据库（信息科技辑）》, 22 August 2023 (2023-08-22), pages 138 - 730 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117635479B (en) | 2024-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115640506B (en) | Magnetic particle distribution model reconstruction method and system based on time-frequency spectrum signal enhancement | |
| WO2021041772A1 (en) | Dilated convolutional neural network system and method for positron emission tomography (pet) image denoising | |
| Deora et al. | Structure preserving compressive sensing MRI reconstruction using generative adversarial networks | |
| CN116503507B (en) | Magnetic particle image reconstruction method based on pre-training model | |
| CN113628296B (en) | Magnetic particle imaging reconstruction method from time-frequency domain signal to two-dimensional image | |
| CN114998471B (en) | Magnetic particle imaging reconstruction method based on RecNet model | |
| CN117078792B (en) | Magnetic particle image reconstruction system, method and equipment with regular term self-adaptive optimization | |
| CN114820849A (en) | Magnetic resonance CEST image reconstruction method, device and equipment based on deep learning | |
| US12045961B2 (en) | Image denoising method and apparatus based on wavelet high-frequency channel synthesis | |
| CN116503506B (en) | Image reconstruction method, system, device and storage medium | |
| CN117291835A (en) | Denoising network model based on image content perception priori and attention drive | |
| CN117635479B (en) | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model | |
| CN116843779A (en) | Linear scanning detector differential BPF reconstructed image sparse artifact correction method | |
| Zhuge et al. | Single image denoising with a feature-enhanced network | |
| CN114820636A (en) | Three-dimensional medical image segmentation model and training method and application thereof | |
| CN115471580A (en) | Physical intelligent high-definition magnetic resonance diffusion imaging method | |
| CN112258508B (en) | Image processing analysis segmentation method, system and storage medium for four-dimensional flow data | |
| CN115100069A (en) | Ultrasonic image reconstruction method, system, device and medium | |
| CN104318521A (en) | Medical image denoising method of multilinear subspace learning | |
| Dumitrescu et al. | Methods for improving image quality for contour and textures analysis using new wavelet methods | |
| Hu et al. | Remote Sensing Image Reconstruction Method Based on Parameter Adaptive Dual-Channel Pulse-Coupled Neural Network to Optimize Multiscale Decomposition | |
| CN117672463B (en) | Data processing system and method for radiation therapy | |
| Shi et al. | Blind CT Image Quality Assessment Using DDPM-derived Content and Transformer-based Evaluator | |
| CN116977473B (en) | Sparse angle CT reconstruction method and device based on projection domain and image domain | |
| CN117582204A (en) | MPI acceleration calibration method based on system matrix super-resolution network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |