CN115526946A - Method, system and equipment for denoising magnetic particle imaging image based on feature fusion - Google Patents
Method, system and equipment for denoising magnetic particle imaging image based on feature fusion Download PDFInfo
- Publication number
- CN115526946A CN115526946A CN202211254765.9A CN202211254765A CN115526946A CN 115526946 A CN115526946 A CN 115526946A CN 202211254765 A CN202211254765 A CN 202211254765A CN 115526946 A CN115526946 A CN 115526946A
- Authority
- CN
- China
- Prior art keywords
- image
- feature
- noise
- mpi
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 title claims abstract description 61
- 239000006249 magnetic particle Substances 0.000 title claims abstract description 41
- 238000003384 imaging method Methods 0.000 title claims abstract description 38
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims description 147
- 238000011176 pooling Methods 0.000 claims description 42
- 238000012545 processing Methods 0.000 claims description 33
- 238000012549 training Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 15
- 238000004088 simulation Methods 0.000 claims description 12
- 239000002245 particle Substances 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005065 mining Methods 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 3
- 238000004321 preservation Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000002591 computed tomography Methods 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000002122 magnetic nanoparticle Substances 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the field of magnetic particle imaging, and particularly relates to a method, a system and equipment for denoising a magnetic particle imaging image based on feature fusion, aiming at solving the problem that the existing MPI denoising method is difficult to give consideration to both noise removal and image detail preservation. The method comprises the following steps: acquiring an MPI image to be denoised as an input image; denoising the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image; the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module; the feature extraction module comprises a noise feature extractor and a content feature extractor; the feature fusion module comprises two channel attention sub-modules, two convolution layers and a space attention sub-module; the feature regression module is constructed based on convolutional layers. The method can achieve good denoising effect and simultaneously keep the structural details of the MPI image.
Description
Technical Field
The invention belongs to the field of magnetic particle imaging, and particularly relates to a method, a system and equipment for denoising a magnetic particle imaging image based on feature fusion.
Background
Magnetic Particle Imaging (MPI) is a tomographic method that images lesions or other markers in real time dynamically in three dimensions through the nonlinear response of Magnetic nanoparticles. Compared with the existing medical Imaging technologies such as Magnetic Resonance Imaging (MRI), computed Tomography (CT), and the like, MPI has the advantages of high sensitivity, no depth limitation, high safety, and the like, and is a new medical Imaging method with a wide application prospect.
In the current practical application scenario, due to the influence of the device itself and external environmental factors, the signal-to-noise ratio of the MPI image is low. In multi-color MPI, the lower signal-to-noise ratio when imaging for multi-concentration samples cannot meet clinical requirements. Therefore, how to improve the signal-to-noise ratio of the MPI image is a challenge and a difficult problem of the current MPI device. Existing methods improve MPI imaging quality mainly by processing noise in the signal domain. Due to the high sensitivity and real-time performance of MPI imaging, noise in imaging is complex and variable, and it is difficult to construct a statistical model in a signal domain for quantitative analysis of effective information and noise. In addition, the existing method is difficult to consider the removal of image noise and the maintenance of image structure details. Based on the method, the invention provides a magnetic particle imaging image denoising method based on feature fusion learning, and aims to effectively remove interference noise in an image, maintain the structural details of the image and improve the imaging quality of MPI.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem that the existing MPI denoising method is difficult to combine noise removal and image detail preservation, the present invention provides a magnetic particle imaging image denoising method based on feature fusion, which includes:
s100, collecting an MPI image to be denoised as an input image;
s200, denoising the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise-carrying MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first down-sampling layer and the third sampling layer, and the second down-sampling layer and the second up-sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention sub-modules, two convolution layers and a space attention sub-module; the inputs of the two channel attention sub-modules are respectively the outputs of the noise feature extractor and the content feature extractor; splicing and combining the outputs of the two channel attention sub-modules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention sub-module for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for performing convolution operation on the output of the feature fusion module and outputting a denoised MPI image.
In some preferred embodiments, each of the three down-sampling layers and the three up-sampling layers is composed based on a3 × 3 convolution operation, batch normalization, and Dropout operation.
In some preferred embodiments, the training method of the feature fusion denoising network model is as follows:
a100, acquiring a noiseless MPI simulation image and a real noise image of MPI equipment, randomly cutting the real noise image, and performing image processing to obtain an amplified noise image; the image processing comprises rotation, white balance transformation and gamma transformation;
a200, multiplying the amplified noise image by a set intensity coefficient, and superposing the amplified noise image with the noiseless simulated MPI image to obtain an MPI image with real noise; constructing a training data set based on an MPI image with real noise and a true value image corresponding to the MPI image;
a300, inputting the MPI image with real noise in the training data set into the feature fusion denoising network model to obtain a denoised MPI image as a feature fusion learning output image, and acquiring the features extracted by a noise feature extractor and a content feature extractor in the feature fusion denoising network model as noise features and content features;
a400, processing the noise characteristics through a pre-constructed noise characteristic regression device, and subtracting the processed noise characteristics from an MPI image with real noise input by the characteristic fusion denoising network model to obtain a noise learning output image; processing the content features through a pre-constructed content feature regression device to obtain a content learning output image; the noise characteristic regressor and the content characteristic regressor are constructed on the basis of convolution layers;
a500, calculating a loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image and the feature fusion learning output image and in combination with the true value image, and updating model parameters of a feature fusion denoising network model;
and A600, circulating the steps A300-A500 until a trained model of the feature fusion denoising network model is obtained.
In some preferred embodiments, the noise-free MPI simulation image is obtained by:
randomly generating a particle distribution image; wherein, the gray value size in the particle distribution image represents the magnetic particle concentration size;
and generating a voltage signal corresponding to the particle distribution image according to an MPI principle, and reconstructing the image through an x-space reconstruction algorithm to obtain a noiseless MPI simulation image.
In some preferred embodiments, the real noise image of the MPI apparatus is obtained by: and placing the dummy without the injected magnetic particles into MPI equipment for empty acquisition to obtain a real noise image of the MPI equipment.
In some preferred embodiments, the pre-constructed loss function is:
wherein, I content Is a true value image, I 1 ，I 2 ，I 3 Learning output images, features for noise, respectivelyAnd fusing the learning output image and the content learning output image.
In a second aspect of the present invention, a magnetic particle imaging image denoising system based on feature fusion is provided, the system comprising: the device comprises an image acquisition module and an image denoising module;
the image acquisition module is configured to acquire an MPI image to be denoised as an input image;
the image denoising module is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise-carrying MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first down-sampling layer and the third sampling layer, and the second down-sampling layer and the second up-sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention sub-modules, two convolution layers and a space attention sub-module; the input of the two channel attention submodules is the output of a noise feature extractor and the output of a content feature extractor respectively; splicing and combining the outputs of the two channel attention submodules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention submodule for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for performing convolution operation on the output of the feature fusion module and outputting a denoised MPI image.
In a third aspect of the present invention, an electronic device is provided, including: at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the above-described method for denoising magnetic particle imaging images based on feature fusion.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions for being executed by a computer to implement the above-mentioned method for denoising magnetic particle imaging image based on feature fusion.
The invention has the beneficial effects that:
the method can achieve a good denoising effect and simultaneously keep the structural details of the MPI image.
The method processes noise generated in the magnetic particle imaging process in an image domain, respectively extracts noise characteristics and content characteristics of the MPI image through two characteristic extractors and corresponding loss function designs, introduces an attention mechanism to realize efficient fusion of the two characteristics, performs image prediction by utilizing the fusion characteristics, can realize MPI image noise removal, simultaneously considers maintenance of image details, and particularly has obvious noise removal effect under the condition of low magnetic particle concentration.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a flow chart of a magnetic particle imaging image denoising method based on feature fusion according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a frame of a magnetic particle imaging image denoising system based on feature fusion according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a feature fusion denoising network model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of the structure of the feature extraction module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the convolution attention module of one embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a feature fusion module according to an embodiment of the present invention;
FIG. 7 is a simplified flowchart of a feature fusion denoising network model training process according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict.
A method for denoising a magnetic particle imaging image based on feature fusion according to a first embodiment of the present invention, as shown in fig. 1, includes:
s100, collecting an MPI image to be denoised as an input image;
s200, denoising the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first down-sampling layer and the third sampling layer, and the second down-sampling layer and the second up-sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention submodules, two convolution layers and a space attention submodule; the inputs of the two channel attention sub-modules are respectively the outputs of the noise feature extractor and the content feature extractor; splicing and combining the outputs of the two channel attention sub-modules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention sub-module for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for performing convolution operation on the output of the feature fusion module and outputting a denoised MPI image.
In order to more clearly explain the method for denoising magnetic particle imaging images based on feature fusion, the steps in one embodiment of the method of the present invention are described in detail below with reference to the accompanying drawings.
In the following embodiments, the structure and the training process of the feature fusion denoising network model are detailed first, and then the process of denoising the MPI image to be denoised by the feature fusion based magnetic particle imaging image denoising method is detailed.
1. The structure and training process of the feature fusion denoising network model are shown in FIG. 7
A100, acquiring a noiseless MPI simulation image and a real noise image of MPI equipment, randomly cutting the real noise image, and performing image processing to obtain an amplified noise image; the image processing comprises rotation, white balance transformation and gamma transformation;
in the present embodiment, an MPI simulation program is used to obtain an MPI simulation image without noise, and a real noise image is obtained by adding real noise to the MPI simulation image, so as to construct a corresponding training data set for training the model. The method comprises the following specific steps:
a110, simulation of magnetic particle imaging process: the MPI simulation program adopted by the invention mainly comprises four parts of data reading, image reconstruction, image post-processing and image derivation. The raw input data is a randomly generated particle distribution image, the size of the grey values in the image representing the magnetic particle density size. After data reading, corresponding voltage signals are generated according to the MPI principle, and the voltage signals of the magnetic particles are subjected to image reconstruction by using an x-space reconstruction algorithm.
A120, constructing a real noise image data set: in order to obtain a real noise image of the MPI equipment, commercial MPI equipment is used for empty acquisition, a dummy body without magnetic particles is placed, a default scanning mode is set, the average times are 1, the empty acquisition of the noise image is started, the acquired noise image is randomly cut into 128 x 128 size, and sample amplification is carried out through image processing operations such as rotation, white balance conversion, gamma conversion and the like, so that an amplified noise image is obtained.
A200, multiplying the amplified noise image by a set intensity coefficient, and superposing the amplified noise image with the noiseless simulated MPI image to obtain an MPI image with real noise; constructing a training data set based on an MPI image with real noise and a true value image corresponding to the MPI image;
in this embodiment, an amplified noise sample (i.e., a noise image) is multiplied by a set intensity coefficient σ and then superimposed on a noise-free simulated MPI image to obtain an MPI image with real noise, and a training data set is constructed based on the MPI image with real noise and a corresponding true value image thereof. Namely, a simulation image with superimposed equipment noise (namely, an MPI image with real noise) is used as a network input, and a noise-free simulation image is used as a training true value (namely, a true value image). During training, the invention preferably compares the training data set to 7:1: and 2, dividing the training set, the verification set and the test set and inputting the training set into a model for training.
A300, inputting the MPI image with real noise in the training data set into the feature fusion denoising network model to obtain a denoised MPI image as a feature fusion learning output image, and acquiring features extracted by a noise feature extractor and a content feature extractor in the feature fusion denoising network model as noise features and content features;
in this embodiment, the feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module;
the feature extraction module is composed of two feature extractors based on a self-coding network architecture, the two feature extractors are constrained by a loss function, so that noise features and content features in the MPI image are respectively extracted, and the two feature extractors are respectively a noise feature extractor and a content feature extractor, as shown in FIG. 3; the input of the noise feature extractor and the input of the content feature extractor are both noise MPI images;
as shown in fig. 4, the noise feature extractor and the content feature extractor are both constructed based on three down-sampling layers and three up-sampling layers, and after each down-sampling, the size of the output feature map is half of the size of the input feature map. After each upsampling, the size of the output characteristic diagram is twice that of the input characteristic diagram so as to realize multi-scale characteristic extraction (as shown by 1, 1/2 and 1/4 in figure 4); the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer; each of the three down-sampling layers and the three up-sampling layers is composed (i.e., sequentially connected) based on a3 × 3 convolution operation, batch normalization and Dropout operation.
The first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first lower sampling layer and the third sampling layer are connected in a cross-layer mode, and the second lower sampling layer and the second upper sampling layer are connected in a cross-layer mode through a convolution attention module;
as shown in fig. 5, the convolution attention module is constructed based on a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
in this way, downsampling is performed by convolution operations to preserve more features. Finally, the low-order detail features and the high-order semantic features are combined through a convolution attention mechanism to improve the characterization capability of the extractor.
As shown in fig. 6, the feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule; the input of the two channel attention submodules is the output of a noise feature extractor and the output of a content feature extractor respectively, and the two channel attention submodules are used for screening out important feature channels; secondly, the outputs of the two channel attention sub-modules are spliced and combined (namely, connected according to the channels in the figure 6), fusion features are extracted through the two convolution layers, finally the fusion features are input into the space attention sub-modules to be processed so as to determine the most important region in the feature map, and the processed fusion features are used as the outputs of the feature fusion module;
the feature regression module is constructed based on the convolution layer and is used for carrying out convolution operation on the output of the feature fusion module and outputting a denoised MPI image, namely, the conversion from fusion features to predicted images is realized.
Inputting the MPI image with real noise in the training data set into the feature fusion denoising network model to obtain a denoised MPI image as a feature fusion learning output image, and acquiring features extracted by a noise feature extractor and a content feature extractor in the feature fusion denoising network model as noise features and content features;
a400, processing the noise characteristics through a pre-constructed noise characteristic regression device, and subtracting the processed noise characteristics from an MPI image with real noise input by the characteristic fusion denoising network model to obtain a noise learning output image; processing the content features through a pre-constructed content feature regression device to obtain a content learning output image; the noise characteristic regressor and the content characteristic regressor are constructed on the basis of convolution layers;
in this embodiment, the noise features are processed by a pre-constructed noise feature regressor, and the processed noise features are subtracted from the MPI image with real noise input by the feature fusion denoising network model to obtain a noise learning output image; and processing the content features through a pre-constructed content feature regression processor to obtain a content learning output image, as shown in fig. 3.
A500, calculating a loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image and the feature fusion learning output image and in combination with the true value image, and updating model parameters of a feature fusion denoising network model;
in this embodiment, the pre-constructed loss function is:
wherein, I content Is a true value image, I 1 ，I 2 ，I 3 Respectively a noise learning output image, a feature fusion learning output image and a content learning output image.
And A600, circulating the steps A300-A500 until a trained model of the feature fusion denoising network model is obtained.
In this embodiment, the model of the feature fusion denoising network model is trained in a loop until the trained model of the feature fusion denoising network model is obtained.
2. Magnetic particle imaging image denoising method based on feature fusion
And S100, acquiring an MPI image to be denoised as an input image.
In the present embodiment, an MPI image to be denoised is acquired.
S200, denoising the input image based on the pre-trained feature fusion denoising network model to obtain a denoised MPI image.
In this embodiment, an MPI image to be denoised is input into a trained feature fusion denoising network model for denoising, so as to obtain a denoised MPI image.
A magnetic particle imaging image denoising system based on feature fusion according to a second embodiment of the present invention, as shown in fig. 2, includes: the image acquisition module 100 and the image denoising module 200;
the image acquisition module 100 is configured to acquire an MPI image to be denoised as an input image;
the image denoising module 200 is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first down-sampling layer and the third sampling layer, and the second down-sampling layer and the second up-sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention submodules, two convolution layers and a space attention submodule; the inputs of the two channel attention sub-modules are respectively the outputs of the noise feature extractor and the content feature extractor; splicing and combining the outputs of the two channel attention sub-modules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention sub-module for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for performing convolution operation on the output of the feature fusion module and outputting a denoised MPI image.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, reference may be made to the corresponding process in the foregoing method embodiment for the specific working process and related description of the system described above, and details are not described herein again.
It should be noted that, the system for denoising a magnetic particle imaging image based on feature fusion provided in the foregoing embodiment is only illustrated by the division of the above functional modules, and in practical applications, the above functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the above described functions. Names of the modules and steps related in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
An electronic device according to a third embodiment of the present invention includes at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for performing the method for denoising magnetic particle imaging images based on feature fusion as claimed above.
A computer readable storage medium of a fourth embodiment of the present invention stores computer instructions for being executed by a computer to implement the method for denoising magnetic particle imaging image based on feature fusion as claimed above.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the electronic device and the computer-readable storage medium described above may refer to corresponding processes in the foregoing method examples, and are not described herein again.
Referring now to FIG. 8, shown is a block diagram of a computer system suitable for use as a server in implementing embodiments of the present systems, methods, and apparatus. The server shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 8, the computer system includes a Central Processing Unit (CPU) 801 which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data necessary for system operation are also stored. The CPU801, ROM802, and RAM803 are connected to each other via a bus 804. An Input/Output (I/O) interface 805 is also connected to the bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a cathode ray tube, a liquid crystal display, and the like, and a speaker and the like; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a local area network card, modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that the computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809 and/or installed from the removable medium 811. The computer program, when executed by the CPU801, performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer-readable storage medium may be, for example but not limited to: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network or a wide area network, or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is apparent to those skilled in the art that the scope of the present invention is not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (9)
1. A magnetic particle imaging image denoising method based on feature fusion is characterized by comprising the following steps:
s100, collecting an MPI image to be denoised as an input image;
s200, denoising the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise-carrying MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first lower sampling layer and the third sampling layer are connected in a cross-layer mode, and the second lower sampling layer and the second upper sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention sub-modules, two convolution layers and a space attention sub-module; the input of the two channel attention submodules is the output of a noise feature extractor and the output of a content feature extractor respectively; splicing and combining the outputs of the two channel attention sub-modules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention sub-module for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for carrying out convolution operation on the output of the feature fusion module and outputting the denoised MPI image.
2. The method for denoising a magnetic particle imaging image based on feature fusion of claim 1, wherein each of the three down-sampling layers and the three up-sampling layers is composed based on a3 x 3 convolution operation, a batch normalization and a Dropout operation.
3. The feature fusion based magnetic particle imaging image denoising method according to claim 2, wherein the feature fusion denoising network model is trained by the method comprising:
a100, acquiring a noiseless MPI simulation image and a real noise image of MPI equipment, randomly cutting the real noise image, and performing image processing to obtain an amplified noise image; the image processing comprises rotation, white balance transformation and gamma transformation;
a200, multiplying the amplified noise image by a set intensity coefficient, and superposing the amplified noise image with the noiseless simulated MPI image to obtain an MPI image with real noise; constructing a training data set based on an MPI image with real noise and a true value image corresponding to the MPI image;
a300, inputting the MPI image with real noise in the training data set into the feature fusion denoising network model to obtain a denoised MPI image as a feature fusion learning output image, and acquiring the features extracted by a noise feature extractor and a content feature extractor in the feature fusion denoising network model as noise features and content features;
a400, processing the noise features through a pre-constructed noise feature regression processor, and subtracting the processed noise features from an MPI image with real noise input by the feature fusion denoising network model to obtain a noise learning output image; processing the content features through a pre-constructed content feature regression device to obtain a content learning output image; the noise characteristic regressor and the content characteristic regressor are constructed on the basis of convolution layers;
a500, calculating a loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image and the feature fusion learning output image and in combination with the true value image, and updating model parameters of a feature fusion denoising network model;
and A600, circulating the steps A300-A500 until a trained model of the feature fusion denoising network model is obtained.
4. The method for denoising the magnetic particle imaging image based on feature fusion according to claim 3, wherein the noise-free MPI simulation image is obtained by:
randomly generating a particle distribution image; wherein, the gray value size in the particle distribution image represents the magnetic particle concentration size;
and generating a voltage signal corresponding to the particle distribution image according to an MPI principle, and reconstructing the image through an x-space reconstruction algorithm to obtain a noiseless MPI simulation image.
5. The feature fusion based magnetic particle imaging image denoising method of claim 3, wherein the true noise image of the MPI device is obtained by: and placing the dummy without the injected magnetic particles into MPI equipment for empty mining to obtain a real noise image of the MPI equipment.
6. The feature fusion based magnetic particle imaging image denoising method of claim 3, wherein the pre-constructed loss function is:
wherein, I content Is a true value image, I 1 ，I 2 ，I 3 Respectively a noise learning output image, a feature fusion learning output image and a content learning output image.
7. A magnetic particle imaging image denoising system based on feature fusion is characterized by comprising: the device comprises an image acquisition module and an image denoising module;
the image acquisition module is configured to acquire an MPI image to be denoised as an input image;
the image denoising module is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;
the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module;
the feature extraction module comprises a noise feature extractor and a content feature extractor; the input of the noise feature extractor and the input of the content feature extractor are both noise MPI images;
the noise feature extractor and the content feature extractor are constructed on the basis of three down-sampling layers and three up-sampling layers; the three down-sampling layers are respectively used as a first down-sampling layer, a second down-sampling layer and a third down-sampling layer; the three upper sampling layers are respectively used as a first upper sampling layer, a second upper sampling layer and a third upper sampling layer;
the first down-sampling layer, the second down-sampling layer, the third down-sampling layer, the first up-sampling layer, the second up-sampling layer and the third up-sampling layer are sequentially connected; the first down-sampling layer and the third sampling layer, and the second down-sampling layer and the second up-sampling layer are connected in a cross-layer mode through a convolution attention module;
the convolution attention module is constructed on the basis of a channel attention submodule and a space attention submodule which are connected in sequence;
the channel attention submodule is used for carrying out average pooling on the input features, multiplying the features before the average pooling with the features after the average pooling and taking the multiplied result as the output of the channel attention submodule;
the space attention submodule is used for performing convolution on the input features, performing maximum pooling on the features after convolution processing, multiplying the features before the maximum pooling with the features after the maximum pooling, and taking the multiplied result as the output of the space attention submodule;
the feature fusion module comprises two channel attention sub-modules, two convolution layers and a space attention sub-module; the inputs of the two channel attention sub-modules are respectively the outputs of the noise feature extractor and the content feature extractor; splicing and combining the outputs of the two channel attention submodules, extracting fusion characteristics through the two convolution layers, inputting the fusion characteristics into the space attention submodule for processing, and taking the processed fusion characteristics as the output of the characteristic fusion module;
the feature regression module is constructed based on the convolution layer and is used for carrying out convolution operation on the output of the feature fusion module and outputting the denoised MPI image.
8. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for performing the method for denoising feature fusion based magnetic particle imaging images according to any one of claims 1-6.
9. A computer-readable storage medium storing computer instructions for execution by a computer to implement the feature fusion based magnetic particle imaging image denoising method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211254765.9A CN115526946A (en) | 2022-10-13 | 2022-10-13 | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211254765.9A CN115526946A (en) | 2022-10-13 | 2022-10-13 | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115526946A true CN115526946A (en) | 2022-12-27 |
Family
ID=84701764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211254765.9A Pending CN115526946A (en) | 2022-10-13 | 2022-10-13 | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115526946A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880440A (en) * | 2023-01-31 | 2023-03-31 | 中国科学院自动化研究所 | Magnetic particle three-dimensional reconstruction imaging method based on generation of countermeasure network |
| CN116977650A (en) * | 2023-07-31 | 2023-10-31 | 西北工业大学深圳研究院 | Image denoising method, image denoising device, electronic equipment and storage medium |
| CN117635479A (en) * | 2024-01-25 | 2024-03-01 | 北京航空航天大学 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
| CN117689761A (en) * | 2024-02-02 | 2024-03-12 | 北京航空航天大学 | Plug-and-play magnetic particle imaging reconstruction method and system based on diffusion model |
-
2022
- 2022-10-13 CN CN202211254765.9A patent/CN115526946A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115880440A (en) * | 2023-01-31 | 2023-03-31 | 中国科学院自动化研究所 | Magnetic particle three-dimensional reconstruction imaging method based on generation of countermeasure network |
| CN115880440B (en) * | 2023-01-31 | 2023-04-28 | 中国科学院自动化研究所 | Magnetic particle three-dimensional reconstruction imaging method based on generation countermeasure network |
| CN116977650A (en) * | 2023-07-31 | 2023-10-31 | 西北工业大学深圳研究院 | Image denoising method, image denoising device, electronic equipment and storage medium |
| CN117635479A (en) * | 2024-01-25 | 2024-03-01 | 北京航空航天大学 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
| CN117635479B (en) * | 2024-01-25 | 2024-04-23 | 北京航空航天大学 | Magnetic particle image denoising method, system and equipment based on double-stage diffusion model |
| CN117689761A (en) * | 2024-02-02 | 2024-03-12 | 北京航空航天大学 | Plug-and-play magnetic particle imaging reconstruction method and system based on diffusion model |
| CN117689761B (en) * | 2024-02-02 | 2024-04-26 | 北京航空航天大学 | Plug-and-play magnetic particle imaging reconstruction method and system based on diffusion model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115526946A (en) | Method, system and equipment for denoising magnetic particle imaging image based on feature fusion | |
| Eun et al. | Deep-learning-based image quality enhancement of compressed sensing magnetic resonance imaging of vessel wall: comparison of self-supervised and unsupervised approaches | |
| CN114998471B (en) | Magnetic particle imaging reconstruction method based on RecNet model | |
| CN116503507B (en) | Magnetic particle image reconstruction method based on pre-training model | |
| CN115640506B (en) | Magnetic particle distribution model reconstruction method and system based on time-frequency spectrum signal enhancement | |
| CN111723728A (en) | Pedestrian searching method, system and device based on bidirectional interactive network | |
| CN113538277B (en) | Neural network-based tomography image noise reduction method and device | |
| CN114944229A (en) | Brain age prediction method based on deep learning and magnetic resonance structure brain image | |
| CN117495714A (en) | Face image restoration method and device based on diffusion generation priori and readable medium | |
| CN115239655A (en) | Thyroid ultrasonic image tumor segmentation and classification method and device | |
| Zhang et al. | A multimodal fusion method for Alzheimer’s disease based on DCT convolutional sparse representation | |
| CN115830317A (en) | Skin cancer image segmentation method and device based on U-Net attention enhancement module of polar coordinate conversion | |
| CN114795247A (en) | Electroencephalogram signal analysis method and device, electronic equipment and storage medium | |
| CN116934597B (en) | FDNN model-based magnetic particle imaging spatial resolution improvement method | |
| Xu et al. | Hyperspectral image super-resolution reconstruction based on image partition and detail enhancement | |
| CN115063500B (en) | Magnetic nanoparticle imaging reconstruction method based on generation countermeasure network | |
| CN117689761B (en) | Plug-and-play magnetic particle imaging reconstruction method and system based on diffusion model | |
| CN116501904B (en) | Distributed storage method, device, equipment and medium | |
| CN116030152A (en) | Magnetic susceptibility image reconstruction method and system based on multisource information fusion network | |
| Chen et al. | Medical image fusion based on multi-scale co-occurrence filter and ResNet152 | |
| CN117934650A (en) | Domain-adaptation-based intra-voxel incoherent motion magnetic resonance imaging reconstruction method and device | |
| CN116777780A (en) | Near infrared fluorescence imaging method based on gram matrix and style domain conversion | |
| Zhang | Weak illumination image enhancement algorithm based on cyclic generation countermeasure network | |
| CN116645439A (en) | Light-weight susceptibility graph reconstruction method, system and electronic equipment with pre-estimated introduction | |
| CN115272506A (en) | Image reconstruction method of large-aperture magnetic particle imaging system based on multiple prior characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |