CN114331911A - Fourier laminated microscopic image denoising method based on convolutional neural network - Google Patents
Fourier laminated microscopic image denoising method based on convolutional neural network Download PDFInfo
- Publication number
- CN114331911A CN114331911A CN202210007222.0A CN202210007222A CN114331911A CN 114331911 A CN114331911 A CN 114331911A CN 202210007222 A CN202210007222 A CN 202210007222A CN 114331911 A CN114331911 A CN 114331911A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- layer
- fourier
- denoising
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a Fourier laminated microscopic image denoising method based on a convolutional neural network, which comprises the following steps of: s1, imaging the human blood cell sample by using a Fourier laminated microscope system to manufacture a data set; s2, building a convolutional neural network; s3, inputting the training set into a coding module, a denoising module and a decoding module of the convolutional neural network, and inhibiting the expression of a noise information channel by using the characteristics of the convolutional neural network; s4, use of L1The loss function repeatedly iterates and optimizes the convolutional neural network to complete the training of the convolutional neural network; s5, denoising the actually collected Fourier microscopic image by using the convolutional neural network to obtain a high-quality reconstruction imageLike this. The method combines the intensity image and the phase image obtained by the Fourier laminated microscope system, utilizes the advantages of a deep learning method, improves the noise problem of the traditional reconstruction algorithm, and provides an accurate and scientific algorithm for medical high-quality imaging.
Description
Technical Field
The invention belongs to the technical field of computer microscopic imaging, and particularly relates to a Fourier laminated microscopic image denoising method based on a convolutional neural network.
Background
The Fourier stacked microscopic imaging technology is a new computational imaging technology which realizes reconstruction of high-resolution images under a large field of view in recent years, has wide application in aspects of digital pathology, cell counting, surface defect detection and the like, and is widely concerned and researched. The method combines the related concepts of phase recovery and synthetic aperture, has various modes such as bright field imaging, dark field imaging, phase contrast imaging and the like, can realize quantitative phase imaging, can obtain information about cell structures, positions and the like, and solves the problem that a transparent sample cannot be imaged. The technique utilizes a programmable Light Emitting Diode (LED) array to illuminate a sample from different angles to achieve frequency domain scanning, collects intensity information of a low-resolution image as spatial domain amplitude constraint, uses a circular pupil function as Fourier domain constraint, and iterates repeatedly on the basis of the two constraints to obtain high-resolution complex amplitude information of the sample.
Despite the advantages of the fourier stacked microscopy imaging technology, the conventional fourier stacked microscopy imaging technology still faces many challenges in the imaging and reconstruction processes, such as various aberrations inevitably introduced by using lenses, position errors of the light emitting diode array inevitably caused by the process, various system noises which are difficult to avoid in the imaging system, and low speed of the reconstruction algorithm. Furthermore, the phase map is more significantly affected by system noise than the intensity map of the sample.
With the development of deep learning, the convolution neural network is utilized to process medical images to become a research hotspot, the neural network is trained on the basis of a large number of data sets by means of huge computing power of a computer, the medical images are optimized in an automatic mode, background noise of the images is effectively removed, image definition is improved, richer detail information is obtained, image imaging quality is improved, and more accurate and reliable results are obtained.
Disclosure of Invention
Aiming at the defects in the prior art, the Fourier laminated microscopic image denoising method based on the convolutional neural network solves the problems of large noise and low medical imaging quality of the traditional reconstruction algorithm.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a Fourier laminated microscopic image denoising method based on a convolutional neural network comprises the following steps:
s1, imaging the human blood cell sample by using a Fourier laminated microscope system, and making a data set comprising a training set and a verification set;
s2, building a convolutional neural network, which comprises an encoding module, a denoising module and a decoding module;
s3, inputting the training set into a coding module, a denoising module and a decoding module of the convolutional neural network, automatically learning the target information and the noise information of the image by using the characteristics of the convolutional neural network, giving a larger weight to the target information channel of the characteristic diagram and giving a smaller weight to the noise information channel of the characteristic diagram, thereby inhibiting the expression of the noise information channel;
s4, use of L1The loss function is iterated repeatedly to optimize the convolutional neural network, after each iteration, a training model is tested by using a verification set, the peak signal-to-noise ratio and the structural similarity are used as evaluation indexes, and when the loss function, the peak signal-to-noise ratio and the structural similarity are not changed obviously, the training of the convolutional neural network is completed;
and S5, denoising the actually acquired Fourier microscopic image by using the convolutional neural network to obtain a high-quality reconstructed image.
Further: the specific steps of step S1 are:
s11, imaging the human blood cell sample by using a Fourier laminated microscopic imaging system, and collecting 400 intensity maps under a 20-time objective lens;
s12, randomly selecting a high-resolution intensity map as an intensity map and a phase map, and obtaining 1600 groups of high-resolution complex amplitudes as truth value data of the neural network in a random combination mode; combining a Fourier laminated imaging simulation algorithm and adding random noise to obtain a low-resolution intensity map;
s13, iterating the low-resolution intensity map once by using a traditional Fourier laminated micro-reconstruction algorithm to obtain low-resolution complex amplitude serving as input of a neural network;
s14, cutting input and truth data in the simulation process to obtain 25600 sets of input and truth data, randomly selecting 23040 sets of input and truth data as a training set, and taking the rest 2560 sets of input and truth data as a verification set.
Further: the encoding module in step S2 is composed of 4 convolutional pooling blocks, and performs feature extraction and downsampling on the input image.
Further: the convolution pooling block comprises a 1 × 1 convolution layer, a batch normalization layer, an activation function layer, a 3 × 3 convolution layer, a batch normalization layer, an activation function layer and a maximum pooling layer which are connected in sequence.
Further: the denoising module in the step S2 includes a global averaging pooling layer, a 1 × 1 convolution layer, an activation function layer, and a 1 × 1 convolution layer and an activation function layer, which are connected in sequence.
Further: the input of the denoising module in the step S2 is feature maps of different levels obtained before the pooling operation of the coding module, the feature maps of different levels are subjected to global pooling operation, the channel features of the feature maps of different levels are fused, the fused channel feature information is subjected to optimization of the convolutional layer and the activation function, and then is divided according to the number of channels of the feature maps of different levels, and the divided channel feature information is subjected to multiplication operation corresponding to the feature maps of the levels, so that channel feature weight redistribution is realized, and the expression of the information channel is highlighted and the expression of the noise information is suppressed.
Further: in step S2, the decoding module is composed of 4 convolution upsampling blocks, performs feature extraction on the high-level feature map, fuses with the same-level feature map optimized by the denoising module, performs upsampling together, and finally obtains high-resolution complex amplitude including intensity information and phase information.
Further: the convolution up-sampling block comprises an up-sampling layer, a 1 x 1 convolution layer, a batch normalization layer, an activation function layer, a 3 x 3 convolution layer, a batch normalization layer and an activation function layer which are sequentially connected.
The invention has the beneficial effects that: the method utilizes the Fourier laminated microscope system to image the human blood cell sample to obtain the intensity image and the phase image of the sample, and the intensity image and the phase image are input into the convolutional neural network as dual-channel characteristics to perform image denoising, so that background noise is removed, and the imaging quality is improved. The multi-level channel attention mechanism convolutional neural network provided by the invention combines an intensity map and a phase map obtained by a Fourier laminated microscope system, utilizes the advantages of a deep learning method, improves the noise problem of the traditional reconstruction algorithm, and provides an accurate and scientific algorithm for medical high-quality imaging.
Drawings
FIG. 1 is an overall flow chart of the present invention;
FIG. 2 is a diagram of a convolutional neural network structure constructed in accordance with the present invention;
FIG. 3 is a block diagram of the coding module of the convolutional neural network of the present invention;
FIG. 4 is a block diagram of a denoising module of the convolutional neural network of the present invention;
FIG. 5 is a block diagram of a decoding module of the convolutional neural network of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, a fourier stacked microscopic image denoising method based on a convolutional neural network includes the following steps:
s1, imaging the human blood cell sample by using a Fourier laminated microscope system, and making a data set comprising a training set and a verification set; the method comprises the following specific steps:
s11, imaging the human blood cell sample by using a Fourier laminated microscopic imaging system, and collecting 400 intensity maps under a 20-time objective lens;
s12, randomly selecting a high-resolution intensity map as an intensity map and a phase map, and obtaining 1600 groups of high-resolution complex amplitudes as truth value data of the neural network in a random combination mode; combining a Fourier laminated imaging simulation algorithm and adding random noise to obtain a low-resolution intensity map;
s13, iterating the low-resolution intensity map once by using a traditional Fourier laminated micro-reconstruction algorithm to obtain low-resolution complex amplitude serving as input of a neural network;
s14, cutting input and truth data in the simulation process to obtain 25600 sets of input and truth data, randomly selecting 23040 sets of input and truth data as a training set, and taking the rest 2560 sets of input and truth data as a verification set.
S2, building a convolutional neural network, which comprises an encoding module, a denoising module and a decoding module; as shown in fig. 2, the size of the input image is 192 × 192. The coding module consists of 4 convolution pooling blocks and mainly performs feature extraction and downsampling on an input image; the input of the denoising module is the feature maps of different levels obtained before the pooling operation of the coding module, the feature maps of different levels are subjected to global pooling operation respectively, channel feature information of the feature maps of different levels is fused, the fused channel feature information is subjected to optimization of a convolutional layer and an activation function and then is divided according to the number of channels of the feature maps of different levels, and the divided channel feature information is subjected to multiplication operation corresponding to the feature maps of the levels respectively. The characteristics of the information channel and the noise channel are adaptively learned through a deep learning framework, and the weight redistribution of the characteristic channel is realized through the multiplication operation, so that the expression of the information channel is highlighted and the expression of the noise information is inhibited; the decoding module is composed of 4 convolution up-sampling blocks, mainly extracts features of a high-level feature map, fuses with the same-level feature map optimized by the denoising module, performs up-sampling operation together, and finally obtains high-resolution complex amplitude including intensity information and phase information.
S3, inputting the training set into a coding module, a denoising module and a decoding module of the convolutional neural network, automatically learning the target information and the noise information of the image by using the characteristics of the convolutional neural network, giving a larger weight to the target information channel of the characteristic diagram and giving a smaller weight to the noise information channel of the characteristic diagram, thereby inhibiting the expression of the noise information channel;
s4, use of L1The loss function is iterated repeatedly to optimize the convolutional neural network, after each iteration, a training model is tested by using a verification set, the peak signal-to-noise ratio and the structural similarity are used as evaluation indexes, and when the loss function, the peak signal-to-noise ratio and the structural similarity are not changed obviously, the training of the convolutional neural network is completed;
and S5, denoising the actually acquired Fourier microscopic image by using the convolutional neural network to obtain a high-quality reconstructed image.
FIG. 3 is a schematic diagram of 1 convolutional pooling block of coding modules in a convolutional neural network, including two convolutional layers, two batch normalization layers, two activation function layers, and one max pooling layer. Firstly, a 1 x 1 convolutional layer is used for carrying out dimension increasing operation on an input feature map, except that a first convolution pooling block increases the input dimension of two channels to 64, the number of the channels of the input feature map is doubled by the convolution pooling block, and then a 3 x 3 convolutional layer is used for carrying out feature extraction on an input image. In addition, a batch normalization layer is added to accelerate the training and convergence speed of the convolutional neural network and prevent overfitting, and an activation function layer is added to increase the nonlinearity of the convolutional neural network. Finally, the feature map is downsampled by using the maximum pooling layer, and 4 different levels of feature maps can be generated after four convolution pooling blocks.
Fig. 4 is a schematic diagram of a denoising module in a convolutional neural network, which shows the optimization of the denoising module on a feature map at a certain level, and includes a global average pooling layer, two 1 × 1 convolutional layers, and two activation functions. And respectively carrying out global average pooling on the feature maps of different levels obtained by the coding module, so that the heights and the widths of the feature maps of different levels are both 1, and the number of channels is not changed, therefore, splicing operation can be carried out on the feature maps of different levels in channel dimensions, and the fused features jointly enter an optimization module consisting of a convolutional layer and an activation function layer, thereby realizing feature sharing of the feature maps of different levels in the channel dimensions. The optimized features are segmented according to the original channel number of feature maps of different levels, and then multiplied by the input feature maps of corresponding levels, so that the weight redistribution of the information channel and the noise channel is realized, the prominent expression of the information channel is realized, the display of noise channel information is inhibited, and the purpose of removing background noise is achieved.
FIG. 5 is a schematic diagram of 1 convolutional upsampling block of a decoding module in a convolutional neural network, including one upsampling layer, two convolutional layers, two batch normalization layers, and two activation function layers. And the upper sampling layer performs upper sampling on the feature map of a higher level, fuses with the feature map of a corresponding level optimized by the denoising module, and jointly enters the convolutional layer, the batch-classification layer and the activation function layer to realize the dimension reduction and feature extraction of the feature map. And when the feature maps of the lowest level are fused, no up-sampling is performed, and dimension reduction is performed to double channels by using the 1 x 1 convolutional layer, so that the two channels are used as final outputs of the convolutional neural network, namely a high-resolution intensity map and a phase map.
Claims (8)
1. A Fourier laminated microscopic image denoising method based on a convolutional neural network is characterized by comprising the following steps:
s1, imaging the human blood cell sample by using a Fourier laminated microscope system, and making a data set comprising a training set and a verification set;
s2, building a convolutional neural network, which comprises an encoding module, a denoising module and a decoding module;
s3, inputting the training set into a coding module, a denoising module and a decoding module of the convolutional neural network, automatically learning the target information and the noise information of the image by using the characteristics of the convolutional neural network, giving a larger weight to the target information channel of the characteristic diagram and giving a smaller weight to the noise information channel of the characteristic diagram, thereby inhibiting the expression of the noise information channel;
s4, use of L1The loss function is iterated repeatedly to optimize the convolutional neural network, after each iteration, a training model is tested by using a verification set, the peak signal-to-noise ratio and the structural similarity are used as evaluation indexes, and when the loss function, the peak signal-to-noise ratio and the structural similarity are not changed obviously, the training of the convolutional neural network is completed;
and S5, denoising the actually acquired Fourier microscopic image by using the convolutional neural network to obtain a high-quality reconstructed image.
2. The method for denoising the fourier stacked microscopy image based on the convolutional neural network as claimed in claim 1, wherein the specific steps of step S1 are:
s11, imaging the human blood cell sample by using a Fourier laminated microscopic imaging system, and collecting 400 intensity maps under a 20-time objective lens;
s12, randomly selecting a high-resolution intensity map as an intensity map and a phase map, and obtaining 1600 groups of high-resolution complex amplitudes as truth value data of the neural network in a random combination mode; combining a Fourier laminated imaging simulation algorithm and adding random noise to obtain a low-resolution intensity map;
s13, iterating the low-resolution intensity map once by using a traditional Fourier laminated micro-reconstruction algorithm to obtain low-resolution complex amplitude serving as input of a neural network;
s14, cutting input and truth data in the simulation process to obtain 25600 sets of input and truth data, randomly selecting 23040 sets of input and truth data as a training set, and taking the rest 2560 sets of input and truth data as a verification set.
3. The method for denoising fourier-stacked microscopy images based on convolutional neural network as claimed in claim 1, wherein the coding module in step S2 is composed of 4 convolutional pooling blocks, and performs feature extraction and downsampling on the input image.
4. The convolutional neural network-based Fourier stacked microscopy image denoising method according to claim 3, wherein the convolutional pooling block comprises a 1 x 1 convolutional layer, a batch normalization layer, an activation function layer, a 3 x 3 convolutional layer, a batch normalization layer, an activation function layer and a maximum pooling layer which are connected in sequence.
5. The method for denoising a fourier stacked microscopy image based on a convolutional neural network as claimed in claim 1, wherein the denoising module in the step S2 comprises a global mean pooling layer, a 1 × 1 convolutional layer, an activation function layer, and a 1 × 1 convolutional layer and an activation function layer connected in sequence.
6. The Fourier stacked microscopic image denoising method based on the convolutional neural network as claimed in claim 5, wherein the input of the denoising module in step S2 is feature maps of different levels obtained before the pooling operation of the coding module, the feature maps of different levels are subjected to global pooling operation respectively, the channel features of the feature maps of different levels are fused, the fused channel feature information is subjected to optimization of a convolutional layer and an activation function, and then is divided according to the number of channels of the feature maps of different levels, and the divided channel feature information is subjected to multiplication operation corresponding to the feature maps of the levels respectively, so that channel feature weight redistribution is realized, and the expression of an information channel is highlighted and the expression of noise information is suppressed.
7. The method for denoising the fourier stacked microscopy image based on the convolutional neural network as claimed in claim 1, wherein the decoding module in step S2 is composed of 4 convolutional upsampling blocks, performs feature extraction on the feature map of the high level, fuses with the feature map of the same level optimized by the denoising module, performs upsampling operation together, and finally obtains high-resolution complex amplitude including intensity information and phase information.
8. The convolutional neural network-based Fourier stacked microscopy image denoising method of claim 7, wherein the convolutional upsampling block comprises an upsampling layer, a 1 x 1 convolutional layer, a batch normalization layer, an activation function layer, a 3 x 3 convolutional layer, a batch normalization layer and an activation function layer which are connected in sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210007222.0A CN114331911A (en) | 2022-01-05 | 2022-01-05 | Fourier laminated microscopic image denoising method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210007222.0A CN114331911A (en) | 2022-01-05 | 2022-01-05 | Fourier laminated microscopic image denoising method based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114331911A true CN114331911A (en) | 2022-04-12 |
Family
ID=81024842
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210007222.0A Pending CN114331911A (en) | 2022-01-05 | 2022-01-05 | Fourier laminated microscopic image denoising method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114331911A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114972981A (en) * | 2022-04-19 | 2022-08-30 | 国网江苏省电力有限公司电力科学研究院 | Power grid power transmission environment observation image denoising method, terminal and storage medium |
| CN115984107A (en) * | 2022-12-21 | 2023-04-18 | 中国科学院生物物理研究所 | Self-supervision multi-mode structure light microscopic reconstruction method and system |
| CN115984406A (en) * | 2023-03-20 | 2023-04-18 | 始终(无锡)医疗科技有限公司 | SS-OCT compression imaging method for deep learning and spectral domain and spatial domain combined sub-sampling |
| CN116246025A (en) * | 2023-04-28 | 2023-06-09 | 中南大学 | CT image synthesis method and imaging method based on three-dimensional modeling and MR image |
| CN116544146A (en) * | 2023-05-22 | 2023-08-04 | 浙江固驰电子有限公司 | Vacuum sintering equipment and method for power semiconductor device |
-
2022
- 2022-01-05 CN CN202210007222.0A patent/CN114331911A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114972981A (en) * | 2022-04-19 | 2022-08-30 | 国网江苏省电力有限公司电力科学研究院 | Power grid power transmission environment observation image denoising method, terminal and storage medium |
| CN115984107A (en) * | 2022-12-21 | 2023-04-18 | 中国科学院生物物理研究所 | Self-supervision multi-mode structure light microscopic reconstruction method and system |
| CN115984107B (en) * | 2022-12-21 | 2023-08-11 | 中国科学院生物物理研究所 | Self-supervision multi-mode structure light microscopic reconstruction method and system |
| CN115984406A (en) * | 2023-03-20 | 2023-04-18 | 始终(无锡)医疗科技有限公司 | SS-OCT compression imaging method for deep learning and spectral domain and spatial domain combined sub-sampling |
| CN115984406B (en) * | 2023-03-20 | 2023-06-20 | 始终(无锡)医疗科技有限公司 | SS-OCT compression imaging method for deep learning and spectral domain airspace combined sub-sampling |
| CN116246025A (en) * | 2023-04-28 | 2023-06-09 | 中南大学 | CT image synthesis method and imaging method based on three-dimensional modeling and MR image |
| CN116246025B (en) * | 2023-04-28 | 2023-07-11 | 中南大学 | CT image synthesis method and imaging method based on three-dimensional modeling and MR image |
| CN116544146A (en) * | 2023-05-22 | 2023-08-04 | 浙江固驰电子有限公司 | Vacuum sintering equipment and method for power semiconductor device |
| CN116544146B (en) * | 2023-05-22 | 2024-04-09 | 浙江固驰电子有限公司 | Vacuum sintering equipment and method for power semiconductor device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114331911A (en) | Fourier laminated microscopic image denoising method based on convolutional neural network | |
| CN111325751B (en) | CT image segmentation system based on attention convolution neural network | |
| CN111462013B (en) | Single-image rain removing method based on structured residual learning | |
| CN115546032B (en) | Single-frame image super-resolution method based on feature fusion and attention mechanism | |
| CN109214989A (en) | Single image super resolution ratio reconstruction method based on Orientation Features prediction priori | |
| CN112215199B (en) | SAR image ship detection method based on multi-receptive field and dense feature aggregation network | |
| CN110599502A (en) | Skin lesion segmentation method based on deep learning | |
| CN111461978A (en) | Attention mechanism-based resolution-by-resolution enhanced image super-resolution restoration method | |
| CN116363527A (en) | Remote sensing image change detection method based on interaction feature perception | |
| Niu et al. | GRAN: ghost residual attention network for single image super resolution | |
| CN113436198A (en) | Remote sensing image semantic segmentation method for collaborative image super-resolution reconstruction | |
| CN115619646B (en) | Deep learning optical illumination super-resolution imaging method for sub-fifty nano-structure | |
| Yang et al. | RSAMSR: A deep neural network based on residual self-encoding and attention mechanism for image super-resolution | |
| CN116091319A (en) | Image super-resolution reconstruction method and system based on long-distance context dependence | |
| CN116029905A (en) | Face super-resolution reconstruction method and system based on progressive difference complementation | |
| Li et al. | Automatic classification of galaxy morphology based on the RegNetX-CBAM3 algorithm | |
| CN116091763A (en) | Apple leaf disease image semantic segmentation system, segmentation method, device and medium | |
| Wang et al. | Image quality enhancement using hybrid attention networks | |
| CN114219738A (en) | Single-image multi-scale super-resolution reconstruction network structure and method | |
| CN114037843A (en) | Method for improving resolution of underwater image based on improved generation countermeasure network | |
| CN110969117A (en) | Fundus image segmentation method based on Attention mechanism and full convolution neural network | |
| Huang et al. | TARN: a lightweight two-branch adaptive residual network for image super-resolution | |
| CN112270265B (en) | SAR image ship detection method based on split-aggregation convolution network | |
| CN117994135B (en) | Method for SOHO/MDI magnetic map super-resolution reconstruction based on deep learning | |
| CN117455774B (en) | Image reconstruction method and system based on differential output |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |