CN111161180A - Deep learning ultrasonic image denoising method based on migration and structure prior - Google Patents
Deep learning ultrasonic image denoising method based on migration and structure prior Download PDFInfo
- Publication number
- CN111161180A CN111161180A CN201911364562.3A CN201911364562A CN111161180A CN 111161180 A CN111161180 A CN 111161180A CN 201911364562 A CN201911364562 A CN 201911364562A CN 111161180 A CN111161180 A CN 111161180A
- Authority
- CN
- China
- Prior art keywords
- denoising
- image
- network
- ultrasonic image
- speckle noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000013135 deep learning Methods 0.000 title claims abstract description 12
- 230000005012 migration Effects 0.000 title claims abstract description 11
- 238000013508 migration Methods 0.000 title claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000002604 ultrasonography Methods 0.000 claims description 29
- 210000004872 soft tissue Anatomy 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 4
- 230000008520 organization Effects 0.000 claims description 4
- 238000007476 Maximum Likelihood Methods 0.000 claims description 3
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000011551 log transformation method Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deep learning ultrasonic image denoising method based on migration and structure prior. Comprises the following steps: according to the priori knowledge that the ultrasonic image speckle noise obeys Gaussian distribution in a logarithmic domain, estimating the distribution parameters of the speckle noise in the logarithmic domain by using a traditional ultrasonic image filter and a parameter estimation method; constructing a network training data set by using the estimated distribution parameters and combining with natural images, wherein the network training data set is used for pre-training the denoising network; and respectively extracting ultrasonic image detail information before and after denoising by using the two image detail extraction networks, optimizing parameters of the denoising network by using the ultrasonic image detail information, and denoising the ultrasonic image by using the optimized denoising network. The invention can effectively remove speckle noise, has simple structure and high practicability.
Description
Technical Field
The invention relates to a medical ultrasonic speckle denoising technology, in particular to a deep learning ultrasonic image denoising method based on migration and structure prior.
Background
Due to the fact that the ultrasonic beam is scattered in the non-uniform microstructure of the human body, the beam interference is caused, and the acquired ultrasonic image naturally contains speckle noise. When the speckle noise of the ultrasonic image is removed, the consistent area is required to be smoothed, and the structural information of human tissues is also required to be protected, so that the subsequent image analysis is facilitated. The conventional ultrasonic image speckle noise suppression method is mainly classified into a speckle noise removal method based on a spatial domain, a speckle noise removal method based on a partial differential equation, and a speckle noise removal method based on a transform domain. The method for removing Speckle noise of an ultrasonic Image Based on a spatial domain mainly considers local information of the Image, and lacks consideration on the whole Image structure, which can result in the loss of important edge information (cope P, Hellier P, Kervran C, et al. non-local Means-Based speed filtration for ultrasonic Images [ J ]. IEEE Transactions on Image Processing,2009,18(10): 2221-. The method for removing speckle noise of an ultrasonic image based on a partial differential equation combines image detail information with a denoising frame to form the partial differential equation, but the solution of the result is an iterative process, a closed solution cannot be obtained, and the construction of the equation needs the guidance of prior knowledge (Eom K. speed Reduction in Ultrasound Images Using Nonisotropic adaptive filtering [ J ]. Ultrasound in Medicine & Biology,2011,37 (10)). The transform domain-based ultrasound image speckle noise removal method mainly converts an image into a Wavelet domain, and then performs threshold suppression on the domain to achieve noise removal, but causes discontinuity of sharp edges (Mustafa N, Li J P, Khan S A, et al. medical image de-noising scheme using a wave threshold with varied noises [ C ]// International Computer Conference on wave Active Media Technology & Information processing. IEEE, 2016).
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art and provide a deep learning ultrasonic image denoising method based on migration and structure prior.
The purpose of the invention is realized by at least one of the following technical solutions.
A deep learning ultrasonic image denoising method based on migration and structure prior comprises the following steps:
s1, estimating the distribution parameters of the speckle noise in the logarithmic domain by using a traditional ultrasonic image filter and a parameter estimation method according to the prior knowledge that the ultrasonic speckle noise obeys Gaussian distribution in the logarithmic domain;
s2, constructing a network training data set by using the estimated distribution parameters and combining with the natural images, and using the network training data set to pre-train the denoising network;
and S3, extracting the ultrasonic image detail information before and after denoising respectively by using the two image detail extraction networks, optimizing the parameters of the denoising network by using the two image detail extraction networks, and denoising the ultrasonic image by using the optimized denoising network.
Further, in step S1, since different conventional ultrasound image filters may consider different image characteristics when performing speckle reduction, when estimating speckle noise distribution parameters, it is necessary to filter the ultrasound image by using different conventional filters to obtain a noise-reduced ultrasound image.
Further, the conventional filters include Lee filters, Frost filters, bilateral filters, and non-local mean filters.
Further, in step S1, since the speckle noise follows gaussian distribution in the logarithmic domain, the noisy ultrasound image and the corresponding noise-removed ultrasound image need to be pairwise transformed into the logarithmic domain for subtraction to obtain the speckle noise data set, and then the distribution parameters thereof are calculated by using the maximum likelihood estimation method.
Further, in step S2, after obtaining the estimated speckle noise distribution parameter, adding gaussian distributed noise in a log domain in combination with the natural image data set PASCAL-VOC2012, and performing inverse log transformation to generate a training data set, and training the denoising network by using a random gradient descent algorithm, thereby solving the problem of lack of the labeling data training network.
Further, in step S2, the denoising network for removing speckle noise of the ultrasound image adopts a symmetric encoding and decoding structure, and denoising by using the symmetric structure can effectively enhance the filtering effect by using the shallow image;
the denoising network comprises a 13-layer structure, wherein the first 6 layers are called as an encoding structure, the subsequent 6 layers are called as a decoding structure, the last layer is an output layer and is a common convolution layer, and all convolution kernels are 3 x 3 in size; wherein every two convolution layers in the first 12 layers are a module, and the total number of the convolution layers is 6; the output of the first module is connected to the output of the data of the sixth module, the output of the second module is connected to the output of the fifth module, the output of the third module is connected to the output of the fourth module, the convolutional layers inside each module have the same number of channels, and the modules whose outputs are connected also have the same number of channels.
Further, in step S3, the speckle noise removal of the ultrasound image needs to ensure that the image detail information is protected while the noise is removed, but the natural image does not include the soft tissue structure information of the ultrasound image, and in order to effectively maintain the information of the ultrasound image soft tissue structure, the two image detail extraction networks are used to respectively extract the ultrasound image soft tissue structure information before and after the noise removal, and the noise removal network parameters are adjusted by means of a random gradient descent algorithm, so as to protect the details of the ultrasound image.
Further, the image detail extraction network selects VGG 16; the VGG16 is selected to be used as an image detail extraction network structure to acquire an ultrasound image structure prior, wherein the output result of the second pooling of the VGG16 is used as an organization prior to adjusting the denoising network parameters.
Compared with the prior art, the invention has the following advantages and effects:
1) the invention can combine the advantages of a plurality of traditional ultrasonic image filters to estimate the speckle noise distribution.
2) The invention utilizes the highly nonlinear function relation of deep learning to smooth the image consistency area and utilizes the image global information.
3) The invention extracts ultrasonic image organization structure information by using the two image detail extraction network structures to adjust the denoising network parameters, thereby strengthening the detail retention capability of the denoising network image.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a denoising network.
Fig. 2 is a schematic diagram of an embodiment of a denoising network according to an embodiment of the present invention.
Detailed Description
Specific embodiments of the present invention will be described in further detail below with reference to specific examples and drawings, but the embodiments of the present invention are not limited thereto.
Example (b):
a deep learning ultrasound image denoising method based on migration and structure prior, as shown in fig. 1, includes the following steps:
s1, estimating the distribution parameters of the speckle noise in the logarithmic domain by using a traditional ultrasonic image filter and a parameter estimation method according to the prior knowledge that the ultrasonic speckle noise obeys Gaussian distribution in the logarithmic domain;
because different traditional ultrasonic image filters can consider different image characteristics when performing speckle elimination, when estimating speckle noise distribution parameters, different traditional filters are required to be used for filtering ultrasonic images, and the ultrasonic images after the speckle elimination are obtained.
The conventional filters include Lee filters, Frost filters, bilateral filters, and non-local mean filters.
Since the speckle noise follows gaussian distribution in the logarithmic domain, the noisy ultrasound image and the corresponding denoising ultrasound image need to be transformed in pairs into the logarithmic domain for subtraction to obtain a speckle noise data set, and then the distribution parameters of the speckle noise data set are calculated by using a maximum likelihood estimation method.
S2, constructing a network training data set by using the estimated distribution parameters and combining with the natural images, and using the network training data set to pre-train the denoising network;
after the estimated speckle noise distribution parameters are obtained, Gaussian distribution noise is added in a log domain in combination with a natural image data set PASCAL-VOC2012, inverse log transformation is carried out to generate a training data set, a denoising network is trained by means of a random gradient descent algorithm, and the problem that the labeling data training network is lacked is solved.
The denoising network for removing the speckle noise of the ultrasonic image adopts a symmetrical coding and decoding structure, and denoising by using the symmetrical structure can effectively utilize the shallow image to enhance the filtering effect;
the denoising network comprises a 13-layer structure, wherein the first 6 layers are called as an encoding structure, the subsequent 6 layers are called as a decoding structure, the last layer is an output layer and is a common convolution layer, and all convolution kernels are 3 x 3 in size; wherein every two convolution layers in the first 12 layers are a module, and the total number of the convolution layers is 6; the output of the first module is connected to the output of the data of the sixth module, in this embodiment, the number of channels of all convolutional layers in the first and sixth modules is 32; the output of the second module is connected to the output of the fifth module, in this embodiment, the number of channels of all the convolutional layers in the second and fifth modules is 64, the output of the third module is connected to the output of the fourth module, in this embodiment, the number of channels of all the convolutional layers in the third and fourth modules is 128, and the number of channels of the last output layer is 1.
S3, extracting the ultrasonic image detail information before and after denoising respectively by using the two image detail extraction networks, optimizing the parameters of the denoising network by using the two image detail extraction networks, and denoising the ultrasonic image by using the optimized denoising network;
the ultrasonic image speckle noise removal needs to ensure that image detail information is protected while noise removal is carried out, however, natural images do not include soft tissue structure information of ultrasonic images, in order to effectively maintain the information of the ultrasonic image soft tissue structure, two image detail extraction networks are used for respectively extracting ultrasonic image soft tissue structure information before and after noise removal, noise removal network parameters are adjusted by means of a random gradient descent algorithm, and details of the ultrasonic images are protected.
The image detail extraction network selects VGG 16; the VGG16 is selected to be used as an image detail extraction network structure to acquire an ultrasound image structure prior, wherein the output result of the second pooling of the VGG16 is used as an organization prior to adjusting the denoising network parameters.
In this embodiment, the overall network structure is shown in fig. 2.
The present invention can be preferably realized as described above.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (8)
1. A deep learning ultrasonic image denoising method based on migration and structure prior is characterized by comprising the following steps:
s1, estimating the distribution parameters of the speckle noise in the logarithmic domain by using a traditional ultrasonic image filter and a parameter estimation method according to the prior knowledge that the ultrasonic speckle noise obeys Gaussian distribution in the logarithmic domain;
s2, constructing a network training data set by using the estimated distribution parameters and combining with the natural images, and using the network training data set to pre-train the denoising network;
and S3, extracting the ultrasonic image detail information before and after denoising respectively by using the two image detail extraction networks, optimizing the parameters of the denoising network by using the two image detail extraction networks, and denoising the ultrasonic image by using the optimized denoising network.
2. The method according to claim 1, wherein in step S1, different traditional ultrasound image filters are used to filter the ultrasound image when estimating speckle noise distribution parameters because different image characteristics are considered when performing speckle removal, so as to obtain the ultrasound image after speckle removal.
3. The method of claim 2, wherein the conventional filters include a Lee filter, a Frost filter, a bilateral filter and a non-local mean filter.
4. The method for denoising deep learning ultrasound images based on migration and structure prior as claimed in claim 1, wherein in step S1, since speckle noise follows gaussian distribution in logarithmic domain, the noisy ultrasound image and the corresponding denoising ultrasound image need to be transformed into logarithmic domain in pair for subtraction to obtain speckle noise data set, and then the distribution parameters are calculated by maximum likelihood estimation method.
5. The method as claimed in claim 1, wherein in step S2, after obtaining the estimated speckle noise distribution parameters, the method combines the natural image data set PASCAL-VOC2012 to add gaussian noise distribution in log domain and perform inverse log transformation to generate a training data set, and trains the denoising network by means of stochastic gradient descent algorithm.
6. The method for denoising deep learning ultrasound images based on migration and structure prior as claimed in claim 1, wherein in step S2, the denoising network for removing ultrasound image speckle noise adopts a symmetric coding and decoding structure, and denoising using the symmetric structure can effectively use shallow images to enhance filtering effect;
the denoising network comprises a 13-layer structure, wherein the first 6 layers are called as an encoding structure, the subsequent 6 layers are called as a decoding structure, the last layer is an output layer and is a common convolution layer, and all convolution kernels are 3 x 3 in size; wherein every two convolution layers in the first 12 layers are a module, and the total number of the convolution layers is 6; the output of the first module is connected to the output of the data of the sixth module, the output of the second module is connected to the output of the fifth module, the output of the third module is connected to the output of the fourth module, the convolutional layers inside each module have the same number of channels, and the modules whose outputs are connected also have the same number of channels.
7. The method according to claim 1, wherein in step S3, two image detail extraction networks are used to respectively extract the ultrasonic image soft tissue structure information before and after the noise removal, and the noise removal network parameters are adjusted by means of a stochastic gradient descent algorithm to protect the details of the ultrasonic image.
8. The deep learning ultrasound image denoising method based on migration and structure prior of claim 7, wherein the image detail extraction network selects VGG 16; the VGG16 is selected to be used as an image detail extraction network structure to acquire an ultrasound image structure prior, wherein the output result of the second pooling of the VGG16 is used as an organization prior to adjusting the denoising network parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911364562.3A CN111161180B (en) | 2019-12-26 | 2019-12-26 | Deep learning ultrasonic image de-noising method based on migration and structure priori |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911364562.3A CN111161180B (en) | 2019-12-26 | 2019-12-26 | Deep learning ultrasonic image de-noising method based on migration and structure priori |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111161180A true CN111161180A (en) | 2020-05-15 |
| CN111161180B CN111161180B (en) | 2023-09-26 |
Family
ID=70556695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911364562.3A Active CN111161180B (en) | 2019-12-26 | 2019-12-26 | Deep learning ultrasonic image de-noising method based on migration and structure priori |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111161180B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101396279A (en) * | 2007-09-29 | 2009-04-01 | 深圳市蓝韵实业有限公司 | Method of removing real-time ultrasound pattern speckle noise |
| CN103971346A (en) * | 2014-05-28 | 2014-08-06 | 西安电子科技大学 | SAR (Synthetic Aperture Radar) image spot-inhibiting method based on spare domain noise distribution constraint |
| US20170278546A1 (en) * | 2016-03-25 | 2017-09-28 | Samsung Electronics Co., Ltd. | Method and device for processing multimedia information |
| CN109859147A (en) * | 2019-03-01 | 2019-06-07 | 武汉大学 | A kind of true picture denoising method based on generation confrontation network noise modeling |
| CN109872288A (en) * | 2019-01-31 | 2019-06-11 | 深圳大学 | For the network training method of image denoising, device, terminal and storage medium |
-
2019
- 2019-12-26 CN CN201911364562.3A patent/CN111161180B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101396279A (en) * | 2007-09-29 | 2009-04-01 | 深圳市蓝韵实业有限公司 | Method of removing real-time ultrasound pattern speckle noise |
| CN103971346A (en) * | 2014-05-28 | 2014-08-06 | 西安电子科技大学 | SAR (Synthetic Aperture Radar) image spot-inhibiting method based on spare domain noise distribution constraint |
| US20170278546A1 (en) * | 2016-03-25 | 2017-09-28 | Samsung Electronics Co., Ltd. | Method and device for processing multimedia information |
| CN109872288A (en) * | 2019-01-31 | 2019-06-11 | 深圳大学 | For the network training method of image denoising, device, terminal and storage medium |
| CN109859147A (en) * | 2019-03-01 | 2019-06-07 | 武汉大学 | A kind of true picture denoising method based on generation confrontation network noise modeling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111161180B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114140353B (en) | Swin-Transformer image denoising method and system based on channel attention | |
| CN111028163A (en) | Convolution neural network-based combined image denoising and weak light enhancement method | |
| CN103854264A (en) | Improved threshold function-based wavelet transformation image denoising method | |
| CN107705260B (en) | The denoising system and calculation method of medical X-ray image | |
| CN110992295B (en) | Low-dose CT reconstruction method based on wavelet-RED convolution neural network | |
| CN112270654A (en) | Image denoising method based on multi-channel GAN | |
| CN107403416B (en) | NSCT-based medical ultrasonic image denoising method with improved filtering and threshold function | |
| KR102106073B1 (en) | Method and apparatus for processing images | |
| CN114240797A (en) | OCT image denoising method, device, equipment and medium | |
| CN104835127A (en) | Adaptive smooth filtering method | |
| Thanh | Medical images denoising method based on total variation regularization and anscombe transform | |
| CN110503608B (en) | Image denoising method based on multi-view convolutional neural network | |
| CN111047537A (en) | System for recovering details in image denoising | |
| Chawla et al. | Hybrid filtering technique for image denoising using artificial neural network | |
| CN111161180B (en) | Deep learning ultrasonic image de-noising method based on migration and structure priori | |
| CN116385312A (en) | Low-illumination image denoising method based on phase correlation | |
| Rashedi et al. | Noise filtering in ultrasound images using Gravitational Search Algorithm | |
| Kaur et al. | Image de-noising techniques: a review paper | |
| CN114387173A (en) | OCT image noise reduction method, electronic device and storage medium | |
| Du et al. | Dual tree complex wavelet transform and bayesian estimation based denoising of poission-corrupted x-ray images | |
| Qiaoman et al. | Application of adaptive median filter and wavelet transform to dongba manuscript images denoising | |
| Dhiman et al. | An improved threshold estimation technique for image denoising using wavelet thresholding techniques | |
| Saoji et al. | Speckle and rician noise removal from medical images and Ultrasound images | |
| Shin et al. | Image denoiser using convolutional neural network with deconvolution and modified residual network | |
| Zhao et al. | Approach of image denoising based on discrete multi-wavelet transform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |