CN113160072B - Robust self-adaptive frame correlation method and system based on image pyramid - Google Patents

Robust self-adaptive frame correlation method and system based on image pyramid Download PDF

Info

Publication number
CN113160072B
CN113160072B CN202110294761.2A CN202110294761A CN113160072B CN 113160072 B CN113160072 B CN 113160072B CN 202110294761 A CN202110294761 A CN 202110294761A CN 113160072 B CN113160072 B CN 113160072B
Authority
CN
China
Prior art keywords
image
pyramid
frame correlation
correlation
coefficient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110294761.2A
Other languages
Chinese (zh)
Other versions
CN113160072A (en
Inventor
肖梦楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jurong Medical Technology Hangzhou Co ltd
Original Assignee
Jurong Medical Technology Hangzhou Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jurong Medical Technology Hangzhou Co ltd filed Critical Jurong Medical Technology Hangzhou Co ltd
Priority to CN202110294761.2A priority Critical patent/CN113160072B/en
Publication of CN113160072A publication Critical patent/CN113160072A/en
Application granted granted Critical
Publication of CN113160072B publication Critical patent/CN113160072B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10132Ultrasound image

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of ultrasonic imaging, and particularly relates to a robust self-adaptive frame correlation method and a robust self-adaptive frame correlation system based on an image pyramid, wherein the robust self-adaptive frame correlation method based on the image pyramid comprises the following steps: s1, constructing a pyramid according to a noise image; s2, calculating a frame correlation coefficient template according to the top-level image of the pyramid; s3, carrying out scale transformation according to the coefficient template; s4, realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation; according to the invention, the frame correlation coefficient is calculated through the pyramid top layer image, so that the complexity of the algorithm can be greatly reduced; and smoothing the convolution kernel of each layer of image of the pyramid in the construction process so as to enable the image pyramid to have a good denoising effect.

Description

Robust self-adaptive frame correlation method and system based on image pyramid
Technical Field
The invention belongs to the technical field of ultrasonic imaging, and particularly relates to a robust self-adaptive frame correlation method and system based on an image pyramid.
Background
Medical ultrasound is a low-cost, non-invasive and real-time clinical examination means, however, the actual medical ultrasound image often has large noise, and the existence of the noise can reduce the signal-to-noise ratio and the contrast resolution of the image, and finally, the image quality is reduced.
The noise in the ultrasonic image mainly comes from two types, one is random noise caused by internal circuits, thermal effects and the like, and the other is speckle noise formed by coherent scattering of ultrasonic echoes. The existing denoising methods are mainly classified into three categories, the first category is a composite-based method, such as frequency composite, spatial composite and time composite. The second category is a noise model-based method, which has the disadvantage that the noise characteristics are too complex and the construction of the noise model is too idealized. The last category is based on post-processing methods, which can be divided into single-scale and multi-scale methods.
The frame correlation, i.e. the time compounding method, can realize better denoising effect on the basis of not increasing the system complexity remarkably. The basic model of frame correlation is: y is t =ay t-1 +(1-a)x t In the formula, x and y represent images before and after frame correlation, subscripts represent time, and α is a frame correlation coefficient.
The existing frame correlation methods mainly include two types, namely non-self-adaption and self-adaption, wherein the non-self-adaption is that alpha in a model is a fixed value, and the self-adaption is that alpha is calculated in real time according to images at different moments, and specifically, the self-adaption can be calculated based on two or more frames.
The existing self-adaptive frame correlation method is used for calculating the size of an original image, and the original image contains more noise before denoising, so that the correlation is not accurately calculated, and finally the frame correlation effect is not good; the existing solution is to carry out preprocessing such as spatial smoothing, digital filtering and the like on an image before frame correlation, but the method has the defect that the good balance between noise removal and image feature retention is difficult to achieve; therefore, it is necessary to improve this to overcome the disadvantages in practical applications.
Disclosure of Invention
Based on the above-mentioned shortcomings and drawbacks of the prior art, it is an object of the present invention to at least solve one or more of the above-mentioned problems of the prior art, in other words, to provide a robust adaptive frame correlation method and system based on image pyramid that meets one or more of the above-mentioned needs.
In order to achieve the purpose, the invention adopts the following technical scheme:
a robust adaptive frame correlation method based on an image pyramid comprises the following steps:
s1, constructing a pyramid according to a noise image;
s2, calculating a frame correlation coefficient template according to the top-level image of the pyramid;
s3, carrying out scale transformation according to the coefficient template;
and S4, realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation.
Preferably, the step S1 specifically includes:
s11, setting the number L of pyramid layers to be decomposed, and converting the original noise image I 0 Defined as layer 0;
s12, calculating a convolution kernel, and performing normalization processing on the kernel;
Figure GDA0003100816550000021
x=-3σ,-3σ+1,...,3σ-1,3σ
wherein σ is the standard deviation of the Gaussian kernel;
s13, initializing I =0, and carrying out pyramid ith layer image I i Is convolved with a kernel, and the kernel is a single-layer kernel,i.e. I i =conv(I i ,kernel);
S14, extracting I i The x-th row and the y-th column are taken as an image I of the I +1 th layer of the pyramid i+1 (ii) a Sequentially calculating the images of the next layer of the pyramid until the calculation of the L layer of the pyramid is completed, so as to complete the construction of the whole image pyramid;
s15, constructing corresponding image pyramids according to the previous frame and the current frame respectively, and recording the pyramid top layer images of the previous frame and the current frame as IP respectively t-1 And IP t
As a preferable scheme, the step S2 specifically includes:
s21, according to IP t-1 And IP t Calculating the correlation C of the current pixel;
s22, calculating a frame correlation coefficient a according to the correlation degree, and sequentially calculating frame correlation coefficients corresponding to the next pixel to obtain the frame correlation coefficients of all the pixels;
s23, forming a coefficient template A by the frame correlation coefficients of all the pixels, wherein A is mapped between 0 and 1.
Preferably, in step S21, the correlation is calculated by mean absolute difference, mean square error, root mean square error, correlation coefficient and similarity function.
Preferably, in step S22, the conversion between the correlation C and the frame correlation coefficient a is implemented by a decreasing function, and the decreasing function includes a linear function, a piecewise linear function, an exponential function, and a logarithmic function.
Preferably, the step S3 specifically includes:
s31, denoising the coefficient template A;
s32, transforming the denoised coefficient template into the size of a pyramid bottom layer image, wherein the transformed coefficient template is A'.
As a preferred scheme, the step S4 specifically includes:
and carrying out frame correlation processing on the transformed coefficient template A', wherein a frame correlation formula is as follows:
Figure GDA0003100816550000031
where the superscript plus f denotes the image after frame correlation and the subscript t denotes the time instant.
The invention also provides a robust self-adaptive frame correlation system based on the image pyramid, which comprises the following steps:
the building module is used for building a pyramid according to the noise image;
the first calculation module is used for calculating a frame correlation coefficient template according to the top-level image of the pyramid;
the transformation module is used for carrying out scale transformation according to the coefficient template;
and the processing module is used for realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation.
Preferably, the first calculation module includes:
a second calculation module for calculating according to IP t-1 And IP t Calculating the correlation C of the current pixel;
and the third calculation module is used for calculating the frame correlation coefficient a according to the correlation degree and sequentially calculating the frame correlation coefficient corresponding to the next pixel so as to obtain the frame correlation coefficients of all the pixels.
Preferably, the second calculating module calculates the correlation by using the mean absolute difference, the mean square error, the root mean square error, the correlation coefficient and the similarity function.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the frame correlation coefficient is calculated through the pyramid top-layer image, so that the complexity of the algorithm can be greatly reduced.
In the construction process of each layer of image of the pyramid, the smoothing of the convolution kernel is carried out, so that the image pyramid has a good denoising effect.
Drawings
Fig. 1 is a flowchart of a robust adaptive frame correlation method based on an image pyramid according to a first embodiment of the present invention;
fig. 2 is a structural diagram of a robust adaptive frame correlation system based on an image pyramid according to a second embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention, the following description will explain specific embodiments of the present invention with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
The first embodiment is as follows:
the embodiment provides a robust adaptive frame correlation method based on an image pyramid, as shown in fig. 1, including the following steps:
s1, constructing a pyramid according to a noise image;
s2, calculating a frame correlation coefficient template according to the top-level image of the pyramid;
s3, carrying out scale transformation according to the coefficient template;
and S4, carrying out frame correlation processing on the noise image after the scale transformation.
Specifically, step S1 constructs a pyramid from the noise image, specifically:
s11, setting pyramid layer number L =3 to be decomposed, and setting an original noise image I 0 Is defined as layer 0.
S12, calculating a Gaussian convolution kernel, wherein the calculation formula is as follows:
Figure GDA0003100816550000051
wherein, the first and the second end of the pipe are connected with each other,
Figure GDA0003100816550000052
represents rounding up, taking σ =1.5;
normalizing the kernel, namely enabling the sum of all elements of the kernel to be equal to 1;
s13, initializing I =0, and converting the image I of the ith layer i Convolved with kernel, formulated as:
I i =conv(I i ,kernel)。
s14, extracting I i The x row and the y column are taken as an image I of an I +1 layer i+1 (ii) a If I i If the number of rows M is even, x =2p, p =0,1,2,., M/2-1, otherwise
Figure GDA0003100816550000053
In the same way if I i Is even, y =2q, q =0,1,2, · M/2-1, otherwise · r>
Figure GDA0003100816550000054
Symbol->
Figure GDA0003100816550000055
Represents rounding down; sequentially calculating the images of the next layer of the pyramid until the calculation of the L layer of the pyramid is completed so as to complete the construction of the image pyramid;
s15, constructing corresponding image pyramids according to the previous frame and the current frame, performing the steps S13 to S14 on the current frame and the previous frame to complete construction of the corresponding image pyramids, and recording top layer images of the pyramids of the previous frame and the current frame as IP respectively t-1 And IP t
Step S2, calculating a frame correlation coefficient template according to the top-level image of the pyramid, specifically:
s21, according to IP t-1 And IP t Calculating the correlation C of the current pixel, calculating the correlation C by adopting Mean Square Error (MSE), and calculating the neighborhood size by 3 × 3, wherein the formula is as follows:
Figure GDA0003100816550000056
in the formula (x) 0 ,y 0 ) Is the current pixel position coordinate;
the correlation degree of the current pixel can be calculated through the average absolute difference, the mean square error, the root mean square error, the correlation coefficient and the similarity function;
s22, calculating a frame correlation coefficient a according to the correlation C, wherein the conversion between the correlation C and the frame correlation coefficient a is realized through a decreasing function, and the decreasing function comprises a linear function, a piecewise linear function, an exponential function and a logarithmic function;
the calculation formula of this embodiment is as follows:
a=e -C
calculating a frame correlation coefficient a corresponding to the next pixel according to the steps until the traversal of the whole pyramid top-layer image is completed so as to obtain the frame correlation coefficients of all the pixels, wherein the frame correlation coefficients a of all the pixels jointly form a coefficient template A;
s23, forming a coefficient template A by using the frame correlation coefficients of all the pixels, wherein A is mapped between 0 and 1; if the value of the coefficient template A is not between 0 and 1, the coefficient transformation of A is needed to be carried out, and the A is mapped to between 0 and 1.
Step S3, carrying out scale transformation according to the coefficient template, specifically:
s31, carrying out spatial smoothing or digital filtering operation on the coefficient template to further remove the influence of noise;
s32, transforming the denoised coefficient template into a pyramid bottom layer image I 0 The scaling may be implemented by interpolation, and the scaled coefficient template is marked as a'.
Step S4, realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation, specifically comprising the following steps:
according to the coefficient template A' after the scale transformation, the frame correlation operation of the noise image is realized, and in order to reduce the algorithm complexity, the image is subjected to block frame correlation, and the frame correlation is expressed by a formula as follows:
Figure GDA0003100816550000061
where the superscript plus f denotes the image after frame correlation and the subscript denotes the time instant.
Compared with the prior art, the embodiment has the following beneficial effects:
in the embodiment, the complexity of the algorithm can be reduced by calculating the frame correlation coefficient through the top-level image of the pyramid, and each layer of image of the pyramid is subjected to smoothing of a convolution kernel in the construction process, so that the image pyramid realizes the denoising function, and a good balance effect is achieved between denoising and image feature retention.
The second embodiment:
the embodiment provides a robust adaptive frame correlation system based on an image pyramid, as shown in fig. 2, including:
the building module 11 is used for building a pyramid according to the noise image;
the first calculating module 12 is configured to calculate a frame correlation coefficient template according to the top-level image of the pyramid;
a transformation module 13, configured to perform scale transformation according to the coefficient template;
and the processing module 14 is configured to implement frame correlation processing on the noise image according to the coefficient template after the scale transformation.
Further, the first calculation module 12 includes:
a second calculation module for calculating according to IP t-1 And IP t Calculating the correlation C of the current pixel;
and the third calculating module is used for calculating the frame correlation coefficient a according to the correlation degree and sequentially calculating the frame correlation coefficient corresponding to the next pixel so as to obtain the frame correlation coefficients of all the pixels.
Further, the second calculation module calculates the correlation degree by the mean absolute difference, the mean square error, the root mean square error, the correlation coefficient and the similarity function.
It should be noted that the robust adaptive frame correlation system based on the image pyramid provided in this embodiment corresponds to the first embodiment, and details are not repeated here.
Compared with the prior art, the system of the embodiment has the following beneficial effects:
the complexity of the algorithm can be reduced by calculating the frame correlation coefficient through the top image of the pyramid, and each layer of image of the pyramid can be smoothed by a convolution kernel in the construction process, so that the image pyramid realizes the denoising function, and a good balance effect is achieved between denoising and image feature retention.
The foregoing has outlined rather broadly the preferred embodiments and principles of the present invention and it will be appreciated that those skilled in the art may devise variations of the present invention that are within the spirit and scope of the appended claims.

Claims (6)

1. A robust adaptive frame correlation method based on an image pyramid is characterized by comprising the following steps:
s1, constructing a pyramid according to a noise image;
s2, calculating a frame correlation coefficient template according to the top-level image of the pyramid;
s3, carrying out scale transformation according to the coefficient template;
s4, realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation;
the step S1 specifically comprises the following steps:
s11, setting the pyramid layer number L to be decomposed, and converting the original noise image I 0 Defined as layer 0;
s12, calculating a convolution kernel, and carrying out normalization processing on the kernel;
Figure FDA0003840772630000011
x=-3σ,-3σ+1,...,3σ-1,3σ
wherein σ is the standard deviation of the Gaussian kernel;
s13, initializing I =0, and carrying out pyramid ith layer image I i Convolution with kernel, i.e. I i =conv(I i ,ker nel);
S14, extracting I i Image I with x row and y column as pyramid layer I +1 i+1 (ii) a Sequentially calculating the images of the next layer of the pyramid until the calculation of the L layer of the pyramid is completed, so as to complete the construction of the whole image pyramid;
s15, constructing corresponding image pyramids according to the previous frame and the current frame respectively,and respectively recording the pyramid top layer images of the previous frame and the current frame as IP t-1 And IP t
The step S2 specifically comprises the following steps:
s21 according to IP t-1 And IP t Calculating the correlation C of the current pixel;
s22, calculating a frame correlation coefficient a according to the correlation degree, and sequentially calculating frame correlation coefficients corresponding to the next pixel to obtain the frame correlation coefficients of all the pixels;
s23, forming a coefficient template A by using the frame correlation coefficients of all the pixels, wherein A is mapped between 0 and 1;
the step S3 specifically comprises the following steps:
s31, denoising the coefficient template A;
s32, transforming the denoised coefficient template into the size of a bottom image of the pyramid, wherein the transformed coefficient template is A';
the step S4 specifically comprises the following steps:
and carrying out frame correlation processing on the transformed coefficient template A', wherein a frame correlation formula is as follows:
Figure FDA0003840772630000021
where the superscript plus f denotes the image after the frame correlation, and the subscript t denotes the time instant.
2. The robust adaptive frame correlation method based on image pyramid as claimed in claim 1, wherein the correlation degree is calculated in step S21 by mean absolute difference, mean square error, root mean square error, correlation coefficient and similarity function.
3. The robust adaptive frame correlation method based on image pyramid as claimed in claim 1, wherein the conversion between the correlation degree C and the frame correlation coefficient a is implemented by a decreasing function in step S22, and the decreasing function includes a linear function, a piecewise linear function, an exponential function and a logarithmic function.
4. An image pyramid based robust adaptive frame correlation system for implementing an image pyramid based robust adaptive frame correlation method as claimed in any one of claims 1 to 3, wherein the image pyramid based robust adaptive frame correlation system comprises:
the building module is used for building a pyramid according to the noise image;
the first calculation module is used for calculating a frame correlation coefficient template according to the top-level image of the pyramid;
the transformation module is used for carrying out scale transformation according to the coefficient template;
and the processing module is used for realizing frame correlation processing on the noise image according to the coefficient template after the scale transformation.
5. The image pyramid based robust adaptive frame correlation system of claim 4, wherein the first computation module comprises:
a second calculation module for calculating according to IP t-1 And IP t Calculating the correlation C of the current pixel;
and the third calculation module is used for calculating the frame correlation coefficient a according to the correlation degree and sequentially calculating the frame correlation coefficient corresponding to the next pixel so as to obtain the frame correlation coefficients of all the pixels.
6. The image pyramid based robust adaptive frame correlation system of claim 5, wherein the second computation module computes the correlation by averaging absolute difference, mean square error, root mean square error, correlation coefficient and similarity function.
CN202110294761.2A 2021-03-19 2021-03-19 Robust self-adaptive frame correlation method and system based on image pyramid Active CN113160072B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110294761.2A CN113160072B (en) 2021-03-19 2021-03-19 Robust self-adaptive frame correlation method and system based on image pyramid

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110294761.2A CN113160072B (en) 2021-03-19 2021-03-19 Robust self-adaptive frame correlation method and system based on image pyramid

Publications (2)

Publication Number Publication Date
CN113160072A CN113160072A (en) 2021-07-23
CN113160072B true CN113160072B (en) 2023-04-07

Family

ID=76888183

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110294761.2A Active CN113160072B (en) 2021-03-19 2021-03-19 Robust self-adaptive frame correlation method and system based on image pyramid

Country Status (1)

Country Link
CN (1) CN113160072B (en)

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7856150B2 (en) * 2007-04-10 2010-12-21 Arcsoft, Inc. Denoise method on image pyramid
CN101527034B (en) * 2008-03-07 2013-01-23 深圳迈瑞生物医疗电子股份有限公司 Method and device for correlating adaptive frames
US8406569B2 (en) * 2009-01-19 2013-03-26 Sharp Laboratories Of America, Inc. Methods and systems for enhanced dynamic range images and video from multiple exposures
CN102908163B (en) * 2011-08-02 2016-01-20 深圳迈瑞生物医疗电子股份有限公司 A kind of frame correlation technique and device thereof and ultrasonic image-forming system
EA017302B1 (en) * 2011-10-07 2012-11-30 Закрытое Акционерное Общество "Импульс" Method of noise reduction of digital x-ray image series
US9262684B2 (en) * 2013-06-06 2016-02-16 Apple Inc. Methods of image fusion for image stabilization
CN104680495B (en) * 2015-03-17 2017-12-22 武汉倍尔生生物科技有限公司 The self-adaptive solution method of ultrasonoscopy
CN110047058B (en) * 2019-03-25 2021-04-30 杭州电子科技大学 Image fusion method based on residual pyramid

Also Published As

Publication number Publication date
CN113160072A (en) 2021-07-23

Similar Documents

Publication Publication Date Title
CN109919838B (en) Ultrasonic image super-resolution reconstruction method for improving outline definition based on attention mechanism
CN109410127B (en) Image denoising method based on deep learning and multi-scale image enhancement
CN110992292B (en) Enhanced low-rank sparse decomposition model medical CT image denoising method
CN113673590B (en) Rain removing method, system and medium based on multi-scale hourglass dense connection network
CN104933678B (en) A kind of image super-resolution rebuilding method based on image pixel intensities
JP5229575B2 (en) Image processing apparatus and method, and program
CN108765330B (en) Image denoising method and device based on global and local prior joint constraint
CN110796622B (en) Image bit enhancement method based on multi-layer characteristics of series neural network
CN111145102A (en) Synthetic aperture radar image denoising method based on convolutional neural network
CN108932699B (en) Three-dimensional matching harmonic filtering image denoising method based on transform domain
CN107194873B (en) Low-rank nuclear norm regular face image super-resolution method based on coupled dictionary learning
CN113727141B (en) Interpolation device and method for video frames
CN115170410A (en) Image enhancement method and device integrating wavelet transformation and attention mechanism
CN110634103A (en) Image demosaicing method based on generation of countermeasure network
JP2020086891A (en) Image processing device, image processing system, imaging device, image processing method, program, and storage medium
CN107945119B (en) Method for estimating correlated noise in image based on Bayer pattern
CN115082336A (en) SAR image speckle suppression method based on machine learning
CN109064394B (en) Image super-resolution reconstruction method based on convolutional neural network
CN113160072B (en) Robust self-adaptive frame correlation method and system based on image pyramid
CN114022392A (en) Serial attention-enhancing UNet + + defogging network for defogging single image
KR100698828B1 (en) An illumination reflectance model based image distortion elimination method
CN108876736A (en) A kind of image alias removing method based on FPGA
CN109447952B (en) Semi-reference image quality evaluation method based on Gabor differential box weighting dimension
CN103903239A (en) Video super-resolution reconstruction method and system based on sparse principal component analysis and continued fraction interpolation technology
CN116612032A (en) Sonar image denoising method and device based on self-adaptive wiener filtering and 2D-VMD

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant