WO2024066890A1 - 视频降噪方法及装置、电子设备及计算机可读存储介质 - Google Patents

视频降噪方法及装置、电子设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2024066890A1
WO2024066890A1 PCT/CN2023/115871 CN2023115871W WO2024066890A1 WO 2024066890 A1 WO2024066890 A1 WO 2024066890A1 CN 2023115871 W CN2023115871 W CN 2023115871W WO 2024066890 A1 WO2024066890 A1 WO 2024066890A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
resolution scale
scale
fusion
current resolution
Prior art date
Application number
PCT/CN2023/115871
Other languages
English (en)
French (fr)
Inventor
陈亚南
赵丙山
杨易华
周玮
Original Assignee
深圳市中兴微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中兴微电子技术有限公司 filed Critical 深圳市中兴微电子技术有限公司
Publication of WO2024066890A1 publication Critical patent/WO2024066890A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the field of video processing technology, and in particular to a video noise reduction method and device, an electronic device, and a computer-readable storage medium.
  • the present application provides a video denoising method, comprising: acquiring a video image, performing multi-scale decomposition on the video image to obtain a sub-band image of at least one resolution scale; and performing multi-scale fusion of a spatial domain filtering image and a temporal domain filtering image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image.
  • the present application also provides a video noise reduction device, the video noise reduction device comprising: a plurality of The scale decomposition module is configured to acquire a video image, perform multi-scale decomposition on the video image, and obtain a sub-band image of at least one resolution scale; the multi-scale fusion module is configured to perform multi-scale fusion on the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image.
  • the scale decomposition module is configured to acquire a video image, perform multi-scale decomposition on the video image, and obtain a sub-band image of at least one resolution scale
  • the multi-scale fusion module is configured to perform multi-scale fusion on the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image.
  • the present application also provides an electronic device, which is a physical device, and includes: a memory, a processor, and a program of the video noise reduction method stored in the memory and executable on the processor, wherein the program of the video noise reduction method is executed by the processor, so that the processor can implement the above-mentioned video noise reduction method.
  • the present application also provides a computer-readable storage medium, on which a program for implementing the video noise reduction method is stored.
  • the program for the video noise reduction method is executed by a processor, so that the processor implements the above-mentioned video noise reduction method.
  • FIG1 is a schematic diagram of a flow chart of a first embodiment of a video noise reduction method of the present application
  • FIG2 is a flow chart of a second embodiment of the video noise reduction method of the present application.
  • FIG3 is a schematic diagram of a process of performing spatiotemporal fusion based on motion estimation results in an embodiment of a video noise reduction method of the present application;
  • FIG4 is a schematic diagram of a process of weighted fusion of the spatiotemporal filtered fusion image at the current resolution scale and the prior denoised image based on edge detection in an embodiment of the video denoising method of the present application;
  • FIG5 is a flow chart of a third embodiment of the video noise reduction method of the present application.
  • FIG6 is a schematic diagram of a process of performing video image noise reduction in an embodiment of a video noise reduction method of the present application
  • FIG7 is a schematic diagram of a device for video noise reduction according to an embodiment of the present application.
  • FIG8 is a schematic diagram of the device structure of the hardware operating environment involved in the video noise reduction method in an embodiment of the present application.
  • an embodiment of the present application provides a video noise reduction method.
  • the video noise reduction method includes the following steps S10 to S20.
  • step S10 a video image is acquired, and multi-scale decomposition is performed on the video image to obtain a sub-band image of at least one resolution scale.
  • step S20 the spatial domain filtered images and the temporal domain filtered images corresponding to the sub-band images are multi-scale fused at the resolution scales to generate a target denoised image corresponding to the video image.
  • multi-scale decomposition refers to decomposing a video image from an original resolution scale into images of multiple resolution scales.
  • the video image of the original resolution scale may be downsampled multiple times at different downsampling intervals to achieve multi-scale decomposition and obtain images of different resolution scales.
  • the downsampling interval may be the interval distance between sampled pixels in the video image.
  • the sub-band image is an image obtained by multi-scale decomposition, for example, it may be an image obtained by downsampling the video image.
  • Multi-scale fusion refers to fusing the denoised images corresponding to the sub-band images of the video image at multiple resolution scales. These denoised images may be one or more of a spatial domain filtered image, a temporal domain filtered image, and a spatiotemporal domain filtered fusion image.
  • step S10 to step S20 include: acquiring a video image, performing multi-scale decomposition on the video image by downsampling the video image, and obtaining sub-band images at different resolution scales, wherein the sub-band images can form an image pyramid; performing spatial domain filtering on each of the sub-band images to obtain a spatial domain filtered image corresponding to each of the sub-band images; performing temporal domain filtering on each of the sub-band images to obtain a temporal domain filtered image corresponding to each of the sub-band images; performing motion estimation and edge detection on each of the sub-band images to obtain a motion estimation result and an edge map corresponding to each of the sub-band images; performing multi-scale fusion on each of the spatial domain filtered images and each of the temporal domain filtered images according to each of the motion estimation results and each of the edge maps to generate a target denoising map corresponding to the video image.
  • the embodiment of the present application uses the motion estimation result and edge map of the video image as fusion reference information, and performs multi-scale fusion of the spatial domain filtered image and the temporal domain filtered image of the video image at different resolution scales, which can control the denoising effect of the video image at different resolution scales, and realize a more refined spatiotemporal fusion denoising process, thereby improving the denoising effect of the video denoising.
  • the step of downsampling the video image includes: estimating the noise of the video image to obtain estimated noise, determining the downsampling frequency according to the noise amplitude of the estimated noise, for example, the larger the noise amplitude, the higher the downsampling frequency; and downsampling the video image according to the downsampling frequency.
  • the estimated noise can be obtained by high-pass filtering the video image, or by performing frame difference processing on the video image and the video noise reduction image of the previous frame.
  • the multi-scale fusion of the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image includes the following steps S21 to S25.
  • step S21 a spatial domain filtered image and a temporal domain filtered image at a current resolution scale are obtained.
  • step S22 if the current resolution scale is the minimum resolution scale, the spatial domain filtered image and the temporal domain filtered image at the current resolution scale are fused according to the motion estimation result of the sub-band image at the current resolution scale to obtain a denoised image at the current resolution scale.
  • step S23 if the current resolution scale is not the minimum resolution scale, a previous denoised image of a lower resolution scale is obtained.
  • step S24 based on the motion estimation result and edge map of the sub-band image at the current resolution scale, the prior denoised image, the spatial domain filtered image at the current resolution scale and the temporal domain filtered image are fused to obtain the denoised image at the current resolution scale.
  • step S25 the current resolution scale is updated to the next higher resolution scale, and the process returns to step S21: obtaining a spatial domain filtered image and a temporal domain filtered image at the current resolution scale, until a denoised image at the original resolution scale is obtained as the target denoised image.
  • steps S21 to S25 include: obtaining the current resolution scale
  • the sub-band image at the current resolution scale is subjected to spatial filtering and temporal filtering respectively to obtain the spatial filtered image and the temporal filtered image at the current resolution scale; it is determined whether the current resolution scale is the minimum resolution scale among the resolution scales; if the current resolution scale is the minimum resolution scale, motion estimation is performed on the sub-band image at the current resolution scale to obtain the motion estimation result; according to the first fusion weight matrix determined by the motion estimation result, the spatial filtered image and the temporal filtered image at the current resolution scale are weightedly fused to obtain the denoised image at the current resolution scale; if the current resolution scale is not the minimum resolution scale, the previous denoised image at the previous lower resolution scale is obtained, and motion estimation and edge detection are performed on the sub-band image at the current resolution scale respectively.
  • the embodiment of the present application performs spatiotemporal fusion of the spatial filtered image and the temporal filtered image based on the fusion weight corresponding to the motion estimation result of each pixel in the sub-band image, thereby improving the denoising effect of the video image, and for the video image that is not at the minimum resolution scale, determines the edge strength corresponding to each pixel in the sub-band image according to the edge map, and then fuses the video denoised image at a lower resolution scale and the spatiotemporal fused filtered image at the current resolution scale according to the fusion weight corresponding to the edge strength of each pixel, thereby realizing the fusion of denoised images at different resolution scales.
  • the embodiment of the present application realizes multi-scale fusion of the temporal filtered image and the spatial filtered image, and can control the denoising effect of the video image at different resolution scales, thereby realizing a more refined video image denoising process and improving the video denoising effect.
  • the first fusion weight matrix includes the first fusion weight and the second fusion weight corresponding to each pixel in the sub-band image.
  • the first fusion weight is used to weight the pixel value corresponding to the pixel point in the time domain filtered image
  • the second fusion weight is used to weight the pixel value corresponding to the pixel point in the spatial domain filtered image.
  • the embodiment of the present application provides a video denoising method.
  • the embodiment of the present application obtains a video image, performs multi-scale decomposition on the video image, and obtains a sub-band image of at least one resolution scale; performs multi-scale fusion on the spatial domain filtered image and the time domain filtered image corresponding to each of the sub-band images at each of the resolution scales, and generates a target denoised image corresponding to the video image to perform video denoising.
  • the embodiment of the present application decomposes each video image into multiple sub-band images of different resolution scales, and then performs multi-scale fusion on the spatial domain filter images and the temporal domain filter images corresponding to the multiple sub-band images of different resolution scales, thereby achieving spatiotemporal fusion denoising in a multi-resolution scale manner for each video image.
  • multiple target denoised images obtained by spatiotemporal fusion denoising are obtained to form a denoised video, so that the denoised video obtained can minimize the loss of image information and remove video noise, and can control the effect of video denoising at different resolution scales, thereby achieving a more refined video denoising process, thereby improving the effect of video denoising.
  • the same or similar contents as those in the above embodiment can be referred to the above description and will not be described in detail later.
  • the prior denoised image, the spatial domain filtered image and the temporal domain filtered image at the current resolution scale are fused according to the motion estimation result and the edge map of the sub-band image at the current resolution scale to obtain the denoised image at the current resolution scale, including the following steps S241 to S244.
  • step S241 motion estimation is performed on the sub-band image of the current resolution scale to obtain a motion estimation result.
  • step S242 according to the motion estimation result, the spatial domain filtered image and the temporal domain filtered image at the current resolution scale are fused to obtain a spatial-temporal fused filtered image at the current resolution scale.
  • step S243 edge detection is performed on the sub-band image of the current resolution scale. Get the edge map.
  • step S244 the spatiotemporal filtering fusion image at the current resolution scale and the prior denoised image are fused according to the edge map to obtain a denoised image at the current resolution scale.
  • motion estimation can detect the motion vector of each pixel in the sub-band image, and the motion difference of each pixel compared with the previous frame image can be determined based on the motion vector.
  • the larger the motion difference the more accurate the temporal feature of the pixel can be, and the pixel value corresponding to the pixel in the temporal filtered image should be given a larger fusion weight.
  • the smaller the motion difference the more accurate the spatial feature of the pixel can be, and the pixel value corresponding to the pixel in the spatial filtered image should be given a larger fusion weight.
  • Edge detection can detect the edge strength of each pixel in the sub-band image.
  • the stronger the edge strength, the more obvious the detail feature of the image at the pixel, and the pixel value in the higher resolution image should be given a larger fusion weight.
  • the motion estimation result includes the motion difference corresponding to each pixel point in the sub-band image
  • the edge map includes the edge intensity corresponding to each pixel point in the sub-band image.
  • steps S241 to S244 include: performing motion estimation on the sub-band image at the current resolution scale to obtain the motion difference corresponding to each pixel in the sub-band image; determining the first fusion weight matrix corresponding to each pixel according to the size of the motion difference corresponding to each pixel; performing weighted fusion on the spatial domain filter image and the temporal domain filter image at the current resolution scale according to the first fusion weight matrix to obtain the spatiotemporal fusion filter image at the current resolution scale; performing edge detection on the sub-band image at the current resolution scale to obtain the edge strength corresponding to each pixel in the sub-band image; determining the second fusion weight matrix corresponding to each pixel according to the size of the edge strength corresponding to each pixel; and fusing the spatiotemporal domain filter fusion image at the current resolution scale with the prior denoised image according to the second fusion weight matrix to obtain the denoised image at the current resolution scale.
  • the embodiment of the present application realizes multi-scale fusion of temporal domain filter images and spatial domain filter
  • the fusing of the spatial domain filtered image and the temporal domain filtered image at the current resolution scale according to the motion estimation result includes the following steps A10 to A20.
  • step A10 a first fusion weight matrix at the current resolution scale is determined according to the motion estimation result.
  • step A20 weighted fusion is performed on the spatial domain filtered image and the temporal domain filtered image at the current resolution scale according to the first fusion weight matrix.
  • the first fusion weight matrix includes first fusion weights and second fusion weights corresponding to each pixel point in the sub-band image.
  • the first fusion weights are used to weight the pixel values corresponding to the pixel points in the spatial domain filtered image
  • the second fusion weights are used to weight the pixel values corresponding to the pixel points in the temporal domain filtered image.
  • step A10 to step A20 include: determining the first fusion weight and the second fusion weight corresponding to each pixel point according to the size of the motion difference corresponding to each pixel point in the sub-band image.
  • the larger the motion difference the smaller the first fusion weight and the larger the second fusion weight; the smaller the motion difference, the larger the first fusion weight and the smaller the second fusion weight; according to each of the first fusion weights and each of the second fusion weights, weighted aggregation is performed on the pixel value corresponding to each pixel point in the spatial domain filtered image and the pixel value corresponding to each pixel point in the temporal domain filtered image to obtain a spatiotemporal fusion filtered image.
  • the weighted aggregation method can be weighted summation or weighted average.
  • the embodiment of the present application realizes setting the weighted value of spatiotemporal fusion based on the motion difference obtained by motion estimation, and can choose to retain more spatial domain features or temporal features of each pixel point during spatiotemporal fusion. That is, when the motion difference of the pixel points is large, by setting a larger second fusion weight and a smaller first fusion weight, more temporal features of the pixel points that can be accurately described are selected for spatiotemporal fusion.
  • the first fusion weight matrix includes at least a spatial filter map The first fusion weight corresponding to the pixel value in the time domain filtered image and the second fusion weight corresponding to the pixel value in the time domain filtered image.
  • Determining the first fusion weight matrix at the current resolution scale according to the motion estimation result includes the following steps A11 to A13.
  • step A11 based on the motion estimation result, it is determined whether each pixel in the sub-band image at the current resolution scale is a moving pixel or a stationary pixel.
  • step A12 if the pixel point is a moving pixel point, a first preset weight is obtained as a second fusion weight corresponding to the pixel point, and a second preset weight is obtained as a first fusion weight corresponding to the pixel point.
  • step A13 if the pixel point is a stationary pixel point, a first preset weight is obtained as a first fusion weight corresponding to the pixel point, and a second preset weight is obtained as a second fusion weight corresponding to the pixel point; the second preset weight is less than the first preset weight.
  • step A11 to step A13 include: judging whether each pixel in the sub-band image at the current resolution scale is a moving pixel or a stationary pixel according to the motion difference corresponding to each pixel; if the pixel is a moving pixel, obtaining a first preset weight with a larger value as the second fusion weight corresponding to the pixel and obtaining a second preset weight with a smaller value as the first fusion weight corresponding to the pixel; if the pixel is a stationary pixel, obtaining a first preset weight with a larger value as the first fusion weight corresponding to the pixel and obtaining a second preset weight with a smaller value as the second fusion weight corresponding to the pixel.
  • the first preset weight may be set to 1, and the second preset weight may be set to 0.
  • the spatial domain filtered image at the i-th resolution scale is the time domain filtered image at the i-th resolution scale, is the sub-band image at the i-th resolution scale, ⁇ is the first fusion weight, and 1- ⁇ is the second fusion weight Heavy,
  • the spatiotemporal fusion filtered image at the i-th resolution scale "x" represents the weighting, and the 0th layer represents the original resolution scale corresponding to the video image.
  • the step of fusing the spatiotemporal filtered fusion image at the current resolution scale with the prior denoised image according to the edge map comprises the following steps B10 to B20.
  • step B10 a second fusion weight matrix at the current resolution scale is determined according to the edge map.
  • step B20 weighted fusion is performed on the spatiotemporal filtered fusion image at the current resolution scale and the prior denoised image according to the second fusion weight matrix.
  • the second fusion weight matrix includes a third fusion weight and a fourth fusion weight corresponding to each pixel in the sub-band image.
  • the third fusion weight is used to weight the pixel value of the pixel in the spatiotemporal fusion filtered image at the current resolution scale
  • the fourth fusion weight is used to weight the pixel value of the pixel in a prior denoised image at a previous lower resolution scale.
  • the prior denoised image is a denoised image at a previous lower resolution scale.
  • step B10 to step B20 include: determining the third fusion weight and the fourth fusion weight corresponding to each pixel point according to the size of the edge strength corresponding to each pixel point in the sub-band image.
  • the greater the edge strength the greater the third fusion weight and the smaller the fourth fusion weight; the smaller the edge strength, the smaller the third fusion weight and the larger the fourth fusion weight; according to each of the third fusion weights and each of the fourth fusion weights, weighted fusion is performed on the pixel values corresponding to each pixel point in the spatiotemporal fusion filter image and the pixel values corresponding to each pixel point in the prior denoised image to obtain a target denoised image.
  • the embodiment of the present application realizes setting the fusion weight of multi-scale fusion according to the edge strength of each pixel point obtained by edge detection, and can select to retain high-resolution image features with more detail features and low-resolution image features with less noise during multi-scale fusion. That is, when the edge strength is high, in order to accurately describe the pixel point to the greatest extent, by setting a larger third fusion weight and a smaller fourth fusion weight, mainly select the image features in the spatiotemporal fusion filtering image at the current resolution scale with more detail features for multi-scale fusion.
  • the fourth fusion weight is large to mainly select the image features in the previous denoised image with less noise at the previous lower resolution scale for multi-scale fusion. This can not only ensure the accurate description of the detailed features in the image, but also reduce the noise in the image to the greatest extent, thereby improving the denoising effect of the video image.
  • the second fusion weight matrix includes at least a third fusion weight corresponding to a pixel value in the spatiotemporal filtering fusion image and a fourth fusion weight corresponding to a pixel value in the prior denoised image.
  • Determining the second fusion weight matrix at the current resolution scale according to the edge map includes the following steps B11 to B12.
  • step B11 the edge strength corresponding to each pixel point in the edge map is determined.
  • step B12 the third fusion weight and the fourth fusion weight corresponding to each pixel point are determined according to the size of each edge strength; the size of the third fusion weight is proportional to the size of the edge strength, and the size of the fourth fusion weight is inversely proportional to the size of the edge strength.
  • step B11 to step B12 include: determining the edge strength corresponding to each pixel in the edge map, querying the third fusion weight of each pixel at each edge strength according to the first mapping relationship between the edge strength and the third fusion weight; querying the fourth fusion weight of each pixel at each edge strength according to the second mapping relationship between the edge strength and the fourth fusion weight.
  • the size of the third fusion weight is proportional to the size of the edge strength
  • the size of the fourth fusion weight is inversely proportional to the size of the edge strength.
  • the spatiotemporal fusion filter image at the i-th resolution scale "x" represents the weight, is the sub-band image at the i-th resolution scale, is the prior denoised image at the i+1th resolution scale, that is, the prior denoised image at the previous lower resolution scale, is the denoised image at the i-th resolution scale, ⁇ is the third fusion weight, 1- ⁇ is the fourth fusion weight, and the 0th layer represents the original resolution scale corresponding to the video image.
  • the present application embodiment provides a multi-scale image fusion method, that is, The method comprises the steps of: performing motion estimation on a sub-band image at a current resolution scale to obtain a motion estimation result; fusing a spatial domain filtered image and a temporal domain filtered image at the current resolution scale according to the motion estimation result to obtain a spatiotemporal fused filtered image at the current resolution scale; performing edge detection on the sub-band image at the current resolution scale to obtain an edge map; and fusing the spatiotemporal filtered fused image at the current resolution scale and the prior denoised image according to the edge map to obtain a denoised image at the current resolution scale.
  • the motion difference corresponding to the pixel point when the motion difference corresponding to the pixel point is large, more time domain features of the pixel point that can be accurately described are selected for time-space domain fusion, and when the motion difference of the pixel point is small, more spatial domain features of the pixel point that can be accurately described are selected for time-space domain fusion, which can improve the accuracy of each pixel value in the time-space domain fusion filter image to describe the pixel point, and can improve the denoising effect of time-space domain fusion denoising; on the other hand, based on the edge detection result, when the edge intensity of the pixel point is high, in order to accurately describe the pixel point to the greatest extent, mainly select the image features in the time-space domain fusion filter image with more detail features at the current resolution scale for multi-scale fusion, and when the edge intensity is low, in order to reduce the noise on the pixel point to the greatest extent, mainly select the image features in the previous denoised image with less noise at the
  • the motion estimation of the sub-band image of the current resolution scale includes the following steps C10 to C30.
  • step C10 a previous motion estimation result corresponding to a sub-band image of a previous smaller resolution scale is obtained.
  • step C20 a motion estimation search range corresponding to the sub-band image of the current resolution scale is determined according to the previous motion estimation result.
  • step C30 motion estimation is performed on the sub-band image of the current resolution scale according to the motion estimation search range.
  • the motion estimation method can be based on the motion estimation method of the block matching algorithm.
  • the specific process is: each frame of the image sequence is divided into a plurality of interconnected Non-overlapping blocks are considered, and the displacement of all pixels in the block is assumed to be the same. Then, for each macroblock to a given specific search range of the reference frame, a block that is most similar to the current block is found according to a certain block matching criterion, i.e., a matching block.
  • the relative displacement between the matching block and the current block is the motion vector.
  • the prior motion estimation result is the motion estimation result corresponding to the sub-band image of the previous smaller resolution scale.
  • step C10 to step C30 include: obtaining a previous motion estimation result corresponding to a sub-band image of a previous smaller resolution scale, in some embodiments, the previous motion estimation result includes the image position of a previous matching block corresponding to a block in the sub-band image of a previous smaller resolution scale; locating a motion estimation search range corresponding to each block in the sub-band image of the current resolution scale according to the image position of the previous matching block; and performing motion estimation on the sub-band image of the current resolution scale by searching for matching blocks corresponding to each block in the sub-band image of the current resolution scale in each of the motion estimation search ranges.
  • the embodiment of the present application realizes the reuse of the previous motion estimation result of the previous lower resolution image to perform motion estimation on the sub-band image at the current resolution scale, which can reduce the motion estimation search range corresponding to each block in the sub-band image at the current resolution scale, thereby improving the efficiency and accuracy of motion estimation.
  • the step of locating the motion estimation search range corresponding to each block in the sub-band image of the current resolution scale according to the image position of the previous matching block includes: locating the image position range of the previous matching block at the current resolution scale according to the image position of the previous matching block in the previous sub-band image with a lower resolution; locating the motion estimation search range corresponding to each block in the sub-band image of the current resolution scale according to the image position range.
  • the image position coordinate range (A, B) can be used as the central area to divide the image position range of a preset size as the motion estimation search range corresponding to each block in the sub-band image of the current resolution scale.
  • the denoised image at the i-th resolution scale of the previous frame is the motion estimation result at the i+1th layer resolution scale, is the sub-band image at the i-th resolution scale, is the prior denoised image at the i+1th resolution scale, is the motion estimation result at the i-th resolution scale, is the spatial domain filtered image at the i-th resolution scale, is the time domain filtered image at the i-th resolution scale,
  • the spatiotemporal fusion filtered image at the i-th resolution scale is a denoised image at the i-th layer resolution scale.
  • the i+1-th layer resolution scale is lower than the i-th layer resolution scale.
  • the 0th layer represents the original resolution scale corresponding to the video image.
  • the embodiment of the present application provides a method for motion estimation at multiple resolution scales, that is, obtaining a previous motion estimation result corresponding to a sub-band image at a previous smaller resolution scale; determining a motion estimation search range corresponding to a sub-band image at a current resolution scale according to the previous motion estimation result; and performing motion estimation on the sub-band image at the current resolution scale according to the motion estimation search range.
  • the previous motion estimation result at the previous resolution scale can be reused.
  • the motion estimation search range for motion estimation at the current resolution scale can be reduced, thereby improving the efficiency of motion estimation.
  • motion estimation is first performed at a low resolution scale. In the process of reusing the motion estimation result at the low resolution scale to the motion estimation at the high resolution scale, the efficiency of motion estimation at a large scale can be improved, thereby improving the efficiency of motion estimation at multiple resolution scales and improving the efficiency of video denoising.
  • the present application also provides a video denoising device, which includes: a multi-scale decomposition module 10, configured to obtain a video image, perform multi-scale decomposition on the video image, and obtain a sub-band image of at least one resolution scale; a multi-scale fusion module 20, configured to perform multi-scale fusion on the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image.
  • a multi-scale decomposition module 10 configured to obtain a video image, perform multi-scale decomposition on the video image, and obtain a sub-band image of at least one resolution scale
  • a multi-scale fusion module 20 configured to perform multi-scale fusion on the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales to generate a target denoised image corresponding to the video image.
  • the multi-scale fusion module 20 is further configured to: obtain a spatial domain filtered image and a temporal domain filtered image at a current resolution scale; if the current resolution scale is a minimum resolution scale, then according to the motion estimation result of the sub-band image at the current resolution scale, fuse the spatial domain filtered image and the temporal domain filtered image at the current resolution scale to obtain a denoised image at the current resolution scale; if the current resolution scale is not the minimum resolution scale, then obtain a prior denoised image at a lower resolution scale; according to the motion estimation result and edge map of the sub-band image at the current resolution scale, fuse the prior denoised image, the spatial domain filtered image and the temporal domain filtered image at the current resolution scale to obtain the current resolution scale.
  • the denoised image at the original resolution scale is obtained; the current resolution scale is updated to the next higher resolution scale, and the execution step is returned to: the spatial domain filtered image and the temporal domain filtered image at the current resolution scale are obtained, until the denoised image at the original resolution scale is obtained as the target denoised image.
  • the multi-scale fusion module 20 is further configured to: perform motion estimation on the sub-band image of the current resolution scale to obtain a motion estimation result; based on the motion estimation result, fuse the spatial domain filtered image and the temporal domain filtered image at the current resolution scale to obtain a spatiotemporal fused filtered image at the current resolution scale; perform edge detection on the sub-band image of the current resolution scale to obtain an edge map; based on the edge map, fuse the spatiotemporal domain filtered fused image at the current resolution scale with the prior denoised image to obtain a denoised image at the current resolution scale.
  • the multi-scale fusion module 20 is further configured to: determine a first fusion weight matrix at the current resolution scale based on the motion estimation result; and perform weighted fusion of the spatial domain filtered image and the temporal domain filtered image at the current resolution scale based on the first fusion weight matrix.
  • the first fusion weight matrix includes at least a first fusion weight corresponding to a pixel value in a spatial domain filtering image and a second fusion weight corresponding to a pixel value in a temporal domain filtering image
  • the multi-scale fusion module 20 is further configured to: determine whether each pixel in the sub-band image at the current resolution scale is a moving pixel or a stationary pixel according to the motion estimation result; if the pixel is a moving pixel, obtain a first preset weight as the second fusion weight corresponding to the pixel and obtain a second preset weight as the first fusion weight corresponding to the pixel; if the pixel is a stationary pixel, obtain the first preset weight as the first fusion weight corresponding to the pixel and obtain the second preset weight as the second fusion weight corresponding to the pixel; the second preset weight is less than the first preset weight.
  • the multi-scale fusion module 20 is further configured to: determine a second fusion weight matrix at the current resolution scale based on the edge map; and perform weighted fusion of the spatiotemporal domain filtered fusion image at the current resolution scale and the prior denoised image based on the second fusion weight matrix.
  • the second fusion weight matrix includes at least spatiotemporal filtering
  • the third fusion weight corresponding to the pixel value in the fused image and the fourth fusion weight corresponding to the pixel value in the prior denoised image, the multi-scale fusion module 20 is also configured to: determine the edge strength corresponding to each pixel point in the edge map; determine the third fusion weight and the fourth fusion weight corresponding to each pixel point according to the size of each edge strength; the size of the third fusion weight is proportional to the size of the edge strength, and the size of the fourth fusion weight is inversely proportional to the size of the edge strength.
  • the multi-scale fusion module 20 is further configured to: obtain a previous motion estimation result corresponding to a sub-band image of a previous smaller resolution scale; determine a motion estimation search range corresponding to a sub-band image of a current resolution scale based on the previous motion estimation result; and perform motion estimation on the sub-band image of the current resolution scale based on the motion estimation search range.
  • the video noise reduction device provided by the present application adopts the video noise reduction method in the above embodiment to solve the technical problem of poor video noise reduction effect.
  • the beneficial effects of the video noise reduction device provided by the embodiment of the present application are the same as the beneficial effects of the video noise reduction method provided by the above embodiment, and other technical features in the video noise reduction device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
  • An embodiment of the present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video noise reduction method in the above-mentioned embodiment 1.
  • FIG8 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers.
  • PDAs personal digital assistants
  • PADs Portable multimedia players
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs and desktop computers.
  • the electronic device shown in FIG8 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which may be configured to process the data according to a program stored in a read-only memory (ROM) or
  • ROM read-only memory
  • the processing device, ROM and RAM are connected to each other through a bus.
  • the input/output (I/O) interface is also connected to the bus.
  • the following systems can be connected to the I/O interface: input devices (e.g., touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.); output devices (e.g., liquid crystal displays (LCDs), speakers, vibrators, etc.); storage devices (e.g., magnetic tapes, hard disks, etc.); and communication devices.
  • input devices e.g., touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.
  • output devices e.g., liquid crystal displays (LCDs), speakers, vibrators, etc.
  • storage devices e.g., magnetic tapes, hard disks, etc.
  • communication devices can allow electronic devices to communicate with other devices wirelessly or by wire to exchange data.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM.
  • the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the electronic device provided by the present application adopts the video noise reduction method in the above embodiment to solve the technical problem of poor video noise reduction effect.
  • the beneficial effects of the electronic device provided by the embodiment of the present application are the same as the beneficial effects of the video noise reduction method provided by the above embodiment, and other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
  • This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the video noise reduction method in the above-mentioned embodiment 1.
  • the computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device.
  • the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.
  • the computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.
  • the computer-readable storage medium carries one or more programs.
  • the electronic device obtains a video image, performs multi-scale decomposition on the video image, and obtains a sub-band image of at least one resolution scale; performs multi-scale fusion of the spatial domain filtered image and the temporal domain filtered image corresponding to each of the sub-band images at each of the resolution scales, and generates a target denoised image corresponding to the video image.
  • Computer program code for performing operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, either a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (eg, through the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g, AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • modules involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a module does not limit the unit itself in some cases.
  • the computer-readable storage medium provided by the present application stores computer-readable program instructions for executing the above-mentioned video noise reduction method, which solves the technical problem of poor video noise reduction effect.
  • the beneficial effects of the computer-readable storage medium provided in the embodiment of the present application are the same as the beneficial effects of the video noise reduction method provided in the above-mentioned embodiment, and will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Picture Signal Circuits (AREA)

Abstract

本申请公开了视频降噪方法及装置、电子设备及计算机可读存储介质，所述视频降噪方法包括：获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；以及将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。

Description

视频降噪方法及装置、电子设备及计算机可读存储介质
相关申请的交叉引用
本申请要求于2022年9月30日提交的中国专利申请NO.202211209506.4的优先权，该中国专利申请的内容通过引用的方式整体合并于此。
技术领域
本申请涉及视频处理技术领域，尤其涉及视频降噪方法及装置、电子设备及计算机可读存储介质。
背景技术
随着图像显示设备的发展，人们对于高质量、高清晰的视频图像信息要求越来越高，但现实中的视频图像在数字化和传输过程中常会受到噪声干扰，影响视频图像的质量和清晰度，因此如何有效地抑制视频图像中的噪声至关重要。目前通常是根据每张视频图像的图像特征信息，来选择性地采用时域滤波或者空域滤波的方式对单张图像进行降噪，从而每张视频图像对应的降噪图像可以组成时空域融合降噪的降噪视频，但是不论是时域滤波还是空域滤波均会使得每张视频图像产生信息丢失，从而影响视频降噪的效果。
发明内容
本申请提供一种视频降噪方法，包括：获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；以及将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
本申请还提供一种视频降噪装置，所述视频降噪装置包括：多 尺度分解模块，配置为获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；多尺度融合模块，配置为将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
本申请还提供一种电子设备，所述电子设备为实体设备，所述电子设备包括：存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的所述视频降噪方法的程序，所述视频降噪方法的程序被处理器执行，使得所述处理器可实现上述的视频降噪方法。
本申请还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有实现视频降噪方法的程序，所述视频降噪方法的程序被处理器执行，使得所述处理器实现上述的视频降噪方法。
附图说明
图1为本申请视频降噪方法第一实施例的流程示意图；
图2为本申请视频降噪方法第二实施例的流程示意图；
图3为本申请视频降噪方法一实施例中基于运动估计结果进行时空域融合的流程示意图；
图4为本申请视频降噪方法一实施例中基于边缘检测将对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行加权融合的流程示意图；
图5为本申请视频降噪方法第三实施例的流程示意图；
图6为本申请视频降噪方法一实施例中进行视频图像降噪的流程示意图；
图7为本申请视频降噪一实施例的装置示意图；以及
图8为本申请实施例中视频降噪方法涉及的硬件运行环境的设备结构示意图。
具体实施方式
为使本申请的上述目的、特征和优点能够更加明显易懂，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清 楚、完整地描述。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动的前提下所获得的所有其它实施例，均属于本申请保护的范围。
参照图1，本申请实施例提供一种视频降噪方法，在本申请视频降噪方法的第一实施例中，所述视频降噪方法包括如下步骤S 10至S20。
在步骤S10中，获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像。
在步骤S20中，将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
在本实施例中，需要说明的是，多尺度分解是指将视频图像从原始分辨率尺度分解为多个分辨率尺度的图像，例如可以以不同的下采样间隔，对原始分辨率尺度的视频图像进行多次下采样，从而实现多尺度分解，得到不同分辨率尺度的图像，下采样间隔可以为在视频图像中采样的像素点之间的间隔距离。子带图像为多尺度分解得到的图像，例如可以为对视频图像进行下采样得到的图像。多尺度融合是指将视频图像在多个分辨率尺度下的子带图像对应的降噪图像进行融合，这些降噪图像可以为空域滤波图像、时域滤波图像和时空域滤波融合图像中的一种或者多种。
作为一种示例，步骤S10至步骤S20包括：获取视频图像，通过对所述视频图像进行下采样，对所述视频图像进行多尺度分解，得到不同分辨率尺度下的子带图像，这些子带图像可以组成图像金字塔；分别对各所述子带图像进行空域滤波，得到各所述子带图像对应的空域滤波图像；分别对各所述子带图像进行时域滤波，得到各所述子带图像对应的时域滤波图像；分别对各所述子带图像进行运动估计和边缘检测，得到各所述子带图像对应的运动估计结果和边缘图；根据各所述运动估计结果和各所述边缘图，将各所述空域滤波图像和各所述时域滤波图像进行多尺度融合，生成所述视频图像对应的目标降噪图 像，以进行视频降噪。本申请实施例实现了以视频图像的运动估计结果和边缘图为融合参考信息，将视频图像在不同分辨率尺度下的空域滤波图像和时域滤波图像进行多尺度融合，可以在不同分辨率尺度下控制视频图像的降噪效果，实现了更加精细化地时空域融合降噪过程，因此可以提升视频降噪的降噪效果。
作为一种示例，对所述视频图像进行下采样的步骤包括：对所述视频图像进行噪声估计，得到估计噪声，根据估计噪声的噪声幅度，确定下采样频率，例如可以设置噪声幅度越大，则下采样频率越高；根据所述下采样频率，对所述视频图像进行下采样。估计噪声可以通过对视频图像进行高通滤波得到，也可以将视频图像和上一帧的视频降噪图像进行帧差处理得到。
在一些实施例中，所述将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像，包括如下步骤S21至S25。
在步骤S21中，获取当前分辨率尺度下的空域滤波图像和时域滤波图像。
在步骤S22中，若当前分辨率尺度为最小分辨率尺度，则依据当前分辨率尺度下的子带图像的运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像。
在步骤S23中，若当前分辨率尺度不为最小分辨率尺度，则获取上一更低分辨率尺度的在先降噪图像。
在步骤S24中，依据当前分辨率尺度下的子带图像的运动估计结果和边缘图，对所述在先降噪图像、当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像。
在步骤S25中，将所述当前分辨率尺度更新为下一更高分辨率尺度，并返回执行步骤S21：获取当前分辨率尺度下的空域滤波图像和时域滤波图像，直至得到原始分辨率尺度下的降噪图像作为所述目标降噪图像。
作为一种示例，步骤S21至步骤S25包括：获取当前分辨率尺 度下的子带图像，分别对当前分辨率尺度下的子带图像进行空域滤波和时域滤波，得到当前分辨率尺度下的空域滤波图像和时域滤波图像；判断当前分辨率尺度是否为各分辨率尺度中的最小分辨率尺度，若当前分辨率尺度为最小分辨率尺度，则对当前分辨率尺度下的子带图像进行运动估计，得到运动估计结果，根据运动估计结果确定的第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合，得到当前分辨率尺度下的降噪图像；若当前分辨率尺度不为最小分辨率尺度，则获取上一更低分辨率尺度的在先降噪图像，分别对当前分辨率尺度下的子带图像进行运动估计和边缘检测，得到运动估计结果和边缘图，根据运动估计结果确定的第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合，得到时空域融合滤波图像，根据边缘图确定的第二融合权重矩阵，对当前分辨率尺度下的时空域融合滤波图像和上一更低分辨率尺度的在先降噪图像进行加权融合，得到当前分辨率尺度下的降噪图像；将所述当前分辨率尺度更新为下一更高分辨率尺度，并返回执行步骤：获取当前分辨率尺度下的空域滤波图像和时域滤波图像，直至得到原始分辨率尺度下的降噪图像作为所述目标降噪图像，在一些实施例中，所述原始分辨率尺度对应所述视频图像的原始分辨率。本申请实施例依据对子带图像中每个像素点的运动估计结果对应的融合权重，对空域滤波图像和时域滤波图像的时空域融合，从而提升视频图像降噪效果，并对于不为最小分辨率尺度的视频图像，根据边缘图确定子带图像中每个像素点对应的边缘强度，从而根据各个像素点的边缘强度对应的融合权重，对更低分辨率尺度的视频降噪图像和当前分辨率尺度下时空域融合滤波图像进行融合，实现了将不同分辨率尺度下的降噪图像进行融合，所以本申请实施例实现了对时域滤波图像和空域滤波图像进行多尺度融合，可以在不同分辨率尺度上控制视频图像的降噪效果，因此实现了更加精细化地视频图像降噪过程，可以提升视频降噪的效果。
在本实施例中，需要说明的是，所述第一融合权重矩阵包括子带图像中各像素点对应的第一融合权重和第二融合权重，在一些实施 例中，所述第一融合权重用于为像素点在时域滤波图像中对应的像素值进行加权的权重，所述第二融合权重用于为像素点在空域滤波图像中对应的像素值进行加权的权重。
本申请实施例提供了一种视频降噪方法，相比于现有技术中根据每张视频图像的图像特征信息，来选择性地采用时域滤波或者空域滤波的方式对单张图像进行降噪，从而使得每张视频图像对应的降噪图像可以组成时空域融合降噪的降噪视频的技术手段，本申请实施例获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像，以进行视频降噪。本申请实施例针对于每一张视频图像，将视频图像分解为多个不同分辨率尺度的子带图像，然后对多个不同分辨率尺度的子带图像对应的空域滤波图像和时域滤波图像进行多尺度融合，实现了针对于每张视频图像以多分辨率尺度的方式进行时空域融合降噪，最后得到的多张时空域融合降噪得到的目标降噪图像以组成降噪视频，使得得到的降噪视频可以在最大程度上减少图像信息丢失以及去除视频噪声，且可以在不同分辨率尺度上控制视频降噪的效果，实现了更加精细化地视频降噪过程，因此提升了视频降噪的效果。
进一步地，参照图2，在本申请另一实施例中，与上述实施例相同或相似的内容，可以参考上文介绍，后续不再赘述。所述依据当前分辨率尺度下的子带图像的运动估计结果和边缘图，对所述在先降噪图像、当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像，包括如下步骤S241至S244。
在步骤S241中，对当前分辨率尺度的子带图像进行运动估计，得到运动估计结果。
在步骤S242中，根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的时空域融合滤波图像。
在步骤S243中，对当前分辨率尺度的子带图像进行边缘检测， 得到边缘图。
在步骤S244中，根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，得到当前分辨率尺度下的降噪图像。
在本实施例中，需要说明的是，运动估计可以检测出子带图像中每个像素点的运动向量，根据运动向量可以确定每个像素点相比于上一帧图像的运动差异，运动差异越大，则表明该像素点的时域特征更能准确描述该像素点，应当给予像素点在时域滤波图像中对应的像素值更大的融合权重，运动差异越小，则表明该像素点的空域特征更能准确描述该像素点，应当给予像素点在空域滤波图像中对应的像素值更大的融合权重；边缘检测可以检测出子带图像中每个像素点的边缘强度，边缘强度越强，则表明图像在该像素点处的细节特征越明显，应当给予像素点在更高分辨率的图像中的像素值更大的融合权重，边缘强度越小，表明图像在该像素点处的细节特征越不明显，而由于低分辨率图像中的噪声信息更少，应当给予像素点在更低分辨率的图像中的像素值更大的融合权重。
作为一种示例，所述运动估计结果包括子带图像中各像素点对应的运动差异，所述边缘图包括子带图像中各像素点对应的边缘强度。
作为一种示例，步骤S241至步骤S244包括：对当前分辨率尺度下的子带图像进行运动估计，得到子带图像中各像素点对应的运动差异；根据各像素点对应的运动差异的大小，确定各像素点对应的第一融合权重矩阵；根据第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合，得到当前分辨率尺度下的时空域融合滤波图像；对当前分辨率尺度的子带图像进行边缘检测，得到子带图像中各像素点对应的边缘强度；根据各像素点对应的边缘强度的大小，确定各像素点对应的第二融合权重矩阵；根据第二融合权重矩阵，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，得到当前分辨率尺度下的降噪图像。本申请实施例实现了结合运动估计和边缘检测，将不同分辨率尺度下的时域滤波图像和空域滤波图像进行多尺度融合。
在一些实施例中，所述根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，包括如下步骤A10至A20。
在步骤A10中，根据所述运动估计结果，确定所述当前分辨率尺度下的第一融合权重矩阵。
在步骤A20中，根据所述第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合。
作为一种示例，所述第一融合权重矩阵包括子带图像中各像素点对应的第一融合权重和第二融合权重，在一些实施例中，所述第一融合权重用于为像素点在空域滤波图像中对应的像素值进行加权，所述第二融合权重用于为像素点在时域滤波图像中对应的像素值进行加权。
作为一种示例，步骤A10至步骤A20包括：根据子带图像中各像素点对应的运动差异的大小，确定各像素点对应的第一融合权重和第二融合权重，在一些实施例中，所述运动差异越大，则所述第一融合权重越小且所述第二融合权重越大，所述运动差异越小，则所述第一融合权重越大且所述第二融合权重越小；根据各所述第一融合权重和各所述第二融合权重，对各像素点在空域滤波图像中对应的像素值和各像素点在时域滤波图像中对应的像素值进行加权聚合，得到时空域融合滤波图像，在一些实施例中，加权聚合的方式可以为加权求和或者加权平均等。本申请实施例实现了依据运动估计得到的运动差异，设置时空域融合的加权值，可以在时空域融合时选择保留每个像素点更多的空域特征还是时域特征，也即在像素点的运动差异较大时，通过设置更大的第二融合权重以及更小的第一融合权重，来更多选择能准确描述的像素点的时域特征进行时空域融合，在像素点的运动差异较小时，通过设置更小的第二融合权重以及更大的第一融合权重，来更多选择能准确描述的像素点的空域特征进行时空域融合，这样可以提升时空域融合滤波图像中各像素值描述像素点的准确度，因此可以提升视频图像的降噪效果。
在一些实施例中，所述第一融合权重矩阵至少包括空域滤波图 像中像素值对应的第一融合权重和时域滤波图像中像素值对应的第二融合权重。
所述根据所述运动估计结果，确定所述当前分辨率尺度下的第一融合权重矩阵，包括如下步骤A11至A13。
在步骤A11中，根据所述运动估计结果，分别判断当前分辨率尺度下的子带图像中各像素点是运动像素点还是静止像素点。
在步骤A12中，若所述像素点为运动像素点，则获取第一预设权重作为所述像素点对应的第二融合权重以及获取第二预设权重作为所述像素点对应的第一融合权重。
在步骤A13中，若所述像素点为静止像素点，则获取第一预设权重作为所述像素点对应的第一融合权重以及获取第二预设权重作为所述像素点对应的第二融合权重；所述第二预设权重小于所述第一预设权重。
作为一种示例，步骤A11至步骤A13包括：根据各像素点对应的运动差异，分别判断当前分辨率尺度下的子带图像中各像素点是运动像素点还是静止像素点；若所述像素点为运动像素点，则获取取值更大的第一预设权重作为所述像素点对应的第二融合权重以及获取取值更小的第二预设权重作为所述像素点对应的第一融合权重；若所述像素点为静止像素点，则获取取值更大的第一预设权重作为所述像素点对应的第一融合权重以及获取取值更小的第二预设权重作为所述像素点对应的第二融合权重。本申请实施例中在依据运动估计结果进行时空域融合时，无需根据运动差异，一一计算每张子带图像中每个像素点对应的融合权重，而是直接进行权重赋予，因此可以提升时空域融合的效率，从而提升视频降噪的效率。
作为一种示例，所述第一预设权重可以设置为1，所述第二预设权重可以设置为0。
参照图3，图3为本申请一实施例中基于运动估计结果进行时空域融合的流程示意图，在一些实施例中，为第i层分辨率尺度下的空域滤波图像，为第i层分辨率尺度下的时域滤波图像，为第i层分辨率尺度下的子带图像，β为第一融合权重，1-β为第二融合权 重，第i层分辨率尺度下的时空域融合滤波图像，“x”表示加权，第0层表示视频图像对应的原始分辨率尺度。
在一些实施例中，所述根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，包括如下步骤B10至B20。
在步骤B10中，根据所述边缘图，确定所述当前分辨率尺度下的第二融合权重矩阵。
在步骤B20中，根据所述第二融合权重矩阵，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行加权融合。
作为一种示例，所述第二融合权重矩阵包括子带图像中各像素点对应的第三融合权重和第四融合权重，在一些实施例中，所述第三融合权重用于为像素点在当前分辨率尺度下的时空域融合滤波图像中的像素值进行加权，所述第四融合权重用于为像素点在上一更低分辨率尺度下的在先降噪图像中的像素值进行加权，在一些实施例中，所述在先降噪图像为上一更低分辨率尺度下的降噪图像。
作为一种示例，步骤B10至步骤B20包括：根据子带图像中各像素点对应的边缘强度的大小，确定各像素点对应的第三融合权重和第四融合权重，在一些实施例中，所述边缘强度越大，则所述第三融合权重越大且所述第四融合权重越小，所述边缘强度越小，则所述第三融合权重越小且所述第四融合权重越大；根据各所述第三融合权重和各所述第四融合权重，对各像素点在时空域融合滤波图像中对应的像素值和各像素点在所述在先降噪图像中对应的像素值进行加权融合，得到目标降噪图像。本申请实施例实现了依据边缘检测得到的各像素点的边缘强度，设置多尺度融合的融合权重，可以在多尺度融合时选择保留细节特征更多的高分辨率图像特征和噪声更少的低分辨率图像特征，也即在边缘强度较高时，为了能够最大程度上准确描述该像素点，通过设置更大的第三融合权重以及更小的第四融合权重，来主要选择细节特征更多的当前分辨率尺度下的时空域融合滤波图像中的图像特征进行多尺度融合，在边缘强度较低时，为了能够最大程度上降低该像素点上的噪声，通过设置更小的第三融合权重以及更 大的第四融合权重，来主要选择噪声更少的上一更低分辨率尺度下的在先降噪图像中的图像特征进行多尺度融合，这样既可保证准确描述图像中的细节特征，又能在最大程度下降低图像中的噪声，因此可以提升视频图像的降噪效果。
在一些实施例中，所述第二融合权重矩阵至少包括时空域滤波融合图像中像素值对应的第三融合权重和在先降噪图像中像素值对应的第四融合权重。
所述根据所述边缘图，确定所述当前分辨率尺度下的第二融合权重矩阵，包括如下步骤B11至B12。
在步骤B11中，确定所述边缘图中各像素点对应的边缘强度。
在步骤B12中，根据各所述边缘强度的大小，确定各所述像素点对应的第三融合权重和第四融合权重；所述第三融合权重的大小与所述边缘强度的大小成正比，所述第四融合权重的大小与所述边缘强度的大小成反比。
作为一种示例，步骤B11至步骤B12包括：确定所述边缘图中各像素点对应的边缘强度，根据边缘强度与第三融合权重的第一映射关系，查询各像素点在各所述边缘强度下的第三融合权重；根据边缘强度与第四融合权重的第二映射关系，查询各像素点在各所述边缘强度下的第四融合权重。在一些实施例中，所述第三融合权重的大小与所述边缘强度的大小成正比，所述第四融合权重的大小与所述边缘强度的大小成反比。
参照图4，图4为本申请一实施例中基于边缘检测将对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行加权融合的流程示意图，在一些实施例中，第i层分辨率尺度下的时空域融合滤波图像，“x”表示加权，为第i层分辨率尺度下的子带图像，为第i+1层分辨率尺度下的在先降噪图像，也即为上一更低分辨率尺度下的在先降噪图像，为第i层分辨率尺度下的降噪图像，α为第三融合权重，1-α为第四融合权重，第0层表示视频图像对应的原始分辨率尺度。
本申请实施例提供了一种多尺度图像融合方法，也即对当前分 辨率尺度的子带图像进行运动估计，得到运动估计结果；根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的时空域融合滤波图像；对当前分辨率尺度的子带图像进行边缘检测，得到边缘图；根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，得到当前分辨率尺度下的降噪图像。本申请实施例中一方面基于运动估计结果，在像素点对应的运动差异较大时更多选择能准确描述的像素点的时域特征进行时空域融合，在像素点的运动差异较小时，更多选择能准确描述的像素点的空域特征进行时空域融合，这样可以提升时空域融合滤波图像中各像素值描述像素点的准确度，可以提升时空域融合降噪的降噪效果；另一方面基于边缘检测结果，在像素点的边缘强度较高时，为了能够最大程度上准确描述该像素点，来主要选择细节特征更多的当前分辨率尺度下的时空域融合滤波图像中的图像特征进行多尺度融合，在边缘强度较低时，为了能够最大程度上降低该像素点上的噪声，来主要选择噪声更少的上一更低分辨率尺度下的在先降噪图像中的图像特征进行多尺度融合，这样既可保证准确描述图像中的细节特征，又能在最大程度下降低图像中的噪声，可以提升多尺度融合降噪的降噪效果。因此本申请实施例提升视频图像的降噪效果。
进一步地，参照图5，在本申请另一实施例中，与上述实施例相同或相似的内容，可以参考上文介绍，后续不再赘述。所述对当前分辨率尺度的子带图像进行运动估计，包括如下步骤C10至C30。
在步骤C10中，获取上一更小分辨率尺度的子带图像对应的在先运动估计结果。
在步骤C20中，根据所述在先运动估计结果，确定当前分辨率尺度的子带图像对应的运动估计搜索范围。
在步骤C30中，根据所述运动估计搜索范围，对当前分辨率尺度的子带图像进行运动估计。
在本实施例中，需要说明的是，运动估计的方法可以基于块匹配算法的运动估计法，具体过程为：将图像序列的每一帧分成许多互 不重叠的块，并认为块内所有像素的位移量都相同，然后对每个宏块到参考帧某一给定特定搜索范围内，根据一定的块匹配准则找出与当前块最相似的块，即匹配块，匹配块与当前块的相对位移即为运动矢量。所述在先运动估计结果为上一更小分辨率尺度的子带图像对应的运动估计结果。
作为一种示例，步骤C10至步骤C30包括：获取上一更小分辨率尺度的子带图像对应的在先运动估计结果，在一些实施例中，所述在先运动估计结果包括上一更小分辨率尺度的子带图像中块对应的在先匹配块所处的图像位置；根据在先匹配块所处的图像位置，定位当前分辨率尺度的子带图像中各块对应的运动估计搜索范围；通过分别在各所述运动估计搜索范围中搜索当前分辨率尺度的子带图像中各块对应的匹配块，对当前分辨率尺度的子带图像进行运动估计。本申请实施例实现了复用上一更低分辨率图像的在先运动估计结果，来对当前分辨率尺度下的子带图像进行运动估计，可以降低当前分辨率尺度下的子带图像中各块对应的运动估计搜索范围，因此可以提升运动估计的效率与精度。
作为一种示例，所述根据在先匹配块所处的图像位置，定位当前分辨率尺度的子带图像中各块对应的运动估计搜索范围的步骤包括：根据在先匹配块在上一更低分辨率的子带图像中所处的图像位置，定位在先匹配块在当前分辨率尺度下的图像位置范围；根据所述图像位置范围，定位当前分辨率尺度的子带图像中各块对应的运动估计搜索范围。例如假设在先匹配块在上一更低分辨率的子带图像中所处的图像位置坐标为(x，y)，该图像位置坐标(x，y)在当前分辨率尺度下对应的图像位置坐标范围(A，B)，则可以图像位置坐标范围(A，B)为中心区域，划分预设大小的图像位置范围作为当前分辨率尺度的子带图像中各块对应的运动估计搜索范围。
参照图6，图6为本申请一实施例中进行视频图像降噪的流程示意图，在一些实施例中，为上一帧第i层分辨率尺度下的降噪图像，为第i+1层分辨率尺度下的运动估计结果，为第i层分辨率尺度下的子带图像，为第i+1层分辨率尺度下的在先降噪图像， 为第i层分辨率尺度下的运动估计结果，为第i层分辨率尺度下的空域滤波图像，为第i层分辨率尺度下的时域滤波图像，第i层分辨率尺度下的时空域融合滤波图像，为第i层分辨率尺度下的降噪图像，在一些实施例中，第i+1层的分辨率尺度低于第i层的分辨率尺度，第0层表示视频图像对应的原始分辨率尺度。
本申请实施例提供了一种多分辨率尺度下的运动估计方法，也即获取上一更小分辨率尺度的子带图像对应的在先运动估计结果；根据所述在先运动估计结果，确定当前分辨率尺度的子带图像对应的运动估计搜索范围；根据所述运动估计搜索范围，对当前分辨率尺度的子带图像进行运动估计。本申请实施例中可以复用上一分辨率尺度下的在先运动估计结果，利用上一分辨率尺度下的在先运动估计结果，可以降低当前分辨率尺度下进行运动估计的运动估计搜索范围，因此可以提升运动估计的效率，且本申请实施例中先在低分辨率尺度下进行运动估计，在将低分辨率尺度下的运动估计结果复用至高分辨率尺度下的运动估计过程中，可以提升大尺度下进行运动估计的效率，从而提升在多分辨率尺度下进行运动估计的效率，提升视频降噪的效率。
为实现上述目的，参照图7，本申请还提供一种视频降噪装置，所述视频降噪装置包括：多尺度分解模块10，配置为获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；多尺度融合模块20，配置为将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
可选地，所述多尺度融合模块20还被配置为：获取当前分辨率尺度下的空域滤波图像和时域滤波图像；若当前分辨率尺度为最小分辨率尺度，则依据当前分辨率尺度下的子带图像的运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像；若当前分辨率尺度不为最小分辨率尺度，则获取上一更低分辨率尺度的在先降噪图像；依据当前分辨率尺度下的子带图像的运动估计结果和边缘图，对所述在先降噪图像、当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨 率尺度下的降噪图像；将所述当前分辨率尺度更新为下一更高分辨率尺度，并返回执行步骤：获取当前分辨率尺度下的空域滤波图像和时域滤波图像，直至得到原始分辨率尺度下的降噪图像作为所述目标降噪图像。
在一些实施例中，所述多尺度融合模块20还被配置为：对当前分辨率尺度的子带图像进行运动估计，得到运动估计结果；根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的时空域融合滤波图像；对当前分辨率尺度的子带图像进行边缘检测，得到边缘图；根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，得到当前分辨率尺度下的降噪图像。
在一些实施例中，所述多尺度融合模块20还被配置为：根据所述运动估计结果，确定所述当前分辨率尺度下的第一融合权重矩阵；根据所述第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合。
在一些实施例中，所述第一融合权重矩阵至少包括空域滤波图像中像素值对应的第一融合权重和时域滤波图像中像素值对应的第二融合权重，所述多尺度融合模块20还被配置为：根据所述运动估计结果，分别判断当前分辨率尺度下的子带图像中各像素点是运动像素点还是静止像素点；若所述像素点为运动像素点，则获取第一预设权重作为所述像素点对应的第二融合权重以及获取第二预设权重作为所述像素点对应的第一融合权重；若所述像素点为静止像素点，则获取第一预设权重作为所述像素点对应的第一融合权重以及获取第二预设权重作为所述像素点对应的第二融合权重；所述第二预设权重小于所述第一预设权重。
在一些实施例中，所述多尺度融合模块20还被配置为：根据所述边缘图，确定所述当前分辨率尺度下的第二融合权重矩阵；根据所述第二融合权重矩阵，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行加权融合。
在一些实施例中，所述第二融合权重矩阵至少包括时空域滤波 融合图像中像素值对应的第三融合权重和在先降噪图像中像素值对应的第四融合权重，所述多尺度融合模块20还被配置为：确定所述边缘图中各像素点对应的边缘强度；根据各所述边缘强度的大小，确定各所述像素点对应的第三融合权重和第四融合权重；所述第三融合权重的大小与所述边缘强度的大小成正比，所述第四融合权重的大小与所述边缘强度的大小成反比。
在一些实施例中，所述多尺度融合模块20还被配置为：获取上一更小分辨率尺度的子带图像对应的在先运动估计结果；根据所述在先运动估计结果，确定当前分辨率尺度的子带图像对应的运动估计搜索范围；根据所述运动估计搜索范围，对当前分辨率尺度的子带图像进行运动估计。
本申请提供的视频降噪装置，采用上述实施例中的视频降噪方法，解决了视频降噪效果差的技术问题。与现有技术相比，本申请实施例提供的视频降噪装置的有益效果与上述实施例提供的视频降噪方法的有益效果相同，且该视频降噪装置中的其他技术特征与上述实施例方法公开的特征相同，在此不做赘述。
本申请实施例提供一种电子设备，电子设备包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行上述实施例一中的视频降噪方法。
下面参考图8，其示出了适于用来实现本公开实施例的电子设备的结构示意图。本公开实施例中的电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等的移动终端以及诸如数字TV、台式计算机等的固定终端。图8示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备可以包括处理装置(例如中央处理器、图形处理器等)，其可以根据存储在只读存储器(ROM)中的程序或 者从存储装置加载到随机访问存储器(RAM)中的程序而执行各种适当的动作和处理。在RAM中，还存储有电子设备操作所需的各种程序和数据。处理装置、ROM以及RAM通过总线彼此相连。输入/输出(I/O)接口也连接至总线。
通常，以下***可以连接至I/O接口：输入装置(例如触摸屏、触摸板、键盘、鼠标、图像传感器、麦克风、加速度计、陀螺仪等)；输出装置(例如液晶显示器(LCD)、扬声器、振动器等)；存储装置(例如磁带、硬盘等)；以及通信装置。通信装置可以允许电子设备与其他设备进行无线或有线通信以交换数据。虽然图中示出了具有各种***的电子设备，但是应理解的是，本公开的实施例并不要求实施或具备所有示出的***，可以替代地实施或具备更多或更少的***。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置从网络上被下载和安装，或者从存储装置被安装，或者从ROM被安装。在该计算机程序被处理装置执行时，执行本公开实施例的方法中限定的上述功能。
本申请提供的电子设备，采用上述实施例中的视频降噪方法，解决了视频降噪效果差的技术问题。与现有技术相比，本申请实施例提供的电子设备的有益效果与上述实施例提供的视频降噪方法的有益效果相同，且该电子设备中的其他技术特征与上述实施例方法公开的特征相同，在此不做赘述。
应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式的描述中，具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。
本实施例提供一种计算机可读存储介质，具有存储在其上的计算机可读程序指令，计算机可读程序指令用于执行上述实施例一中的视频降噪的方法。
本申请实施例提供的计算机可读存储介质例如可以是U盘，但不限于电、磁、光、电磁、红外线、或半导体的***、***或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本实施例中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行***、***或者器件使用或者与其结合使用。计算机可读存储介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
上述计算机可读存储介质可以是电子设备中所包含的；也可以是单独存在，而未装配入电子设备中。
上述计算机可读存储介质承载有一个或者多个程序，当上述一个或者多个程序被电子设备执行时，使得电子设备：获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smal ltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中， 远程计算机可以通过任意种类的网络(局域网(LAN)或广域网(WAN))连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本申请各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块可以通过软件的方式实现，也可以通过硬件的方式来实现。模块的名称在某种情况下并不构成对该单元本身的限定。
本申请提供的计算机可读存储介质，存储有用于执行上述视频降噪方法的计算机可读程序指令，解决了视频降噪效果差的技术问题。与现有技术相比，本申请实施例提供的计算机可读存储介质的有益效果与上述实施例提供的视频降噪方法的有益效果相同，在此不做赘述。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利处理范围内。

Claims (11)

  1. 一种视频降噪方法，包括：
    获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；以及
    将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
  2. 如权利要求1所述视频降噪方法，其中，所述将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像，包括：
    获取当前分辨率尺度下的空域滤波图像和时域滤波图像；
    若当前分辨率尺度为最小分辨率尺度，则依据当前分辨率尺度下的子带图像的运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像；
    若当前分辨率尺度不为最小分辨率尺度，则获取上一更低分辨率尺度的在先降噪图像；
    依据当前分辨率尺度下的子带图像的运动估计结果和边缘图，对所述在先降噪图像、当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像；以及
    将所述当前分辨率尺度更新为下一更高分辨率尺度，并返回执行步骤：获取当前分辨率尺度下的空域滤波图像和时域滤波图像，直至得到原始分辨率尺度下的降噪图像作为所述目标降噪图像。
  3. 如权利要求2所述视频降噪方法，其中，所述依据当前分辨率尺度下的子带图像的运动估计结果和边缘图，对所述在先降噪图像、当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的降噪图像，包括：
    对当前分辨率尺度的子带图像进行运动估计，得到运动估计结 果；
    根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，得到当前分辨率尺度下的时空域融合滤波图像；
    对当前分辨率尺度的子带图像进行边缘检测，得到边缘图；以及
    根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，得到当前分辨率尺度下的降噪图像。
  4. 如权利要求3所述视频降噪方法，其中，所述根据所述运动估计结果，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行融合，包括：
    根据所述运动估计结果，确定所述当前分辨率尺度下的第一融合权重矩阵；以及
    根据所述第一融合权重矩阵，对当前分辨率尺度下的空域滤波图像和时域滤波图像进行加权融合。
  5. 如权利要求4所述视频降噪方法，其中，所述第一融合权重矩阵至少包括空域滤波图像中像素值对应的第一融合权重和时域滤波图像中像素值对应的第二融合权重，
    所述根据所述运动估计结果，确定所述当前分辨率尺度下的第一融合权重矩阵，包括：
    根据所述运动估计结果，分别判断当前分辨率尺度下的子带图像中各像素点是运动像素点还是静止像素点；
    若所述像素点为运动像素点，则获取第一预设权重作为所述像素点对应的第二融合权重以及获取第二预设权重作为所述像素点对应的第一融合权重；以及
    若所述像素点为静止像素点，则获取第一预设权重作为所述像素点对应的第一融合权重以及获取第二预设权重作为所述像素点对应的第二融合权重；
    其中，所述第二预设权重小于所述第一预设权重。
  6. 如权利要求3所述视频降噪方法，其中，所述根据所述边缘图，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行融合，包括：
    根据所述边缘图，确定所述当前分辨率尺度下的第二融合权重矩阵；以及
    根据所述第二融合权重矩阵，对当前分辨率尺度下的时空域滤波融合图像和所述在先降噪图像进行加权融合。
  7. 如权利要求6所述视频降噪方法，其中，所述第二融合权重矩阵至少包括时空域滤波融合图像中像素值对应的第三融合权重和在先降噪图像中像素值对应的第四融合权重，
    所述根据所述边缘图，确定所述当前分辨率尺度下的第二融合权重矩阵，包括：
    确定所述边缘图中各像素点对应的边缘强度；以及
    根据各所述边缘强度的大小，确定各所述像素点对应的第三融合权重和第四融合权重；
    其中，所述第三融合权重的大小与所述边缘强度的大小成正比，所述第四融合权重的大小与所述边缘强度的大小成反比。
  8. 如权利要求3所述视频降噪方法，其中，所述对当前分辨率尺度的子带图像进行运动估计，包括：
    获取上一更小分辨率尺度的子带图像对应的在先运动估计结果；
    根据所述在先运动估计结果，确定当前分辨率尺度的子带图像对应的运动估计搜索范围；以及
    根据所述运动估计搜索范围，对当前分辨率尺度的子带图像进行运动估计。
  9. 一种视频降噪装置，包括：
    多尺度分解模块，配置为获取视频图像，对所述视频图像进行多尺度分解，得到至少一个分辨率尺度的子带图像；以及
    多尺度融合模块，配置为将各所述子带图像对应的空域滤波图像和时域滤波图像在各所述分辨率尺度下进行多尺度融合，生成所述视频图像对应的目标降噪图像。
  10. 一种电子设备，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1至8中任一项所述的视频降噪方法。
  11. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有实现视频降噪方法的计算机程序，所述实现视频降噪方法的计算机程序被处理器执行，以使得所述处理器实现如权利要求1至8中任一项所述视频降噪方法。
PCT/CN2023/115871 2022-09-30 2023-08-30 视频降噪方法及装置、电子设备及计算机可读存储介质 WO2024066890A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211209506.4A CN117876243A (zh) 2022-09-30 2022-09-30 视频降噪方法、电子设备及计算机可读存储介质
CN202211209506.4 2022-09-30

Publications (1)

Publication Number Publication Date
WO2024066890A1 true WO2024066890A1 (zh) 2024-04-04

Family

ID=90476074

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/115871 WO2024066890A1 (zh) 2022-09-30 2023-08-30 视频降噪方法及装置、电子设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN117876243A (zh)
WO (1) WO2024066890A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101887580A (zh) * 2010-07-23 2010-11-17 扬州万方电子技术有限责任公司 非下采样轮廓波变换域的图像降噪方法
CN102014240A (zh) * 2010-12-01 2011-04-13 深圳市蓝韵实业有限公司 一种实时医学视频图像去噪方法
CN103108109A (zh) * 2013-01-31 2013-05-15 深圳英飞拓科技股份有限公司 一种数字视频降噪***与方法
CN109743473A (zh) * 2019-01-11 2019-05-10 珠海全志科技股份有限公司 视频图像3d降噪方法、计算机装置及计算机可读存储介质
CN110933334A (zh) * 2019-12-12 2020-03-27 腾讯科技（深圳）有限公司 视频降噪方法、装置、终端及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101887580A (zh) * 2010-07-23 2010-11-17 扬州万方电子技术有限责任公司 非下采样轮廓波变换域的图像降噪方法
CN102014240A (zh) * 2010-12-01 2011-04-13 深圳市蓝韵实业有限公司 一种实时医学视频图像去噪方法
CN103108109A (zh) * 2013-01-31 2013-05-15 深圳英飞拓科技股份有限公司 一种数字视频降噪***与方法
CN109743473A (zh) * 2019-01-11 2019-05-10 珠海全志科技股份有限公司 视频图像3d降噪方法、计算机装置及计算机可读存储介质
CN110933334A (zh) * 2019-12-12 2020-03-27 腾讯科技（深圳）有限公司 视频降噪方法、装置、终端及存储介质

Also Published As

Publication number Publication date
CN117876243A (zh) 2024-04-12

Similar Documents

Publication Publication Date Title
CN110276346B (zh) 目标区域识别模型训练方法、装置和计算机可读存储介质
US20140133780A1 (en) Nonlocality based super resolution reconstruction method and device
JP2011257503A (ja) 画像安定化装置、画像安定化方法、及びプログラム
CN110288520B (zh) 图像美化方法、装置及电子设备
US20200272308A1 (en) Shake Event Detection System
WO2022105622A1 (zh) 图像分割方法、装置、可读介质及电子设备
WO2021031847A1 (zh) 图像处理方法、装置、电子设备和计算机可读存储介质
WO2024016923A1 (zh) 特效图的生成方法、装置、设备及存储介质
WO2022247630A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2024140154A1 (zh) 视线检测方法、装置、电子设备及存储介质
WO2024149078A1 (zh) 一种支持多相机动态输入的感知方法、***及车辆
WO2024066890A1 (zh) 视频降噪方法及装置、电子设备及计算机可读存储介质
JP2015191630A (ja) 情報処理装置、変更検出方法および変更検出プログラム
CN114422698B (zh) 视频生成方法、装置、设备及存储介质
US20200097729A1 (en) Method and apparatus for determining target rotation direction, computer readable medium and electronic device
CN115319760A (zh) 机械臂末端抖动抑制方法、装置、电子设备及存储介质
CN110349109B (zh) 基于鱼眼畸变校正方法及其***、电子设备
CN113808020A (zh) 图像处理方法及设备
US8917956B1 (en) Enhancing spatial resolution of an image
WO2021004171A1 (zh) 水波纹图像实现方法及装置
CN114724528B (zh) 显示设备的显示控制方法、装置、电子设备及存储介质
CN112818748B (zh) 确定视频中的平面的方法和装置、存储介质和电子设备
WO2021018178A1 (zh) 文字特效处理方法及装置
CN113808050B (zh) 3d点云的去噪方法、装置、设备及存储介质
WO2023005341A1 (zh) 一种姿态估计方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23870130

Country of ref document: EP

Kind code of ref document: A1