WO2015085922A1 - 一种帧率上采样的方法和装置 - Google Patents

一种帧率上采样的方法和装置 Download PDF

Info

Publication number
WO2015085922A1
WO2015085922A1 PCT/CN2014/093448 CN2014093448W WO2015085922A1 WO 2015085922 A1 WO2015085922 A1 WO 2015085922A1 CN 2014093448 W CN2014093448 W CN 2014093448W WO 2015085922 A1 WO2015085922 A1 WO 2015085922A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
pixel value
interpolated
current block
value
Prior art date
Application number
PCT/CN2014/093448
Other languages
English (en)
French (fr)
Inventor
李凯
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2015085922A1 publication Critical patent/WO2015085922A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation

Definitions

  • the present invention relates to the field of image processing, and in particular, to a method and apparatus for frame rate upsampling.
  • the frame rate refers to the number of frames per second (FPS).
  • FPS frames per second
  • the video frame rate can be increased by inserting one or more interpolated frames between two adjacent frames of the video to be played. This method can be referred to as Frame Rate Up Conversion (FRUC).
  • FRUC Frame Rate Up Conversion
  • Motion Compensated Frame Rate Up Conversion is a method of frame rate upsampling, which mainly includes Motion Estimation (ME) process and Motion Compensated Interpolation (MCI). The two parts of the process.
  • ME Motion Estimation
  • MCI Motion Compensated Interpolation
  • the motion estimation process may specifically be: dividing each frame image in the video stream into a plurality of non-overlapping macroblocks, and considering that all pixels in the macroblock have the same amount of displacement; and the previous frame of the current frame (n-1) One macroblock in the frame image is used as the current block, and the matching block that best matches the current block is found according to a certain matching criterion in a certain search range of the current frame (the nth frame image); the current block relative matching block is calculated.
  • Motion vector The motion compensation interpolation may specifically be: generating an interpolation frame according to the result of the motion estimation and a certain interpolation method.
  • the moving block interpolation method is a commonly used interpolation method. Specifically, the pixel value of the current block is inserted into the position L from the current block, where L is half of the motion vector (mv) of the current block relative to the matching block, such as Figure 1 shows.
  • the moving block interpolation method easily causes a problem of serious blockiness in the interpolation frame, thereby affecting the viewing experience of the user.
  • Embodiments of the present invention provide a method and apparatus for frame rate upsampling, which are used to reduce blockiness in an interpolated frame, thereby improving the viewing experience when a user views a video.
  • a method for frame rate upsampling including:
  • the method before the obtaining a weight value of a pixel value of the similar block relative to a pixel value of the current block, the method further includes: acquiring the similar block The absolute value of the difference between the pixel value and the pixel value of the current block;
  • the obtaining, by the weighting algorithm, the weight value of the pixel value of the similar block relative to the pixel value of the current block comprising: weighting an absolute value of the difference value based on the weighting algorithm, to obtain the similar block A weight value of a pixel value relative to a pixel value of the current block.
  • the current block in the frame to be interpolated is obtained according to the location of the current block and the motion vector of the current block relative to the matching block.
  • the method further includes:
  • the method further includes:
  • the similarity block includes: the matching block; or m macroblocks in the first region of the first image, m ⁇ 1, and m is an integer.
  • the similar block is the matching block; the method further includes:
  • the modified block is a virtual macro block, and is used to correct a pixel value of the block to be interpolated;
  • the similar block is the m macroblocks in the first area of the first image
  • the obtaining the weight value of the pixel value of the similar block relative to the pixel value of the current block includes:
  • a device for upsampling a frame rate including:
  • a determining unit configured to determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;
  • a location acquiring unit configured to acquire, according to a location of the current block and a motion vector of the current block with respect to the matching block, a location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated;
  • a weight value obtaining unit configured to acquire, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;
  • a pixel value obtaining unit configured to multiply a pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated
  • An interpolation unit configured to insert a pixel value of the block to be interpolated into a position of the block to be interpolated.
  • the weight value obtaining unit is further configured to acquire an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;
  • the weight value obtaining unit is configured to perform a weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.
  • the device further includes:
  • An expansion unit configured to separately expand the current block and the similar block according to the same expansion ratio
  • the location obtaining unit is configured to: obtain, according to the extended location of the current block, and the motion vector of the current block relative to the matching block, a to-be-interpolated frame corresponding to the extended current block.
  • the pixel value obtaining unit is configured to multiply the pixel value of the extended similar block by the weight value to obtain a pixel value of the block to be interpolated.
  • the pixel value obtaining unit is further configured to:
  • the similarity block includes: the matching block; or m macroblocks in the first region of the first image, m ⁇ 1, and m is an integer.
  • the similar block is the matching block
  • the pixel value obtaining unit is further configured to: acquire a pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block; and the correction block is a virtual macro block, and the block to be interpolated is modified. Pixel value;
  • the pixel value obtaining unit is configured to obtain a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the modified block, and the weight value.
  • the pixel value acquiring unit is specifically configured to: the pixel value of the current block and the pixel value of the matching block
  • the mean value is used as the pixel value of the correction block.
  • the similar block is m macroblocks in a first region of the first image
  • the weight value obtaining unit is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m ⁇ n ⁇ 1, where n is an integer ;
  • the method and apparatus for frame rate upsizing provided by the embodiment of the present invention, by determining a current block and a similar block corresponding to the current block, acquiring a weight value of a pixel value of a similar block corresponding to the current block relative to a pixel value of the current block, and The obtained weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • FIG. 1 is a schematic diagram of a moving block interpolation method in the prior art
  • FIG. 2 is a schematic flowchart of a method for sampling a frame rate up according to Embodiment 1;
  • Embodiment 3 is a schematic flowchart of a method for sampling a frame rate up by Embodiment 1;
  • FIG. 4 is a schematic diagram showing a relationship between a weight value and an absolute error according to Embodiment 1;
  • FIG. 5 is a schematic diagram of a moving block interpolation method according to Embodiment 1;
  • FIG. 5 is a schematic diagram of a moving block interpolation method according to Embodiment 1;
  • FIG. 6 is a schematic flowchart diagram of another method for sampling frame rate up by Embodiment 1;
  • FIG. 7 is a schematic flowchart diagram of a method for frame rate upsizing provided in Embodiment 2;
  • FIG. 8 is a schematic diagram showing a positional relationship between an extended current block and a current block according to Embodiment 2;
  • FIG. 9 is a schematic diagram of a mobile block interpolation method according to Embodiment 2.
  • FIG. 10 is a schematic flowchart diagram of another method for sampling frame rate up by Embodiment 2;
  • FIG. 11 is a schematic flowchart diagram of a method for frame rate upsizing provided by Embodiment 3;
  • FIG. 12 is a schematic diagram showing a positional relationship between a current block and a corresponding similar block according to Embodiment 3;
  • FIG. 13 is a schematic diagram of a moving block interpolation method according to Embodiment 3.
  • FIG. 14 is a schematic structural diagram of an apparatus for frame rate upsampling provided in Embodiment 2;
  • FIG. 15 is a schematic structural diagram of another apparatus for sampling frame rate up according to Embodiment 2;
  • FIG. 15 is a schematic structural diagram of another apparatus for sampling frame rate up according to Embodiment 2;
  • FIG. 16 is a schematic structural diagram of an apparatus for frame rate upsampling provided in Embodiment 3.
  • the current frame refers to the nth frame image; the previous frame of the previous frame refers to the n-1th frame image. Where n ⁇ 2, n is an integer.
  • An interpolated frame refers to an image that has been interpolated; an interpolated frame refers to an image that has not been interpolated. Both the interpolated frame and the frame to be interpolated may be referred to as a virtual frame (ie, not an image in the video stream itself, but an image obtained by a frame rate upsampling method).
  • the first image and the second image may be any two frames in a video stream.
  • the two adjacent frames are taken as an example for description.
  • the first image may be the previous frame image of the second image, or may be the next frame image of the second image.
  • the first image is the current frame image
  • the second image is the previous frame image of the current frame; for example, the first image is the previous frame image of the current frame, and the second image is the current frame image.
  • the first image and the second image may be images of the video stream itself; in a scene in which a plurality of frames of images are inserted between adjacent frames of the video stream, The first image and the second image may be images of the video stream itself, or may be images after the video stream is interpolated (ie, interpolated frames).
  • each macroblock is a square. If the width and/or height of the image cannot divide the side length of the macroblock, the image width is expanded to an integral multiple of the macroblock.
  • a macroblock that is being processed at a current time in one frame of an image is referred to as a current block.
  • the image in which the current block is located is the first image.
  • macroblocks in the image are processed sequentially in order from top to bottom and left to right in the position of the macroblock.
  • the matching block is relative to the current block and can be described as a matching block corresponding to the current block.
  • the image in which the matching block is located is the second image.
  • the matching block may be a complete macroblock or a block composed of a plurality of partial macroblocks; generally, the matching block is a square whose side length is equal to the side length of the macroblock.
  • the block to be interpolated is relative to the current block and can be described as the block to be interpolated corresponding to the current block.
  • the image to be interpolated is the frame to be interpolated.
  • the similarity block is relative to the current block and can be described as a similar block corresponding to the current block. Specifically, it refers to a block other than the current block used when acquiring pixels of the block to be interpolated corresponding to the current block.
  • the correction block is a virtual macroblock for correcting the pixel value of the block to be interpolated.
  • the pixel value of a block refers to the sum of the pixel values of all points in the block.
  • the pixel value of the matching block refers to the sum of the pixel values of all the points in the matching block; for example, the pixel value of the current block refers to the sum of the pixel values of all the points in the current block.
  • the term “and/or” in the embodiment of the present invention is merely an association relationship describing an associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, and A exists at the same time. And B, there are three cases of B alone.
  • the character "/" generally indicates that the contextual object is an "or" relationship.
  • a method for sampling a frame rate includes:
  • the embodiment of the present invention is an improvement of the method for up-sampling the motion compensation frame rate, and is mainly an improvement of the motion compensation interpolation process in the method of motion compensation frame rate upsampling.
  • the position of the current block can be represented by coordinates, and can be expressed by the coordinates of the center point of the current block, any vertex, or other points in the current block. It should be noted that, in a specific implementation, it is necessary to reasonably select the coordinates of a certain point in the current block to represent the position of the current block.
  • a matching block corresponding to the current block in the second image, and a motion vector of the current block relative to the matching block may be obtained in the motion estimation process.
  • the method for implementing motion estimation is not limited in the embodiment of the present invention, and may be any method in the prior art.
  • a matching block corresponding to the current block in the second image may be acquired using a full search method (Full Search), a diamond search method, or the like.
  • the motion vector after the motion estimation process, in order to improve the accuracy of the motion vector obtained by the motion estimation process, the motion vector may be optimized, which may include: a motion vector decision optimization process, a motion vector smoothing process. Wait. Therefore, the “motion vector of a current block relative to a matching block in the second image” in step 201 may also be a motion vector obtained after performing optimization processing on the motion vector obtained by the motion estimation process.
  • the specific optimization processing method is not limited in the embodiment of the present invention. The motion vectors mentioned in the following embodiments are all described by taking the motion vector obtained during the motion estimation process as an example.
  • the process of motion estimation may be described as: searching for a matching block corresponding to the current block in the second image with reference to the current block in the first image. Therefore, the first image can also be referred to as a reference frame, and the second image is referred to as a search frame. For example, if the first image is the current frame (the nth frame image) and the second image is the previous frame of the current frame (the n-1th frame image), the nth frame is the reference frame and the n-1th frame is the search.
  • the n-1th frame is the reference frame
  • the nth The frame is a search frame.
  • the similar blocks include but are not limited to the following two types:
  • the matching block (2) m macroblocks in the first region of the first image, m ⁇ 1, and m is an integer.
  • the first area may be formed by the neighborhood of the current block and/or the neighborhood of the macroblock corresponding to the time domain.
  • the value of m is not limited in the embodiment of the present invention.
  • the step 202 may be: determining, by using the position of the frame to be interpolated from the current block as the position of the block to be interpolated corresponding to the current block in the frame to be interpolated, where L means that the current block is relative to the current block. Matches half of the motion vector of the block.
  • the coordinates of the current block are (x1, y1), and the motion vector of the current block relative to the matching block is (a, b)
  • the first image is the previous frame image of the second image
  • the value to be interpolated The coordinates of the position of the block to be interpolated corresponding to the current block in the frame are:
  • the coordinates of the position of the block to be interpolated corresponding to the current block in the frame to be interpolated are:
  • the reference index used in the embodiment of the present invention is not limited.
  • the reference indicator may be an absolute value (SAD of Absolute Difference, SAD) of a pixel value of a similar block relative to a pixel value of the current block, and a pixel value of a certain point in the similar block corresponds to the point in the current block.
  • a point in the similar block relative to the point in the current block may be specifically: the upper left vertex in the similar block is relative to the upper left in the current block The vertex, or the center point in the similar block, is relative to the center point in the current block.
  • the embodiment of the present invention does not limit the specific weighting algorithm, and may be any weighting algorithm in the prior art that satisfies the rule that the reference indicator (such as SAD or SD) has a smaller weight value.
  • it may be: a Gaussian weighting algorithm, a linear weighting algorithm, a trigonometric function weighting algorithm, a custom window function weighting algorithm, and the like.
  • the method may further include: acquiring an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block.
  • the step 203 may include: performing a weighting operation on the absolute value of the difference based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.
  • the invention provides the following embodiments, specifically:
  • the method before step 202, further comprises: expanding the current block and the similar block respectively according to the same expansion ratio.
  • the step 202 includes: obtaining the to-be-interpolated frame corresponding to the extended current block according to the expanded position of the current block and the motion vector of the current block relative to the matching block.
  • Step 204 specifically includes: multiplying the expanded pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.
  • the value of the expansion ratio of the embodiment of the present invention is not limited, and may be set according to experience and a specific scenario. For example, after repeated experiments, it is concluded that in the small-magnification motion scene, when the expansion ratio is 2, the effect of removing the hole is better, and the distortion and ghosting at the boundary of the moving object are smaller.
  • step 203 may be: obtaining a weight value of the pixel of the current block that is expanded relative to the expanded similar block, and using the weight value as a pixel of the similar block relative to the current block pixel. The weight value of the value.
  • the similarity block is m macroblocks in the first region of the first image; and the step 203 specifically includes: acquiring pixel values of n macroblocks in the m macroblocks, respectively. n weight values relative to the pixel values of the current block; m ⁇ n ⁇ 1, the n being an integer.
  • the step 204 includes: multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain the The pixel value of the interpolation block.
  • each macroblock in the first image is taken as the current block.
  • the motion compensation interpolation process is completed, and the frame to be interpolated becomes an interpolation frame.
  • the pixel value of any point in the interpolated frame is the sum of all pixels covering the point.
  • the motion vector determined by the motion estimation process is often inaccurate.
  • the motion estimation process using two frames cannot search for the matching block of the occlusion portion, in which case The obtained matching block may be wrong.
  • the similarity block is the matching block; the method may further include: acquiring a pixel value of the modified block according to the pixel value of the current block and the pixel value of the matching block;
  • the block is a virtual macroblock for correcting pixel values of the block to be interpolated.
  • the step 204 specifically includes: obtaining, according to the pixel value of the matching block, the pixel value of the correction block, and the weight value, a pixel value of the block to be interpolated.
  • the specific method for obtaining the pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block is not limited.
  • the method may further include:
  • overlapping pixels refer to pixels that are repeatedly covered. Since the current block in step 201 can be any macroblock in the first image, there may be overlapping pixels in the frame to be interpolated after performing step 205.
  • step A) and step B) may include, but is not limited to, the following two modes: mode 1, after performing steps 201-204 as any current block in the first image, performing step A) and step B); Mode 2, after performing steps 201-204 for each macroblock in the first image as the current block, performing steps A) and B). Since a repeated pixel may be covered multiple times during the process of generating an interpolation frame in the frame to be interpolated, according to mode 1, Steps A) and B) need to be performed multiple times. Therefore, in the specific implementation, the mode 2 is generally used, and the mode 2 in the following specific embodiments is taken as an example for description.
  • the frame to be interpolated obtained by the mode 2 is referred to as an interpolated frame. Since there may be a hole in the interpolated frame, the method may further include: performing a filling hole processing on the interpolated frame, where The method of processing the holes can be implemented using methods in the prior art and will not be described here.
  • the frame rate upsampling method obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block.
  • the weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • the similar block is a matching block, and the current block and the similar block are not expanded; in addition, the first image and the second image are two adjacent frames in a video stream.
  • a method for frame rate upsampling includes:
  • the step 302 may be referred to as a motion estimation process, and the specific implementation manner is not limited in this embodiment.
  • the i-th macro block in the first image is used as a current block; determining a position of the current block, a matching block corresponding to the current block, and a current block relatively matching the motion vector of the block.
  • the “matching block” in the “moving vector of the current block relative to the matching block” refers to: the matching block corresponding to the current block
  • the “matching block” in the following steps refers to: the matching block corresponding to the current block.
  • the coordinates of the center point of the current block can be used to represent the position of the current block
  • the coordinates of the center point of the matching block are used to indicate the position of the matching block.
  • the motion vector of the current block relative to the matching block is: (x2-x1, y2-y1); when the first image is the nth frame image and the second image is the n-1th frame image, the current block is relatively matched to the block.
  • the motion vector is: (x1-x2, y1-y2).
  • step 304 may be specifically:
  • the position of the interpolated block corresponding to the current block in the interpolated frame may be expressed as: which is
  • the position of the interpolated block corresponding to the current block in the interpolated frame may be expressed as: which is
  • the first image is the previous frame image or the subsequent frame image of the second image
  • the coordinates of the block to be interpolated corresponding to the current block can be determined.
  • the absolute error AD(s) of the pixel value of the sth pixel in the matching block relative to the pixel value of the sth pixel in the current block can be calculated by the following formula:
  • AD(s)
  • MB ref (s) refers to the pixel value of the sth pixel in the current block (MB ref ) of the first image (ie, the reference frame)
  • MB search (s) refers to the second image (ie, the search frame)
  • the pixel value of the sth pixel in the matching block (MB search ) corresponding to the current block.
  • the absolute value SAD of the difference between the pixel value of the matching block and the pixel value of the current block can be calculated by the following formula:
  • step 306 is described below as a Gaussian weighting algorithm by a weighting algorithm:
  • the weight value w(s) of the pixel value of the sth pixel in the matching block relative to the pixel value of the sth pixel in the current block can be obtained according to the following formula:
  • ⁇ means that the expected value can be fixedly taken as 0; ⁇ means the variance, and generally can take a fixed value of 5 or 10, of course, not limited to this.
  • the weight value of the pixel value of the matching block relative to the pixel value of the current block can be obtained according to the following formula:
  • the pixel value MB interpolate (s) of the sth pixel in the block to be interpolated can be obtained according to the following formula:
  • MB interpolate (s) MB search (s)w(s);
  • Pixel value of the block to be interpolated It can be obtained according to the following formula:
  • the pixel value of the block to be interpolated is obtained by combining the pixel value of the current block and the pixel value of the matching block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, as shown in FIG. 5 , wherein In FIG. 5, the first image is the n-1th frame image, and the second image is the nth frame image.
  • step 310 is performed; if yes, step 311 is performed.
  • the determination result of step 309 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.
  • step 310 the process returns to step 303.
  • 311 Perform weighted accumulation processing on pixel values of each overlapping pixel in the interpolation frame, respectively, to obtain an intermediate pixel value of each overlapping pixel.
  • the intermediate pixel value ws(t) of the pixel t can be obtained according to the following formula:
  • J is the number of times the pixel t is repeatedly covered, that is, the number of blocks to be interpolated that cover the pixel t;
  • MB int erpolate (s) j refers to the jth block to be interpolated (MB int ) covering the pixel t Erpolate ),
  • w(t) j refers to the weight value of the pixel value of the corresponding pixel in the current block corresponding to the j-th interpolated block corresponding to the j-th interpolated block (MB int erpolate ) of the pixel t.
  • step 312 Determine a target pixel value of each overlapping pixel according to the weighting algorithm used in step 306 and the intermediate pixel value of each overlapping pixel, to obtain an interpolated frame.
  • step 312 may be specifically implemented as:
  • the weight of the pixel t and sw(t) can be obtained by the following formula:
  • Target pixel value of pixel t It can be obtained by the following formula:
  • step 307 can be replaced by the following steps 307A-307B:
  • the pixel values of each pixel in the current block and the pixel values of the corresponding pixels in the matching block are respectively averaged to obtain pixel values of corresponding pixels in the modified block.
  • the pixel value of the third pixel in the current block and the pixel value of the third pixel in the matching block are averaged to obtain the pixel value of the third pixel in the correction block.
  • step 307B may include, but is not limited to, being implemented in the following manner:
  • the pixel value MB int erpolate (s) of the sth pixel in the block to be interpolated can be obtained according to the following formula:
  • MB int erpolate (s) MB search (s)w(s)+MB correct (1-w(s)); where MB correct refers to the correction block.
  • Pixel value of the block to be interpolated It can be obtained according to the following formula:
  • step 307 by replacing the scheme of step 307 with steps 307A-307B, the influence of the inaccurate motion vector on the result of the frame rate upsampling can be reduced.
  • the frame rate up-sampling method dynamically obtains the weight value of the pixel value of the matching block corresponding to the current block relative to the pixel value of the current block, and uses the dynamically obtained weight value to obtain the block to be interpolated.
  • frame rate upsampling is achieved.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • the similar block is a matching block, and the current block and the similar block are expanded; in addition, the first image and the second image are two adjacent frames in a video stream.
  • the first image is divided into R macroblocks, each of which contains 16 ⁇ 16 pixels.
  • a method for sampling a frame rate includes:
  • the i-th macroblock in the first image is used as a current block; the location of the current block, the matching block corresponding to the current block, and the motion vector of the current block relative to the matching block are determined.
  • 1 ⁇ i ⁇ R, i is an integer, and the initial value of i is 1.
  • steps 701-703 can refer to steps 301-303 of the above embodiment 1, and only the differences from the embodiment 1 are explained below:
  • the "extension” herein generally refers to expanding the pixels of the current block and the matching block centering on the origin, and expanding one block before and after (including the current block or matching).
  • the coordinates of the center point of the block are not changed. Therefore, the coordinates of the center point of the block (including the current block or the matching block) are generally not used to indicate the position of the block.
  • the coordinates of the left vertex of the current block are used to represent the current block. The position, using the coordinates of the left vertex of the matching block to indicate the position of the matching block.
  • the left vertex of the current block is A
  • the position of the current block may specifically be the coordinates of point A
  • the left vertex of the expanded current block is B
  • the position of the expanded current block is Specifically, the coordinates of point B.
  • step 706 may be replaced by: obtaining an absolute value of a difference between a pixel value of the extended matching block and a pixel value of the expanded current block; in this case, step 707 may be replaced by the following steps: The weighting algorithm and the absolute value of the difference value obtain the weight value of the pixel value of the current block of the expanded matching block relative to the current block.
  • Steps 706-707 in this embodiment are the same as steps 305-306 in Embodiment 1.
  • steps 305-306 in Embodiment 1 For a specific example in the alternative manner, reference may be made to steps 305-306 in Embodiment 1, except that the values of S are different.
  • the pixel value of the expanded current block is combined with the pixel value of the expanded matching block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, such as As shown in FIG. 9, the first image in FIG. 9 is the n-1th frame image, and the second image is the nth frame image.
  • step 711 is performed; if yes, step 712 is performed.
  • the determination result of step 710 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.
  • step 711 the process returns to step 703.
  • step 713 Determine a target pixel value of each overlapping pixel according to the weighting algorithm used in step 707 and the intermediate pixel value of each overlapping pixel, to obtain an interpolation frame.
  • steps 710-714 may refer to steps 309-313 in Embodiment 1, and details are not described herein again.
  • step 708 can be replaced with the following steps 708A-708B:
  • the average value of the pixel value of the expanded current block and the pixel value of the expanded matching block is used as the pixel value of the correction block.
  • step 708 replacing the scheme of step 708 with steps 708A-708B can reduce the impact of inaccurate motion vectors on the result of frame rate upsampling.
  • Embodiment 1 For example, related explanations and specific examples in steps 708-714, 708A, and 708B in this embodiment may be referred to in Embodiment 1, wherein, in specific implementation, the "matching block” in Embodiment 1 is required. Replaced with “expanded matching block”, the "current block” in Embodiment 1 is replaced with “expanded current block”; the value of S in Embodiment 1 is replaced by 16x16 to 32x32.
  • each of the blocks to be interpolated obtained in Embodiment 1 contains 16 ⁇ 16 pixels
  • each block to be interpolated obtained in Embodiment 2 contains 32 ⁇ 32 pixels.
  • the frame rate upsampling method dynamically obtains the weight value of the pixel value of the matching block corresponding to the current block relative to the pixel value of the current block, and expands the current block and the matching block, and combines the dynamic acquisition.
  • the weight value, the pixel value of the pixel of the expanded current block, and the pixel value of the pixel of the expanded matching block acquire the pixel value of the block to be interpolated, thereby achieving frame rate upsampling.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • the similar block is m macroblocks in the first region of the first image, and is not Extend similar blocks.
  • the first image and the second image are two adjacent frames in a video stream.
  • another method for sampling a frame rate includes:
  • the step 1102 may be referred to as a motion estimation process, and the specific implementation manner is not limited in this embodiment.
  • the i-th macroblock in the first image is used as a current block; determining a location of the current block, a motion vector of a current block relative to a matching block in the second image, and a similar block corresponding to the current block, where the similar block is m macroblocks in the first region of an image.
  • the first region may be composed of a neighborhood of a current block and/or a neighborhood of a macroblock corresponding to the time domain. As shown in FIG. 12, it is an example of a similar block corresponding to the current block. Of course, the specific implementation is not limited thereto.
  • the initial value of j is 1, 1 ⁇ j ⁇ m, and j is an integer.
  • steps 1105-1106 may refer to steps 305-306 in the above embodiment 1.
  • the “pixel value of the matching block” in steps 305-306 is replaced with “the image value element of the jth similar block”.
  • the “jth pixel value of the interpolation block” specifically means that the pixel value of the block to be interpolated corresponding to the i th current block is obtained in the jth time.
  • the pixel value of the block to be interpolated corresponding to the i th current block is acquired only once for the i th current block; in this embodiment, for the i th current Block, obtaining pixel values of the block to be interpolated corresponding to the i-th current block m times.
  • step 1109 If no, go to step 1109; if yes, go to step 1111.
  • the determination result of step 1107 is “Yes”, indicating that the pixel values of the block to be interpolated corresponding to the i-th current block have been acquired m times.
  • step 1109 After step 1109 is performed, the process returns to step 1105.
  • the pixel value of the block to be interpolated is obtained by combining the pixel value of the current block and the pixel value of the similar block around the current block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, as shown in FIG.
  • the first image in FIG. 13 is the n-1th frame image
  • the second image is the nth frame image.
  • step 1113 If no, go to step 1113; if yes, go to step 1114.
  • the determination result of step 1112 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.
  • step 1113 the process returns to step 1103.
  • step 1115 Determine a target pixel value of each overlapping pixel according to the weighting algorithm used in step 1106 and the intermediate pixel value of each overlapping pixel, to obtain an interpolated frame.
  • steps 1112-1116 may refer to steps 309-313 in Embodiment 1, and details are not described herein again.
  • the frame rate upsampling method provided by the embodiment of the present invention dynamically obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block, and uses the dynamically obtained weight value to obtain the block to be interpolated.
  • frame rate upsampling is achieved.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • a frame rate upsampling apparatus 14 for performing the frame rate upsampling method shown in FIG. 1 is provided.
  • the apparatus 14 includes:
  • a determining unit 141 configured to determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;
  • a location obtaining unit 142 configured to acquire, according to a location of the current block and a motion vector of the current block with respect to the matching block, a location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated;
  • the weight value obtaining unit 143 is configured to acquire, according to the weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;
  • a pixel value obtaining unit 144 configured to multiply a pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated
  • the interpolation unit 145 is configured to insert a pixel value of the block to be interpolated into a position of the block to be interpolated.
  • the weight value obtaining unit 143 is further configured to acquire an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;
  • the weight value obtaining unit 143 is specifically configured to perform weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.
  • the device 14 further includes:
  • the expansion unit 146 is configured to separately expand the current block and the similar block according to the same expansion ratio
  • the location obtaining unit 142 is specifically configured to: obtain, according to the expanded location of the current block, and a motion vector of the current block relative to the matching block, a corresponding to the extended current block in the to-be-interpolated frame. The position of the block to be interpolated;
  • the pixel value obtaining unit 144 is specifically configured to multiply the pixel value of the extended similar block by the weight value to obtain a pixel value of the block to be interpolated.
  • the pixel value obtaining unit 144 is further configured to:
  • the similar block includes: the matching block; or m macroblocks in the first region of the first image, m ⁇ 1, where m is an integer.
  • the similar block is the matching block
  • the pixel value obtaining unit 144 is further configured to: acquire, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct the to-be-interpolated value.
  • the pixel value obtaining unit 144 is specifically configured to: according to the pixel value of the matching block, The pixel value of the correction block and the weight value obtain a pixel value of the block to be interpolated.
  • the pixel value obtaining unit 144 is specifically configured to use, as a pixel value of the modified block, a mean value of a pixel value of the current block and a pixel value of the matching block.
  • the similar block is m macroblocks in the first region of the first image
  • the weight value obtaining unit 143 is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m ⁇ n ⁇ 1, where n is Integer
  • the frame rate upsizing apparatus obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block.
  • the weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • a frame rate upsampling apparatus 14 for performing the frame rate upsampling method shown in FIG. 1 is provided.
  • the apparatus 14 includes a memory 161 and a processor 162. among them,
  • the memory 161 is used to store a set of codes for controlling the processor 162 to perform the following actions:
  • the processor 162 is further configured to: obtain an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;
  • the processor 162 is specifically configured to perform weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.
  • the processor 162 is further configured to separately expand the current block and the similar block according to the same expansion ratio
  • the processor 162 is specifically configured to:
  • processor 162 is further configured to:
  • the similar block includes: the matching block; or m macroblocks in the first region of the first image, m ⁇ 1, where m is an integer.
  • the similar block is the matching block
  • the processor 162 is further configured to: obtain, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct a pixel value of the block to be interpolated ;
  • the processor 162 is specifically configured to obtain a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the modified block, and the weight value.
  • the processor 162 is specifically configured to use, as a pixel value of the modified block, a mean value of a pixel value of the current block and a pixel value of the matching block.
  • the similar block is m macroblocks in the first region of the first image
  • the processor 162 is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m ⁇ n ⁇ 1, where n is an integer;
  • the frame rate upsizing apparatus obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block.
  • the weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained.
  • the pixel value of the current block is directly used as the pixel of the block to be interpolated.
  • the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the device described above The embodiment is merely illustrative.
  • the division of the unit is only a logical function division, and the actual implementation may have another division manner.
  • multiple units or components may be combined or integrated into another system. Or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the software functional units described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform portions of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Television Systems (AREA)

Abstract

本发明实施例公开了一种帧率上采样的方法和装置，涉及图像处理领域，用以减少插值帧中的块效应，从而提高用户观赏视频时的观赏体验。本发明实施例提供的方法包括：确定第一图像中的当前块的位置、当前块相对第二图像中的一匹配块的运动向量以及与当前块对应的相似块；根据当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与当前块对应的待插值块的位置；基于加权算法获取相似块的像素值相对当前块的像素值的权重值；将相似块的像素值乘以权重值，得到待插值块的像素值；将待插值块的像素值***待插值块的位置。

Description

一种帧率上采样的方法和装置
本申请要求于2013年12月10日提交中国专利局、申请号为201310669856.3、发明名称为“一种帧率上采样的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及图像处理领域，尤其涉及一种帧率上采样的方法和装置。
背景技术
视频帧率(Frame rate)是指每秒显示的帧数(Frames per Second，FPS)。视频帧率越大，运动画面的平顺度就越好，这样，用户的观赏体验就会越好。一般地，可以通过在待播放视频中相邻的两帧图像之间***一个或者多个插值帧来提高视频帧率，可以将该方法称为帧率上采样(Frame Rate Up Conversion，FRUC)。
运动补偿帧率上采样(Motion Compensated Frame Rate Up Conversion，MC-FRUC)是一种帧率上采样的方法，主要包括运动估计(Motion Estimation，ME)过程和运动补偿插值(Motion Compensated Interpolation，MCI)过程两大部分。
运动估计过程具体可以为：将视频流中的每一帧图像分成许多互不重叠的宏块，并认为宏块内所有像素的位移量都相同；将当前帧的前一帧(第n-1帧图像)中的一个宏块作为当前块，在当前帧(第n帧图像)的某一特定搜索范围内根据一定的匹配准则找出与当前块最匹配的匹配块；计算当前块相对匹配块的运动向量。运动补偿插值具体可以为：根据运动估计的结果和一定的插值方法生成插值帧。移动块插值方法是一种常用的插值方法，具体的：将当前块的像素值***与当前块相距L的位置，其中，L是指当前块相对匹配块的运动向量(mv)的一半，如图1所示。
由于相邻宏块(当前块)的运动向量一般不同，因此，移动块插值方法容易导致插值帧中块效应严重的问题，从而影响用户的观赏体验。
发明内容
本发明实施例提供了一种帧率上采样的方法和装置，用以减少插值帧中的块效应，从而提高用户观赏视频时的观赏体验。
第一方面，提供一种帧率上采样的方法，包括：
确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；
基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
将所述待插值块的像素值***所述待插值块的位置。
结合第一方面，在第一种可能的实现方式中，在所述获取所述相似块的像素值相对所述当前块的像素值的权重值之前，所述方法还包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
所述基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
结合第一方面，在第二种可能的实现方式中，在所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置之前，所述方法还包括：
按照相同的扩展倍率分别扩展所述当前块和所述相似块；
所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置，包括：
根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位 置；
所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
结合第一方面，在第三种可能的实现方式中，在所述将所述待插值块的像素值***所述待插值块的位置之后，所述方法还包括：
对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
结合第一方面或者第一方面的第一种可能的实现方式至第三种可能的实现方式任一种，在第四种可能的实现方式中，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
结合第一方面的第四种可能的实现方式，在第五种可能的实现方式中，所述相似块为所述匹配块；所述方法还包括：
根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
结合第一方面的第五种可能的实现方式，在第六种可能的实现方式中，所述根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值，包括：
将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的 像素值。
结合第一方面的第四种可能的实现方式，在第七种可能的实现方式中，所述相似块为所述第一图像的第一区域中的m个宏块；
所述获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：
分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
第二方面，提供一种帧率上采样的装置，包括：
确定单元，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
位置获取单元，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；
权重值获取单元，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
像素值获取单元，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
插值单元，用于将所述待插值块的像素值***所述待插值块的位置。
结合第二方面，在第一种可能的实现方式中，所述权重值获取单元还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
所述权重值获取单元具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
结合第二方面，在第二种可能的实现方式中，所述装置还包括：
扩展单元，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；
所述位置获取单元具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；
所述像素值获取单元具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
结合第二方面，在第三种可能的实现方式中，所述像素值获取单元还用于：
对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
结合第二方面或者第二方面的第一种可能的实现方式至第三种可能的实现方式任一种，在第四种可能的实现方式中，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
结合第二方面的第四种可能的实现方式，在第五种可能的实现方式中，所述相似块为所述匹配块；
所述像素值获取单元还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
所述像素值获取单元具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
结合第二方面的第五种可能的实现方式，在第六种可能的实现方式中，所述像素值获取单元具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
结合第二方面的第四种可能的实现方式，在第七种可能的实现方式中，所述相似块为所述第一图像的第一区域中的m个宏块；
所述权重值获取单元具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
本发明实施例提供的帧率上采样的方法和装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
附图说明
为了更清楚地说明本发明实施例或现有技术的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为现有技术中的一种移动块插值方法的示意图；
图2为实施例一提供的一种帧率上采样的方法的流程示意图；
图3为实施例1提供的一种帧率上采样的方法的流程示意图；
图4为实施例1提供的一种表示权重值与绝对误差的变化关系的示意图；
图5为实施例1提供的一种移动块插值方法的示意图；
图6为实施例1提供的另一种帧率上采样的方法的流程示意图；
图7为实施例2提供的一种帧率上采样的方法的流程示意图；
图8为实施例2提供的一种表示扩展后的当前块与当前块的位置关系的示意图；
图9为实施例2提供的一种移动块插值方法的示意图；
图10为实施例2提供的另一种帧率上采样的方法的流程示意图；
图11为实施例3提供的一种帧率上采样的方法的流程示意图；
图12为实施例3提供的一种表示当前块与其对应的相似块的位置关系的示意图；
图13为实施例3提供的一种移动块插值方法的示意图；
图14为实施例二提供的一种帧率上采样的装置的结构示意图；
图15为实施例二提供的另一种帧率上采样的装置的结构示意图；
图16为实施例三提供的一种帧率上采样的装置的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
为了清楚理解本发明实施例，下面对本发明实施例中的一些要素进行解释：
(1)当前帧、当前帧的前一帧
当前帧是指第n帧图像；前帧的前一帧是指第n-1帧图像。其中，n≥2，n为整数。
(2)插值帧、待插值帧
插值帧是指已经完成插值的图像；待插值帧是指还未完成插值的图像。插值帧和待插值帧均可以称为虚拟帧(即并非视频流本身中的图像，而是通过帧率上采样方法得到的图像)。
(3)第一图像、第二图像
第一图像和第二图像可以为一视频流中的任意两帧图像，本发明实施例中均以其为相邻的两帧图像为例进行说明。其中，第一图像可以为第二图像的前一帧图像，也可以为第二图像的后一帧图像。例如，第一图像为当前帧图像，第二图像为当前帧的前一帧图像；又如，第一图像为当前帧的前一帧图像，第二图像为当前帧图像。
在视频流的相邻帧之间***一帧图像的场景中，第一图像和第二图像可以为该视频流本身的图像；在视频流的相邻帧之间***多帧图像的场景中，第一图像和第二图像可以为该视频流本身的图像，也可以为该视频流插值完成后的图像(即插值帧)。
(4)宏块
将一帧图像等分成若干部分，每一部分称为一个宏块(Macro Block)。一般地，每个宏块为一正方形，若该图像的宽(width)和/或高(height)不能整除该宏块的边长，则将图像宽高扩展成宏块的整数倍。
(5)当前块
将一帧图像中、当前时刻正在处理的的宏块称为当前块。本发明实施例中，当前块所在的图像为第一图像。一般地，按照宏块在图像中所处位置从上到下、从左到右的顺序依次处理该图像中的宏块。
(6)匹配块
匹配块是相对当前块而言的，可以描述为与当前块对应的匹配块。本发明实施例中，匹配块所在的图像为第二图像。匹配块可以为一完整宏块，也可以为由多个部分宏块组合而成的一个块；一般地，匹配块为一正方形，其边长与宏块的边长相等。
(7)待插值块
待插值块是相对当前块而言的，可以描述为与当前块对应的待插值块。待插值块所在的图像为待插值帧，
(8)相似块
相似块是相对当前块而言的，可以描述为与当前块对应的相似块。具体是指，获取与当前块对应的待插值块的像素时所使用的、除当前块之外的块。
(9)修正块
修正块为一虚拟宏块，用于修正待插值块的像素值。
(10)块的像素值
块的像素值是指该块中的所有点的像素值之和。例如，匹配块的像素值是指该匹配块中所有点的像素值之和；又如，当前块的像素值是指当前块中的所有点的像素值之和。
另外，本发明实施例中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。本发明实施例中字符“/”，一般表示前后关联对象是一种“或”的关系。
实施例一
如图2所示，为本发明实施例提供的一种帧率上采样的方法，包括：
201：确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块。
示例性的，本发明实施例是对运动补偿帧率上采样的方法的改进，主要是对运动补偿帧率上采样的方法中的运动补偿插值过程的改进。
当前块的位置可以用坐标表示，具体可以用当前块的中心点、任一顶点或者该当前块中的其他点的坐标表示。需要说明的是，在具体实现时，需要合理选择使用当前块中的某一个点的坐标表示该当前块的位置。
第二图像中与当前块对应的匹配块，以及当前块相对该匹配块的运动向量可以在运动估计过程中获得。本发明实施例对实现运动估计的方法不进行限定，可以为现有技术中的任一种方法。例如，可以使用全搜索法(Full Search)、钻石搜索法等获取第二图像中与当前块对应的匹配块。
需要说明的是，具体实现时，在运动估计过程之后，为了提高运动估计过程获得的运动向量的精确度，可以对运动向量进行优化处理，具体可以包括：运动向量判定优化过程、运动向量平滑过程等。因此，步骤201中的“当前块相对第二图像中的一匹配块的运动向量”还可以为：对运动估计过程获得的运动向量进行优化处理之后，得到的运动向量。本发明实施例对具体的优化处理方法不进行限定。以下实施例中所提到的运动向量，均以运动估计过程中得到的运动向量为例进行说明。
另外需要说明的是，由于运动估计的过程可以描述为：以第一图像中的当前块为参考，在第二图像中搜索与当前块对应的匹配块。因此，还可以将第一图像称为参考帧，第二图像称为搜索帧。例如，若第一图像为当前帧(第n帧图像)、第二图像为当前帧的前一帧(第n-1帧图像)，则第n帧为参考帧、第n-1帧为搜索帧；又如，若第一图像为当前帧的前一帧(第n-1帧图像)，第二图像为当前帧(第n帧图像)，则第n-1帧为参考帧，第n帧为搜索帧。
可选的，所述相似块包括但不限于以下两种：
(1)所述匹配块；(2)所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
示例性的，第一区域可以由当前块的邻域和/或时域对应的宏块的邻域构成，本发明实施例对m的取值不进行限定。
202：根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置。
示例性的，步骤202具体可以为：将待插值帧中与当前块相距L 的位置作为待插值帧中的与该当前块对应的待插值块的位置，其中，L是指当前块相对所述匹配块的运动向量的一半。一般地，假设当前块的坐标为(x1，y1)，当前块相对所述匹配块的运动向量为(a，b)，则当第一图像为第二图像的前一帧图像时，待插值帧中的与该当前块对应的待插值块的位置的坐标为：
Figure PCTCN2014093448-appb-000001
当第一图像为第二图像的后一帧图像时，待插值帧中的与该当前块对应的待插值块的位置的坐标为：
Figure PCTCN2014093448-appb-000002
203：基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值。
示例性的，本发明实施例对确定相似块的像素值相对当前块的像素值的权重值时，所使用的参考指标不进行限定。例如，该参考指标可以为相似块的像素值相对当前块的像素值的差值的绝对值(Sum of Absolute Difference，SAD)，相似块中的某一点的像素值相对该点在当前块中对应的点的像素值的绝对误差(Absolute Difference，AD)等，其中，相似块中的某一点相对该点在当前块中对应的点可以具体为：相似块中的左上顶点相对当前块中的左上顶点，或者，相似块中的中心点相对当前块中的中心点等。
本发明实施例对具体的加权算法不进行限定，可以为现有技术中的满足“参考指标(例如SAD或者SD)越大权重值越小”的规律的任一种加权算法。例如，可以为：高斯加权算法、线性加权算法、三角函数加权算法、自定义窗函数加权算法等。
在本发明的一个实施例中，在步骤203之前，该方法还可以包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值。该情况下，步骤203具体可以包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
进一步地，为了减少插值帧中的空洞(空洞是指没有***像素值的部分)，从而提高用户的观赏体验，发明提供下述实施例，具体的：
在本发明的一个实施例中，在步骤202之前，该方法还包括：按照相同的扩展倍率分别扩展所述当前块和所述相似块。
该情况下，步骤202具体包括：根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置。步骤204具体包括：将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
示例性的，本发明实施例扩展倍率的取值不进行限定，可以根据经验和具体场景进行设置。例如，经反复实验得出：在智真小幅度运动场景中，扩展倍率为2时，去除空洞的效果较好，且运动物体边界处扭曲和重影较小。
另外，需要说明的是，该情况下，步骤203还可以为：获取扩展后的相似块的像素相对扩展后的当前块像素值的权重值，将该权重值作为相似块的像素相对当前块像素值的权重值。
204：将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
在本发明的一个实施例中，所述相似块为所述第一图像的第一区域中的m个宏块；步骤203具体包括：分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数。步骤204具体包括：分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
205：将所述待插值块的像素值***所述待插值块的位置。
示例性的，将第一图像中的每个宏块作为当前块，执行步骤201-205之后，运动补偿插值过程完成，待插值帧成为了插值帧。该插值帧中的任一点的像素值为覆盖在该点的所有像素的和。
进一步地，在图像(包括第一图像和第二图像)中包含两个/多个运动物体，或者运动物体旋转、缩放、光照变化的场景中，运动估计过程确定的运动向量往往不精确。另外，当图像中运动物体出现遮挡时，由于被遮挡的块只存在于当前帧或者当前帧的前一帧中，所以利用两帧进行运动估计过程搜索不到遮挡部分的匹配块，该情况下，获得的匹配块可能是错误的。为了减少不精确的运动向量对帧率上采样的结果产生的影响，本发明提供下述实施例，具体的：
在本发明的一个实施例中，所述相似块为所述匹配块；该方法还可以包括：根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值。该情况下，步骤204具体包括：根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
示例性的，本发明实施例对根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值的具体方法不进行限定。例如，可以具体为：将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
可选的，在步骤205之后，该方法还可以包括：
A)对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
B)根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
示例性的，重叠像素是指被重复覆盖的像素。由于步骤201中的当前块可以为第一图像中的任一宏块，因此，在执行步骤205之后，待插值帧中可能存在重叠像素。
关于步骤A)和步骤B)的应用可以包括但不限于以下两种方式：方式1、在将第一图像中的任一宏块作为当前块执行步骤201-204之后，执行步骤A)和步骤B)；方式2、在将第一图像中的每个宏块作为当前块执行步骤201-204之后，执行步骤A)和步骤B)。由于在待插值帧生成插值帧的过程中，一重复像素可能被覆盖多次，若按照方式1， 需要多次执行步骤A)和步骤B)，因此，具体实现时，一般使用方式2，下述具体实施例中以方式2为例进行说明。
进一步地，本实施例中将方式2得到的待插值帧称为插值帧，由于插值帧中可能存在空洞，具体实现时，该方法还可以包括：对插值帧进行填补空洞的处理，其中，填补空洞的处理方法可以使用现有技术中的方法实现，此处不再描述。
本发明实施例提供的帧率上采样的方法，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
下面通过具体的实施例对实施例一中描述的帧率上采样的方法进行示例性说明。
实施例1
本实施例中，相似块为匹配块，且不对当前块和相似块进行扩展；另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含S个像素，S＝16×16。
如图3所示，为本发明实施例提供的一种帧率上采样的方法，包括：
301：输入第一图像和第二图像。
302：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。
示例性的，该步骤302可以称为运动估计过程，本实施例对具体的实现方式不进行限定。
303：将第一图像中的第i个宏块作为当前块；确定当前块的位置、与当前块对应的匹配块，当前块相对匹配块的运动向量。
其中，1≤i≤R，i为整数，i的初始值为1。
示例性的，“当前块相对匹配块的运动向量”中的“匹配块”是指：与当前块对应的匹配块，以下步骤中的“匹配块”均是指：与当前块对应的匹配块。
由于本实施例中不对当前块和相似块(匹配块)进行扩展，因此，可以使用当前块的中心点的坐标表示当前块的位置，使用匹配块的中心点的坐标表示匹配块的位置。
假设当前块的中心点的坐标为(x1，y1)，匹配块的中心点的坐标为(x2，y2)，则当第一图像为第n-1帧图像、第二图像为第n帧图像时，当前块相对匹配块的运动向量为：(x2-x1，y2-y1)；当第一图像为第n帧图像、第二图像为第n-1帧图像时，当前块相对匹配块的运动向量为：(x1-x2，y1-y2)。
304：根据当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与当前块对应的待插值块的位置。
示例性的，按照实施例一中的描述，针对步骤303中的示例，步骤304可以具体为：
当第一图像为第n-1帧图像、第二图像为第n帧图像时，插值帧中的与当前块对应的待插值块的位置可以表示为:
Figure PCTCN2014093448-appb-000003
Figure PCTCN2014093448-appb-000004
即
Figure PCTCN2014093448-appb-000005
当第一图像为第n帧图像、第二图像为第n-1帧图像时，插值帧中的与当前块对应的待插值块的位置可以表示为:
Figure PCTCN2014093448-appb-000006
Figure PCTCN2014093448-appb-000007
即
Figure PCTCN2014093448-appb-000008
由此可知，无论第一图像是第二图像的前一帧图像还是后一帧图 像，只要确定当前块的坐标和匹配块的坐标，即可确定与当前块对应的待插值块的坐标。
305：获取匹配块的像素值相对当前块的像素值的差值的绝对值。
示例性的，匹配块中的第s个像素的像素值相对当前块中的第s个像素的像素值的绝对误差AD(s)可以通过如下公式计算：
AD(s)＝|MBref(s)-MBsearch(s)|；
其中，MBref(s)是指第一图像(即参考帧)的当前块(MBref)中的第s个像素的像素值，MBsearch(s)是指第二图像(即搜索帧)中与当前块对应的匹配块(MBsearch)中的第s个像素的像素值。另外，按照步骤302中的示例，s∈{1，2，…，S}。
按照步骤303中的示例，匹配块的像素值相对当前块的像素值的差值的绝对值SAD可以通过如下公式计算：
Figure PCTCN2014093448-appb-000009
306：基于加权算法对该差值的绝对值进行加权运算，得到匹配块的像素值相对当前块的像素值的权重值。
示例性的，下面以加权算法为高斯加权算法对步骤306进行说明：
匹配块中的第s个像素的像素值相对当前块中的第s个像素的像素值的权重值w(s)可以根据以下公式获得：
Figure PCTCN2014093448-appb-000010
其中，μ是指期望值可以固定取0；σ是指方差，一般可取固定值5或者10，当然不限于此。
匹配块的像素值相对当前块的像素值的权重值可以根据以下公式获得：
Figure PCTCN2014093448-appb-000011
如图4所示，为σ取不同值时，w(s)与AD(s)之间的关系。由该图4可以看出，w(s)随着AD(s)的增加而减小，当AD(s)增大到一定程度时，w(s)趋于0。也就是说，当匹配块中的第s个像素的像素值与当前块中的第s个像素的像素值的差超过一定数值时，当前块中的第s个像素的像素值不会***插值帧中。例如，取σ＝10，由图4可知，当AD(s)超过30时，w(s)几乎降为0，也就是说，当匹配块中的第s个像素的像素值与当前块中的第s个像素的像素值的差超过30时，当前块中的第s个像素的像素值不会***插值帧中。由此可知，当匹配块的像素值与当前块的像素值的差超过一定数值时，当前块中的像素值不会***插值帧中。
307：将匹配块的像素值乘以权重值，得到待插值块的像素值。
示例性的，待插值块中的第s个像素的像素值MBinterpolate(s)可以根据以下公式获得：
MBinterpolate(s)＝MBsearch(s)w(s)；
待插值块的像素值
Figure PCTCN2014093448-appb-000012
可以根据以下公式获得：
Figure PCTCN2014093448-appb-000013
308：将待插值块的像素值***待插值块的位置。
示例性的，本实施例中结合当前块的像素值、匹配块的像素值得到待插值块的像素值，并将待插值块的像素值***待插值块的位置，如图5所示，其中，图5中第一图像为第n-1帧图像，第二图像为第n帧图像。
309：判断i是否大于等于R。
若否，则执行步骤310；若是，则执行步骤311。
示例性的，步骤309的判断结果为“是”，说明：已经在待插值帧中***了第一图像中的每个宏块对应的待***块。
310：i自加1。
步骤310之后，返回步骤303。
311：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。
假设像素t为一重叠像素，则像素t的中间像素值ws(t)可以根据以下公式获得：
Figure PCTCN2014093448-appb-000014
其中，J是指该像素t被重复覆盖的次数，即覆盖该像素t的待插值块的个数；MBint erpolate(s)j是指覆盖该像素t的第j个待插值块(MBint erpolate)，w(t)j是指覆盖像素t的第j个待插值块(MBint erpolate)相对与第j个待插值块对应的当前块中相应像素的像素值的权重值。
312：根据步骤306中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。
示例性的，按照步骤311中的示例，步骤312具体可以实现为：
像素t的权重和sw(t)可以通过以下公式获得：
Figure PCTCN2014093448-appb-000015
像素t的目标像素值
Figure PCTCN2014093448-appb-000016
可以通过以下公式获得：
Figure PCTCN2014093448-appb-000017
313：输出该插值帧。
可选的，如图6所示，步骤307可以用以下步骤307A-307B替换：
307A：将当前块的像素值与匹配块的像素值的均值作为修正块的 像素值。
示例性的，分别将当前块中的每个像素的像素值与匹配块中相应像素的像素值求均值，得到修正块中相应像素的像素值。例如，将当前块中的第3个像素的像素值与匹配块中的第3个像素的像素值求均值，得到修正块中第3个像素的像素值。
307B：根据匹配块的像素值、修正块的像素值和步骤306中获得的权重值，得到待插值块的像素值。
示例性的，步骤307B可以包括但不限于通过以下方式实现：
待插值块中的第s个像素的像素值MBint erpolate(s)可以根据以下公式获得：
MBint erpolate(s)＝MBsearch(s)w(s)+MBcorrect(1-w(s))；其中，MBcorrect是指修正块。
待插值块的像素值
Figure PCTCN2014093448-appb-000018
可以根据以下公式获得：
Figure PCTCN2014093448-appb-000019
需要说明的是，用步骤307A-307B替换步骤307的方案，可以减少不精确的运动向量对帧率上采样的结果产生的影响。
本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的匹配块的像素值相对当前块的像素值的权重值，并将动态获取的权重值用于获得待插值块的像素值的过程中，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
实施例2
本实施例中，相似块为匹配块，且对当前块和相似块进行扩展；另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含16×16个像素。
如图7所示，为本发明实施例提供的一种帧率上采样的方法，包括：
701：输入第一图像和第二图像。
702：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。
703：将第一图像中的第i个宏块作为当前块；确定当前块的位置、与当前块对应的匹配块，当前块相对匹配块的运动向量。其中，1≤i≤R，i为整数，i的初始值为1。
示例性的，步骤701-703的相关解释和具体示例均可以参考上述实施例1的步骤301-303，下面仅对与实施例1的不同之处进行说明：
由于本实施例中对当前块和相似块(匹配块)进行扩展，这里的“扩展”一般是指以原点为中心扩展当前块和匹配块的像素，扩展前后一个块(包括当前块块或者匹配块)的中心点的坐标不改变，因此，一般不使用块(包括当前块块或者匹配块)的中心点的坐标表示该块的位置，本实施例中使用当前块的左顶点坐标表示当前块的位置，使用匹配块的左顶点坐标表示匹配块的位置。
704：按照相同的扩展倍率分别扩展当前块和匹配块。
示例性的，本实施例中以扩展倍率为2进行说明，具体是指，将宏块的边长扩展2倍。按照扩展倍率为2进行扩展后，每个宏块中包含S个像素，S＝32×32。
705：根据扩展后的当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与扩展后的当前块对应的待插值块的位置。
示例性的，如图8所示，假设当前块的左顶点为A，当前块的位置具体可以为A点的坐标，扩展后的当前块的左顶点为B，则扩展后的当前块的位置具体为B点的坐标。
706：获取匹配块的像素值相对当前块的像素值的差值的绝对值。
707：基于加权算法对该差值的绝对值进行加权云端，得到匹配块 的像素相对当前块的像素值的权重值。
可选的，步骤706可以用以下步骤替换：获取扩展后的匹配块的像素值相对扩展后的当前块的像素值的差值的绝对值；该情况下，步骤707可以用以下步骤替换：根据加权算法和该差值的绝对值，获取扩展后的匹配块的像素相对扩展后的当前块的像素值的权重值。
本实施例中的步骤706-707与实施例1中的步骤305-306相同。可选的方式中的具体示例可以参考实施例1中的步骤305-306，不同之处在于，S的取值不同。
708：将扩展后的匹配块的像素值乘以该权重值，得到待插值块的像素值。
709：将待插值块的像素值***待插值块的位置。
示例性的，本实施例中结合扩展后的当前块的像素值、扩展后的匹配块的像素值得到待插值块的像素值，并将待插值块的像素值***待插值块的位置，如图9所示，其中，图9中第一图像为第n-1帧图像，第二图像为第n帧图像。
710：判断i是否大于等于R。
若否，则执行步骤711；若是，则执行步骤712。
示例性的，步骤710的判断结果为“是”，说明：已经在待插值帧中***了第一图像中的每个宏块对应的待***块。
711：i自加1。
步骤711之后，返回步骤703。
712：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。
713：根据步骤707中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。
714：输出该插值帧。
示例性的，步骤710-714的具体示例可以参考实施例1中的步骤309-313，此处不再赘述。
可选的，如图10所示，步骤708可以用以下步骤708A-708B替换：
708A：将扩展后的当前块的像素值与扩展后的匹配块的像素值的均值作为修正块的像素值。
708B：根据扩展后的匹配块的像素值、修正块的像素值和步骤707中获得的权重值，得到待插值块的像素值。
需要说明的是，用步骤708A-708B替换步骤708的方案，可以减少不精确的运动向量对帧率上采样的结果产生的影响。
示例性的，本实施例中的步骤708-714、708A、708B中的相关解释和具体示例可以参考实施例1中相关，其中，具体实现时，需要将实施例1中的“匹配块”的替换为“扩展后的匹配块”，将实施例1中的“当前块”替换为“扩展后的当前块”；将实施例1中S的取值由16×16替换为32×32。
2)S的取值不同。
另外，实施例1中获得的每个待插值块中包含16×16个像素，实施例2中获得的每个待插值块中包含32×32个像素。
本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的匹配块的像素值相对当前块的像素值的权重值，并对当前块和匹配块进行扩展，结合动态获取的权重值、扩展后的当前块的像素的像素值和扩展后的匹配块的像素的像素值获取待插值块的像素值，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
实施例3
本实施例中，相似块为第一图像的第一区域中的m个宏块，且不 对相似块进行扩展。另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含S个像素，S＝16×16。
如图11所示，为本发明实施例提供的另一种帧率上采样的方法，包括：
1101：输入第一图像和第二图像。
1102：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。
示例性的，该步骤1102可以称为运动估计过程，本实施例对具体的实现方式不进行限定。
1103：将第一图像中的第i个宏块作为当前块；确定当前块的位置、当前块相对第二图像中一匹配块的运动向量、当前块对应的相似块，其中，相似块为第一图像的第一区域中的m个宏块。
其中，1≤i≤R，i为整数，i的初始值为1；m≥1，m为整数。
示例性的，第一区域可以由当前块的邻域和/或时域对应的宏块的邻域构成。如图12所示，为当前块对应的相似块的示例，当然，具体实现时不限于此。
1104：根据当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与当前块对应的待插值块的位置。
1105：获取第j个相似块的像素值相对当前块的像素值的差值的绝对值。
1106：基于加权算法，对该差值的绝对值进行加权运算，得到第j个相似块的像素值相对当前块的像素值的权重值。
其中，j的初始值为1，1≤j≤m，j为整数。
示例性的，步骤1105-1106的具体示例可以参考上述实施例1中的步骤305-306。其中，具体实现时，将步骤305-306中的“匹配块的像素值”替换为“第j个相似块的像值素”。
1107：将第j个相似块的像素值乘以该权重值，得到待插值块的第j像素值。
需要说明的是，为了清楚说明本实施例的方法，引入“插值块的第j像素值”的概念。“插值块的第j像素值”具体是指，第j次获得与第i个当前块对应的待插值块的像素值。
另外需要说明，在实施例1和实施例2中，针对第i个当前块，只获取一次与该第i个当前块对应的待插值块的像素值；本实施例中，针对第i个当前块，获取m次与该第i个当前块对应的待插值块的像素值。
1108：判断j是否大于或者等于m。
若否，则执行步骤1109；若是，则执行步骤1111。
示例性的，步骤1107的判断结果为“是”，说明：已经获取了m次与第i个当前块对应的待插值块的像素值。
1109：j自加1。
执行步骤1109之后，返回步骤1105。
1110：对待插值块的m个像素值进行求和运算，得到待插值块的像素值。
1111：将待插值块的像素值***待插值块的位置。
示例性的，本实施例中结合当前块的像素值、当前块周围的相似块的像素值得到待插值块的像素值，并将待插值块的像素值***待插值块的位置，如图13所示，其中，图13中第一图像为第n-1帧图像，第二图像为第n帧图像。
1112：判断i是否大于等于R。
若否，则执行步骤1113；若是，则执行步骤1114。
示例性的，步骤1112的判断结果为“是”，说明：已经在待插值帧中***了第一图像中的每个宏块对应的待***块。
1113：i自加1。
步骤1113之后，返回步骤1103。
1114：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。
1115：根据步骤1106中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。
1116：输出该插值帧。
示例性的，步骤1112-1116的具体示例可以参考实施例1中的步骤309-313，此处不再赘述。
本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将动态获取的权重值用于获得待插值块的像素值的过程中，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
实施例二
如图14所示，为本发明实施例提供的一种帧率上采样的装置14，用以执行图1所示的帧率上采样的方法，该装置14包括：
确定单元141，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
位置获取单元142，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；
权重值获取单元143，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
像素值获取单元144，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
插值单元145，用于将所述待插值块的像素值***所述待插值块的位置。
可选的，所述权重值获取单元143还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
所述权重值获取单元143具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
可选的，如图15所示，所述装置14还包括：
扩展单元146，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；
所述位置获取单元142具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；
所述像素值获取单元144具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
可选的，所述像素值获取单元144还用于：
对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
可选的，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
可选的，所述相似块为所述匹配块；
所述像素值获取单元144还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
所述像素值获取单元144具体用于，根据所述匹配块的像素值、 所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
可选的，所述像素值获取单元144具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
可选的，所述相似块为所述第一图像的第一区域中的m个宏块；
所述权重值获取单元143具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
本发明实施例提供的帧率上采样的装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
实施例三
如图16所示，为本发明实施例提供的一种帧率上采样的装置14，用以执行图1所示的帧率上采样的方法，该装置14包括：存储器161和处理器162，其中，
存储器161用于存储一组代码，该代码用于控制处理器162执行以下动作：
确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
根据所述当前块的位置和所述当前块相对所述匹配块的运动向 量，获取待插值帧中的与所述当前块对应的待插值块的位置；
基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
将所述待插值块的像素值***所述待插值块的位置。
可选的，处理器162还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
处理器162具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
可选的，处理器162还用于，按照相同的扩展倍率分别扩展所述当前块和所述相似块；
处理器162具体用于：
根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；
将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
可选的，处理器162还用于：
对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
可选的，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
可选的，所述相似块为所述匹配块；
处理器162还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
处理器162具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
可选的，处理器162具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
可选的，所述相似块为所述第一图像的第一区域中的m个宏块；
处理器162具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
本发明实施例提供的帧率上采样的装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的***，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实 施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理包括，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。

Claims (16)

  1. 一种帧率上采样的方法，其特征在于，包括：
    确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
    根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；
    基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
    将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
    将所述待插值块的像素值***所述待插值块的位置。
  2. 根据权利要求1所述的方法，其特征在于，
    在所述获取所述相似块的像素值相对所述当前块的像素值的权重值之前，所述方法还包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
    所述基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
  3. 根据权利要求1所述的方法，其特征在于，在所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置之前，所述方法还包括：
    按照相同的扩展倍率分别扩展所述当前块和所述相似块；
    所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置，包括：
    根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块 的位置；
    所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
    将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
  4. 根据权利要求1所述的方法，其特征在于，在所述将所述待插值块的像素值***所述待插值块的位置之后，所述方法还包括：
    对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
    根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，
    所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
  6. 根据权利要求5所述的方法，其特征在于，所述相似块为所述匹配块；所述方法还包括：
    根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
    所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
    根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值，包括：
    将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
  8. 根据权利要求5所述的方法，其特征在于，所述相似块为所述第一图像的第一区域中的m个宏块；
    所述获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：
    分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
    所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：
    分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
  9. 一种帧率上采样的装置，其特征在于，包括：
    确定单元，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；
    位置获取单元，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；
    权重值获取单元，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；
    像素值获取单元，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；
    插值单元，用于将所述待插值块的像素值***所述待插值块的位置。
  10. 根据权利要求9所述的装置，其特征在于，
    所述权重值获取单元还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；
    所述权重值获取单元具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。
  11. 根据权利要求9所述的装置，其特征在于，所述装置还包括：
    扩展单元，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；
    所述位置获取单元具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；
    所述像素值获取单元具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。
  12. 根据权利要求9所述的装置，其特征在于，所述像素值获取单元还用于：
    对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；
    根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。
  13. 根据权利要求9-12任一项所述的装置，其特征在于，
    所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。
  14. 根据权利要求13所述的装置，其特征在于，所述相似块为所述匹配块；
    所述像素值获取单元还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；
    所述像素值获取单元具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。
  15. 根据权利要求14所述的装置，其特征在于，
    所述像素值获取单元具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。
  16. 根据权利要求13所述的装置，其特征在于，所述相似块为所述第一图像的第一区域中的m个宏块；
    所述权重值获取单元具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；
    分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。
PCT/CN2014/093448 2013-12-10 2014-12-10 一种帧率上采样的方法和装置 WO2015085922A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310669856.3 2013-12-10
CN201310669856.3A CN103647973B (zh) 2013-12-10 2013-12-10 一种帧率上采样的方法和装置

Publications (1)

Publication Number Publication Date
WO2015085922A1 true WO2015085922A1 (zh) 2015-06-18

Family

ID=50253116

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/093448 WO2015085922A1 (zh) 2013-12-10 2014-12-10 一种帧率上采样的方法和装置

Country Status (2)

Country Link
CN (1) CN103647973B (zh)
WO (1) WO2015085922A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104079950A (zh) * 2014-07-04 2014-10-01 福建天晴数码有限公司 视频输出处理、视频接收处理的方法、装置和***
US9978180B2 (en) 2016-01-25 2018-05-22 Microsoft Technology Licensing, Llc Frame projection for augmented reality environments
CN113014928A (zh) * 2021-02-08 2021-06-22 北京奇艺世纪科技有限公司 一种补偿帧生成方法及装置
CN113556581A (zh) * 2021-07-20 2021-10-26 深圳市爱协生科技有限公司 视频插值帧的生成方法、装置及电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103647973B (zh) * 2013-12-10 2017-01-04 华为技术有限公司 一种帧率上采样的方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101073256A (zh) * 2004-12-09 2007-11-14 汤姆森许可贸易公司 产生运动补偿图像的方法和设备
CN101277419A (zh) * 2007-03-27 2008-10-01 株式会社东芝 帧内插设备与方法
CN101361369A (zh) * 2006-01-06 2009-02-04 国际商业机器公司 用于视觉信号外插或内插的***和方法
CN101483771A (zh) * 2008-01-11 2009-07-15 奇景光电股份有限公司 提升帧速率的方法与装置
CN102868879A (zh) * 2011-07-05 2013-01-09 北京大学 一种视频帧速率上转换方法及***
CN103647973A (zh) * 2013-12-10 2014-03-19 华为技术有限公司 一种帧率上采样的方法和装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100541953B1 (ko) * 2003-06-16 2006-01-10 삼성전자주식회사 움직임 보상을 위한 화소값 선택 장치 및 방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101073256A (zh) * 2004-12-09 2007-11-14 汤姆森许可贸易公司 产生运动补偿图像的方法和设备
CN101361369A (zh) * 2006-01-06 2009-02-04 国际商业机器公司 用于视觉信号外插或内插的***和方法
CN101277419A (zh) * 2007-03-27 2008-10-01 株式会社东芝 帧内插设备与方法
CN101483771A (zh) * 2008-01-11 2009-07-15 奇景光电股份有限公司 提升帧速率的方法与装置
CN102868879A (zh) * 2011-07-05 2013-01-09 北京大学 一种视频帧速率上转换方法及***
CN103647973A (zh) * 2013-12-10 2014-03-19 华为技术有限公司 一种帧率上采样的方法和装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104079950A (zh) * 2014-07-04 2014-10-01 福建天晴数码有限公司 视频输出处理、视频接收处理的方法、装置和***
US9978180B2 (en) 2016-01-25 2018-05-22 Microsoft Technology Licensing, Llc Frame projection for augmented reality environments
CN113014928A (zh) * 2021-02-08 2021-06-22 北京奇艺世纪科技有限公司 一种补偿帧生成方法及装置
CN113014928B (zh) * 2021-02-08 2023-01-20 北京奇艺世纪科技有限公司 一种补偿帧生成方法及装置
CN113556581A (zh) * 2021-07-20 2021-10-26 深圳市爱协生科技有限公司 视频插值帧的生成方法、装置及电子设备
CN113556581B (zh) * 2021-07-20 2023-11-03 深圳市爱协生科技股份有限公司 插值帧的生成方法、装置及电子设备

Also Published As

Publication number Publication date
CN103647973B (zh) 2017-01-04
CN103647973A (zh) 2014-03-19

Similar Documents

Publication Publication Date Title
KR101393048B1 (ko) 이미지들의 수퍼 해상도를 위한 방법 및 장치
US7899122B2 (en) Method, apparatus and computer program product for generating interpolation frame
WO2015085922A1 (zh) 一种帧率上采样的方法和装置
US9055217B2 (en) Image compositing apparatus, image compositing method and program recording device
TW201146011A (en) Bi-directional, local and global motion estimation based frame rate conversion
JP5089610B2 (ja) ブロック基盤の動き推定方法及び装置
JP2003274416A (ja) 適応型動き推定装置及び推定方法
JP2007181674A (ja) ブロック整合及び動き補償補間を利用した映像形成方法
JP2010288110A (ja) 画像処理装置および画像処理方法
US20160080770A1 (en) Encoding system using motion estimation and encoding method using motion estimation
TWI408620B (zh) 用以決定插補畫面之移動向量的影像處理方法及其相關裝置
JP2005012797A (ja) 動き補償のための画素値選択装置および方法
US8406305B2 (en) Method and system for creating an interpolated image using up-conversion vector with uncovering-covering detection
US20120093231A1 (en) Image processing apparatus and image processing method
JP2010148037A (ja) 動画像再生装置、動画像再生方法および動画像再生プログラム
KR100855976B1 (ko) 배경과 이동 객체를 분리하여 움직임을 추정하는 프레임보간 장치 및 프레임 보간 방법
JP5927051B2 (ja) 画像処理装置、画像処理方法及びプログラム
Guo et al. Frame rate up-conversion using linear quadratic motion estimation and trilateral filtering motion smoothing
JP2019067169A (ja) 移動ベクトル算出方法、装置、プログラム、及びノイズ除去処理を含む移動ベクトル算出方法
JP2015216459A (ja) 画像処理装置、撮像装置及び画像処理方法
US8305500B2 (en) Method of block-based motion estimation
JP5478533B2 (ja) 全方位画像生成方法、画像生成装置およびプログラム
JP5448983B2 (ja) 解像度変換装置及び方法、走査線補間装置及び方法、並びに映像表示装置及び方法
JP2006215657A (ja) 動きベクトル検出方法、動きベクトル検出装置、動きベクトル検出プログラム及びプログラム記録媒体
JP5927053B2 (ja) 画像処理装置、画像処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14870056

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14870056

Country of ref document: EP

Kind code of ref document: A1