CN1250000C - Method for determining repeated pattern frame interpolating method and frame interpolating apparatus - Google Patents
Method for determining repeated pattern frame interpolating method and frame interpolating apparatus Download PDFInfo
- Publication number
- CN1250000C CN1250000C CNB2003101205871A CN200310120587A CN1250000C CN 1250000 C CN1250000 C CN 1250000C CN B2003101205871 A CNB2003101205871 A CN B2003101205871A CN 200310120587 A CN200310120587 A CN 200310120587A CN 1250000 C CN1250000 C CN 1250000C
- Authority
- CN
- China
- Prior art keywords
- repeat patterns
- piece
- block
- reference block
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
- H04N7/0147—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes the interpolation using an indication of film mode or an indication of a specific pattern, e.g. 3:2 pull-down pattern
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
- H04N7/014—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes involving the use of motion vectors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Television Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A repeated pattern judging method, a frame interpolation method using the same and an apparatus suitable for the same are provided to discriminate that a repeated pattern moves between frames. An MAE(Mean Ratio Error) calculating unit(1202) obtains MAEs with respect to respective reference blocks, which exist in a search range with an (M+2P)x(N+2P) size of a frame which refers for searching a reference block and a motion vector, by a full search method. An MAE map storing unit(1204) stores (M+P)x(N+P) MAEs as a map type with an (M+P)x(N+P) size. A motion vector extracting unit(1206) judges positions(x,y) of horizontal and vertical directions of a reference block having the least MAE among the MAEs stored in the MAE map storing unit(1204) as an interframe motion direction with respect to a current block. A repeated pattern recognizing unit(1210) discriminates whether a repeated pattern exists with reference to the MAEs stored in the MAE map storing unit(1204). A frame interpolation unit(1212) performs frame interpolation using a motion vector extracted in the motion vector extracting unit(1206) and linear interpolation.
Description
Technical field
The present invention relates to a kind of converting frame rate device, relate in particular to a kind of method, this method determines whether piece is included in the repeat patterns, is used to have the valid frame interpolation of the image of repeat patterns, and frame interpolation method and frame interpolation are put.
Background technology
The resolution of image depends on the pixel count that is comprised in the frame.1,920 * 1,080 ultimate resolution, a frame is made up of 1,920 pixel on each horizontal line and 1,080 pixel on each vertical line.Frame rate is represented the number of the frame that per second sends.When the picture signal that sends as TV signal, determine frame rate based on people's visual characteristic.
Usually, according to the requirement of locality, with the picture signal of various frequency broadcasting from image output device output.For example, in Europe and China, should export the picture signal that vertical frequency is 50Hz, and, should export the picture signal that vertical frequency is 60Hz in Korea S and North America.
When the picture signal of the various frequencies of output, image output device needs frequency translation.This frequency translation is called as converting frame rate.Particularly, when low frequency transform is arrived high frequency, must increase the number of frame.
In the past, by using the motion vector of estimating based on the difference between the consecutive frame, by repeating consecutive frame or creating the number that new frame increases frame.
High-resolution system checks the motion of present frame according to the movement tendency of image, constructs the image that seems nature to use the motion vector correction technology.This motion vector correction technology comprises: extract the piece with a plurality of motion vectors (motion vector that comprises correction target), and proofread and correct with this piece in the different motion vector of those nearby motion vectors directions, so the motion vector after the correction is identical with the nearby motion vectors direction.
As mentioned above, the conventional frame interpolating method uses the correlation of interframe to be provided for the superperformance of proper motion image or rest image.Yet, when the image with repeat patterns when interframe moves, the conventional frame interpolating method can't guarantee enough frame interpolation performances, because be difficult to accurate estimating motion vector.
For example, when the pattern that periodically repeats moved, for example striped shirt, striped tablecloth or the building of a string window is arranged were then compared with proper motion image or rest image, are difficult to the motion vector between accurate estimated frame.This is because the correlation of interframe changes significantly.
Therefore, need a kind of method, be used for the situation of determining that repeat patterns moves in interframe, with and effective frame interpolation method.
Summary of the invention
The invention provides a kind of method, be used for when the image with repeat patterns moves, determine the pattern that repeats.
The present invention also provides a kind of frame interpolation method, uses when the image with repeat patterns moves.
The present invention also provides a kind of frame interpolation to put, and uses this frame interpolation method.
According to an aspect of the present invention, a kind of method of definite repeat patterns is provided, be used for when the image with repeat patterns moves, whether the reference block of determining M * N is included in the described repeat patterns, this method may further comprise the steps: use search fully, obtain the error of calibrated bolck and reference block, each reference block is comprised in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process; According to the order of reference block, arrange (M+P) * (N+P) individual error with the form of (M+P) * (N+P) figure; In the left diagonal and right diagonal of this figure, obtain each deviation between current block and the adjacent block, and be accumulated in deviation that obtains in the left diagonal and the deviation that in right diagonal, obtains respectively; The deviation of accumulating in deviation of accumulating in the more left diagonal and the right diagonal, and choose bigger deviation; Selected deviation and threshold value 1 are compared; And if selected deviation determines then that greater than threshold value 1 this reference block is included in the repeat patterns.
This method also comprises: the sub-piece that this figure is divided into identical size; Calculate the ratio of the worst error and the minimal error of each sub-piece; Counting has the sum greater than the sub-piece of the ratio of threshold value 2; And if should sum greater than threshold value 3, then determine this a little and be included in the repeat patterns.
This method also comprises: inspection has the distribution greater than the sub-piece of the ratio of threshold value 2, and determines whether this sub-piece is included in the pseudo-repeat patterns.
In the method, by relatively on the level of this figure, vertical and diagonal, concentrating the pseudo-repeat patterns of the sub-piece that distributes, determine whether this sub-piece is included in this puppet repeat patterns.
This method also comprises: check the repetition degree that is confirmed as being included in the piece in the repeat patterns in reference block and the adjacent block; And, determine that reference block is included in this repeat patterns based on the degree that repeats.
According to a further aspect in the invention, a kind of frame interpolation method is provided, may further comprise the steps: (a) use search fully, obtain the error of calibrated bolck and reference block, each reference block is comprised in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame that is referenced in the motion vector sought process; (b) estimating motion vector is the positional information with reference block of minimal error; (c), determine whether described calibrated bolck and reference block are included in the repeat patterns according to the error that is obtained; (d) calculate the current block that will be interpolated and around the correlation between the adjacent block of this current block; And (e) according to the correlation of being calculated, by mixing piece that forms by linear interpolation and the piece that forms by estimation and motion compensation, the image after the acquisition interpolation.
Step (d) further comprises: count the number that all center on reference block and are included in all adjacent blocks in the repeat patterns; And calculate around the ratio of the number of the sum of the adjacent block of reference block and counting.
By interpolation the current block that is formed is represented as
Wherein, (α represents to be included in the number of the adjacent block in the repeat patterns to f for i, the j) location of pixels of expression current block, and total represents the sum of adjacent block.
According to a further aspect in the invention, provide a kind of frame interpolation to put, comprise: the mean absolute error computing unit, it uses search fully, mean absolute error between basis of calculation piece and reference block, each reference block is comprised in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process; Mean absolute error figure memory cell, it is according to the order of reference block, with form storage (M+P) * (N+P) individual error of (M+P) * (N+P) figure; Motion vector extraction unit, its direction of motion with current block be identified as the reference block with the minimum average B configuration absolute error in each mean absolute error that is stored among this mean absolute error figure the position (x, y); The repeat patterns determining unit, it determines with reference to the mean absolute error figure that is stored in the mean absolute error figure memory cell whether piece is included in the repeat patterns; And the frame interpolation unit, use the motion vector and the linear interpolation of extracting to come interpolation frame.
Description of drawings
More than of the present invention and other aspects and advantage will become apparent when describing one exemplary embodiment with reference to the accompanying drawings, wherein:
Fig. 1 is the schematic diagram of symmetry blocks matching process;
The block diagram that Fig. 2 puts for the frame interpolation that uses conventional motion estimation and motion compensation (ME/MC);
Fig. 3 is the flow chart according to the method for definite repeat patterns of the present invention;
Fig. 4 illustrates being included in the piece in the repeat patterns, the figure that is formed by the MAE in the left and right sides diagonal that is presented on MAE figure;
Fig. 5 illustrates the MAE figure of general pattern;
Fig. 6 illustrates the MAE figure that is included in the piece in the repeat patterns, and wherein, each MAE figure is divided into 16 sub-pieces, and marks the zone that has greater than 3.0 MAE ratio with stain;
Fig. 7 illustrates the image with pseudo-repeat patterns;
Fig. 8 illustrates MAE figure, wherein marks the zone that has greater than 3.0 MAE ratio;
Fig. 9 illustrates for the sub-piece that has greater than 3.0 MAE ratio and carries out the arrowband classification, and described sub-piece concentrates on certain direction;
Figure 10 is for describing the flow chart according to frame interpolation of the present invention;
Figure 11 is the figure that explains the correlation between current block and the adjacent block;
The block diagram that Figure 12 puts for frame interpolation according to the present invention;
Figure 13 illustrates and is used for the adjacent block that correlation is checked;
Figure 14 A to 14D illustrates the test pattern with repeat patterns;
Figure 15 illustrates the image of handling by traditional MC, and the image that wherein has repeat patterns is disperseed;
Figure 16 illustrates the image of handling by according to the method for interpolation frame of the present invention, and wherein false picture is suppressed visibly.
Embodiment
Describe the present invention in detail referring now to accompanying drawing, wherein accompanying drawing shows the preferred embodiments of the present invention.
The block matching method that is used for estimation can be categorized as forward block matching process, back to block matching method and symmetry blocks matching process.Because the track of motion is very important to converting frame rate, thus generally use the symmetry blocks matching process to be used for converting frame rate, as shown in Figure 1.
Fig. 1 is the schematic diagram of symmetry blocks matching process.In Fig. 1, in the promptly symmetrical matching process, obtain each error of calibrated bolck with the reference block in the motion vector sought zone (M+2P) * (N+2P) that is positioned at reference frame of present frame.In the symmetry blocks matching process, this reference frame refers to former frame or back one frame.
Referring to Fig. 1, frame k-1 represents former frame, and frame k+1 represents back one frame, and frame K represents the present frame that will be interpolated.The calibrated bolck of present frame is by label 102 expressions.The motion vector sought zone of former frame is by label 110 expressions, and then the motion vector sought zone of a frame is by label 120 expressions.The area of tentative standard piece 102 equals M * N, and then the motion vector sought zone of calibrated bolck 102 will equal (M+2P) * (N+2P), wherein P represent from calibrated bolck to+/-x and+/-number of pixels that the y direction extends out.
Usually, the area of calibrated bolck 102 equals 8 * 8 pixels (comprising 64 pixels altogether), and the motion vector sought zone of calibrated bolck 102+/-the x direction accounts for 4 pixels ,+/-the y direction accounts for 4 pixels, promptly equals 16 * 16 pixels (comprising 256 pixels altogether).
Here, with classification for search for fully the search or diamond search.Search fully commonly used.
In search fully, be that unit moves and has and calibrated bolck reference block of the same area with a pixel, and compare with calibrated bolck.Thus, have 256 reference blocks.
The symmetry blocks matching process uses absolute difference sum (SAD) to obtain each error of the reference block in the motion vector sought zone of reference block in the motion vector sought zone of former frame and back one frame.
According to the symmetry blocks matching process, mutual benchmark piece 112 and reference block 122, the upper left side in the motion vector sought zone 120 of reference block 112 in frame k-1, and the lower right side in the motion vector sought zone 120 of reference block 122 in frame k+1.Equally, mutually benchmark piece 114 and reference block 124, reference block 114 be than reference block 112 pixel to the right, reference block 124 than reference block 122 to the first from left pixel.
Obtain each SAD by mutual more symmetrical reference block, and the result be stored in (M+P) * (N+P) SAD figure in.
The block diagram that Fig. 2 puts for the frame interpolation that uses conventional motion estimation and motion compensation (ME/MC).
SAD computing unit 202 uses symmetric motion to amount estimation method, the SAD between the symmetrical reference block in the motion vector sought zone 110 of calculating frame k-1 and the motion vector sought zone 120 of frame k+1.
Wherein, M and N represent the wide and high of calibrated bolck, and f
1(i, j) and f
2(i, j) location of pixels in the expression former frame and the location of pixels in one frame of back.
Because the number of the reference block in each motion vector sought zone equals 256, so the number of the SAD that obtains equals 256.
This SAD figure of SAD figure memory cell 204 storages.576 SAD that this SAD figure storage is tried to achieve by SAD computing unit 202.
Motion vector extraction unit 206 is determined to have in the SAD figure memory cell 204 among each SAD of storage, and (x, y), and (x is y) as the direction of image motion in the current block to discern this position in the position of the reference block of the minimum SAD among the candidate that promptly moves.
As mentioned above, referring to Fig. 1 and Fig. 2, employed conventional motion estimation is identified as optimal motion vectors with the motion vector that has minimum SAD in the motion vector sought zone in converting frame rate.
Yet, even this optimal motion vectors can comprise the error of relevant motion total amount or direction.So, the correlation between motion filter 208 checking current blocks and the adjacent block.
Can use median filter or average filter as motion filter 208.Median filter is by the sequence arrangement current block from the minimum to the maximum and the motion vector of adjacent block, and the median in the motion vector of extraction arrangement.The mean value of all motion vectors of mean value filter calculations.
The final motion vector of being handled by motion filter 208 is considered to have the best information of current block, and comprised the motion vector of former frame and back one frame mean value (dx, dy).
Following equation 2 these frame interpolation methods of expression.
Wherein, f
k(i, j) location of pixels of the frame of expression interpolation.
The motion vector that use obtains by conventional motion estimation, conventional frame interpolating method have shown enough performances when supporting generally mobile or rest image.But, when the image with repeat patterns when interframe moves, be difficult to use the conventional motion estimation algorithm to obtain motion vector accurately.
For example, when having the image that periodically repeats pattern when moving, for example striped shirt, striped tablecloth or the building of a string window is arranged are difficult to accurate estimating motion vector.This is because the correlation of interframe changes significantly.
Briefly, use conventional motion estimation to be difficult to accurately estimate to have the movable information that periodically repeats the image of pattern, this has just caused the motion artifacts (artifact) of scattering as image.
Therefore, the present invention proposes a kind of algorithm, it allows to determine the pattern of repetition.The invention allows for a kind of frame interpolation method, by this method, be contained in piece in the repeat patterns, and use motion vector to handle other pieces by traditional ME/MC by the linear interpolation pack processing.In addition, the invention allows for a kind of frame interpolation of this frame interpolation method that uses puts.
Fig. 3 is for determining the flow chart of the method for repeat patterns according to the present invention.
This algorithm according to definite repeat patterns of the present invention carried out with three steps.
Change this fact largely based on the correlation between current block and the adjacent block, can make following two hypothesis.
The first, in repeat patterns, the correlation marked change between the mean absolute error among the MAE figure (MAE).Compare with normal situation mobile or rest image, the left side and the deviation between the MAE in the right diagonal that are included in the piece in the repeat patterns are bigger.
The second, the marked change of the correlation between MAE evenly distributes on the whole motion vector sought zone of this piece.
Under these hypothesis, determine that the algorithm of repeat patterns carried out with three steps.In the following description, the MAE map distance is represented the deviation between the MAE, and each piece error is represented MAE or SAD.
The MAE of the mean value of the difference sum between the remarked pixel can obtain with following formula 3
Wherein, M and N represent the wide and high of piece, and f
1And f
2Location of pixels in location of pixels in the expression former frame and back one frame.
In the step of determining the MAE map distance, according to first hypothesis, from the regional MAE figure (step S302) that obtains of the motion vector sought of former frame and back one frame.Then, each motion vector sought zone is divided into sub-piece (step S304).Obtain maximum MAE and minimum MAE from each sub-piece, and calculate the ratio of maximum MAE and minimum MAE, promptly MAE is than (Max/Min) (step S306).
Compare with proper motion or rest image, the correlation that has between image each MAE in MAE figure of repeat patterns is changed to bigger degree.By the left side in searching moving vector search zone and the motion in the right diagonal, can discern the motion of repeat patterns.
Can use following method to measure the degree that the correlation between each MAE changes among the MAE figure.
According to the order of reference block, be form arrangement (M+P) * (N+P) individual MAE of the MAE figure of (M+P) * (N+P) with size.In this MAE figure, try to achieve and accumulate the deviation between the adjacent MAE in left diagonal and the right diagonal.More left cornerwise accumulated value and right cornerwise accumulated value are obtained higher value, and this higher value and threshold value 1 are compared mutually.
Then MAE figure is divided into sub-piece.Obtain the MAE ratio from each sub-piece.Add up to the sub-piece number that has greater than the MAE ratio of threshold value 2.This count value and threshold value 3 are compared.
In this embodiment of the present invention, the motion vector sought zone+/-x and+/-the y direction on scope from-8~+ 7.The number of MAE among the MAE figure equals 256, comprises 16 MAE on each horizontal line and 16 MAE on each vertical line.The number of sub-piece equals 16, comprises 4 sub-pieces on each horizontal line and 4 sub-pieces on each vertical line.
Obtain and accumulate difference (step S308) between the adjacent MAE in left diagonal and the right diagonal.This accumulated value is represented total MAE map distance (step S310).
Obtain after total MAE map distance, the total MAE map distance of maximum is set to the MAE map distance of current block.If should the total MAE map distance of maximum greater than threshold value 1, then this handles according to second hypothesis and carries out (step S312).
Fig. 4 illustrates for the piece that is included in the repeat patterns, the figure that is formed by the MAE that presents in the left side of MAE figure and the right diagonal.As shown in Figure 4, the correlation marked change between MAE.
Fig. 5 illustrates the MAE figure to general pattern.As shown in Figure 5, the correlation between the MAE in the diagonal of the left and right sides does not have marked change.
In second step of the algorithm of determining repeat patterns, promptly in MAE figure classification, abandon pseudo-repeat patterns.This puppet repeat patterns is not real repeat patterns, but comprises and the similar characteristic of repeat patterns.If the distribution that has greater than the zone of 3.0 MAE ratio is twisted, promptly concentrate in any direction to present these zones, then really stator block is comprised in the pseudo-repeat patterns.
Have greater than threshold value 2 as the fruit piece, promptly be 3.0 MAE ratio in the present invention, and occupied more than 50% of whole sub-pieces, then really stator block is comprised in the repeat patterns, and this process proceeds to the distribution inspection of step 314.
Shown in equation 4, the zone of being represented by " 1 " is called as most zones, and other zones are called as a few regions.Each number that adds up to most zones and a few regions.
Wherein f (i) represents threshold value 2.
In step 314, count value is used to equation 5 to determine pseudo-repeat patterns.
(^ of minority_count 〉=α * region_count) (minority_count≤β * (16-region_count) ... (5), wherein, region_count represents the number of the sub-piece in most zones, is set to " 8 " in the present invention.Factor alpha is set to 0.5, and factor beta is set to 0.25.
Fig. 6 illustrates for the MAE figure that is included in the piece in the repeat patterns, and wherein, each MAE figure is divided into 16 sub-pieces, and marks the zone that has greater than 3.0 MAE ratio with stain.Referring to Fig. 6, the sub-piece that has greater than the MAE ratio of threshold value 3 (promptly 7.0) is evenly distributed on the MAE figure.According to experiment, because the MAE of general pattern ratio is less than 2.0, so the image that threshold value 2 allows clearly to distinguish general pattern and has repeat patterns.
In step 316, check the distribution of MAE ratio, and in step 318, determine that the piece with arrowband is included in the pseudo-repeat patterns.
Fig. 7 illustrates the image with pseudo-repeat patterns.In Fig. 7, the part that is marked by circle is pseudo-repeat patterns.As shown in Figure 8, has regional centralized greater than 3.0 MAE ratio in diagonal.This result does not conform to the hypothesis of the characteristics of relevant image with repeat patterns.Therefore, must remove pseudo-repeat patterns.
Fig. 9 illustrates for the sub-piece that has greater than 3.0 MAE ratio and carries out the arrowband classification, and this a little concentrates on certain direction.Have zone greater than 3.0 MAE ratio by " 1 " expression, and mark, and other zones are represented by " 0 " with stain.
Determining of repeat patterns with improved median filter end.In improved median filter, not that (dependent) of subordinate but this fact of concentrating are checked the repetition of adjacent block, thereby only just approve the repetition of current block when adjacent block when being shown repetition greater than predeterminated level based on repeat patterns.In other words, be aggregated in 24 adjacent blocks and be included in 5 pieces and 5 pieces on each vertical line on each horizontal line, be included in the number of the piece in the repeat patterns.Adding up to greater than threshold value 4 if be included in the number of the piece in the repeat patterns, promptly is 6 in the present invention, determines that then current block is included in the repeat patterns.
In the present invention, can use the MAE ratio of adjacent block, rather than the MAE of sub-piece ratio.In other words, obtain the MAE ratio of adjacent block in the diagonal, add up to number, and this count value and threshold value 2 are compared greater than the MAE ratio of threshold value 4.
Used the symmetry blocks coupling in the present invention, but also may use additive method, mated to piece as forward block matching process or back.
In addition, the method for definite repeat patterns of the present invention can be used in motion vector estimation and the frame interpolation.
Figure 10 is for describing the flow chart according to frame interpolation method of the present invention.
At first step S1002, obtain MAE figure.
At next step S1004, based on this MAE figure estimating motion vector.
Then, at step S1006, determine whether the sub-piece of MAE figure is included in the repeat patterns.The flow chart of Fig. 3 has been described determining of repeat patterns.
In step S1008, check the correlation between current block and the adjacent block, with the ratio that obtains to be included in the piece in the repeat patterns and not have other pieces of repeat patterns, i.e. correlation between current block and the adjacent block.
Figure 11 has explained the correlation between current block and the adjacent block.As shown in figure 11, place piece 1102 between repeat patterns zone and the non-repeat patterns zone to be subjected to the two the influence of repeat patterns and non-repeat patterns.Therefore, the piece that must generate by linear interpolation and mix by the piece that ME/MC generates is with inserted block 1102 in correct.
In step S1110, mix piece that generates by linear interpolation and the piece that generates by ME/MC by ratio, and obtain the image after the interpolation based on acquisition.In other words, carry out the frame interpolation by soft handover.
The block diagram that Figure 12 puts for frame interpolation according to the present invention.
MAE computing unit 1202 uses the symmetric motion vector to estimate, the MAE between the reference block in the region of search of calculating former frame k-1 and back one frame k+1.
Because the number of the reference block in each motion vector sought zone is 256, so the number of the MAE that obtains is 256.
MAE figure memory cell 1204 storages one MAE figure.This MAE figure comprises 256 MAE that obtained by MAE computing unit 1202.
In the MAE that is stored in MAE figure memory cell 1204, among the candidate that promptly moves, motion vector extraction unit 1206 with the moving direction of current block be identified as the reference block with minimum MAE the position (x, y).
Motion filter 1208 is used to consider the correlation between current block and the adjacent block, and is used for preventing the motion vector mistake.General median filter or the mean value filter of using is as motion filter 1208.
With reference to the MAE figure that is stored in the MAE figure memory cell 1204, repeat patterns determining unit 1210 determines whether piece is included in the repeat patterns.Repeat patterns determining unit 1210 comprises MAE map distance computing unit 1210a, MAE figure taxon 1210b and improved median filter 1210c.
MAE map distance computing unit 1210a calculates the difference between each MAE of the adjacent block that centers on current block on a left side and the right diagonal, and with result's accumulation, to calculate the distance between each MAE in a left side and right diagonal.
Ultimate range in institute's calculated distance is set to the MAE map distance, and compares with threshold value 1.If the MAE map distance is greater than threshold value 1, then MAE map distance computing unit 1210a determines to satisfy first hypothesis.
MAE map distance computing unit 1210a is divided into sub-piece with MAE figure then, and calculates minimum MAE and maximum MAE in each sub-piece, to obtain MAE than (Max/Min).
In the present invention, the motion vector sought zone+/-x and+/-the y direction is from-8-+7 expansion.The sum of MAE in MAE figure equals 256, is included in 16 MAE on each horizontal line and 16 MAE on each vertical line.MAE figure is divided into 16 sub-pieces, is included in 4 pieces on each horizontal line and 4 pieces on each vertical line.
MAE figure taxon 1210b removes pseudo-repeat patterns.Have greater than threshold value 2 as the fruit piece, promptly of the present invention 3.0 MAE ratio, and accounted for more than 50% of all sub-pieces, and then MAE figure taxon 1210b determines that this a little is included in the pseudo-repeat patterns, wherein threshold value 3 is set to 7.
MAE figure taxon 1210b checks the distribution have greater than the sub-piece of the MAE ratio of threshold value 2 then.Have the arrowband as the fruit piece, determine that then it is included in the pseudo-repeat patterns.
At step S320, improved median filter 1210c checks the repetition of adjacent block, only just approves the repetition of current block during greater than the repetition of predeterminated level when the adjacent block demonstration.In other words, at 24 adjacent blocks and be included in 5 pieces and 5 pieces on each vertical line on each horizontal line, add up to the number of piece with repeat patterns.Adding up to greater than threshold value 4 if be included in the number of the piece in the repeat patterns, promptly is 6 in the present invention, determines that then current block is included in the repeat patterns.
In the present invention, can use the MAE ratio of adjacent block, rather than the MAE of sub-piece ratio.In other words, obtain the MAE ratio of adjacent block in a left side and the right diagonal, add up to number, and this count value and threshold value 2 are compared greater than the MAE ratio of threshold value 4.
Frame interpolation unit 1212 uses the final motion vector of being handled by motion filter 1208 and uses linear interpolation that frame is carried out interpolation.
The adjacent block that frame interpolation unit 1212 is checked around current block is with the ratio that obtains to be included in the piece in the repeat patterns and not have other pieces of repeat patterns, i.e. correlation between current block and the adjacent block.Frame interpolation unit 1212 is by mixing piece that generates by linear interpolation and the piece that generates by ME/MC, the image after the acquisition interpolation then.
For example, by adding up to, check the correlation between current block and the adjacent block around the repetition of the adjacent block of current block.
Figure 13 illustrates and is used for the adjacent block that correlation is checked.In Figure 13,25 adjacent blocks are around current block.Add up to the number that is included in the adjacent block in the repeat patterns.
Frame interpolation based on the correlation that obtains is represented by equation 6.
Wherein, (α represents to be included in the number of the adjacent block in the repeat patterns to f for i, the j) location of pixels of expression current block, and scope is 0≤α≤25 in the present invention.
According to the present invention, the image with repeat patterns is handled by linear interpolation, and does not have another image of repeat patterns to handle by ME/MC.
Yet simple hardware switches and can't show superior performance at the boundary between the piece in being included in repeat patterns zone and non-repeat patterns zone.Like this, based on the correlation between current block and the adjacent block, use soft handover.
In equation 6, α represents to be included in the number of the adjacent block in the repeat patterns.If α is very big, that is,, then use linear interpolation if image has many repeat patterns.Otherwise, then be used to mean value from the motion vector of former frame and back one frame, use interpolation based on MC.
Figure 14 A to 14D illustrates the test pattern with repeat patterns.Figure 14 A is illustrated in the grating image that the center has the repetition sliver.Figure 14 B illustrates has the radially Snell Wilcox of repeat patterns.Figure 14 C illustrates a Melco image, presents the building of the window with repetition.Figure 14 D illustrates a restaurant image, presents the tablecloth with repetition striped.
In order to compare the present invention and traditional MC, equation 7 and 8 defined mean square deviations (MSE, mean square error) or Y-PSNR (PSNR, peak signal to noise) have been used.
Wherein, M and N represent the wide and high of piece, and f
1(i, j) and f
2(i, j) location of pixels of the location of pixels of expression former frame and back one frame.
Table 1 shows the PSNR that use equation 7 and 8 acquisitions.The PSNR that obtains is the mean value from the PSNR of frame.
[table 1]
Image | Tradition MC | The algorithm that proposes | Improve |
Grid | 23.6db | 26.2db | 2.6db |
Snell Wilcox | 15.8db | 24.9db | 9.1db |
Melco | 26.1db | 26.5db | 0.4db |
The restaurant | 28.9db | 30.1db | 1.2db |
Shown in top table 1, PSNR changes with the degree that the form of repeat patterns, motion total amount between the frame and repeat patterns occupy image.
Below, the vision difference that is obtained when using according to frame interpolation method of the present invention and using traditional MC is described with reference to Figure 15 and 16.
Figure 15 illustrates the image of handling by traditional MC, and the image that wherein has repeat patterns is disperseed.
Figure 16 illustrates the image of handling by according to the method for interpolation frame of the present invention, and wherein motion artifacts is suppressed visibly.
Result as a comparison, frame interpolation method according to the present invention has been eliminated the shown serious vacation picture of traditional MC.
As mentioned above, when the image with repeat patterns moves, allow effective frame interpolation according to the method for definite repeat patterns of the present invention.
When the image with repeat patterns moves, allow to use linear interpolation and traditional MC according to frame interpolation method of the present invention, thereby the image after the interpolation that does not have motion artifacts is provided.
In addition, when the image with repeat patterns moved, frame interpolation according to the present invention was put frame that mixing handles by linear interpolation and the frame by traditional MC use motion compensation process, and the image after the interpolation that does not have motion artifacts is provided thus.
Though concrete demonstration of the present invention and description it should be appreciated by those skilled in the art under the prerequisite of scope that does not break away from claim and spirit with reference to its exemplary embodiment, can make the various modifications on form and the details.
Claims (26)
1. the method for a definite repeat patterns is used for when the image with repeat patterns moves, and determines whether the reference block of M * N is included in the repeat patterns, and this method may further comprise the steps:
Use search fully, obtain the error of calibrated bolck and reference block, each reference block is included in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process;
According to the order of reference block, arrange (M+P) * (N+P) individual error with the form of (M+P) * (N+P) figure;
In the left diagonal and right diagonal of this figure, obtain each deviation between current block and the adjacent block, and be accumulated in deviation that obtains in the left diagonal and the deviation that in right diagonal, obtains respectively;
Compare deviation of in left diagonal, accumulating and the deviation of in right diagonal, accumulating, and select bigger deviation;
Deviation and the threshold value 1 selected are compared; And
If the deviation of selecting, determines then that this reference block is included in the repeat patterns greater than threshold value 1.
2. the method for claim 1, wherein the error of calibrated bolck and reference block is called as mean absolute error, because this mean absolute error is to obtain by the absolute difference sum between average each reference block.
3. the method for claim 1 also comprises:
The sub-piece that this figure is divided into identical size;
Calculate the ratio of the worst error and the minimal error of each sub-piece;
Counting has the sum greater than the sub-piece of the ratio of threshold value 2; And
If should sum greater than threshold value 3, then really stator block is included in the repeat patterns.
4. method as claimed in claim 3 also comprises:
Inspection has the distribution greater than the sub-piece of the ratio of threshold value 2, and determines whether this sub-piece that has greater than the ratio of threshold value 2 is included in the pseudo-repeat patterns.
5. method as claimed in claim 4 wherein, by relatively concentrate the pseudo-repeat patterns of the sub-piece that distributes on the level of this figure, vertical and diagonal, determines whether the described sub-piece that has greater than the ratio of threshold value 2 is included in the described pseudo-repeat patterns.
6. method as claimed in claim 5 also comprises:
Inspection is included in the reference block in the repeat patterns and the repetition degree of adjacent block; And
Based on the degree that repeats, determine whether reference block is included in the described repeat patterns.
7. method as claimed in claim 6, wherein, the degree of repetition is represented by being included in the reference block in the repeat patterns and the sum of adjacent block.
8. frame interpolation method may further comprise the steps:
(a) use search fully, obtain the error of calibrated bolck and reference block, each reference block is included in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process;
(b) estimating motion vector is the positional information with reference block of minimal error;
(c), determine whether described calibrated bolck and reference block are included in the repeat patterns according to the error that is obtained;
(d) calculate the current block that will be interpolated and around the correlation between the adjacent block of this current block; And
(e),, obtain the image after the interpolation by mixing piece that forms by linear interpolation and the piece that forms by estimation and motion compensation according to the correlation of calculating.
9. frame interpolation method as claimed in claim 8, wherein, step (d) further comprises:
Counting centers on described reference block and is comprised in the number of all adjacent blocks in the repeat patterns; And
Calculate around the ratio of the number of the sum of the adjacent block of reference block and counting.
10. frame interpolation method as claimed in claim 9, wherein, the current block that forms by interpolation is expressed from the next,
Wherein, (α represents to be included in the number of the adjacent block in the repeat patterns to f for i, the j) location of pixels of expression current block, and total represents the sum of adjacent block, 0≤α≤25.
11. frame interpolation method as claimed in claim 8, wherein, step (c) further comprises:
When use is searched for fully, obtain the error of calibrated bolck and reference block, each reference block is included in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process;
According to the order of reference block, arrange (M+P) * (N+P) individual error with the form of (M+P) * (N+P) figure;
In the left diagonal and right diagonal of this figure, obtain current block and around each deviation between the adjacent block of this current block, and be accumulated in deviation that obtains in the left diagonal and the deviation that in right diagonal, obtains respectively;
Compare deviation of in left diagonal, accumulating and the deviation of in right diagonal, accumulating, and choose bigger deviation;
Selected deviation and threshold value 1 are compared; And
If the deviation of selecting, determines then that this reference block is included in the repeat patterns greater than threshold value 1.
12. frame interpolation method as claimed in claim 11, wherein, the error of calibrated bolck and reference block is called as mean absolute error, because this mean absolute error is to try to achieve by the absolute difference sum between average each reference block.
13. frame interpolation method as claimed in claim 11 also comprises:
The sub-piece that this figure is divided into identical size;
Calculate the ratio of the worst error and the minimal error of each sub-piece;
Counting has the sum greater than the sub-piece of the ratio of threshold value 2; And
If should sum greater than threshold value 3, determine that then described sub-piece is included in the described repeat patterns.
14. frame interpolation method as claimed in claim 13 also comprises:
Inspection has the distribution greater than the sub-piece of the ratio of threshold value 2, and determines whether this sub-piece that has greater than the ratio of threshold value 2 is included in the pseudo-repeat patterns.
15. frame interpolation method as claimed in claim 14, wherein, by relatively concentrate the pseudo-repeat patterns of the sub-piece that distributes on the level of described figure, vertical and diagonal, whether the sub-piece of determining to have greater than the ratio of threshold value 2 is included in this puppet repeat patterns.
16. frame interpolation method as claimed in claim 15 also comprises:
Inspection is included in the reference block in the repeat patterns and the repetition degree of adjacent block; And
Based on the degree that repeats, determine whether reference block is included in this repeat patterns.
17. frame interpolation method as claimed in claim 16, wherein, the degree of repetition is represented by being included in the reference block in the repeat patterns and the sum of adjacent block.
18. a frame interpolation is put, and comprising:
The mean absolute error computing unit, be used for using search fully, mean absolute error between basis of calculation piece and the reference block, each reference block are comprised in the region of search of (M+2P) * (N+2P), and this region of search belongs to the frame of institute's reference in the motion vector sought process;
Mean absolute error figure memory cell is used for the order according to reference block, with form storage (M+P) * (N+P) individual error of (M+P) * (N+P) figure;
The motion vector extraction unit, be used for the image moving direction of current block be identified as the reference block with the minimum average B configuration absolute error in each mean absolute error that is stored in this mean absolute error figure the position (x, y);
The repeat patterns determining unit is used for determining with reference to the mean absolute error figure that is stored in mean absolute error figure memory cell whether piece is included in the repeat patterns; And
The frame interpolation unit is used to use the motion vector of extraction and linear interpolation to come interpolation frame.
19. frame interpolation as claimed in claim 18 is put, wherein, described repeat patterns determining unit comprises:
Mean absolute error map distance computing unit, be used for described mean absolute error figure is divided into sub-piece, and calculate minimum average B configuration absolute error and maximum mean absolute error in each sub-piece, to obtain mean absolute error ratio, promptly maximum mean error/minimum average B configuration error;
Difference between the mean absolute error of calculating adjacent block in left diagonal and right diagonal;
The difference that accumulation is calculated is to calculate the distance between each mean absolute error in the diagonal of the left and right sides;
From each distance of calculating, choose maximum distance; And
If this ultimate range, determines then that reference block is included in this repeat patterns greater than threshold value 1.
20. frame interpolation as claimed in claim 19 is put, wherein, described repeat patterns determining unit also comprises:
Mean absolute error figure taxon is used for counting left and right sides diagonal and has sum greater than the sub-piece of the mean absolute error ratio of threshold value 2; And have mean absolute error ratio greater than threshold value 2 as the fruit piece, and accounted for more than 50% of whole sub-pieces, determine that then this sub-piece is included in the repeat patterns.
21. frame interpolation as claimed in claim 20 is put, wherein, the inspection of described mean absolute error figure taxon has the distribution greater than the sub-piece of the mean absolute error ratio of threshold value 2.
22. frame interpolation as claimed in claim 21 is put, wherein, described mean absolute error figure taxon is by relatively concentrating the pseudo-repeat patterns of the sub-piece that distributes on the level of this mean absolute error figure, vertical and diagonal, determine whether this sub-piece that has greater than the mean absolute error ratio of threshold value 2 is included in this puppet repeat patterns.
23. frame interpolation as claimed in claim 20 is put, wherein, described repeat patterns determining unit also comprises:
Median filter is used for checking the reference block that is included in repeat patterns and the repetition of adjacent block, and based on the degree that repeats, determines whether reference block is included in this repeat patterns.
24. frame interpolation as claimed in claim 18 is put, wherein, described frame interpolation unit calculates the current block that will be interpolated and around the correlation between the adjacent block of this current block, and according to the correlation of calculating, by mixing piece that forms by linear interpolation and the piece that forms by motion compensation, obtain the image after the interpolation.
25. frame interpolation as claimed in claim 24 is put, wherein, described frame interpolation unit counting is included in this repeat patterns and centers on the number of all adjacent blocks of described calibrated bolck, calculate around the ratio of the number of the sum of the adjacent block of reference block and counting, and the current block that uses this ratio to calculate will to be interpolated and around the correlation between the adjacent block of this current block.
26. frame interpolation as claimed in claim 25 is put, wherein, the current block that forms by interpolation is expressed from the next,
Wherein, (α represents to be included in the number of the adjacent block in the repeat patterns to f for i, the j) location of pixels of expression current block, and total represents the sum of adjacent block, 0≤α≤25.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR79745/2002 | 2002-12-13 | ||
KR79745/02 | 2002-12-13 | ||
KR10-2002-0079745A KR100462629B1 (en) | 2002-12-13 | 2002-12-13 | Method for determining a repeated pattern, a frame interpolation method thereof and a frame interpolation apparatus thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
CN1507274A CN1507274A (en) | 2004-06-23 |
CN1250000C true CN1250000C (en) | 2006-04-05 |
Family
ID=34270545
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CNB2003101205871A Expired - Fee Related CN1250000C (en) | 2002-12-13 | 2003-12-15 | Method for determining repeated pattern frame interpolating method and frame interpolating apparatus |
Country Status (3)
Country | Link |
---|---|
KR (1) | KR100462629B1 (en) |
CN (1) | CN1250000C (en) |
AU (1) | AU2003266803B2 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4398925B2 (en) * | 2005-03-31 | 2010-01-13 | 株式会社東芝 | Interpolation frame generation method, interpolation frame generation apparatus, and interpolation frame generation program |
JP4165580B2 (en) * | 2006-06-29 | 2008-10-15 | トヨタ自動車株式会社 | Image processing apparatus and image processing program |
KR101356613B1 (en) | 2009-08-21 | 2014-02-06 | 에스케이텔레콤 주식회사 | Video Coding Method and Apparatus by Using Adaptive Motion Vector Resolution |
WO2011021915A2 (en) * | 2009-08-21 | 2011-02-24 | 에스케이텔레콤 주식회사 | Method and apparatus for encoding/decoding images using adaptive motion vector resolution |
KR102229192B1 (en) * | 2019-12-06 | 2021-03-17 | 국방과학연구소 | Control device that estimates the synchronous signal of a frame with variable length and control method there of |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100220834B1 (en) * | 1996-11-15 | 1999-09-15 | 전주범 | Apparatus and method of target tracking in image telephone |
KR100301835B1 (en) * | 1998-09-03 | 2001-09-06 | 구자홍 | Method for block matching motion estimation and apparatus for the same |
KR100317279B1 (en) * | 1998-11-04 | 2002-01-15 | 구자홍 | Lossless entropy coder for image coder |
US6377297B1 (en) * | 1999-12-07 | 2002-04-23 | Tektronix, Inc. | Detection of repeated and frozen frames in a video signal |
US6532264B1 (en) * | 2000-03-27 | 2003-03-11 | Teranex, Inc. | Processing sequential video images to detect image motion among interlaced video fields or progressive video images |
EP1387588A2 (en) * | 2002-08-02 | 2004-02-04 | KDDI Corporation | Image matching device and method for motion estimation |
KR100530223B1 (en) * | 2003-05-13 | 2005-11-22 | 삼성전자주식회사 | Frame interpolation method and apparatus at frame rate conversion |
-
2002
- 2002-12-13 KR KR10-2002-0079745A patent/KR100462629B1/en not_active IP Right Cessation
-
2003
- 2003-12-05 AU AU2003266803A patent/AU2003266803B2/en not_active Ceased
- 2003-12-15 CN CNB2003101205871A patent/CN1250000C/en not_active Expired - Fee Related
Also Published As
Publication number | Publication date |
---|---|
CN1507274A (en) | 2004-06-23 |
AU2003266803A1 (en) | 2004-07-01 |
AU2003266803B2 (en) | 2005-01-06 |
KR100462629B1 (en) | 2004-12-23 |
KR20040052026A (en) | 2004-06-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN1252999C (en) | High grade edge self adaption interpolation method and system for interleave to inching change | |
CN1638447A (en) | Image adaptive deinterlacing method and device based on edge | |
CN1581982A (en) | Pattern analysis-based motion vector compensation apparatus and method | |
CN1192614C (en) | Stillness judging device and scanning line interpolating device having it | |
CN100351868C (en) | Generation of still image from a plurality of frame images | |
CN1178474C (en) | Method for detecting horizontal black tape in video image | |
CN1180627C (en) | Image codec method, image coder and image decoder | |
CN101047779A (en) | Apparatus for creating interpolation frame | |
CN1645907A (en) | Hybrid sequential scanning using edge detection and motion compensation | |
US20080246885A1 (en) | Image-processing method and device | |
CN1750656A (en) | Encoding and decoding images | |
CN1283368A (en) | Source coding to provide for robust error recovery during transmission losses | |
CN101047858A (en) | Block noise detecting method and apparatus, and block noise reducing method and apparatus | |
CN1678024A (en) | Improved motion vector estimation at image borders | |
CN1934871A (en) | Mode decision for inter prediction in video coding | |
CN1360437A (en) | Interpolation apparatus and video signal process apparatus comprising said interpolation apparatus | |
CN1816149A (en) | Filtering-wave method of removing blocking-effect in video-frequency picture and loop filter | |
CN1893658A (en) | Motion estimator and motion estimating method thereof | |
CN1859666A (en) | Cell searching method and its device for mobile communication terminal | |
CN1250000C (en) | Method for determining repeated pattern frame interpolating method and frame interpolating apparatus | |
CN1647113A (en) | Motion estimation unit and method of estimating a motion vector | |
CN1343421A (en) | Image interpolation system and image interpolocation method | |
CN1285211C (en) | Device and method for enhancing reproduction image quality | |
CN1245028C (en) | Non-uniform multilayer hexaploid lattice full pixel kinematic search method | |
CN1636406A (en) | Unit for and method of motion estimation and image processing apparatus provided with such motion estimation unit |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20060405 Termination date: 20181215 |