Embodiment
Describe embodiments of the invention below in detail, the example of described embodiment is shown in the drawings, and wherein identical from start to finish or similar label is represented identical or similar elements or the element with identical or similar functions.Below by the embodiment that is described with reference to the drawings is exemplary, only is used to explain the present invention, and can not be interpreted as limitation of the present invention.
In order to realize the present invention's purpose, the invention discloses a kind of formation method of video coding complexity controlling models, Fig. 1 shows the block diagram of this method, in conjunction with illustrated in figures 1 and 2, comprises the steps:
A1: input macro block data.
In video encoder, import macro block data.
A2: macro block data is carried out mapping of infra-frame prediction complexity and the mapping of inter prediction complexity, predictive mode to infra-frame prediction and inter prediction divides into groups, and every group of predictive mode of inter prediction is put in order picture element movement estimate complexity mapping and the mapping of sub-pixel motion estimation complexity.
Specifically, intraframe predictive coding is exactly to predict current pixel value with contiguous pixel value on every side, then predicated error is encoded.This prediction is based on piece, and for luminance component (luma), block size can be selected between 16 * 16 and 4 * 4, and 16 * 16 have 4 kinds of predictive modes, and 4 * 4 have 9 kinds of predictive modes.Except mean prediction, the prediction on the corresponding different directions of other every kind predictive mode.H.264 reference software JM adopts the exhaustive strategy to select the optimum code pattern by the cost function value of calculating various predictive modes, and time complexity is very high.
In the present embodiment, intra prediction mode adopts 4 * 4 luma prediction modes.4 * 4 luma prediction modes: shown in (a) among Fig. 3, the prediction piece P of 4 * 4 sub-pieces is made up of 16 some a-p, and after the sampled point A-M of the left side and top had rebuild, they just can be produced with reference to sub-piece as the prediction pixel.Shown in (b) among Fig. 3,4 * 4 have 9 kinds of prediction mode, are respectively: vertical prediction, and horizontal forecast, mean prediction, the prediction of diagonal angle, lower-left, the lower-right diagonal position prediction, vertical-right prediction, level are predicted downwards, the vertical left prediction, level is prediction upwards.
From the final pattern statistics of each video sequence, each pattern is selected as being more or less the same between the ratio of optimization model, Diagonal_Down_left just, and Vertical_Left, Vertical, the Vertical_Right accounting is high slightly.Thus, 9 kinds of predictive modes of infra-frame prediction are done an overall grouping, be divided into 4 groups.
First group of IG1:Diagonal_Down_left, Vertical_Left, Vertical, Vertical_Right;
Second group of IG2:Diagonal_Down_left, Vertical_Left, Vertical, Vertical_Right, DC;
The 3rd group of IG3:Diagonal_Down left, Vertical_Left Vertical, Vertical_Right, DC, Diagonal_Down_Right, Horizontal;
The 4th group of IG4:Diagonal_Down left, Vertical_Left, Vertical, Vertical_Right, DC, Diagonal_Down_Right, Horizontal, Horizontal_Down, Horizontal_Up.
In the present embodiment, intra prediction mode 16 * 16 luma prediction modes are divided into vertical prediction (Vertical), horizontal forecast (Horizontal), mean prediction (DC) and 4 kinds of patterns of planar prediction (Plane).As shown in Figure 4, vertical prediction, the sampled value of each sub-piece of macro block top is used as the predicted value of the corresponding permutation of macro block; Horizontal forecast, each sub-piece sampled value of the macro block left side are used as the predicted value of the corresponding full line of macro block; Mean prediction, the macro block top is used as the macroblock prediction value with the average of each sub-piece sampled value of the left side; Planar prediction, each sub-piece sampled value of the macroblock prediction value upper right side and the left side is as the macroblock prediction value.The interpolation method of various patterns.
Take all factors into consideration each pattern complexity of infra-frame prediction and to the contribution of code efficiency, all patterns of infra-frame prediction are divided into four classes, respectively the power level (Power Mode) shown in the correspondence table 1.Wherein, power level and level of computational complexity are linear proportional relation.As shown in table 1, power level is that High represents current electric quantity of power supply abundance; Power level is that Normal represents that current electric quantity of power supply is normal; Power level is that Low represents that current power supply is in low state of charge; Power level is that Ultra-Low represents that power supply is in extremely low state of charge.
Table 1
Power level |
The intra prediction mode grouping |
High |
Diagonal_Down?left,Vertical_Left,Vertical, Vertical_Right,DC,Diagonal_Down_Right,Horizontal, Horizontal_Down,Horizontal_Up,Intra_16×16 |
Normal |
Diagonal_Down_left,Vertical_Left,Vertical, Vertical_Right,DC,Diagonal_Down?Right,Horizontal, Intra_16×16 |
Low |
Diagonal_Down_left,Vertical_Left,Vertical, Vertical_Right,DC,Intra_16×16 |
Ultra-Low |
Diagonal_Down_left,Vertical_Left,Vertical, Vertical_Right,Intra_16×16 |
For inter-frame forecast mode, with H.264/AVC 7 kinds of block size patterns and SKIP pattern all mode as inter prediction.By its algorithm complex and to the contribution of code efficiency, all patterns are divided into 4 groups.
First group of PG1:SKIP and Inter16 * 16.If inter prediction is only chosen two kinds of patterns of this group, computation complexity then sharply reduces, but code efficiency is also lower simultaneously;
Second group of PG2:SKIP, Inter16 * 16, Inter16 * 8, Inter8 * 16.This group has increased by two kinds of patterns, can improve code efficiency greatly, and the computational complexity of algorithm increases simultaneously;
The 3rd group of PG3:SKIP, Inter16 * 16, Inter16 * 8, Inter8 * 16, Inter8 * 8, in fact this group coding efficient is with to choose all mode close, but algorithm complex is compared with syntype and is lower;
The 4th PG4:SKIP, Inter16 * 16, Inter16 * 8, Inter8 * 16, Inter8 * 8, Inter8 * 4, Inter4 * 8, code efficiency is the highest, and the computing method complexity is also the highest simultaneously.
According to the grouping of above-mentioned inter-frame forecast mode, adopt whole picture element movement to estimate and the sub-pixel motion estimation to the grouping of each pattern.
Specifically, whole picture element movement estimates that the representative fast algorithm of complexity mapping comprises, TSS algorithm (Three Step Search, three step searching algorithms), DS algorithm (Diamond search, the diamond search algorithm), FSS algorithm (Four step search, four step searching algorithms), NTSS algorithm (the new three step searching algorithms of New Three stepsearch), HEX algorithm (Hexagon search, the hexagon search algorithm) and UMH algorithm (Unsymmetrical-cross Muti-hexagon Search, asymmetric cross multi-level hexagonal point search algorithm) etc.
Wherein, DS algorithm (Diamond search, diamond search algorithm) utilizes the statistical property of motion vector and has adopted bitellos and two kinds of search block patterns of melee.It is static or approaching static Halfway Stopping strategy based on the characteristic and the employing decision block of central distribution that NTSS algorithm (the new three step searching algorithms of New Three stepsearch) utilizes motion vector.In addition, HEX algorithm (Hexagon search, hexagon search algorithm), PMVFAST algorithm (Predictive Motion Vector Field Adaptive SearchTechnique) scheduling algorithm is ought comparatively successful motion estimation algorithm.
Wherein, the DIA algorithm search is counted minimum, so it is fastest, and its code efficiency is also lost little and other algorithms are also very approaching, is the most normal adopted motion estimation algorithm generally.The EXA algorithm is the most consuming time, and is not remarkable for promoting the PSNR contribution simultaneously.Therefore, take all factors into consideration the Time Calculation complexity of code efficiency and algorithm, DIA algorithm, HEX algorithm and UMH algorithm become more preferably to be selected, and above-mentioned algorithm can significantly improve speed under the prerequisite of not losing the PSNR performance substantially.
The displacement of moving object seldom just in time is the integral multiple pixel in the actual video sequence, therefore needs to adopt sub-pixel motion to estimate.Sub-pixel motion is estimated the above-mentioned TSS algorithm of same sampling (Three Step Search, three step searching algorithms), DS algorithm (Diamond search, the diamond search algorithm), FSS algorithm (Fourstep search, four step searching algorithms), NTSS algorithm (the new three step searching algorithms of New Three step search), HEX algorithm (Hexagon search, the hexagon search algorithm) and UMH algorithm (Unsymmetrical-cross Muti-hexagon Search, asymmetric cross multi-level hexagonal point search algorithm).
In the present embodiment,, adopt earlier and estimate to determine starting point, carry out degree of depth fine search at optimal location then and determine final optimum point by whole picture element movement for further saving search time.According to the search that becomes more meticulous after the model selection, behind the discovery optimum position, carry out again based on the deep search of being scheduled to the sub-pix searching times at this position periphery.Carry out sub-pixel motion by this method and estimate, will keep effectively improving the search accuracy under the prerequisite fast, improve code efficiency at algorithm.
Specifically, in conjunction with Fig. 5 and shown in Figure 6, sub-pixel motion estimates that the complexity mapping may further comprise the steps: after estimating through above-mentioned whole picture element movement, determine the starting point 1 that sub-pixel motion is estimated.Select according to the take exercises row mode of estimating to go forward side by side of above-mentioned starting point; According to sub-pix searching times under every kind of pattern (SME ITERATION TIME) and sub-pix fine search number of times (SME REFINEMENT), the periphery of starting point 1 is carried out search based on described sub-pix searching times.Searching for 1/2 picture element at the peripheral position of starting point 1, obtain 1/2 picture element 2, is that starting point is searched for 1/4 picture element at its peripheral position to put 2, obtains 1/4 picture element 3.
Shown in the associative list 2, in the table data definition sub-pix searching times and the sub-pix fine search number of times of 1/2,1/4 pixel.Sub-pix searching times (SME ITERATION TIME) shows any pattern, all will search for by the number of times of definition.Inferior pixel fine search number of times (SMEREFINEMENT) defined by present pattern search after optimum point, the counting of search around it again.By different SME ITERATION TIME and REFINEMENT number of SME, constituted the complexity level and the code efficiency of inferior pixel.On the whole, SME_LEVEL is low more, and search point is few more, and algorithm complex is low more, and code efficiency also can be minimum.As shown in table 2, sub-pixel motion is estimated level from 1 to 7 arrangement, and algorithm complex and code efficiency increase synchronously.
Table 2
Sub-pixel motion is estimated level |
Inferior pel search number of times (1/2 pixel, 1/4 pixel) |
Model selection |
Inferior pixel fine search number of times (1/2 pixel, 1/4 pixel) |
1 |
(0,0) |
SAD |
(1,1) |
2 |
(1,0) |
SATD |
(0,1) |
3 |
(1,1) |
SATD |
(0,0) |
4 |
(1,1) |
SATD |
(0,1) |
5 |
(1,2) |
SATD |
(0,1) |
6 |
(2,2) |
SATD |
(0,0) |
7 |
(2,2) |
SATD+RDO |
(0,0) |
Preferably, in inter-frame forecast mode, video encoding standard adopts a plurality of forward reference frame, thereby increases the estimation frame number of time shaft in the motion vector.By in a plurality of reference frames, carrying out estimation, seek the optimum Match of present encoding piece.Adopt the multi-reference frame estimation can improve code efficiency and fault freedom.Some specific occasions such as exist periodic motion, fast scene switch, when there is masking phenomenon in object, the use of multi-reference frame has effect preferably, but has meanwhile also increased buffer memory capacity and encoder complexity.Multi-reference frame quantity under the different electrical power pattern is as shown in table 3.As shown in table 3, when electric source modes was High, choosing multi-reference frame quantity was 5; When electric source modes was Normal, choosing multi-reference frame quantity was 3; When electric source modes was Low, choosing multi-reference frame quantity was 2; When electric source modes was Ultra-Low, choosing multi-reference frame quantity was 1.
Table 3
Power level |
Reference frame quantity |
High |
|
5 |
Normal |
3 |
Low |
2 |
Ultra-Low |
1 |
For the reference frame number, can choose more than one number.But according to statistics, the reference frame number is greater than 5 having little significance for the raising code efficiency.
The video coding complexity controlling models that the embodiment of the invention relates to has adopted the complexity mapping of variable size block motion compensation pattern.
Specifically, because all pixels in the block-based motion model hypothesis piece have all been done identical translation, can be quite different in motion this hypothesis of edge relatively more violent or moving object with reality, thereby cause bigger predicated error, the block size that reduce motion compensation this moment can make hypothesis still set up.The in addition little blocking effect that block mode caused is relatively also little, so in general little block mode can improve prediction effect.
, H.264/AVC each macro block is cut apart by 4 kinds of modes for this reason: 1 16 * 16, or 2 16 * 8, or 28 * 16 or 48 * 8.Its motion compensation also should have 4 kinds mutually.Each 8 * 8 sub-macro block can also further be cut apart in 4 kinds of modes: 18 * 8, or 28 * 4, or 24 * 8, or 44 * 4.Encoder is searched in reference frame according to picture material, finds the piece with the original block optimum Match to carry out coding transmission.For changing mild zone in the image, adopt bigger block size proper; For the abundant zone of details, adopt less size proper.
This multimodal flexible, trickle macroblock partitions suits the shape of the actual motion object in the image more, has improved the accuracy of estimation greatly.Compare with the conventional method that only adopts 16 * 16 macroblock predictions, use the piece of 7 kinds of different sizes and shape can save code check more than 15%.But the computation complexity of the encoder that the increase of block mode also increases greatly, test data show that also also there is very large gap in different masses pattern shared ratio in the final pattern of selecting.All in all, 16 * 16 patterns are used the most frequent, and several block mode frequencies of utilization below 8 * 8 are lower, and are less in motion, seldom use in the uncomplicated scene of image texture.In view of the above, as shown in table 4 the present invention proposes by adopting the purpose of the incompatible adjusting encoder complexity of different masses modal sets.
Table 4
Packet index number |
The inter-frame forecast mode grouping | Mode packet | |
1 |
SKIP,Inter16×16 |
PG1 |
2 |
SKIP,Inter16×16,Inter8×8 |
PG2 |
3 |
SKIP,Inter16×16,Inter16×8,Inter8×16 |
PG2 |
4 |
SKIP,Inter?16×16,Inter?16×8, Inter?8×16,Inter?8×8 |
PG3 |
5 |
SKIP,Inter?16×16,Inter?16×8, Inter8×16,Inter8×8,Inter4×4 |
PG4 |
6 |
SKIP,Inter16×16,Inter16×8,Inter8×16, Inter8×8,Inter8×4,Inter4×8,Inter4×4 |
PG4 |
A3: according to the mode packet of above-mentioned infra-frame prediction and inter prediction, carry out the cross-packet test, obtain every group computation complexity and code efficiency.
Specifically, according to the algorithms of different and the grouping of steps A 2, carry out the cross-packet test.
In the present embodiment, the cross-packet test realizes by encoder, exports every group computation complexity and code efficiency by encoder.
A4:, when computation complexity equates, judge whether this group is the highest group of code efficiency, if then keep the highest group of code efficiency, otherwise reject this group according to computation complexity and code efficiency.
In conjunction with illustrated in figures 1 and 2, test computation complexity and the code efficiency that obtains according to cross-packet in the steps A 3.At first select the group that computation complexity equates, under the prerequisite of equal computation complexity, if only corresponding one group, then keep this group.If to a plurality of different algorithm combination should be arranged, judge then whether this group is the highest group of code efficiency, if, then keep this group, reject the lower group of other code efficiencies.
In the present embodiment, computation complexity is by clock cycle metering required under equal test condition.
Thus, obtain with respect to the highest combination of code efficiency under each computation complexity by above-mentioned steps.
A5: computation complexity is carried out normalization, normalized computation complexity value is numbered from low to high, obtain level of computational complexity, generate C-R-D table corresponding to the highest combination of the code efficiency under each computation complexity.
According to the highest group of code efficiency under each computation complexity that obtains in the steps A 4, the aforementioned calculation complexity is carried out normalization.Choosing the highest one group of computation complexity in all groupings, is benchmark with this computation complexity, and computation complexity is carried out normalization.All normalized complexity value that obtain are numbered from low to high, obtain level of computational complexity (Complexity Level).The computation complexity C-R-D from low to high that generates corresponding to the algorithms of different combination according to above-mentioned data shows.Table 5 shows above-mentioned C-R-D table.As shown in table 5, the C-R-D table comprises level of computational complexity (Complexity Level), motion compensation block mode (MD Level), sub-pixel motion estimation level (SME Level), average peak signal to noise ratio difference (BDPSNR) and normalization computation complexity value (Normalized Complexity).Wherein, BDPSNR for benchmark group poor for according to the mean P SNR that calculates.The benchmark group is the highest one group of computation complexity.Above-mentioned code efficiency is measured by BDPSNR.
Table 5
Level of computational complexity |
The motion compensation block mode packet |
Sub-pixel motion is estimated level |
BDPSNR(dB) |
Complexity value after the normalization |
0 |
SKIP |
|
|
0 |
1 |
1 |
1 |
-0.76 |
0.228 |
2 |
2 |
1 |
-0.59 |
0.246 |
3 |
3 |
1 |
-0.46 |
0.265 |
4 |
4 |
1 |
-0.44 |
0.283 |
5 |
6 |
1 |
-0.53 |
0.359 |
6 |
3 |
2 |
-0.3 |
0.397 |
7 |
4 |
2 |
-0.26 |
0.438 |
8 |
3 |
3 |
-0.2 |
0.491 |
9 |
3 |
4 |
-0.18 |
0.528 |
10 |
3 |
5 |
-0.16 |
0.544 |
11 |
4 |
3 |
-0.15 |
0.577 |
12 |
4 |
4 |
-0.13 |
0.614 |
13 |
4 |
5 |
-0.12 |
0.637 |
14 |
5 |
3 |
-0.12 |
0.707 |
15 |
5 |
4 |
-0.1 |
0.745 |
16 |
5 |
5 |
-0.08 |
0.775 |
17 |
6 |
3 |
-0.03 |
0.918 |
18 |
6 |
4 |
-0.02 |
0.955 |
19 |
6 |
5 |
0 |
1 |
|
|
|
|
|
A6:,, obtain video coding complexity controlling models according to the parameter of the described video coding complexity of above-mentioned curve calculation controlling models according to normalized computation complexity value and level of computational complexity matched curve.
According to normalization complexity value and level of computational complexity matched curve, as shown in Figure 7, transverse axis is a level of computational complexity, and the longitudinal axis is normalized complexity.Calculate the parameter of video coding complexity controlling models according to above-mentioned matched curve.
Setting video encoder complexity controlling models is y=ax+b, a wherein, and b is a video coding complexity controlling models parameter.In the present embodiment, calculate, obtain a=0.045 according to matched curve among Fig. 7, b=0.1389, can obtain video coding complexity controlling models thus is y=0.045x+0.1389.
According to the video coding complexity controlling models that obtains, when input level of computational complexity x, can calculate normalized computation complexity value y.
Preferably, under different platforms, the matched curve that the method that relates to by present embodiment obtains, parameter a and b also can be embodied as other values, obtain video coding complexity controlling models thus and still belong to protection scope of the present invention.
Because level of computational complexity and power level are linear, by the controlling models that said method obtains, can also be illustrated in the video coding complexity controlling models under the power level (Power), i.e. the P-R-D model.
Fig. 8 is the Foreman cycle tests performance map of video coding complexity controlling models.As shown in Figure 8, transverse axis is the average CYCLE number of saving, and unit is %; The longitudinal axis is BDPSNR, and unit is dB.Cycle tests performance under four kinds of patterns has been shown among Fig. 8.The curve that Diamond spot constitutes is the cycle tests performance under the P-R-D model that the present invention relates to, and the curve that square dot constitutes is the cycle tests performance under the whole picture element movement estimation; The curve that the triangle form point constitutes is the cycle tests performance under sub-pixel motion is estimated; The curve that the cross form point constitutes is the cycle tests performance under whole picture element movement estimation and the sub-pixel motion estimation.As can be seen from Figure 8, under the situation that the average CYCLE number of saving equates, the BDPSNR of the cycle tests performance under the P-R-D model is the highest, and promptly loss is minimum.
Fig. 9 is the Paris cycle tests performance map of video coding complexity controlling models.As shown in Figure 9, transverse axis is the average CYCLE number of saving, and unit is %; The longitudinal axis is BDPSNR, the dB of unit.Cycle tests performance under four kinds of patterns has been shown among Fig. 9.The curve that Diamond spot constitutes is the cycle tests performance under the P-R-D model that the present invention relates to, and the curve that square dot constitutes is the cycle tests performance under the whole picture element movement estimation; The curve that the triangle form point constitutes is the cycle tests performance under sub-pixel motion is estimated; The curve that the cross form point constitutes is the cycle tests performance under whole picture element movement estimation and the sub-pixel motion estimation.As can be seen from Figure 9, under the situation that the average CYCLE number of saving equates, the BDPSNR of the cycle tests performance under the P-R-D model is the highest, and promptly loss is minimum.
To sum up, can find P-R-D model that the embodiment of the invention provides relatively other video coding models under the situation that the average CYCLE number of saving equates, loss distortion minimum.
Pass through embodiments of the invention, exist in handheld device etc. under the application of power supply constraint, adjust video encoder complexity by adopting different whole picture element movement algorithm for estimating or encoder inner parameters such as sub-pixel motion algorithm for estimating, reference frame number and macro block mode, make itself and current power supply power supply state reach coupling, thereby still can guarantee to obtain best reconstruction video quality in the power supply electricity shortage, in the time of must reducing the video coding complexity.
One of ordinary skill in the art will appreciate that and realize that all or part of step that the foregoing description method is carried is to instruct relevant hardware to finish by program, described program can be stored in a kind of computer-readable recording medium, this program comprises one of step or its combination of method embodiment when carrying out.
In addition, each functional unit in each embodiment of the present invention can be integrated in the processing module, also can be that the independent physics in each unit exists, and also can be integrated in the module two or more unit.Above-mentioned integrated module both can adopt the form of hardware to realize, also can adopt the form of software function module to realize.If described integrated module realizes with the form of software function module and during as independently production marketing or use, also can be stored in the computer read/write memory medium.
The above-mentioned storage medium of mentioning can be a read-only memory, disk or CD etc.
The above only is a preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.