Summary of the invention
The invention provides a kind of adaptive frame/field encoding method based on the division of image adaptive band that can overcome the above problems.
In first aspect, the invention provides a kind of adaptive frame/field encoding method based on image adaptive band partitioning technology, it is characterized in that comprising the following steps:
A. judge whether present frame is first frame of video or is in scene switching place;
B. the judged result of step a for the situation of negating under, according to the image motion feature present frame is carried out band and divides, present frame is divided into wherein macro block to band violent for motion and that motion is releived;
C. according to the division result of step b present frame is carried out a frame coding or a coding,, then current band is carried out field and encode if the right motion feature of the macro block in band is violent for motion; If the right motion feature of the macro block in band is releived for motion, then current band is carried out the frame coding.
In a first aspect of the present invention, preferably, step b comprises the following steps:
B1. present frame is carried out slice-group and divide, obtain the right motion feature of macro block in the present frame, the macro block that will have the same movement feature is to being divided into a slice-group;
B2. according to described motion feature described slice-group is divided into band.
In a first aspect of the present invention, preferably, step b1 comprises:
Calculate the right gray variance value of current macro;
Releive for motion if described gray variance value, is then judged the right motion feature of current macro less than first threshold, described first threshold is for preestablishing;
If described gray variance value is greater than first threshold, then to mate most macro block right to seeking for current macro in the former frame image, current macro is compared mate most the right motion excursion amount of macro block and second threshold value with respect to it, judge the right motion feature of current macro according to comparative result, described second threshold value is for preestablishing, if it is violent for motion that described motion excursion amount, is then judged the right motion feature of current macro greater than second threshold value; If described motion excursion amount, is then judged the right motion feature of current macro less than second threshold value and is releived for motion.
In a first aspect of the present invention, preferably, step b2 comprises the following steps:
If b2.1. current macro enters step b2.3 to being that first macro block in the present frame upper left corner is right; Otherwise enter step b2.2;
If b2.2. the right motion feature of the motion feature that current macro is right and last macro block is identical, then they are classified as same band, otherwise with current macro to right as first macro block of new band, enter step 4 then;
B2.3. current macro is right to first macro block as first band in the present frame, enter step b2.4; If b2.4. current macro is then got next macro block to right as current macro to not being that last macro block in the present frame lower right corner is right, enter step b2.1, finish otherwise divide for the band of present frame.
In a first aspect of the present invention, preferably, also comprise step:
F. after step b or d, the band that marks off is merged,
If the macro block in band to quantity less than a threshold value, then incorporate it into last band or back one band according to similitude, wherein, described threshold value sets in advance, represent the minimum macro block that needs in the band to comprise to quantity, described similitude is right motion feature of the macro block in the band or content information.
In second aspect, the invention provides a kind of adaptive frame/field encoding device based on image adaptive band partitioning technology, it is characterized in that comprising:
Judge module is used to judge whether present frame is first frame of video or whether is in scene switching place;
Band is divided module, described judged result for the situation of negating under, according to the image motion feature present frame is carried out band and divides, present frame is divided into wherein macro block to band violent for motion and that motion is releived;
Coding module is used for dividing the result according to described band present frame is carried out a frame coding or a coding, if the right motion feature of the macro block in band is violent for motion, then current band is carried out field and encodes; If the right motion feature of the macro block in band is releived for motion, then current band is carried out the frame coding.
In a second aspect of the present invention, preferably, band is divided module and is comprised:
Present frame is carried out slice-group divide, obtain the right motion feature of macro block in the present frame, the macro block that will have the same movement feature is to being divided into the module of a slice-group;
Described slice-group is divided into the module of band according to described motion feature.
In a second aspect of the present invention, preferably, also comprise: the module that after described band is divided, the band that marks off is merged, if the macro block in band to quantity less than a threshold value, then incorporate it into last band or back one band according to similitude, wherein, described threshold value sets in advance, represent the minimum macro block that needs in the band to comprise to quantity, described similitude is right motion feature of the macro block in the band or content information.
In the third aspect, the invention provides a kind of adaptive frame/field encoding method based on image adaptive band partitioning technology, it is characterized in that comprising the following steps:
A. judge whether present frame is first frame of video or is in scene switching place;
B. the judged result of step a for the situation of negating under, according to the image motion feature present frame is carried out band and divides, present frame is divided into wherein macro block to band violent for motion and that motion is releived;
C. according to the division result of step b present frame is carried out a frame coding or a coding,, then current band is carried out field and encode if the right motion feature of the macro block in band is violent for motion; If the right motion feature of the macro block in band is releived for motion, then current band is carried out the frame coding;
D. under the judged result of step a is sure situation, carries out band according to image content information and divide;
E. the division result according to step b carries out a frame coding and a coding respectively to the band in the present frame, selects a wherein less coding mode as current band of coding distortion cost.
The present invention is by to the content detail of video and the analysis of motion feature, and every two field picture of video is divided into the similar band of content character, takes based on the adaptive frame/field encoding method of image band video to be encoded again.Owing to adopted the image adaptive band to divide, make to have more unified content characteristic in the band, thereby further improved the code efficiency that has now based on the adaptive frame/field encoding method of image band.
Embodiment
By in actual applications a frame coding and an analysis of experiments of the encoding characteristics of coding being drawn, it is less that frame encoding mode is suitable for use in amount of exercise, the big video area of spatial coherence between the adjacent row; And that coding mode is suitable for use in amount of exercise is bigger, the big video area of temporal correlation between the adjacent row.Hence one can see that, and in the middle of the less video of mobility, frame encoding mode is more efficient than the field coding mode; Otherwise in the middle of the bigger video of mobility, a coding mode is more efficient than frame encoding mode.In the two field picture in the middle of video, the amount of exercise of some content is big usually, and the amount of exercise of a part of content is little.
The present invention propose based on the adaptive frame/field encoding method of video content adaptive band partitioning technology moving situation according to video, the content detail feature, every two field picture is divided into the similar band of motion feature, adopts again based on the adaptive frame/field encoding method of image band video is encoded.The present invention is according to the moving situation feature of macro block in the known strips, adopts the fast algorithm of adaptive frame/field coding, under the prerequisite that guarantees encoding compression efficient, can further reduce the operand and the complexity of encoding, and improves coding rate.
In addition, the video standard of some such as h.264 is supported the slice-group partitioning technology.Slice-group is by the macro block in a two field picture or macro block subclass to the whole two field picture formed, and it also is a kind of image segmentation that image is divided into slice-group.A band is macro block or the macro block right set of particular bands group inside according to the integer amount of raster scan order arrangement.Even so, but macro block that is comprised in band or macro block are to might not being continuously arranged according to raster scan order in image inside.Therefore, the employing of slice-group technology can be more flexibly be divided into several regions with a frame of image.
Owing to need to use the adaptive frame/field coding techniques, band of dividing among the present invention and slice-group zone all by macro block to forming.The adaptive frame/field encoding method based on video content adaptive band partitioning technology that the present invention proposes can be according to the moving situation of video, content characteristic etc., more every two field picture is divided into similar slice-group zone, be several strips with each slice-group area dividing then, adopt based on the adaptive frame/field encoding method of image band again video is encoded, code efficiency can be significantly improved.
Fig. 1 is the flow chart according to the adaptive frame/field encoding method based on video content adaptive band partitioning technology of the present invention.
As shown in Figure 1, in step 101, judge at first whether the video data of input is first frame of video or the frame data of scene switching place.At present, video scene switches and to mainly contain following several types: the sudden change scene is switched, melts and is fade-in fade-out etc.
Switch at above-mentioned several main video scenes, can adopt various existing video scene detection methods to judge whether current frame image is in scene switching place.Existing scene change detection algorithm is divided three classes, and is based on gray value respectively and detects, detects and detect based on edge contour based on motion search.Back two class detection algorithms have relative preferable performance, but the high complexity of its algorithm has greatly limited their application, especially to the requirement of real-time than higher application in.Preferably, consider the requirement of computational complexity, we have selected first kind edge detection algorithm, i.e. the algorithm that detects based on gray value.In concrete the application, according to selecting different scene change detection algorithms for different requirements such as real-time coding qualities.
Under the situation of the scene detection that adopts the first kind to detect based on gray value, find by the statistical analysis to video sequence: when scene was switched, present frame and its reference frame had very big difference on gray scale and color information; And when not having scene to switch, the gray scale of whole sequence and color are in steady or gradual state substantially.Specifically, in scene switching place, all or part of appearance sudden change in 3 components (Y is a luminance component, and U, V are chromatic component) average.Adopt present frame and reference frame 3 component averages absolute difference as judge current frame image whether have the difference function of scene switching be Dx (F, Fp)=| mean (F, x)-mean (Fp, x) |.Wherein, mean () is a mean value function, and F and Fp represent present image and reference frame image, and x represents three component Y, U, V.According to this difference function, judged whether the scene switching
D
Y(F,Fp)/mean(Y)+D
U(F,Fp)/mean(U)+D
V(F,Fp)/mean(V)≥t
1h
D
Y(F,Fp)+D
U(F,Fp)+D
V(F,Fp)≥t
2h
When last two formulas satisfy simultaneously, can judge that present frame has scene to switch and takes place.In the following formula: mean (Y), mean (U) and mean (V) are respectively in the current N frame, each component mean of mean of all frames before the present frame; t
1hAnd t
2hBe decision threshold, describe the relative different and the antipode of brightness and colourity average between present frame and the reference frame respectively, and in a given empirical value scope of gray value detection method, choose.Above-mentioned algorithm only need calculate the average of three components in each two field picture, and its algorithm complex is low, is well suited in real-time demanding application.Be almost 0 through emulation experiment proof false drop rate.
After step 101 is made judgement, next adopt content-adaptive band partitioning technology that current video frame is divided.
Be first frame of video or be in scene switching place that then flow process proceeds to step 103 if judge the video data of input in step 101.
In step 103, according to image content information present frame is carried out band and divide.
At first, the macro block that will have identical content information is a slice-group to incorporating into.Because the data of video first frame lack the former frame data, and be in the video data of scene switching place because lower with the former frame correlation of data, so to the analysis of image content method of they employings such as rim detection, extract the right detail textures information of each macro block in the image, use the number of macro block centering details pixel to characterize the right detailed information of macro block, set a certain threshold value T1, with all macro blocks in the two field picture to being divided into two types of smooth and details.If the number of macro block centering details pixel is greater than T1, then with this macro block to being included into the type of details; If the data of macro block centering details pixel are less than T1, then with this macro block to being included into smooth type.Like this, just will put in order two field picture and be divided into two slice-group, the macro block (mb) type that comprises in one of them is a details, and the right type of the macro block that comprises in another is smooth.For the ease of understanding, we can claim wherein to comprise macro block (mb) type is that the slice-group of details is the details slice-group, and claims that another is the flat band group.
Then, more smooth and this slice-group of two types of details are divided into band, detailed process will describe in detail hereinafter.
Not first frame of video or be not in scene switching place that then flow process proceeds to step 102 if judge the video data of input in step 101.
In step 102, according to the right moving situation information of macro block in the image present frame is carried out band and divide.
Divide in order in step 102, to carry out band, at first will obtain the right moving situation information of macro block, the i.e. right motion feature of macro block in the image according to the right motion feature of macro block.The right motion feature of macro block can obtain by following method.
At first, the macro block that will have a same movement feature is a slice-group to incorporating into.Consider that because The noise in the image, the offset value of the match block of static flat site also may be very big.Therefore, at first use half-tone information will be in the macro block of smooth region to separating.Vgray is the right gray variance value of current macro, as Vgray during less than a certain threshold value T2, judges macro block to being in smooth region, and its type of sports is releived for motion; As Vgray during greater than threshold value T2, it is right to the coupling macro block in the former frame image to use the method for piece match search to obtain current macro, uses current macro to mating most macro block to the x of position, the severe degree side-play amount absolute value and that represent the motion that current macro is right of y coordinate figure with it.Then, for above-mentioned side-play amount absolute value with set a certain threshold value T3, can the type of sports that current macro is right be judged as the violent or motion of motion and releive.Particularly, if motion excursion amount absolute value and greater than T3, judge that then the right motion feature of current macro is violent for motion; If motion excursion amount absolute value and less than T3, then judge the right motion feature of current macro for the motion releive.Like this, just will put in order two field picture and be divided into two slice-group, the macro block (mb) type that comprises in one of them is violent for motion, and the right type of the macro block that comprises in another is releived for motion.For the ease of understanding, we can claim wherein to comprise macro block (mb) type and be the violent slice-group of motion for the violent slice-group of motion, and claim that another is the slice-group of releiving of moving.
Then, will move violent again and motion this slice-group of two types of releiving is divided into band, detailed process will describe in detail hereinafter.
It will be appreciated by those skilled in the art that according to the method described above macro block is actually the process that a two field picture is divided into slice-group to the process of classifying.Next, slice-group should be divided into band, so that carry out frame field coding based on band.As mentioned above, first frame or the Frame that is in scene switching place are divided into flat band group and details slice-group, and other Frames are divided into the violent and motion of the motion slice-group of releiving, and no matter be which kind of frame data is divided the slice-group that forms, can adopt following two kinds of methods again it to be divided into band.
On the one hand, the present invention adopts following method that one two field picture is divided into band:
1. if current macro enters step 3 to being that first macro block in the present frame upper left corner is right; Otherwise enter step 2;
If current macro to type and last macro block to type identical (belong to together smooth or details in one type, perhaps belong to one type that moves acutely or move in releiving together), then they are classified as same band, otherwise current macro to right as first macro block of new band, is entered step 4
Current macro is right to first macro block as first band in the present frame 3., enter step 4
4. if current macro is then got next macro block to right as current macro to not being that last macro block in the lower right corner is right in the present frame, flow process enters step 1, otherwise divides end for the band of present frame.
Note that in the band that finally marks off in the said process macro block in a two field picture from the upper left corner to the lower right corner according to the sequence arrangement of raster scan.
On the other hand, also can according to real needs directly slice-group is divided into one or a plurality of band encode.Note that if directly slice-group is divided, the macro block in the band that so finally marks off in a two field picture from the upper left corner to the lower right corner not necessarily according to the sequence arrangement of raster scan.
After present frame is divided into band, can directly carry out encoding based on the adaptive frame/field encoding method of image band.Yet, band divide finish after only to comprise a spot of macro block right owing to might occur each band the inside in actual applications, thereby can cause the decline of code efficiency.For fear of the generation of this situation, the band that needs to have divided merges.
The concrete grammar that merges band is that a parameter Qmin is set, and represents the minimum macro block that needs in each band to comprise to number.If using content-adaptive band partitioning technology to carry out band to the frame data in the video divides, the macro block that resultant a certain strips S comprises to number less than Qmin, then according to similitude it is incorporated among last strips S p or the back one strips S f, described similitude is right motion feature of the macro block in the band or content information.
Judge a band needs merging according to parameter Qmin after, can and be in the frame data of scene switching place for first frame, and not be that first frame and the true data that is not in scene switching place adopt the merging method based on similitude as described below respectively.
So to first frame or be in the frame data of scene switching place, the concrete grammar that merges band is:
Relatively among S, Sp and the Sf each macro block to the difference of comprising of details pixel mean value, with S incorporate into the little band of current band difference in.If the difference of S, Sp and Sf is identical, then S is incorporated into and comprise macro block among Sp, the Sf in the less band of number.
For not being first frame or the frame data that are not in scene switching place, the concrete grammar that merges band is:
1. be motion during violent band as S, if the macro block of the level and smooth type that comprises among the Sp to number greater than Sf, then S is incorporated into Sf, otherwise then incorporates Sp into; If the macroblock number of the level and smooth type that comprises among Sp, the Sf is identical, then the mean value of the right match block offset value of the macro block of Sp, the non-level and smooth type of Sf is relatively incorporated S in the big band of deviant mean value into;
2. be motion when releiving band as S, relatively the mean value of the right match block offset value of the macro block of Sp, the non-level and smooth type of Sf is incorporated S in the little band of deviant mean value into;
3. when current step 1 and step 2 can't determine the ownership of S, incorporate S into Sp, Sf and comprise macro block in the less band of number.
So far, content-adaptive band of the present invention is divided and is finished.Should be appreciated that various types of slice-group are divided into band that again or through after the bar tape merge, last division result can correspondingly be referred to as: details band, flat band, and the motion band of releiving, violent band moves.
Then, in step 104, use and encode based on the adaptive frame/field encoding method of image band.
At step 104-1, so to video first frame or be in scene and switch the video data that, select traditional adaptive frame/field encoding method:, carry out a frame coding and a coding respectively for each image band based on the image band; Select a less coding mode of cost as the image band.
At step 104-2, for not being video first frame or the video data that is not in scene switching place, utilize the adaptive frame/field coding fast algorithm of the moving situation information employing of the interior macro block of image band that obtains:, then select the field to encode as its coding mode for the violent band of motion based on the image band; For the motion band of releiving, select the frame coding as its coding mode.
The adaptive frame/field encoding method that the present invention proposes based on video content adaptive band partitioning technology, by for the content of video own, the abundant analysis of motion feature, every two field picture of video is divided into the similar band of moving situation characteristic, takes again video to be encoded based on the adaptive frame/field encoding method of image band.The feasible video band that uses based on the adaptive frame/field encoding method of image band of the image adaptive band partitioning technology of our design has more unified motion feature, the advantage of adaptive frame/field encoding method can better be used, thereby higher encoding compression efficient can be obtained.In addition, utilize the motion feature of the image band that obtains, the present invention proposes adaptive frame/field coding fast algorithm, in the encoding compression gain that guarantees to obtain based on the image band, can also further improve speed of coding, reduce the computational complexity of coding.
Obviously, under the prerequisite that does not depart from true spirit of the present invention and scope, the present invention described here can have many variations.Therefore, the change that all it will be apparent to those skilled in the art that all should be included within the scope that these claims contain.The present invention's scope required for protection is only limited by described claims.