CN101621683A - Fast stereo video coding method based on AVS - Google Patents

Abstract

The invention proposes a fast stereo video coding method based on AVS. The method comprises the steps of using an AVS coder to code a reference image on the left, predicting a target image on the right simultaneously in a time domain and a spatial domain and using a two-level neural classifier based on an SOFM neural network to quickly determine a prediction mode. The result of first-level classification is to generate candidate objects for segmenting different blocks of each bulk. Second-level classification selects the most appropriate prediction source which is possibly forward motion estimation or disparity estimation. The input of the two-level classifier is simple calculations between a current frame and a reference frame thereof. Experiments show that the method proposed by the invention can save 80 percent of time for negligible quality attenuation, and ensures that the number of occupied coding bytes exceeds the acceptable number by only about 7 percent.

Description

A kind of fast stereo video coding method based on AVS

Technical field

The present invention relates to a kind of method for encoding stereo video, particularly a kind of fast stereo video coding method based on AVS.

Background technology

Three-dimensional video-frequency is a kind of relief video that produces when human eye is watched.Its principle is by the transmitting two paths vision signal, according to the binocular parallax principle of human eye, delivers to human eye in some way, forms three-dimensional effect in brain.Three-dimensional video-frequency can make spectators obtain truer strong visual impact, has a wide range of applications at aspects such as Digital Television, long-distance education, video conference, virtual demonstrations.But the transmitting two paths vision signal can be brought than the twice transmission quantity of original single channel signal.Therefore, must carry out coding transmission to it by effective method.

" information technology advanced audio/video " national standard (being called for short the AVS standard) video section was formally issued by national standardization administration committee in February, 2006, was numbered GB/T 20090.2-2006, and from formal enforcement on March 1 in 2006.The generation of AVS is a historical opportunity, and in the face of MPEG, H.264 wait the expensive patent royalties of standard, China presses for and works out the audio and video standard with independent intellectual property right, and this also helps improving the core competitiveness of China's digital audio/video industry.

Summary of the invention

Purpose of the present invention just is to address the deficiencies of the prior art, and has designed a kind of fast stereo video coding method based on AVS.

Technical scheme of the present invention is:

A kind of fast stereo video coding method based on AVS is characterized in that may further comprise the steps:

A kind of fast stereo video coding method based on AVS is characterized in that may further comprise the steps:

(1) left road reference video coding:

Left side road video sequence is encoded by the AVS video coding technique as the reference sequence, and producing suffix is the bit stream of .avs;

(2) the two-stage neural classifier is handled:

In order to cover all possible piecemeal and prediction source, use the two-stage neural classifier to finish pattern and determine; First order grader finishes 16 * 16 directly, 16 * 16, inner three kinds of classification results, second level grader estimates and adopts forward motion compensation or parallax compensation;

(3) right wing target image coding:

By the data after the processing of two-stage neural classifier, move accordingly or parallax compensation according to the compensation type of determining; Produce the bit stream that suffix is .avs at last.

In described fast stereo video coding method method based on AVS, in the reference video cataloged procedure of described left road, the identifying code that the AVS encoder uses is RM5.0a, coding parameter is selected the 2D-VLC entropy coding, the utilization rate aberration optimizing, 2 frame reference pictures, 2 frame IBBP, picture frame/field self adaptation uses loop filtering.

In described fast stereo video coding method method based on AVS, the grader that is based on self-organizing mapping (SOFM) neural net that described first order grader and second level grader adopt.

In described fast stereo video coding method method based on AVS, described first order grader always has 7 kinds of features, comprises 16 * 16 the average, variance, a left side of variance, top and bottom of average, bulk of bulk and right 16 * 16 average, variance, foreground image shared proportion in bulk.

In described fast stereo video coding method method based on AVS, described second level classifier calculated be input right wing video present frame, forward reference frame and synchronization left side road reference frame.Always have 4 kinds of features, comprise the variance of the average of the sub-piece of forward direction, sub-piece, the average of the sub-piece of reference picture, the variance of sub-piece.

In described fast stereo video coding method method based on AVS, described first order grader is divided into 3 classes with a bulk: 16 * 16 directly, 16 * 16, inside; Use 4,6,3 neurons as input layer, hiding layer and output layer respectively; The aero mode estimation procedure is only selected a kind of K of 3 classes in cutting apart ₁Enter next step assessment, K ₁Output score for first order grader.

In described fast stereo video coding method method based on AVS, described second level grader has used 5,4,2 neurons as input layer, hiding layer and output layer respectively; Similar to the first order, from two outputs, choose little K according to score ₂, K ₂Output score for second level grader.

In described fast stereo video coding method method based on AVS, what the second level classifier calculated parallax that described two-stage neural classifier processing procedure adopts used is block algorithm, and its step is as follows:

(1) calculate the coupling cost:

DSI(x，y，d ₀)＝F ^l(x，y)-F ^r(x+d ₀，y)

Wherein giving tacit consent on the y direction does not have parallax, and this is to use limit constraint, thinks that promptly parallax only appears on the x direction, does not have parallax on the y direction, wherein d ₀∈ (d _Min, d _Max), consider that so all possible parallax situation just can obtain a DSI space later, comprises the cost that all possible parallax determines;

(2) with the cost addition summation that obtains:

$C(x,y,d_0)=\underset{x,y\∈W}{\mathrm{\Σ}}\mathrm{DSI}(x_i,y_j,d_0)$

Suc as formula shown in, be to be unit now with the piece, be exactly that the size of each piece is by then fixing the cost value addition of all pixels in this piece at this parallax value place so calculate the cost of some parallax correspondences of certain piece;

(3) calculate parallax:

Calculated the C space of cost weighting summation of each pixel in previous step after, find out and make the C (x of each pixel ₀, y ₀, d) Zui Xiao d is as the parallax value of this pixel.

Advantage of the present invention and effect are:

1. utilize the AVS technology, improved code efficiency and speed, in practicality, also can avoid paying the patent royalties of great number.

2. use the piece coupling to calculate parallax, computational speed is fast, and amount of calculation is little.

3. by two-stage neural classifier technology, can determine to use which kind of compensation model more effectively, obtain than conventional method faster speed.

Other advantages of the present invention and effect will continue to describe below.

Description of drawings

Fig. 1---based on the fast stereo video encryption algorithm flow chart of AVS

Fig. 2---two-stage neural classifier algorithm block diagram

Fig. 3---target sequence reference frame schematic diagram

Fig. 4---use this method and use common AVS encoding ratio

Fig. 5---use this method and use H.264 encoding ratio

Embodiment

Below in conjunction with drawings and Examples, technical solutions according to the invention are further elaborated.

Fig. 1 has represented the fast stereo video encryption algorithm flow chart based on AVS.This coding method can be finished by following three steps.

One. left road reference video coding

Left side road video is only done the estimation on the time-domain as the reference sequence, and specific implementation is to encode by existing AVS encoder.

Two. the two-stage neural classifier is handled

This step is in order to determine choosing of compensation model, to select the grader based on the SOFM neural net.First order grader is finished the difference of each bulk is cut apart, and second level grader is chosen and adopted propulsion to estimate or the disparity estimation in space.Two-stage neural classifier algorithm block diagram specifies as follows as shown in Figure 2:

1. first order grader feature extraction:

First order classifier calculated present frame and forward reference frame, calculate characteristic value, comprise 7 kinds of features: comprise 16 * 16 the average, variance, a left side of variance, top and bottom of average, bulk of bulk and right 16 * 16 average, variance, foreground image shared proportion in bulk.

2. first order grader piece is cut apart:

First order grader is divided into 3 classes with a bulk: 16 * 16 directly, 16 * 16, inside.Use 4,6,3 neurons as input layer, hiding layer and output layer in the method respectively.After training fully, neural net can calculate the mark K of each output neuron ₁, the value of mark from 0.0 to 1.0.A little K ₁Can accelerated procedure, but bit quantity can be increased.For the aero mode estimation procedure, only select a kind of K of 3 classes in cutting apart ₁Enter next step assessment.Use difference in the different classes processing procedure below, directly do not enter the neural classification in the second level such as 16 * 16, and 16 * 16 classes can be further divided into 16 * 8,8 * 16,8 * 8 three seed block in the nerve classification of the second level with inner two classes.

3. second level grader feature extraction:

Second level classifier calculated present frame and propulsion estimate that space-time function is estimated.Disparity estimation adopts the piece algorithm for estimating:

At first calculate the coupling cost:

DSI (x, y, d ₀)=F ^l(x, y)-F ^r(x+d ₀, y) (formula 1)

Shown in (formula 1), wherein giving tacit consent on the y direction does not have parallax, and this is to use limit constraint, thinks that promptly parallax only appears on the x direction, does not have parallax on the y direction, wherein d ₀∈ (d _Min, d _Max), consider that so all possible parallax situation just can obtain a DSI space later, comprises the cost that all possible parallax determines.The scope that is noted that parallax will reasonably be selected, and so just can obtain a satisfied result.

To calculate cost addition summation then:

$C(x,y,d_0)=\underset{x,y\∈W}{\mathrm{\Σ}}\mathrm{DSI}(x_i,y_j,d_0)$ (formula 2)

Shown in (formula 2), be to be unit now with the piece, be exactly that the size of each piece is by then fixing the cost value addition of all pixels in this piece at this parallax value place so calculate the cost of some parallax correspondences of certain piece.Compare with the DSI space, the size in C space reduces greatly, has raising on the arithmetic speed of program.

Calculate parallax at last: calculated the C space of cost weighting summation of each pixel in a last step after, find out and make the C (x of each pixel ₀, y ₀, d) Zui Xiao d is as the parallax value of this pixel.

The feature that second level classifier calculated goes out always has 4 kinds of features, comprises the variance of the average of the sub-piece of forward direction, sub-piece, the average of the sub-piece of reference picture, the variance of sub-piece.

4. grader predictive mode in the second level is determined:

Second level grader has used 5,4,2 neurons as input layer, hiding layer and output layer respectively.Similar to the first order, from two outputs, choose little K according to mark ₂, K ₂Output score for second level grader.

So just determined that target image frame piece is to adopt forward motion compensation or compensate with space-time function.

Three. right wing target image coding

By the data after the processing of two-stage neural classifier, move accordingly or parallax compensation according to the compensation type of determining, as shown in Figure 3.

Four. performance evaluation:

In order to verify that method that the present invention proposes can validity and rapidity, adopt two experiments to come comparatively validate.First is the method that proposes of the present invention and does not use the neural classification of two-stage to carry out the comparison of coding method based on AVS.Second is the method that proposes of the present invention and based on H.264 using the neural classification of two-stage to carry out the comparison of coding method.Experiment uses 2 groups of video sequences " Puppy ", " Bookseller " as training sequence, uses " Soccer " as the experiment sequence.

1. and based on AVS do not use the neural classification of two-stage to carry out the comparison of coding method

Fig. 4 has shown the method for using the present invention's proposition and the comparative result that does not use the neural classification of two-stage to carry out coding method based on AVS.As can be seen, the method that the present invention proposes has increased bit rate 0.12% for " Puppy " sequence in first order grader, and speed saves 78.41%, and bit rate increases by 2.44% behind the grader of the second level, and speed saves 82.10%.The method that the present invention proposes has increased bit rate 2.52% for " Bookseller " sequence in first order grader, speed saves 77.93%, and bit rate increases by 6.79% behind the grader of the second level, and speed saves 83.28%.The method that the present invention proposes has increased bit rate 2.17% for " Soccer " sequence in first order grader, speed saves 77.43%, and bit rate increases by 7.68% behind the grader of the second level, and speed saves 85.19%.

2. and based on H.264 using the two-stage nerve to classify carry out the comparison of coding method

Fig. 5 has shown and has used method that the present invention proposes and based on the comparative result that H.264 uses the neural classification of two-stage to carry out coding method.That H.264 encoder uses is JM6.1e, and coding parameter is selected the CABAC entropy coding, the utilization rate aberration optimizing, and 2 frame reference pictures, 2 frame IBBP, the macroblock frame self adaptation is used loop filtering.What the quantization step of AVS was selected is 28, is 23 except the first frame step-length H.264, and other all is 26.Can observe out from experimental result, using the AVS encoding ratio to use and H.264 encoding has the performance difference of 0.3～0.5db, and this mainly is because the AVS video standard is only supported image level frame/field adaptive coding at present.But from calculating implementation complexity, the AVS encoder complexity roughly have only H.264 70%.

Claims (8)

1. fast stereo video coding method based on AVS is characterized in that may further comprise the steps:

(1) left road reference video coding:

Left side road video sequence is encoded by the AVS video coding technique as the reference sequence, and producing suffix is the bit stream of .avs;

(2) the two-stage neural classifier is handled:

In order to cover all possible piecemeal and prediction source, use the two-stage neural classifier to finish pattern and determine; First order grader finishes 16 * 16 directly, 16 * 16, inner three kinds of classification results, second level grader estimates and adopts forward motion compensation or parallax compensation;

(3) right wing target image coding:

By the data after the processing of two-stage neural classifier, move accordingly or parallax compensation according to the compensation type of determining; Produce the bit stream that suffix is .avs at last.

2. a kind of fast stereo video coding method according to claim 1 based on AVS, it is characterized in that, in the reference video cataloged procedure of described left road, the identifying code that the AVS encoder uses is RM5.0a, and coding parameter is selected 2D-VLC entropy coding, utilization rate aberration optimizing, 2 frame reference pictures, 2 frame IBBP, picture frame/field self adaptation uses loop filtering.

3. a kind of fast stereo video coding method based on AVS according to claim 1 is characterized in that, the grader that is based on self-organizing mapping (SOFM) neural net that described first order grader and second level grader adopt.

4. a kind of fast stereo video coding method according to claim 1 based on AVS, it is characterized in that, described first order grader always has 7 kinds of features, comprises 16 * 16 the average, variance, a left side of variance, top and bottom of average, bulk of bulk and right 16 * 16 average, variance, foreground image shared proportion in bulk.

5. a kind of fast stereo video coding method based on AVS according to claim 1 is characterized in that, described second level classifier calculated be input right wing video present frame, forward reference frame and synchronization left side road reference frame; Always have 4 kinds of features, comprise the variance of the average of the sub-piece of forward direction, sub-piece, the average of the sub-piece of reference picture, the variance of sub-piece.

6. a kind of fast stereo video coding method based on AVS according to claim 1 is characterized in that, described first order grader is divided into 3 classes with a bulk: 16 * 16 directly, 16 * 16, inside; Use 4,6,3 neurons as input layer, hiding layer and output layer respectively; The aero mode estimation procedure is only selected a kind of K of 3 classes in cutting apart ₁Enter next step assessment, K ₁Output score for first order grader.

7. a kind of fast stereo video coding method based on AVS according to claim 1 is characterized in that, described second level grader has used 5,4,2 neurons as input layer, hiding layer and output layer respectively; Similar to the first order, from two outputs, choose little K according to score ₂, K ₂Output score for second level grader.

8. a kind of fast stereo video coding method based on AVS according to claim 1 is characterized in that, described disparity estimation adopts the piece algorithm for estimating, and concrete steps are as follows:

(1) calculate the coupling cost:

DSI(x，y，d ₀)＝F ^l(x，y)-F ^r(x+d ₀，y)

Wherein giving tacit consent on the y direction does not have parallax, and this is to use limit constraint, thinks that promptly parallax only appears on the x direction, does not have parallax on the y direction, wherein d ₀∈ (d _Min, d _Max), consider that so all possible parallax situation just can obtain a DSI space later, comprises the cost that all possible parallax determines;

(2) with the cost addition summation that obtains:

$C(x,y,d_0)=\underset{x,y\∈W}{\mathrm{\Σ}}\mathrm{DSI}(x_i,y_j,d_0)$

Suc as formula shown in, be to be unit now with the piece, be exactly that the size of each piece is by then fixing the cost value addition of all pixels in this piece at this parallax value place so calculate the cost of some parallax correspondences of certain piece;

(3) calculate parallax:

Calculated the C space of cost weighting summation of each pixel in previous step after, find out and make the C (x of each pixel ₀, y ₀, d) Zui Xiao d is as the parallax value of this pixel.

Images