Background technology
Along with Development of Multimedia Technology, particularly the application based on the video conferencing of long-range video and audio communication constantly enlarges, and in communication process how vision signal is realized that real-time, the integrality of encoding effectively to guarantee communication just seem more and more important.
H.263, existing video compression international standard H.261, all adopt based on block motion compensation (Motion compensation with MPEG1, MPEG2, MPEG4, be called for short MC) and discrete digital cosine transform (Digitalcosine transform, be called for short DCT) the mixing compression method of technology all is to finish compression to vision signal by removal time and spatial redundancy.
But owing to the DCT frequency spectrum does not have reasons such as local characteristics, thus piece conversion and coding techniques shortcoming such as intrinsic blocking artifact, mosquito noises often unsatisfactory; In addition, in the problem that also exists inconvenience to handle aspect the anti-error code.For this reason, industry just begins to adopt the wavelet transformation technique with advantages such as conversion frequency spectrum local characteristics, multiresolution characteristics to come compressed video signal.In the prior art, mainly contain following three types of technology scheme:
One, estimation-prediction difference image-wavelet transformation-coding;
Two, wavelet transformation-estimation-prediction difference image-coding;
Three, 3D wavelet transformation, i.e. spatial domain 2D wavelet transformation+time orientation 1D wavelet transformation.
Because scheme three has bigger time delay, so feasible scheme is preceding two kinds in the realtime graphic communications field.The technology of scheme one with H.261, H.263 reach in the mpeg standard technology type seemingly, the main distinction is to replace DCT with discrete digital wavelet transformation (Digital wavelet transform is called for short DWT).For example, the patent No. is: US5764805, the United States Patent (USP) that name is called " Low bit rate video encoder using overlappingblock motion compensation and zerotree wavelet coding " discloses this technology, this patent adopts overlapped block motion compensation (Overlapping block motioncompensation, being called for short OBMC) technology carries out estimation and prediction asks poor, carry out wavelet transformation then, and adopt zero-tree wavelet coefficient coding method (Embedded image coding using zerotrees ofwavelet coefficients is called for short EZW) to encode.Scheme two is to carry out estimation and motion compensation in the wavelet coefficient territory, adopts multiresolution estimation and motion compensation process usually, and generally will carry out DCT again to the prediction difference image, and then quantizes and entropy coding.For example, the patent No. is: US5495292, name is called the United States Patent (USP) of " Inter-frame wavelet transform coder for color videocompression ".The coding method that this patent adopts is Run-Length Coding RLC (Run lengthcode) and entropy coding method.
Though the technology of above-mentioned two patent disclosures has solved to a certain extent based on the problem in block motion compensation and the discrete digital cosine transform technology, still there is shortcoming separately in above-mentioned two patents.As follows specifically:
The patent No. is the described method of the patent of US5764805, owing to be that whole two field picture is carried out estimation and motion compensation, thereby be not easy to directly carry out space layered coding; The patent No. is the described method of the patent of US5495292, when high-frequency sub-band being carried out estimation calculating, because the characteristics that high frequency coefficient distributes, its accuracy is difficult to be guaranteed, and the prediction piece is carried out dct transform and RLC coding again, computation complexity is bigger, and coding method does not make full use of the correlation of intersubband yet, can not get compression efficiency preferably.
Summary of the invention
The object of the invention is to provide a kind of real-time and method for video coding that have error-resilient performance.
For achieving the above object, method for video coding of the present invention comprises the steps:
(1) whole two field picture is carried out wavelet decomposition;
(2) low frequency sub-band after the wavelet decomposition being made estimation calculates;
(3) difference is asked in each subband prediction, wherein the high-frequency sub-band motion vector adopts with the whole pixel motion estimated result of layer low frequency sub-band;
(4) the high-frequency sub-band prediction difference image to gained in the step (3) quantizes;
(5) respectively quantized result is encoded.
Wherein:
Also comprise before in step (4) the thickest layer low frequency sub-band prediction difference image is carried out wavelet transformation, and in step (4), resulting each subband of this wavelet transformation is quantized.
The thickest layer low frequency sub-band prediction difference image is carried out wavelet transformation, till being not easy to decompose again
Step (2) adopts multiresolution method for estimating (MRME).
Estimation employing to low frequency sub-band is taken thin straton band to by thick straton, the mode of the refinement that makes progress gradually; Or adopt and take thick straton band to, the downward gradually mode of calculating by thin straton.
If the absolute value of subband piece difference of gained when estimation and (SAD) surpass certain thresholding, then to Forecasting Methodology calculated difference data in this piece utilization piece;
Coded system is signal to noise ratio hierarchical coding, space layered coding or time hierarchical coding.
Coding method is embedded zerotrees wavelet algorithm (EZW), layer tree diversity algorithm (SPIHT) or importance coupling assembling parser (SLCCA).
The outstanding advantage of the method for the invention is:
A, not based on the normal blocking artifact phenomenon that occurs in the block coding method, can bigger raising visual quality of images under low bandwidth;
B, owing to only low frequency sub-band is carried out estimation and calculates, its result of calculation is applied to the high-frequency sub-band of layer together, and can adopt multiresolution method for estimating from coarse to fine or from fine to coarse between the different layers low frequency sub-band, can significantly reduce amount of calculation, improve computational efficiency;
C, can realize signal to noise ratio layering, space delamination and time hierarchical coding very naturally;
D, owing to can be similar to mutually between the motion vector of different layers low frequency sub-band and push away, and to their encode respectively packing transmission, under the situation that one deck subband motion vector information is lost, can obtain from another straton band motion vector information is approximate, thereby have error-resilient performance preferably therein;
E, has higher compression efficiency;
F, can realize diminishing and can't harm two kinds of compress modes.
Embodiment
Present embodiment is that example is described in detail encryption algorithm and process with CIF form and 3 grades of discrete digital wavelet transformations (DWT).
Be convenient narration, present embodiment is represented (adding the digital subband of then representing different layers behind letter) with LL, HL, LH and HH respectively with a low frequency and three high-frequency sub-band that image obtains after through a wavelet transformation.
Consult Fig. 1 and Fig. 2:
At first present frame, the predictive frame with input carries out DWT decomposition, i.e. step 10 among Fig. 1.In the present embodiment image is carried out 3 grades of DWT and decompose (being the DWT1 among Fig. 1), establish image after the decomposition and be respectively f (t) and f (t-1), its data structure as shown in Figure 3.
Low frequency sub-band after the DWT decomposition is made estimation calculate, be i.e. step 11 among Fig. 1.At first consider estimation and motion compensation problem between f (t) and f (t-1), the present invention is based on following hypothesis: along with moving of certain target in the image, LL, HL, LH and the HH of same layer have identical displacement vector corresponding to the piece of this target among the f (t).This hypothesis obviously is rational, because they are respectively the low frequency coefficient and the high frequency coefficient of same target after all, therefore, the present invention only carries out estimation calculating to low frequency component LL1, LL2, the LL3 of each layer, its result of calculation is applied to the high-frequency sub-band corresponding blocks, thereby can significantly reduces amount of calculation.Further, because the resolution of adjacent layer LL subband has the multiple relation, therefore, the present invention adopts multiresolution method for estimating MRME that each layer LL subband is carried out estimation, comprises from coarse to fine and two class methods from fine to coarse:
(1) from coarse to fine: as to carry out estimation to 4 * 4 among the LL3 earlier, then, consider corresponding 8 * 8 among the LL2, with 2 times of motion vector corresponding 4 * 4 among the LL3 as initial displacement vector, again around it 8 carry out estimation and calculate, find match block.Similarly, can obtain motion vector result corresponding 16 * 16 among the LL1.
(2) from fine to coarse: earlier carry out estimation to 16 * 16 among the LL1, with its motion vector value divided by 2 obtaining motion vector value corresponding 8 * 8 among the LL2, can obtaining motion vector value corresponding 4 * 4 among the LL3 divided by 4.
In two class methods, can carry out 1/2,1/4 pixel precision estimation, and only need consider that for other subband whole pixel precision gets final product for the LL3 subband.
For the piece that in the hunting zone, can not find accurate matched position, promptly poor absolute value and (Sum ofabsolute difference, being called for short SAD) value surpasses the thresholding set (corresponding to the INTRA piece in H.263), then this piece uses prediction mode in piece, and each pixel predicted value is taken as 0 without exception.
Then, carry out the step 12 among Fig. 1, each subband prediction difference is calculated.In this step was calculated, the high-frequency sub-band motion vector adopted with the whole pixel motion estimated result of layer low frequency sub-band, to reduce amount of calculation significantly.
In order to obtain the more efficient compression effect, the thickest low frequency sub-band LL3 prediction difference is carried out 2 grades of DWT again decompose (being the DWT2 among Fig. 1): LL5, HL5, LH5, HH5, HL4, LH4 and HH4, as shown in Figure 4.To the thickest low frequency sub-band LL3 prediction difference, generally be proceed to be not easy to decompose again till.
Then, carry out the step 13 among Fig. 1, each subband to DWT1 gained high-frequency sub-band prediction difference image and DWT2 gained quantizes respectively, adopts different quantizers to quantize to the subband of twice wavelet decomposition is general.
At last, carry out the step 14 among Fig. 1, respectively each subband after quantizing is encoded.At first LL5, HL5, LH5, HH5, HL4, LH4, HH4 are encoded, again HL3, LH3, HH3, HL2, LH2, HH2, HL1, LH1, HH1 are encoded, coding method can be adopted embedded zerotrees wavelet algorithm (EZW), layer tree diversity algorithm (SPIHT) or importance coupling assembling parser (SLCCA) etc., make full use of the correlation that intersubband exists, improve code efficiency.
Method for video coding of the present invention can be realized signal to noise ratio layering, space delamination and time hierarchical coding function at an easy rate.
(1) signal to noise ratio hierarchical coding.Carry out Bit-Plane Encoding, the scanning sequency on each plane is LL5, HL5, HL4, LH5, LH4, HH5, HH4 and HL3, HL2, HL1, LH3, LH2, LH1, HH3, HH2, HH1, compose with a sign on each plane, can realize signal to noise ratio hierarchical coding function.
(2) space layered coding.LL3 decomposes subband and encodes separately, and the scanning sequency of other subband is HL3, LH3, HH3, HL2, LH2, HH2, HL1, LH1, HH1, and each decomposition layer is composed with a sign, but implementation space hierarchical coding function.
(3) time hierarchical coding.Carry out bi-directional motion estimation and compensation, introduce B frame coding, can realize time hierarchical coding function.
Method for video coding of the present invention has error-resilient performance preferably.Adopt hierarchy encoding method, after losing a certain amount of information, still can obtain acceptable picture quality in decoding end.Owing to need the motion vector information of coding transmission also few, the present invention is with the transmission of packing respectively of the block motion vector information of LL3 and LL1, when the motion vector information of one of them subband is lost, still can obtain approximation by the motion vector information of another subband, gained picture quality generally can also be accepted.
Method for video coding of the present invention can be selected the floating-point small echo, as the two-way orthogonal wavelet of 9-7, also can select integer wavelet, as the 5-3 integer wavelet.When adopting integer wavelet, can carry out lossless compress, thereby can be applicable to the situation of lossless video communication.
Identical with this enforcement for other form with the method for encoding images of multistage discrete digital wavelet transformation (DWT).