Be used to improve the error covering method of video quality
Technical field
The present invention relates to the video communication field, relate in particular to the method for the picture quality that in the Internet, improves the transmission real time video data.
Background technology
Along with the development of social economy and life, people are also more and more higher for the requirement of communication, not only have been satisfied with the communication of simple speech mode, and also increasing for the video requirement, this makes that the video communication technology is developed fast.In the video communication technology, for improving video transmission quality, need suitably handle the mistake in the video transmission, main is exactly error covering and two kinds of methods of error correction.The error covering technology is not increase fully under the prerequisite of transinformation content, the technology of the recovery error message of carrying out at receiving terminal.The purpose of error covering does not lie in reduces mistake fully, but utilizes the only video information of receiving terminal to increase decoded image quality.Compare with error correction, error covering can not reduce the efficient of image encoding.And being easy to generate the wrong transmission environment of continuous long string, error correcting code itself is also disturbed by mistake easily, causes losing the ability of protection, so the error covering technology more can show the practical value that it is potential.How utilizing the only information of receiving terminal to recover image quality to increase under the extraneous information not increasing, is the problem that all error covering technology will be faced, and the technology of using always is the error covering method of time-domain and spatial domain at present.
The principle of time-domain error covering is exactly to utilize the time redundancy of interframe to do recovery.Wrong start point in the piece group may occur in the part of motion vector part and motion-compensated values.When mistake is when occurring in the part of motion-compensated values,, therefore can directly recover wrong macro block with this motion vector because the motion vector of macro block itself is errorless; If mistake be begin from the motion vector sign indicating number or the variable-length code (VLC) of macro block is wrong before, then because motion vector and motion-compensated values all are the mechanism that variable length code and macro block do not have synchronous code yet, so can cause ensuing macro block motion vector and motion-compensated values can't decode, occurring up to next piece group could be synchronous again, must look for suitable motion vector this moment and do and shelter.Because the background major part in the frame is fixed, when macro block was made mistakes, the same position macro block (i.e. zero motion vector) that can get its reference frame was filled up the macro block of loss.Yet be not that each macro block all is fixed,, therefore when the macro block mistake, can be averaged or the mode of intermediate value replaces the motion vector of erroneous macroblock by the motion vector that is close to four or six macro blocks because object is moving of systematicness mostly in the frame.When the macro block of mistake not exclusively with the macro block around contiguous have the identical direction of motion with apart from the time, if motion vector mean value or intermediate value has changed the original direction of motion of object sometimes on the contrary around simple the use, therefore except motion vector, error covering need be used the motion vector that more information decides macro block, and correct pixel value is exactly an adaptable information around the erroneous macroblock, has produced border matching method BMA (Boundary Matching Algorithm) thus.Border matching method and modified border matching method MBMA (Modified Boundary Matching Algorithm) utilize motion vector and two kinds of obtainable information of edge pixel to do recovery.The characteristics of BMA maximum are need complex calculations, utilize the macro block chosen with around correct pixel can select matcher and reach good matching effect between macro block via simple diversity ratio.But its maximum shortcoming is exactly that its coupling is easy to generate inharmonic connection when the loss macro block comprises the object edge of non-perpendicular or level variation.Mate the problem that lacks direction determining though MBMA has improved BMA for the edge, improvement does not exist different objects can produce discontinuous situation when losing in the macro block.
The error covering of spatial domain is to use the correct information that is disturbed in the current erroneous frame to recover.Has very high correlation between a certain pixel and its contiguous pixel in the general pattern picture, so can be with losing near the macro block pixel cooperates method recovery erroneous macroblock such as distance weighting with interpolation, extrapolation pixel value.When gray-value variation is little in the frame of front and back, adopt this method effect can compare nature, make the macro block that recovers that the gradually sensation of layer be arranged but also might change relations such as excessive or weight allocation is inappropriate because of object edge, method commonly used is summit pixel linear interpolation, nearest pixel linear interpolation and the interpolation method with directivity.The basic principle of space domain method is to use pixel to do interpolation to recover, when around pixel value change excessive or when the different objects edge was arranged, the recovery in usage space territory can produce the dim gradually sensation of layer, make the edge blurry of object.When such situation had the edge of non-perpendicular or horizontal direction variation in macro block, gradually the sensation of layer can be obvious especially to use the generation of nearest pixel linear interpolation or summit pixel linear interpolation.The error covering method of spatial domain is not suitable for extensive area is done recovery, because when the recovery scope is excessive, inner pixel is far away excessively from macroblock edges, loses the correlation with the edge pixel value easily.Macro block with a size 16 * 16 is an example, if recover with the method for spatial domain fully, produce fuzzy image in interior macroblocks easily, and near the pixel recovery effects the edge is better.
Time-domain and spatial domain error covering method utilize the redundancy of video on time-domain and spatial domain to do recovery respectively.For the time-domain method because use the relation of motion vector, the interframe object continuously under the situation of neat motion recovery effects fine, but when object in interframe during fixing motion, then can the discontinuous sensation of generation between macro block.Space domain method is to use the adjacent continuous between pixel to do the interpolation recovery, so pixel value changes when little around the macro block, good recovery effects is arranged, yet when around pixel value change excessive or when having the different objects edge to take place, usage space territory method recovers to produce the dim gradually sensation of layer, makes the edge blurry of object.
Summary of the invention
The objective of the invention is to overcome discontinuous or shortcoming that object edge is fuzzy between the macro block that prior art exists, can not keep in touch between macro block during fixing motion and object edge still keeps purpose clearly in interframe at object in the hope of realizing.
For achieving the above object, the present invention proposes a kind of error covering method that is used to improve video quality, it is characterized in that, may further comprise the steps:
(1) determines the erroneous macroblock position of correct macro block all around;
(2) be initial with motion vector in the time-domain principle, according to motion vector match whether principle, the macroblock partition that desire is recovered becomes several zones;
(3) select different error covering modes to recover:, to adopt the cut section BMA to recover according to the edge matching principle for the part of motion vector match; When having certain zone can't adopt the cut section BMA to recover in the macro block, further adopt spatial domain improvement nearest pixel interpolation method to recover.
In described step (2), motion vector is represented the case of motion of most of pixel in the inter-coded macroblocks, judges that according to following formula a certain motion vector of correct macro block of losing around the macro block is pixel portion coupling or whole coupling in the piece:
Abs (CP (i, j)-RP (i+mvi, j+mvj))>=λ motion vector do not match
Abs (CP (i, j)-RP (i+mvi, j+mvj))+<the λ motion vector match
Reference frame, λ when wherein mvi and the mvj component, the CP that are respectively the i of macro block motion vector and j direction represents that present frame, RP represent the present frame coding represent critical value;
In described step (2), the part that will lose macro block is divided into and is directly relatedly recovered that the zone can be recovered in zone, indirect correlation, the part indirect correlation can recover the zone and can't recover the zone;
In described step (3), the described directly related zone of replying is coupling and identical value is arranged with the motion vector of correct macro block adjacent edge all around, utilizes the motion vector that mates to take out the pixel value that the regional compensation of identical size should the zone in reference political affairs frame;
Described indirect correlation can recover the zone and be coupling with the motion vector of correct macro block adjacent edge all around value inequality is only but arranged, take out the candidate region by matched motion vector inequality at reference frame, select a candidate that mates most to recover this regional pixel value with the edge matching method;
Described part indirect correlation can recover the zone and have coupling with the motion vector of correct block adjacent edge all around unmatched situation is also arranged, at first the matched motion vector operation of unmatched edge, when if unmatched edge becomes coupling originally, then this zone converts the directly related zone of recovering to, and reset mode recovers regional identical with directly related; If instead when still not matching, then this zone comprises coupling and unmatched motion vector, utilizes this moment the edge matching method to select the zone of mating most and does recovery;
The described zone of can't recovering does not all have coupling with the motion vector of correct macro block adjacent edge all around, and the motion vector of macro block is for the candidate around utilizing, and extra to increase by zero motion-vector be one of candidate.
If four kinds of zones all take place, then after error covering was finished recovery, the motion vector of selecting the zone of shared largest portion except that can't recovering the zone was as the motion vector that recovers macro block.
In described step (3), described Improvement type nearest pixel interpolation method is improved the nearest pixel interpolation method and can be produced fuzzy situation at object edge, around the correct pixel of each bar all according to judgment value of following formula calculating:
b(i,k)=[1/(tl×dL+tr×dR+tt×dT+tb×dB)]×
[tl×dRbL(k,1)+tr×dLbR(k,N)+tt×dBbT(1,i)+tb×dTbB(N,i)](2)
Go up following judgment value about wherein tl, tr, tt and tb are respectively, when the direction that certain some pixel is recovered in losing macro block comprised different objects, its value was made as 0, otherwise then is made as 1.
In described step (3), time-domain is cut apart macro block error covering method for recovering the motion vector that the zone does not have coupling as described, and then usage space territory Improvement type nearest pixel interpolation method is recovered.
In described step (3), recover in the erroneous macroblock process integrating the territory, in cutting apart macro block, have can't the territory, recovery district the time, integrate the territory method and just can carry out; If there is not to recover the zone in the macroblock partition, then can not carry out the computing of spatial domain, directly recover wrong macro block to cut apart the macroblock match method.
The method of the invention is on the basis of existing time-domain and spatial domain error covering method, the error covering method of a kind of integrating time territory and spatial domain has been proposed in conjunction with both advantage, recover in different ways according to different wrong situations, thereby reach best image quality.Compared with prior art, when the interframe object is not fixed motion, the method of the invention has been improved the discontinuous sensation of generation between macro block significantly, has also improved space domain method and has recovered the fuzzy problem of back object edge, has reached the good result that adopts error covering method answer image.
Description of drawings
Fig. 1 judges the motion vector match schematic diagram in the method for the invention.
Fig. 2 is the directly related area schematic of recovering in the described method of this invention.
Fig. 3 is nearest pixel interpolation method and Improvement type nearest pixel interpolation method comparison diagram.
Fig. 4 is the method for the invention flow chart.
Fig. 5 is a cut section BMA flow chart in the method for the invention.
Fig. 6 is a spatial domain improvement nearest pixel interpolation method flow chart in the method for the invention.
Embodiment
Below in conjunction with accompanying drawing the method for the invention is further described.
Core concept of the present invention is the error covering method of integrating time territory and spatial domain, be initial at first with motion vector in the time-domain principle, according to motion vector match whether principle, the macroblock partition that desire is recovered becomes several zones and selects the different modes error covering method to recover, wherein the part of motion vector match adopts the cut section BMA to recover according to the edge matching principle, when having certain zone to be not suitable for edge coupling recovery in the macro block, then the Improvement type nearest pixel interpolation method in usage space territory is recovered.
The error covering method of integrating time territory and spatial domain comprises the space domain method of cut section BMA and Improvement type nearest pixel interpolation method, is described in detail as follows below.
For the macro block of interframe encode, according to the principle of estimation, motion vector is represented the case of motion of most of pixel in the macro block, can be divided into the situation of motion vector match and mate and partly mate two classes fully.Judge that a certain motion vector of losing correct macro block around the macro block is pixel portion coupling or whole coupling in the piece, can make a decision each pixel substitution one by one (1) formula.
.abs (CP (i, j)-RP (i+mvi, j+mvj))>=λ motion vector do not match
Abs (CP (i, j)-RP (i+mvi, j+mvj)<λ motion vector match (1)
Wherein mvi and mvj are respectively the i of macro block motion vector and the component of j direction.CP represents present frame, the reference frame when RP represents the present frame coding, and λ represents critical value.After the motion vector of the macro block of pixel utilization of a certain correct macro block own is brought (1) formula into, if the pixel value error of estimating with reference frame during more than or equal to critical value λ, is then represented the do not match motion vector of macro block of this pixel value own; Otherwise then be coupling.In order to determine to lose the macro block motion vector match degree of correct macro block all around, get the adjacent row or column pixel of losing macro block of each macro block and take exercises to whether flux matched judgement, as shown in Figure 1.Correct macro block is all judged to determine the situation of coupling through the border matching method around losing macro block.
When losing after motion vector match whether detection has been done at correct edge around the macro block, utilize these information to divide into four kinds of zones to the part of losing macro block, be respectively directly related recover the zone, indirect correlation can recover the zone, the part indirect correlation can recover the zone and can't recover the zone.
The directly related zone of replying: a certain zone is coupling and identical value is arranged with the motion vector of correct macro block adjacent edge all around in losing macro block, then claims this zone to be the directly related zone of recovering.Utilize the motion vector of coupling in reference political affairs frame, to take out this regional pixel value of regional compensation of identical size.With Fig. 2 is example, loses the left of macro block and the zone of top is correct when a certain, if both motion vectors are identical and left edge is all mated its motion-vector, then to lose the upper left gray area of block be the directly related zone of recovering to Fig. 2.
Indirect correlation can recover the zone: a certain zone is coupling with the motion vector of correct macro block adjacent edge all around in losing macro block only but value inequality, then claims this zone can recover regional for indirect correlation.Desire is recovered the pixel value in this class zone, can take out the candidate region at reference frame by matched motion vector inequality, selects a candidate that mates most to recover this regional pixel value with the edge matching method.With Fig. 2 is example, though being, the top of loss block left hand edge and the left side of top edge mate the motion-vector of macro block separately, but two motion vectors are inequality, so the macro block motion vector with the left side and top is sought the candidate that recovers, choose the zone of mating most by the criterion of edge coupling and do recovery.
The part indirect correlation can recover the zone: when a certain zone and the motion vector of correct block adjacent edge all around have coupling that unmatched situation is also arranged in losing macro block, then claim this zone can recover regional for the part indirect correlation.Desire is recovered this regional pixel value at first matched motion vector substitution (1) the formula computing of unmatched edge, when if unmatched edge becomes coupling originally, then this zone converts the directly related zone of recovering to, and reset mode recovers regional identical with directly related; If instead when still not matching, then this zone comprises coupling and unmatched motion vector, utilizes this moment the edge matching method to select the zone of mating most and does recovery.Be example equally with Fig. 2, suppose that the top edge left side is divided into the edge of the left macro block motion vector that do not match for the top of the edge left hand edge of coupling top macro block motion vector, if the top computing with the motion vector substitution left hand edge of top macro block becomes coupling, the grey color part of then losing macro block is transformed into the directly related zone of recovering, and directly does recovery with the motion vector of top macro block; If the top computing of substitution left hand edge still is unmatched words, then search candidate at reference frame with the motion vector of top and left macro block, select a zone of mating most to recover with the edge matching method.
Can't recover the zone: lose the edge that a certain zone in the macro block and the motion vector of correct macro block adjacent edge all around all do not have coupling, then claim this zone for recovering the zone.The motion vector of macro block is for the candidate around utilizing under such situation, and extra increase by zero motion-vector is one of candidate, and reason is that such zone may be a background.
Lose macro block with one, above-mentioned four kinds of zones all may take place, suppose the situation when four kinds of zones all take place, after error covering is finished recovery, which motion vector does this select use for follow-up macro block? the most basic principle is to select the motion vector in the zone of shared largest portion except can't recovering the zone as the motion vector that recovers macro block, represents the principle of movement of the most of pixel of this block to meet motion vector.
Improvement type nearest pixel interpolation method, topmost purpose is exactly will improve the nearest pixel interpolation method can produce fuzzy situation at object edge, the principle of its recovery is identical with the nearest pixel interpolation method, but around the correct pixel of each bar all to a judgment value shown in (2) formula.
b(i,k)=[1/(tl×dL+tr×dR+tt×dT+tb×dB)]×
[tl×dRbL(k,1)+tr×dLbR(k,N)+tt×dBbT(1,i)+tb×dTbB(N,i)](2)
Go up following judgment value about wherein tl, tr, tt and tb are respectively, when the direction that certain some pixel is recovered in losing grand determining comprised different objects, its value was made as 0, otherwise then is made as 1.
Object edge around Fig. 3 (a) assumed lost macro block changes situation, Fig. 3 (b) then shows possible object edge direction, when Fig. 3 (c) shows with zone, the nearest pixel interpolation method recovery upper right corner, use four limit pixel values to do interpolation, because different objects edge betides and will make in the macro block that recovery effects has produced blur effect.Fig. 3 (d) shows Improvement type nearest pixel interpolation method, is example to put 1, and tl and tb value are 0, and tt and tr value are 1, and for point 2, tl and tb value are 1, and tt and tr value are 0.Such reset mode can make that the differentiation of pixel becomes more obvious between the different objects.
Time-domain is cut apart in the macro block error covering method as previously mentioned, for recovering the motion vector that the zone does not have coupling, the mode that is not suitable for territory service time is recovered, can usage space territory Improvement type nearest pixel interpolation method recover the error covering method of Here it is integrating time territory and spatial domain this moment.This method is not that each macro block is done recovery with the time-domain or the mode of spatial domain fully, but equal possibility territory service time, each zone in the macro block or the mode of spatial domain.The order of its integration as shown in Figure 4.
When Fig. 4 explanation is determined in integration territory recovery error macros, the selection priority of time-domain and spatial domain method.In cutting apart macro block, have can't the territory, recovery district the time, integrate the territory method and just can carry out; If there is not to recover the zone in the macroblock partition, then can not carry out the computing of spatial domain, directly recover wrong macro block to cut apart the macroblock match method.The judgement sequential flowchart of various situations is integrated territory error covering method handling process as shown in Figure 6 as shown in Figure 5.
Because that at first carries out in integrating time territory and the spatial domain error covering method cuts apart the information that the macroblock match method partly needs motion vector, so the method is the error covering method based on time-domain, is fit to be used in interframe encode.For the intraframe coding that does not have motion vector, when making a mistake, except using general spatial domain error covering method recovery, also can utilize integrating time territory and spatial domain error covering method, the motion vector of each inter-coded macroblocks can be represented with the motion vector of the macro block of adjacent last inter-frame encoding frame opposite position, carries out recovering intra-coded picture as the same step of Fig. 6.