CN105992012B - Error concealment method and device - Google Patents
Error concealment method and device Download PDFInfo
- Publication number
- CN105992012B CN105992012B CN201510086369.3A CN201510086369A CN105992012B CN 105992012 B CN105992012 B CN 105992012B CN 201510086369 A CN201510086369 A CN 201510086369A CN 105992012 B CN105992012 B CN 105992012B
- Authority
- CN
- China
- Prior art keywords
- block
- spatial
- error
- adjacent
- current error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
- H04N19/895—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides an error concealment method, which comprises the following steps: sequentially taking each error block in the current error coding tree unit CTU as a current error block; determining a space-domain left adjacent block and a space-domain adjacent block of the current error block, wherein the space-domain left adjacent block and the space-domain adjacent block are adjacent to the error block in a space domain; performing temporal and spatial correlation comparisons when both spatial left and spatial neighboring blocks of the current erroneous block are available and neither are intra-predicted blocks; if the spatial correlation of the current error block is larger than a preset intensity threshold value, performing spatial interpolation error concealment; and if the spatial correlation of the current error block is not greater than a preset intensity threshold, performing time domain error concealment. The scheme combines the coding characteristics of the HEVC video, repairs damaged images by using the correlation between error block data and spatial domain time domain adjacent data, and has a good repairing effect on damaged HEVC video code streams.
Description
Technical Field
The invention relates to the field of videos, in particular to a method and a device for error concealment.
Background
The great advances in network technology and video coding technology have led to an increasing popularity of broadcast television, network video and video communications. Inevitable mobile channel fading and network congestion inevitably cause damage to video data during transmission. The error concealment of the video at the decoding end is an important method for improving the user experience.
After detecting video errors at the decoding end, the error concealment technique approximately recovers damaged images by using the correlation between the video information that has been correctly decoded and damaged data. Error concealment techniques can be divided into spatial error concealment and temporal error concealment based on the spatial and temporal correlation of available data with erroneous data.
Spatial error concealment mainly utilizes the spatial correlation of video images, and the data of the damaged area can be reconstructed by interpolating the correct data of the adjacent frame. The current spatial error concealment methods are mainly represented by two types: BI (BI-linear Interpolation) and DI (Directional Interpolation).
Temporal error concealment mainly exploits the temporal correlation of video pictures, i.e. the estimation of erroneous data in a current picture using correctly decoded picture data in a reference picture. A representative method is BMA (Boundary matching algorithm ). The basic idea of the algorithm is to extract a motion vector from a correctly received block around a damaged block as a candidate motion vector of a current damaged block according to a motion consistency principle between adjacent blocks, then select a motion vector which minimizes the Sum of absolute differences (SAD of absolute difference) of boundary pixels of a motion-compensated macroblock and boundary pixels of a lost macroblock as a motion vector of the current lost macroblock, and then perform motion compensation to repair the damaged macroblock.
Error concealment techniques, which typically incorporate specific coding properties, work better because of the correlation between data to be exploited. The damaged CTU (Coding Tree Unit) size in an HEVC (High Efficiency Video Coding) Video is much larger than the size of a macroblock in the past standard, the variable Coding Unit size, multi-angle intra-frame prediction and advanced motion vector prediction methods are one of the major differences between the HEVC Coding standard and h.264, and the existing error concealment technology is directly applied to the HEVC Video and cannot obtain satisfactory Video quality.
The invention content is as follows:
the invention provides an error concealment method and device, which are used for solving the problem that HEVC video cannot obtain satisfactory video quality easily.
To solve the above technical problem, the present invention provides an error concealment method, including:
sequentially taking each error block in the current error coding tree unit CTU as a current error block;
determining an airspace left adjacent block and an airspace adjacent block of the current error block, wherein the airspace left adjacent block and the airspace adjacent block are adjacent to the error block in an airspace;
performing temporal and spatial correlation comparisons when the spatial left neighboring block and the spatial neighboring block of the current erroneous block are both available and are not intra prediction blocks;
if the spatial correlation of the current error block is larger than a preset intensity threshold value, performing spatial interpolation error concealment; and if the spatial correlation of the current error block is not greater than a preset intensity threshold, performing time domain error concealment.
Preferably, the method further comprises:
and when the spatial left adjacent block and the spatial adjacent block are both available and are intra-frame prediction blocks, performing intra-frame prediction error concealment on the current error block.
Preferably, the first and second electrodes are formed of a metal,
the performing the time-domain and spatial-domain correlation comparison comprises:
when SADSIs less than (SAD)T1+SADT2) When the error block is in a/3 state, the spatial correlation of the current error block is strong;
when SADSGreater than or equal to (SAD)T1+SADT2) When the current error block is in a time domain, the time domain correlation of the current error block is strong;
wherein:
wherein n is the frame number, x is the horizontal coordinate of the current error block, y is the vertical coordinate of the current error block, m is the size of the current error block, i, j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the performing spatial interpolation error concealment comprises:
calculating a predicted pixel P' (i, j) of the current error block according to the spatial left adjacent block of the current error block;
wherein, i, j is the pixel coordinate of the current error block; p' (0,0) is the top left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the performing time-domain error concealment comprises:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the intra prediction error concealment of the current erroneous block comprises:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the first and second electrodes are formed of a metal,
before sequentially using each error block in the current error coding tree unit CTU as the current error block, the method further includes:
when any one of an upper spatial adjacent CTU, a lower spatial adjacent CTU, a left spatial adjacent CTU, a right spatial adjacent CTU and a time domain adjacent CTU of the current error CTU is divided, dividing the current error CTU into 4 error blocks;
and when the adjacent CTUs in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTUs in the time domain of the current error CTU are not divided, taking the whole current error CTU as 1 error block.
The present invention also provides an error concealment apparatus, comprising:
the configuration module is used for taking each error block in the current error coding tree unit CTU as a current error block in sequence;
an information obtaining module, configured to determine a spatial left neighboring block and a spatial neighboring block of the current error block, where the spatial left neighboring block and the spatial neighboring block are adjacent to the error block in a spatial domain;
a determining module, configured to perform a time domain and spatial domain correlation comparison when the left spatial neighboring block and the upper spatial neighboring block of the current erroneous block are both available and are not intra-frame prediction blocks;
a concealment processing module, configured to perform spatial interpolation error concealment if the spatial correlation of the current error block is greater than a preset intensity threshold; and the processor is further configured to perform time-domain error concealment if the spatial correlation of the current erroneous block is not greater than a preset strength threshold.
Preferably, the first and second electrodes are formed of a metal,
the concealment processing module is further configured to perform intra prediction error concealment on the current erroneous block when the spatial left neighboring block and the spatial left neighboring block are both available and are intra prediction blocks.
Preferably, the first and second electrodes are formed of a metal,
the determining module is configured to perform temporal and spatial correlation when the spatial left neighboring block and the spatial neighboring block of the current erroneous block are both available and are not intra-prediction blocks, where:
when SADS<(SADT1+SADT2) When the error block is in a/3 state, the spatial correlation of the current error block is strong; wherein:
wherein n is the frame number, x is the horizontal coordinate of the current error block, y is the vertical coordinate of the current error block, m is the size of the current error block, i, j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the hiding processing module is used for carrying out spatial interpolation error hiding, and the hiding processing module is used for:
calculating a predicted pixel P' (i, j) of the current error block according to the spatial left adjacent block of the current error block;
wherein, i, j is the pixel coordinate of the current error block; p' (0,0) is the top left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the hiding processing module is further configured to perform time domain error hiding, where:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the concealment processing module is further configured to perform intra prediction error concealment on the current erroneous block, including:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the apparatus further comprises:
the dividing module is used for dividing the current error CTU into 4 error blocks when any one of an upper spatial adjacent CTU, a lower spatial adjacent CTU, a left spatial adjacent CTU, a right spatial adjacent CTU and a time domain adjacent CTU of the current error CTU is divided; and the method is also used for taking the whole current error CTU as 1 error block when the adjacent CTU in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTU in the time domain of the current error CTU are not divided.
The scheme combines the coding characteristics of the HEVC video, repairs damaged images by using the correlation between error block data and spatial domain time domain adjacent data, and has a good repairing effect on damaged HEVC video code streams.
Drawings
FIG. 1 is a flowchart illustrating a method of error concealment according to a first embodiment of the present invention;
FIG. 2 is a diagram illustrating neighboring blocks of an error block according to an embodiment of the present invention;
FIG. 3 is a block location diagram illustrating a comparison of spatial and temporal correlations in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of spatial interpolation according to a first embodiment of the present invention;
FIG. 5 is a schematic diagram of pixels used to calculate boundary matching differences according to one embodiment of the present invention;
fig. 6 is a schematic structural diagram of an error concealment apparatus according to a first embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the present application more apparent, embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
Example one
The technical scheme of the invention combines the HEVC video coding characteristics, and repairs the damaged image by utilizing the correlation between the damaged image area and the decoded image information so as to solve the problem that the HEVC video cannot obtain satisfactory video quality.
The following provides a detailed embodiment of the present invention, which is further described with reference to the accompanying drawings.
As shown in fig. 1, the present invention provides an error concealment method, which includes:
step S101: sequentially taking each error block in the current error coding tree unit CTU as a current error block;
step S103: determining a spatial left adjacent block and a spatial upper adjacent block of the current error block;
the spatial left adjacent block and the spatial adjacent block are adjacent to the error block in a spatial domain;
step S105: turning to step S107 when both the spatial left neighboring block and the spatial neighboring block of the current erroneous block are available and are not intra prediction blocks;
step S107: performing a time domain and spatial domain correlation comparison;
step S109: if the spatial correlation of the current error block is greater than a preset intensity threshold, turning to step S110; if the spatial correlation of the current error block is not greater than the preset intensity threshold, turning to step S111;
step S110: carrying out spatial interpolation error concealment; step S101 is transferred to determine the next current error block;
step S111: performing time domain error concealment; go to S101 to determine the next current erroneous block.
It should be noted that the execution flow provided by the present embodiment is the same for each current error block, and each error block in one error CTU sequentially executes the operations, i.e., step S101 to step S111. And when one current error block is subjected to error concealment, continuously searching the next error block as a current error, and performing corresponding error concealment. And when all error blocks contained in one error CTU are subjected to error concealment, searching the next error CTU for corresponding error concealment until all the error CTUs are subjected to the error concealment.
Preferably, the method further comprises:
step S106: and when the spatial left adjacent block and the spatial adjacent block are both available and are intra-frame prediction blocks, performing intra-frame prediction error concealment on the current error block.
Preferably, the method further comprises:
step S100: when any one of an upper spatial adjacent CTU, a lower spatial adjacent CTU, a left spatial adjacent CTU, a right spatial adjacent CTU and a time domain adjacent CTU of the current error CTU is divided, dividing the current error CTU into 4 error blocks;
and when the adjacent CTUs in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTUs in the time domain of the current error CTU are not divided, taking the whole current error CTU as 1 error block.
It should be noted that step S105 and step S106 do not limit the order, and in other embodiments, step S106 may be executed first. In addition, the order of step S110 and step S111 is not limited.
The above steps are further explained and illustrated below:
as shown in fig. 2, in this embodiment, for a certain error block, assuming that the pixel coordinate of the upper left corner of the error block is (x, y) and the size is w, the upper neighboring block of the error block is the PU (prediction Unit) where the pixel (x, y-1) is located; the left neighboring block is the PU where pixel (x-1, y) is located; the lower neighboring block is the PU where pixel (x, y + w +1) is located; the right neighboring block is the PU where pixel (x + w +1, y) is located; the lower left neighboring block is the PU where pixel (x-1, y + w +1) is located; the upper right neighboring block is the PU where pixel (x + w +1, y-1) is located;
the temporal neighboring block is a PU where POC (Picture Order Count) is smaller than the POC of the current Picture and the closest pixel (x + w/2-1, y + w/2-1) in the decoded Picture is located.
In steps S105 and S106, when a neighboring block satisfies both of the following two conditions, the neighboring block is considered to be available; otherwise, a neighboring block is considered unavailable when the neighboring block cannot satisfy both of the following conditions:
condition one, the adjacent block does not exceed the image boundary, namely is in the image;
conditional two, the neighboring blocks are decoded correctly, or subjected to error concealment. The error concealment process here means that the error concealment process has been performed by the error concealment method described in the present invention.
The comparing of the time domain correlation and the spatial correlation in step S105 specifically includes:
as shown in fig. 3, a left neighboring block B (n, x-1, y) and an upper neighboring block B (n, x, y-1) are found with the current erroneous block B (n, x, y), a temporal neighboring block B (n-1, x, y) is found with the current erroneous block B (n-1, x-1, y) and a left neighboring block B (n-1, x, y-1) and an upper neighboring block B (n-1, x, y-1) are found with the temporal neighboring block B (n, x, y).
Calculating SAD between neighboring blocks B (n, x-1, y) and B (n, x, y-1)S(ii) a Then respectively calculating SADs of B (n, x-1, y) and B (n-1, x-1, y)T1And anSAD of B (n, x, y-1) and B (n-1, x, y-1)T2。
When SADSIs less than (SAD)T1+SADT2) When the error block is in a/3 state, the spatial correlation of the current error block is strong;
when SADSGreater than or equal to (SAD)T1+SADT2) When the current error block is in a time domain, the time domain correlation of the current error block is strong;
wherein:
wherein n is the frame number, x is the horizontal coordinate of the current error block, y is the vertical coordinate of the current error block, m is the size of the current error block, i, j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
The spatial interpolation error concealment in step S110 specifically includes:
as shown in fig. 4, the predicted pixel P' (i, j) of the current error block is calculated from the spatial left neighboring block of the current error block, and specifically, the predicted pixel of the current error block is obtained by performing interpolation using a row of pixels at the boundary of the spatial left neighboring block and a row of pixels at the boundary of the spatial neighboring block.
Wherein, i, j is the pixel coordinate of the current error block; p' (0,0) represents the upper left pixel value of the current erroneous block.
The step S106 of performing intra prediction error concealment on the current error block specifically includes:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
As shown in fig. 5, when specifically calculating the boundary matching difference of each candidate recovery block, the boundary matching difference is calculated according to a circle of pixels on the inner boundary of the candidate recovery block and adjacent pixels in the adjacent blocks.
The boundary matching difference can be specifically calculated by using the following formula 1:
equation 1: SADE＝∑0≤i＜m||P(i，j)-P(i，j-1)||+∑i＝0，m-1||P(i，j)-P(i-1，j)||
Wherein m is the size of the current error block; i, j is the pixel coordinate of the current error block; p (0,0) is the top left pixel value of the current error block.
The time domain error concealment in step S111 specifically includes:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block, and taking the MV corresponding to the recovery pixel block as the MV of the current error block.
The specific calculated boundary matching difference can be obtained by calculation according to the method of the above formula 1. Meanwhile, in this step, it is necessary to determine whether the difference between the same components of any two MVs in each MV is greater than a predetermined threshold, for example, the predetermined threshold may be set to 4, and then the current error block is divided into 4 error sub-blocks when formula 2 is satisfied;
equation 2 is | MVi(j)-MVi(j-m)|＞4。
As shown in fig. 6, the present invention also provides an error concealment apparatus, which includes:
a configuration module 11, configured to take each error block in the current error coding tree unit CTU as a current error block in sequence;
an information obtaining module 12, configured to determine a left spatial neighboring block and an upper spatial neighboring block of the current error block, where the left spatial neighboring block and the upper spatial neighboring block are adjacent to the error block in a spatial domain;
a determining module 13, configured to perform a time domain and spatial domain correlation comparison when the left spatial neighboring block and the upper spatial neighboring block of the current error block are both available and are not intra-frame prediction blocks;
a concealment processing module 14, configured to perform spatial interpolation error concealment if the spatial correlation of the current error block is greater than a preset intensity threshold; and the processor is further configured to perform time-domain error concealment if the spatial correlation of the current erroneous block is not greater than a preset strength threshold.
Preferably, the first and second electrodes are formed of a metal,
the concealment processing module 14 is further configured to perform intra-prediction error concealment on the current erroneous block when the spatial left neighboring block and the spatial neighboring block are both available and are intra-prediction blocks.
Preferably, the first and second electrodes are formed of a metal,
the determining module 13 is configured to perform temporal and spatial correlation when the left spatial neighboring block and the upper spatial neighboring block of the current erroneous block are both available and are not intra-prediction blocks, where:
when SADS<(SADT1+SADT2) In/3, empty of said current error blockThe domain correlation is strong; wherein:
wherein n is the frame number, x is the horizontal coordinate of the current error block, y is the vertical coordinate of the current error block, m is the size of the current error block, i, j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the hiding processing module 14 is configured to perform spatial interpolation error hiding, where:
calculating a predicted pixel P' (i, j) of the current error block according to the spatial left adjacent block of the current error block;
wherein, i, j is the pixel coordinate of the current error block; p' (0,0) is the top left pixel value of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the concealment processing module 14 is further configured to perform time domain error concealment, where:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the first and second electrodes are formed of a metal,
the concealment processing module 14 is further configured to perform intra-prediction error concealment on the current erroneous block, including:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
Preferably, the apparatus further comprises:
a dividing module 16, configured to divide the current erroneous CTU into 4 error blocks when any one of an upper spatial neighboring CTU, a lower spatial neighboring CTU, a left spatial neighboring CTU, a right spatial neighboring CTU, and a time domain neighboring CTU of the current erroneous CTU is divided; and the method is also used for taking the whole current error CTU as 1 error block when the adjacent CTU in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTU in the time domain of the current error CTU are not divided.
It is emphasized that, those skilled in the art will appreciate that the policies and steps embodied in the present invention can be implemented in a general purpose computing device, they can be centralized on a single computing device or distributed across a network of multiple computing devices, and they can optionally be implemented in program code that is executable by a computing device, such that it can be stored in a memory device and executed by a computing device, or they can be separately fabricated into various integrated circuit modules, or multiple modules or steps thereof can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by instructing the relevant hardware through a program, and the program may be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, and the like. Alternatively, all or part of the steps of the foregoing embodiments may also be implemented by using one or more integrated circuits, and accordingly, each module/module in the foregoing embodiments may be implemented in the form of hardware, and may also be implemented in the form of a software functional module. The present application is not limited to any specific form of hardware or software combination.
Claims (12)
1. A method of error concealment, the method comprising:
sequentially taking each error block in the current error coding tree unit CTU as a current error block;
determining an airspace left adjacent block and an airspace adjacent block of the current error block, wherein the airspace left adjacent block and the airspace adjacent block are adjacent to the error block in an airspace;
performing temporal and spatial correlation comparisons when the spatial left neighboring block and the spatial neighboring block of the current erroneous block are both available and are not intra prediction blocks;
if the spatial correlation of the current error block is larger than a preset intensity threshold value, performing spatial interpolation error concealment; if the spatial correlation of the current error block is not greater than a preset intensity threshold, performing time domain error concealment;
the performing the time-domain and spatial-domain correlation comparison comprises:
when SADSIs less than (SAD)T1+SADT2) When the error block is in a/3 state, the spatial correlation of the current error block is strong;
when SADSGreater than or equal to (SAD)T1+SADT2) When the current error block is in a time domain, the time domain correlation of the current error block is strong;
wherein:
wherein n is the frame number, m is the size of the current error block, i and j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
2. The method of claim 1, wherein the method further comprises:
and when the spatial left adjacent block and the spatial adjacent block are both available and are intra-frame prediction blocks, performing intra-frame prediction error concealment on the current error block.
3. The method of claim 1, wherein:
the performing spatial interpolation error concealment comprises:
calculating a predicted pixel P' (i, j) of the current error block according to the spatial left adjacent block of the current error block;
wherein, i, j is the pixel coordinate of the current error block; p' (0,0) is the top left pixel value of the current error block.
4. The method of claim 1, wherein:
the performing time-domain error concealment comprises:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
5. The method of claim 2, wherein:
the intra prediction error concealment of the current erroneous block comprises:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
6. The method of any of claims 1 to 5, wherein:
before sequentially using each error block in the current error coding tree unit CTU as the current error block, the method further includes:
when any one of an upper spatial adjacent CTU, a lower spatial adjacent CTU, a left spatial adjacent CTU, a right spatial adjacent CTU and a time domain adjacent CTU of the current error CTU is divided, dividing the current error CTU into 4 error blocks;
and when the adjacent CTUs in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTUs in the time domain of the current error CTU are not divided, taking the whole current error CTU as 1 error block.
7. An apparatus for error concealment, the apparatus comprising:
the configuration module is used for taking each error block in the current error coding tree unit CTU as a current error block in sequence;
an information obtaining module, configured to determine a spatial left neighboring block and a spatial neighboring block of the current error block, where the spatial left neighboring block and the spatial neighboring block are adjacent to the error block in a spatial domain;
a determining module, configured to perform a time domain and spatial domain correlation comparison when the left spatial neighboring block and the upper spatial neighboring block of the current erroneous block are both available and are not intra-frame prediction blocks;
a concealment processing module, configured to perform spatial interpolation error concealment if the spatial correlation of the current error block is greater than a preset intensity threshold; the method is further used for performing time domain error concealment if the spatial correlation of the current error block is not greater than a preset intensity threshold;
the determining module is configured to perform temporal and spatial correlation when the spatial left neighboring block and the spatial neighboring block of the current erroneous block are both available and are not intra-prediction blocks, where:
when SADS<(SADT1+SADT2) When the error block is in a/3 state, the spatial correlation of the current error block is strong; wherein:
wherein n is the frame number, m is the size of the current error block, i and j are the pixel coordinates of the current error block, n is the frame number, and P (n,0,0) is the upper left pixel value of the current error block.
8. The apparatus of claim 7, wherein:
the concealment processing module is further configured to perform intra prediction error concealment on the current erroneous block when the spatial left neighboring block and the spatial left neighboring block are both available and are intra prediction blocks.
9. The apparatus of claim 7, wherein:
the hiding processing module is used for carrying out spatial interpolation error hiding, and the hiding processing module is used for:
calculating a predicted pixel P' (i, j) of the current error block according to the spatial left adjacent block of the current error block;
wherein, i, j is the pixel coordinate of the current error block; p' (0,0) is the top left pixel value of the current error block.
10. The apparatus of claim 7, wherein:
the hiding processing module is further configured to perform time domain error hiding, where:
obtaining each motion vector MV in a space domain left adjacent block, a space domain upper adjacent block, a space domain left lower adjacent block and a space domain right upper adjacent block of the current error block;
when the difference value of any two MV identical components in each MV is greater than a preset threshold value, dividing the current error block into 4 error sub-blocks; otherwise, taking the whole current error block as 1 error sub-block;
for each erroneous sub-block, the following operations are performed in turn:
using the available MVs and zero motion MVs in the spatial domain left adjacent subblock, the spatial domain upper adjacent subblock, the spatial domain right adjacent subblock, the spatial domain lower adjacent subblock and the time domain adjacent subblock of the error subblock as MVs of the error subblock, and obtaining each candidate recovery block of the error subblock according to each MV of the error subblock;
and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
11. The apparatus of claim 8, wherein:
the concealment processing module is further configured to perform intra prediction error concealment on the current erroneous block, including:
respectively taking the prediction modes of the spatial left adjacent block and the spatial upper adjacent block as the candidate prediction modes of the current error block;
obtaining each candidate recovery block of the current error block according to each candidate prediction mode; and taking the candidate recovery block with the minimum boundary matching difference in the candidate recovery blocks as the recovery pixel block of the current error block.
12. The apparatus of any of claims 7 to 11, further comprising:
the dividing module is used for dividing the current error CTU into 4 error blocks when any one of an upper spatial adjacent CTU, a lower spatial adjacent CTU, a left spatial adjacent CTU, a right spatial adjacent CTU and a time domain adjacent CTU of the current error CTU is divided; and the method is also used for taking the whole current error CTU as 1 error block when the adjacent CTU in the space domain, the left adjacent CTU in the space domain, the right adjacent CTU in the space domain and the adjacent CTU in the time domain of the current error CTU are not divided.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510086369.3A CN105992012B (en) | 2015-02-17 | 2015-02-17 | Error concealment method and device |
| PCT/CN2015/090813 WO2016131270A1 (en) | 2015-02-17 | 2015-09-25 | Error concealment method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510086369.3A CN105992012B (en) | 2015-02-17 | 2015-02-17 | Error concealment method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105992012A CN105992012A (en) | 2016-10-05 |
| CN105992012B true CN105992012B (en) | 2020-09-15 |
Family
ID=56691977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510086369.3A Active CN105992012B (en) | 2015-02-17 | 2015-02-17 | Error concealment method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105992012B (en) |
| WO (1) | WO2016131270A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107277549B (en) * | 2017-06-07 | 2020-05-12 | 南京邮电大学 | HEVC intra-frame coding frame error concealment method based on texture angle prediction mode |
| CN107241609A (en) * | 2017-07-21 | 2017-10-10 | 东华大学 | The spatial domain error concealing method that filling is shunk with non-iterative is counted based on Gradient Features |
| CN111770346B (en) * | 2020-06-10 | 2022-09-30 | 杭州电子科技大学 | Three-dimensional video error concealment method based on significance |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1463154A (en) * | 2002-05-28 | 2003-12-24 | 华为技术有限公司 | Method for shielding errors happening in video signal |
| CN1964497A (en) * | 2005-11-09 | 2007-05-16 | 上海乐金广电电子有限公司 | A method for error concealment in the case of motive video decoding |
| CN101163250A (en) * | 2006-10-09 | 2008-04-16 | 北京航空航天大学 | Boundary gradient based video stream fault tolerance method |
| CN101304538A (en) * | 2008-06-19 | 2008-11-12 | 宁波中科集成电路设计中心有限公司 | Method for detecting and shielding error of AVS video decode |
| CN102307304A (en) * | 2011-09-16 | 2012-01-04 | 北京航空航天大学 | Image segmentation based error concealment method for entire right frame loss in stereoscopic video |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7738561B2 (en) * | 2004-11-16 | 2010-06-15 | Industrial Technology Research Institute | MPEG-4 streaming system with adaptive error concealment |
| CN102111622B (en) * | 2009-12-23 | 2012-12-19 | 中兴通讯股份有限公司 | Method and device for concealing time domain errors of video images |
| CN102104782A (en) * | 2011-03-30 | 2011-06-22 | 武汉大学 | Decoding end error concealment method suitable for AVS-P2 (Audio Video Standard-P2) video encoding standard |
| CN103313064B (en) * | 2013-06-07 | 2016-04-13 | 国家广播电影电视总局广播科学研究院 | Based on the time domain error hidden method of inter-frame mode and movement repair |
-
2015
- 2015-02-17 CN CN201510086369.3A patent/CN105992012B/en active Active
- 2015-09-25 WO PCT/CN2015/090813 patent/WO2016131270A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1463154A (en) * | 2002-05-28 | 2003-12-24 | 华为技术有限公司 | Method for shielding errors happening in video signal |
| CN1964497A (en) * | 2005-11-09 | 2007-05-16 | 上海乐金广电电子有限公司 | A method for error concealment in the case of motive video decoding |
| CN101163250A (en) * | 2006-10-09 | 2008-04-16 | 北京航空航天大学 | Boundary gradient based video stream fault tolerance method |
| CN101304538A (en) * | 2008-06-19 | 2008-11-12 | 宁波中科集成电路设计中心有限公司 | Method for detecting and shielding error of AVS video decode |
| CN102307304A (en) * | 2011-09-16 | 2012-01-04 | 北京航空航天大学 | Image segmentation based error concealment method for entire right frame loss in stereoscopic video |
Non-Patent Citations (1)
| Title |
|---|
| 视频传输的错误隐藏技术综述;张义云 等;《计算机应用研究》;20150228;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105992012A (en) | 2016-10-05 |
| WO2016131270A1 (en) | 2016-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5254997B2 (en) | Motion estimation for discovered frame regions | |
| US9060177B2 (en) | System and method to process motion vectors of video data | |
| WO2019229705A1 (en) | Weighted interweaved prediction | |
| CN107046645B9 (en) | Image coding and decoding method and device | |
| US9307241B2 (en) | Video encoding method and a video encoding apparatus using the same | |
| US20140241436A1 (en) | Method and device for determining parameters for encoding or decoding of an image of a video sequence | |
| CN106375764B (en) | A kind of bonding position prediction and block replicate the video intra-frame encoding method of prediction | |
| US11363276B2 (en) | Intra-frame prediction method and apparatus, video coding device, and storage medium | |
| WO2012095466A1 (en) | Video encoding and decoding with improved error resilience | |
| US20210211647A1 (en) | Utilization of non-sub block spatial-temporal motion vector prediction in inter mode | |
| CN112385211A (en) | Motion compensation for video encoding and decoding | |
| US20100002771A1 (en) | Seamless Wireless Video Transmission For Multimedia Applications | |
| US20240031576A1 (en) | Method and apparatus for video predictive coding | |
| CN105992012B (en) | Error concealment method and device | |
| US9432694B2 (en) | Signal shaping techniques for video data that is susceptible to banding artifacts | |
| KR100846780B1 (en) | Motion vector derivation method and apparatus | |
| WO2020049447A1 (en) | Fast encoding methods for interweaved prediction | |
| US20200351494A1 (en) | Restricted overlapped block motion compensation | |
| US11778224B1 (en) | Video pre-processing using encoder-aware motion compensated residual reduction | |
| CN112565747B (en) | Decoding and encoding method, device and equipment | |
| JP2005184241A (en) | System for determining moving picture interframe mode | |
| KR101378434B1 (en) | Method for fast intra prediction mode decision | |
| Pradhan et al. | Survey on block matching algorithms for motion estimation | |
| WO2019229682A1 (en) | Application of interweaved prediction | |
| WO2020049446A1 (en) | Partial interweaved prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |