CN1568015A - Multiple visual-angle video coding-decoding prediction compensation method and apparatus - Google Patents
Multiple visual-angle video coding-decoding prediction compensation method and apparatus Download PDFInfo
- Publication number
- CN1568015A CN1568015A CN 03148031 CN03148031A CN1568015A CN 1568015 A CN1568015 A CN 1568015A CN 03148031 CN03148031 CN 03148031 CN 03148031 A CN03148031 A CN 03148031A CN 1568015 A CN1568015 A CN 1568015A
- Authority
- CN
- China
- Prior art keywords
- motion vector
- macro block
- global motion
- target
- obtains
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a prediction compensation method of video frequency coding and decoding of multiple visual angles. The coding includes the following procedures: between 2 views, use one as a reference view, and use the other one as a target view, afterward, then gain the prediction parameters of the global motion through making a global motion prediction on the reference view after transformation; make a prediction compensation on every macro global motion and motion vector; obtain the macro residual images on the basis of the decided prediction compensation; make the transformation, quantization, entropy coding on residual images to output macro codes. The decoding includes the following procedures: first, gain the global motion prediction parameters of the actual pins to the reference view in the target view, and enter the decoding process of every macro; do the entropy decoding, inverse quantization, inverse transformation to obtain the information of the mode of the prediction compensation used by the macro and the residual images. The method can help make full use of the relation between every view in the multiple visual angles, and make the coding and decoding on the multiple visual angles.
Description
Technical field
The present invention relates to digital image processing techniques, particularly relate to the encoding and decoding technique of digital picture, specifically, relate to a kind of method of multi-angle video encoding and decoding.
Background technology
Along with the fast development of network and multimedia technology, increasing image and video occur and transmission with the form of Digital Media, and video coding and decoding technology is the key that realizes digital media storage and transmission efficiently.At present, a kind of new Digital Media form, promptly 3 D video extensively adopts in multimedia application system.So-called 3 D video is meant the time upward synchronously, the set of relevant some video sequences on the space.Compare with traditional two-dimensional video, 3 D video can provide abundanter visual information, therefore can enjoy for the user provides higher-quality subjective vision, and can be applied to video conference, during digital entertainment etc. are used.Multi-angle video (Multiview Video) is a kind of common 3 D video.So-called multi-angle video is meant by the one group video sequence of a plurality of video cameras in different visual angle synchronous acquisition.By certain synthetic technology, multi-angle video can have the scene that three-dimensional visual sense is felt for the user provides.In addition, a key feature of multi-angle video is the interactivity in the scene, and the user can also select his viewpoint in scene.
With respect to traditional two-dimensional video, the data volume of multi-angle video is bigger, a cost that realizes its function be the expression scene video data along with the increase of viewpoint (being number of cameras) with how much speed increments, therefore, multi-angle video processing method of data is efficiently just seemed extremely important.For storage and the transmission that realizes multi-angle video, just need carry out encoding and decoding efficiently to multi-angle video.At present, advanced video coding and decoding technology exists with the form of standard usually, (the Moving Picture Expert Group of Motion Picture Experts Group that typical video compression standard has International Organization for Standardization to divide into, abbreviation MPEG) the MPEG series international standard MPEG-1 that releases, MPEG-2, MPEG-4 etc., and the H.26x series video that International Telecommunication Union proposes compresses recommendation etc.At present, MPEG organizes the standardization effort that carries out three-dimensional audio frequency and video (3DAV), but does not also form concrete standard at present.
A method the simplest of the multi-angle video data being carried out encoding and decoding is to utilize existing encoding and decoding standard, MPEG-1 for example, MPEG-2, MPEG-4, JVT or H.26x wait regards the video data at each visual angle as the common video sequence and carries out encoding and decoding respectively.But this method is not utilized the correlation between each visual angle, and code efficiency is not very high.Though some technology in the mpeg standard can be used for utilizing to a certain extent the correlation between each visual angle to carry out encoding and decoding, but the original intention that these technology propose not is specially at the multi-angle video encoding and decoding, do not take into full account the characteristic of multi-angle video, so code efficiency does not reach optimum.Also there are some coding/decoding systems to utilize the spatial coherence information of multi-angle video between the image of synchronization different visual angles, but, owing to they mainly are that coding at haplopia (2D) video proposes, they do not consider each special relationship between looking of multi-angle video, for example, the scene content major part of correspondence image is all identical between adjacent the looking, and the movement tendency of scene is identical, only angle changes to some extent, this can see (except the example of rapid movement) in many examples, a similitude of looking middle front and back interframe is than the similitude height between different points of view, so, motion compensation interframe prediction substitutes probably looks interchannel potential difference compensation (each look between motion prediction compensation) prediction, only local each is looked and preponderates than encoding respectively at some and look interchannel potential difference compensation, and its code efficiency is not very good.
Summary of the invention
The technical problem to be solved in the present invention is to propose a kind of novel multi-angle video encoding and decoding predictive compensation method and device based on global motion predictive compensation and motion vector prediction compensation, to improve the code efficiency of multi-angle video encoding and decoding.
The method of multi-angle video encoding and decoding predictive compensation of the present invention, two use between looking the global motion predictive compensation extract each look between the spatial coherence of correspondence image, and then raising code efficiency, then on the basis of realizing the global motion predictive compensation, predict that with the motion vector between the front and back frame of looking (with reference to looking) another looks the motion vector between the frame before and after (target is looked).Specifically:
When coding, use one of them to regard as reference frame, another adjacent target of regarding as is looked, the target view picture looks like to carry out the global motion predictive compensation with reference to the reference-view of distortion after (translation, rotation, convergent-divergent etc.), obtain the global motion Prediction Parameters, and then utilize the global motion prediction cost (generally use original macro and the absolute difference that the macro block behind the coding is decoded between each pixel of reconstruct macro block that obtains is weighed) of these each macro blocks of calculation of parameter, obtain the global motion vector of this macro block.Motion vector before and after global motion vector that motion vector prediction compensation utilization obtains and the reference of having encoded are looked between the frame, prediction obtain target look before and after motion vector between the frame, and calculate the motion prediction cost of this method.According to the motion prediction cost, the coded prediction compensation method of selecting current macro to adopt, then according to selected predictive compensation method or carry out the global motion compensation (if global motion predictive compensation method is selected) of macro block or carry out macro block local motion compensated (if the motion vector prediction compensation method is selected), and then obtain the macro block residual image.At last, residual image is carried out conversion, quantification, entropy coding, final output encoder code stream.
During decoding, at first decoding obtains target and looks middle present frame to reference to the global motion Prediction Parameters of looking, and utilizes these calculation of parameter to obtain the global motion vector of current decoded macroblock then, and preserves, for future use.Decoding obtains predictive compensation method and the residual image that macro block uses again.According to different predictive compensation methods,, and merge with the residual image of decoding and to form final macro block decoded picture according to the image of this macro block of motion compensation process reconstruct in when coding; Use said method to decode to each macro block, can obtain the decoded picture of this frame.
The invention allows for a kind of multi-view video coding predictive compensation device, comprise reference-view as coding unit and target view as coding unit, reference-view is encoded through the local motion predictive compensation as coding unit, and code stream is looked in the reference behind the final output encoder; Target view is encoded through global motion predictive compensation and motion vector prediction compensation as coding unit, and the target behind the last output encoder is looked code stream.
Utilization of the present invention is two relevant global motion predictive compensation and motion vector prediction compensation methodes of looking to adopt between the sequence, than existing coding method, can more make full use of the correlation between each is looked in the multi-angle video, therefore can effectively carry out encoding and decoding multi-angle video.
The global motion predictive compensation acts on synchronization by on the multiple image of a plurality of camera acquisitions, its objective is the spatial coherence that extracts these images, characteristic between having taken into full account that multi-angle video is adjacent and looking, particularly the scene content major part is identical, movement tendency is identical, different places be or the angle difference, perhaps there are characteristics such as relative displacement between the whole scene, these characteristics make the global motion predictive compensation can obtain motion prediction more accurately than direct local motion predictive compensation, and it does not need encoding motion vector, reduce the bit of encoding motion vector information, and then improve code efficiency.
The motion vector prediction compensation way is to carry out on the basis of realizing global motion predictive compensation mode, its objective is the movable information of the sequence of certain camera acquisition that utilization is encoded, predict the movable information in the sequence of other camera acquisition, this prediction can adopt identical method to realize in encoder respectively, therefore do not need encoding motion vector, reduce the bit of encoding motion vector information, and then improve code efficiency.
Description of drawings
Fig. 1 is that motion vector concerns schematic diagram;
Fig. 2 is the motion vector central point prediction flow chart that the present invention proposes;
Fig. 3 obtains with reference to the flow chart of looking the global motion vector that target looks by iteration coupling;
Fig. 4 is one of a multi-view video coding predictive compensation device schematic diagram;
Fig. 5 is two schematic diagrames of multi-view video coding predictive compensation device;
Fig. 6 is based on the prediction principle schematic diagram of MPEG-4 time scalable encoding method.
Embodiment
In the multi-angle video sequence, the corresponding frame of two video sequences at adjacent visual angle exists sizable correlation, the corresponding frame scene content major part of two video sequences that is adjacent visual angle is identical, movement tendency is identical, different places be the angle difference, perhaps have relative displacement between the whole scene, the motion vector between this also implicit one front and back frame of looking (target is looked) can look the motion vector prediction between the frame obtains before and after (with reference to looking) by another.They are through after simply being out of shape (translation, rotation, affine transformation etc.), and most of image is closely similar.If can utilize these characteristics, between two corresponding frames, adopt global motion predictive compensation mode, and between the front and back frame that target is looked, adopt the motion vector prediction compensation way, just can improve the code efficiency of multi-angle video.Comprise the steps:
Coding step is as follows:
At first use one of them to regard as with reference to looking, another adjacent looking (looking hereinafter referred to as target) to carrying out the global motion prediction with reference to looking, obtains the global motion Prediction Parameters through the distortion back.Enter the cataloged procedure of each macro block global motion predictive compensation and motion vector prediction compensation then, its step is as follows:
The global motion prediction cost of step 1.1, the global motion Prediction Parameters computing macro block that utilize to obtain (generally use original macro with the macro block behind the coding is decoded between each pixel of reconstruct macro block that obtains absolute difference with weigh);
Step 1.2, calculating obtain the global motion vector of this macro block, with its preservation, have been equipped with the back and have used;
Step 2, motion vector prediction compensation coding process:
Motion vector prediction acquisition target before and after the global motion vector of this macro block that step 2.1, utilization obtain and the global motion vector that former frame has been preserved and the reference of having encoded are looked between the frame is looked the motion vector between the frame of front and back;
The local motion prediction cost of the motion vector calculation macro block that step 2.2, utilization prediction obtain;
Step 3, the global motion of selecting to calculate acquisition are predicted the predictive compensation of predictive compensation less in cost and the local motion prediction cost as macro block;
Step 4, according to selected predictive compensation or carry out the global motion compensation (if global motion predictive compensation method is selected) of macro block or carry out macro block local motion compensated (if the motion vector prediction compensation method is selected), and then do difference with original macro and obtain the macro block residual image;
Step 5, residual image is carried out conversion, quantification, entropy coding, final output macro block encoding code stream.
Decoding step is as follows:
At first decoding obtains target and looks middle present frame to reference to the global motion Prediction Parameters of looking, and enters each macro-block decoding process then, and its step is as follows:
The global motion Prediction Parameters that step 2, utilization obtain is calculated the global motion vector that obtains current decoded macroblock, and preserves, for future use;
Step 3, if macro block adopts is the global motion predictive compensation, then decode as follows:
Step 3.1, will be out of shape according to the global motion Prediction Parameters that obtains with reference to corresponding reference frame in looking;
The global motion vector that step 3.2, utilization obtain is carried out global motion compensation with reference to the reference frame after being out of shape to current decoded macroblock, obtains the macro block compensating images;
The residual image merging of step 3.3, macro block compensating images and decoding obtains the decoded macroblock image.
Step 4, if macro block adopts is the motion prediction compensation, then decode as follows:
The motion vector of interframe before and after decoded global motion vector that step 4.1, utilization have been preserved and decoded reference are looked calculates and obtains the motion vector that target is looked front and back interframe;
The last decoded frame that step 4.2, the motion vector references target of utilizing calculating to obtain are looked carries out motion compensation to current decoded macroblock, obtains the macro block compensating images;
The residual image merging of step 4.3, macro block compensating images and decoding obtains the decoded macroblock image.
Fig. 1 is that motion vector concerns schematic diagram, and target is looked and is that in the multi-angle video sequence two look with reference to looking among Fig. 1, and one is used as with reference to looking, and another is looked as target.Its expression be a certain coding constantly sequence with reference to look and target look before and after relation between the frame (the i-1 frame is represented former frame, the i frame is represented present frame), described with reference to look and target between looking global motion vector (in Fig. 1, using GMC0 and GMC1 to indicate) and the relation between the motion vector (use MV0 and MV1 indicate in Fig. 1) in same the looking, wherein:
Front and back two two field pictures that the top a certain moment target of two grid representations is looked, the a certain moment of two following grid representations is with reference to front and back two two field pictures of looking, each blockage in the grid is represented a macro block (image pixel piece), and the grey square in the target view lattice is represented current coding macro block.
GMC0 represents that the previous target frame of having encoded is to previous global motion vector between the reference coding frame;
GMC1 represents that the present encoding target frame is to the corresponding global motion vector between the reference coding frame;
MV0 represents the motion vector with reference to interframe before and after looking;
MV1 represent target look before and after the motion vector of interframe, it also is to need to carry out the motion vector that motion vector prediction obtains among the present invention.
Fig. 2 is the motion vector central point prediction flow chart that the present invention proposes.What it was described is that target is looked sequence by with reference to the front and back frame motion vector and the acquired global motion vector of looking, and comes target of prediction to look the way of a kind of predictive coding of corresponding front and back frame motion vector.
Specific practice is, certain macro block of looking when the coding target is finished following steps when (representing with the grey square) in Fig. 1:
The central point of step 2, calculating acquisition current macro (CurXcent, CurYcent), (representing with CurCentPos among Fig. 1, i.e. pore in the grey square);
Step 3, by GMC1 motion vector and position (CurXcent, CurYcent) calculate obtain point (CurXcent, CurYcent) position in the reference-view picture (CurRefX, CurRefY), (representing with CurPosRef among Fig. 1);
Step 4, utilize MV0, calculate obtain point (CurRefX, CurRefY) in its former frame reference-view picture corresponding position (PreRefX, PreRefY), (representing with PrePosRef among Fig. 1);
Step 5, utilize GMC0, by the iteration coupling obtain with reference to look the global motion vector that target looks (GMVX, GMVY), (GMC0 that is about among Fig. 1 is reverse), detailed process will be described in Fig. 3;
Step 6, with the global motion vector (GMVX that obtains, GMVY) reverse, and then utilize its calculating location (PreRefX, PreRefY) correspondence position (PreXcent in the former frame image that target is looked, PreYcent), (represent with PrePosCur among Fig. 1);
Step 7, calculating obtain motion vector MV1, MV1X=PreXcent-CurXcent, MV1Y=PreXcent-CurYcent; (vector that thick dashed line is represented among Fig. 1).
It is worthy of note,, except the central point prediction mode that the present invention proposes, also can adopt the multiple spot prediction, ask the mode of its mean value then, but this still belongs to processing thinking of the present invention if in order to pursue higher precision of prediction.
Fig. 3 is that iteration coupling is obtained with reference to the method for looking the global motion vector that target looks.It has described the realization of overstriking functional block among Fig. 2.Its input be through calculating with reference to the correspondence position point in looking (PreRefX, PreRefY), target looks with reference to the global motion vector GMC0 that looks, final output obtain corresponding reference look the global motion vector that target looks (GMVX, GMVY).Wherein,
MBSIZE represents the size of macro block, can get different values according to different environment;
The maximum iterations of N control can be provided with according to actual needs.It is used for guaranteeing that in the time can not obtaining to mate fully by iteration, the control iteration finishes.The scope of its size control iterative search;
During the Minerr initialization, be MAXERROR, MAXERROR can get any max-thresholds according to actual conditions.Minimum iteration error in the Minerr record iterative process, the global motion vector that finally adopts the minimum target of Minerr (iteration error) to look macro block represent with reference to look the global motion vector that target looks (GMVX, GMVY).
Particularly, the Minerr initialization makes iteration error Minerr=MAXERROR;
Calculate to obtain point (PreRefX, PreRefY) macro block position at place (PreMBX, PreMBY), by GMC0 and macro block position (PreMBX, PreMBY) obtain global motion vector (GMVX, GMVY);
With point (PreRefX, PreRefY) deduct global motion vector (GMVX, GMVY), obtain new position (PreCurX, PreCurY); Calculate to obtain again point (PreCurX, PreCurY) macro block position at place (PreCurMBX, PreCurMBY).
Iterative process is such: with the macro block position (PreCurMBX that obtains after the iteration, PreCurMBY) with the preceding macro block position (PreMBX of iteration, PreMBY) relatively, if equate, then expression obtains fully, iteration error Minerr is 0, directly withdraw from iterative process, if it is unequal, if the left side of the macro block position before iteration, then obtain the absolute difference Xerr of the directions X position of the correspondence position point after directions X position, the macro block left side and the iteration, otherwise the absolute difference Xerr of the directions X position of the correspondence position point after acquisition macro block directions X position, the right and the iteration, similarly, if up, then obtain the absolute difference Yerr of the Y direction position of the correspondence position point after Y direction position, macro block top and the iteration, otherwise obtain the absolute difference Yerr of the Y direction position of the correspondence position point after macro block bottom Y direction position and the iteration, last Xerr and Yerr's and with Minerr relatively, get its minimum value and give Minerr.
Fig. 4 is as specific embodiments of the invention, expression be multi-view video coding predictive compensation device 1.The input of this device is any two original video stream of looking (the adjacent effect of looking better) in the multi-angle video, and one of them is looked and is called with reference to looking, and another is looked and is called target and looks.The reference-view picture is encoded through local motion predictive compensation method, and code stream is looked in the reference behind the final output encoder; The target view picture is encoded through global motion predictive compensation and motion vector prediction compensation, and the target behind the last output encoder is looked code stream.Wherein,
Reference-view picture and target view picture are respectively that raw video image is looked in the reference of input and target is looked raw video image;
Store the reference-view picture and the target view picture of the back reconstruct of former frame coding respectively as buffering area as buffering area and target apparent weight composition with reference to the apparent weight composition.
Its job order is at first a frame reference-view to be looked like to encode, and then a frame target view is looked like to encode.Be respectively the process of reference-view picture coding and target view picture coding below:
1, reference-view is as coding unit
It realizes the predictive compensation coding of a frame reference-view picture.It adopts local motion to estimate, and obtains motion vector thus, and it is carried out entropy coding.In the motion vector buffering area, preserve this motion vector in order to the coding target view as the time use.Motion vector and the cooperation of former frame reconstructed reference view picture are finished with reference to apparent motion compensation, obtain the reference picture after the compensation, promptly among Fig. 4 with reference to looking predicted picture, this image is tried to achieve residual image looking like to do poor with the original reference view.Residual image is through conversion, quantification and entropy coding, and final formation is with reference to looking code stream.Pass through inverse quantization and inverse transformation again through the residual image after quantizing, and look predicted picture with reference and add up, and then obtain, and put it into, use during in order to next frame reference-view picture and target view picture coding with reference to apparent weight structure buffering area with reference to apparent weight composition picture.
2, target view is as coding unit
It realizes the predictive compensation coding of a frame target view picture.
The input target look original image and with reference to apparent weight composition picture between carry out overall motion estimation, obtain the global motion parameter, and obtain global motion prediction cost and global motion vector thus, and global motion vector is kept in the global motion vector buffering area, for future use;
The motion vector of preserving when the global motion vector that obtains when utilizing former frame target view picture coding and the global motion vector of current acquisition and corresponding reference view picture coding carries out target apparent motion vector forecasting, obtains the local motion vector that the target view picture is looked with reference to former frame reconstruct target;
Calculate the local motion prediction cost of target view picture, and select it and global motion to predict that less predictive compensation method is carried out following work in the cost by selection control;
According to selected predictive compensation method or carry out global motion compensation (if global motion predictive compensation method is selected) or carry out local motion compensated (if the motion vector prediction compensation method is selected), target image after obtaining to compensate, be that target is looked predicted picture among Fig. 4, this image is tried to achieve residual image looking like to do poor with the original object view;
Residual image finally forms target and looks code stream through conversion, quantification and entropy coding.Pass through inverse quantization and inverse transformation again through the residual image after quantizing, and look predicted picture with target and add up, and then obtain target apparent weight composition picture, and put it into target apparent weight structure buffering area, use during in order to next frame target view picture coding.
Fig. 5 is an alternative embodiment of the invention, expression be multi-view video coding predictive compensation device 2.This device 2 and device 1 difference be the coding target view as the time, global motion predictive compensation and motion vector prediction compensation are finished coding work jointly as two kinds of predictive modes and other predictive mode.Promptly, the target view picture is except using the compensation of global motion predictive compensation and motion vector prediction encodes, can also introduce current other predictive compensation method that has been used (for example, direct predictive compensation method), they help the raising of multi-view video coding efficient.The input of this device 2 is any two original video stream of looking (the adjacent effect of looking better) in the multi-angle video, and one of them is looked and is called with reference to looking, and another is looked and is called target and looks.
The first half of Fig. 5 (part that top rectangle frame comprises) is the cataloged procedure of reference-view picture, and the latter half (part that following rectangle frame comprises) is the cataloged procedure of target view picture; What comprise in the empty bold box is global motion predictive compensation module and motion vector prediction module, and they join in the whole device as two kinds of patterns;
Reference-view picture and target view picture are respectively that raw video image is looked in the reference of input and target is looked raw video image;
Reference frame buffering area and target frame buffering area are stored the reference-view picture and the target view picture of former frame coding back reconstruct respectively.
Its job order is at first a frame reference-view to be looked like to encode, and then a frame target view is looked like to encode.Be respectively the process of reference-view picture coding and target view picture coding below:
1, reference-view is as coding unit
Identical in 1 of this process and device is described in detail referring to reference-view among Fig. 4 as coding key.
2, target view is as coding unit
It realizes the predictive compensation coding of a frame target view picture.
Look original image and carry out overall motion estimation in the target of input, obtain the global motion parameter, and obtain global motion prediction cost and global motion vector thus, and global motion vector is preserved, for future use with reference between looking.In the overall motion estimation module, finish;
The motion vector of preserving when the global motion vector that obtains when utilizing former frame target view picture coding and the global motion vector of current acquisition and corresponding reference view picture coding carries out target apparent motion vector forecasting, obtain the local motion vector that the target view picture is looked with reference to former frame reconstruct target, and calculate the local motion prediction cost of target view picture.In the motion vector prediction module, finish;
Carry out other predictive compensation method, and obtain motion vector thus, it is carried out entropy coding.Calculate the prediction cost of this kind predictive compensation method.In the target frame motion estimation module, finish;
Principle according to prediction cost minimum is selected suitable predictive compensation method.According to selected predictive compensation method or carry out global motion compensation (if global motion predictive compensation method is selected, in the global motion compensation module, finish), perhaps carry out local motion compensated (if the motion vector prediction compensation method is selected, in the target frame motion compensating module, finish), perhaps carry out the compensation (in the target frame motion compensating module, finishing) of alternate manner, target view picture after obtaining to compensate, it is poor that this image and original object view look like to do, and tries to achieve residual image;
Residual image finally forms target and looks code stream through conversion, quantification and entropy coding.Pass through inverse quantization and inverse transformation again through the residual image after quantizing, and look like to add up, and then obtain target apparent weight composition picture, and put it into the target frame buffering area, use during in order to next frame target view picture coding with target view after the compensation.
Fig. 6 has described the prediction principle based on MPEG-4 time scalable encoding method:
Of the present inventionly adopt overall motion estimation prediction and motion vector prediction method can more make full use of correlation between each is looked in the multi-angle video than existing coding method between the sequence, therefore can effectively carry out encoding and decoding multi-angle video two relevant looking.They can be used in combination with the MPEG-4 coding standard, adopt following steps to realize that (with the stereopsis sequence is example for overall motion estimation prediction and motion vector prediction processing, it has only two to look, and be called a left side and look with the right side and look, be the simplest multi-angle video sequence therefore).
One, coding side is handled:
Among Fig. 6, the left view picture is looked sequence as a reference, the right view picture is as working as the forward sight sequence, GMC0 represents the overall motion estimation prediction between the image of the former frame left and right sides, GMC1 represents the overall motion estimation prediction between the image of the present frame left and right sides, MV0 represents that with reference to the motion prediction vectors of looking between the frame of (being the left view picture) front and back MV1 represents that when the motion prediction vectors between the frame before and after the forward sight (being the right view picture) it need obtain by using GMC0, GMC1 and MV0 carry out motion vector prediction.I represents intraframe coding, and P represents predictive coding, and B represents bi-directional predictive coding.For with the MPEG-4 operating such, the present invention adopts the mode that increases the macroblock prediction pattern to embed overall motion estimation prediction and motion vector prediction.Therefore, the B image increases by two kinds of predictive modes again except original predictive mode, is respectively global motion prediction (using GME to represent) and motion vector prediction (using MVP to represent).Its concrete implementation procedure is as follows:
1. the reference-view picture is encoded: this step is former identical in steps with MPEG-4, and the motion vector MV0 between the frame before and after obtaining thus.
When the front view picture not only can be with reference to the reference-view picture, can also be with reference to former frame image when forward sight, therefore except first frame, use the realization of encoding of B two field picture among the MPEG-4 here.Here local motion estimates that prediction mode is former identical in steps with MPEG-4.
3. when between front view picture and reference-view picture, carry out overall motion estimation and compensation, and obtain the global motion vector of global motion parameter GMC1 and macro block.
4. utilize encoded MV0, GMC0 and GMC1 work as motion vector prediction between the frame of forward sight front and back.Adopt the central point forecast method, its Forecasting Methodology is seen accompanying drawing 2.
5. mode selecting method:
1) selects optimumly between the original pattern of B frame in MPEG-4, adopt the original selection strategy of MPEG-4;
2) percent of pass aberration optimizing (RDO-rate-distortion optimization) policy selection optimal prediction modes in original pattern, GME and the MVP pattern of the B frame optimum of selecting.System of selection is a pattern of selecting MSE+ λ Rate minimum, and wherein, MSE is the mean square deviation of macro block original image and macro block reconstructed image, and Rate is the sign indicating number position behind this macroblock coding.λ is a Lagrangian, and λ=(0.85 * 2QP/3) 1/2, QP is a quantization parameter here.
6. carry out motion compensation.The GME pattern adopts the overall motion estimation compensation, and other pattern all adopts the original compensation method of MPEG-4.
7. form the macro block code stream.Adopt the macro block of GME and MVP pattern to need not encoding motion vector, other the original method of employing MPEG-4.
Two, decoding end is handled:
1. decoding reference-view picture adopts the original coding/decoding method of MPEG-4.
2. decoding is adopted following steps when the front view picture:
1) the overall motion estimation parameter of decoding present frame;
2) if current macro is the original predictive mode of MPEG-4, then according to the original coding/decoding method decoding of MPEG-4, if otherwise GME type, then carry out this macro block of global motion compensation reconstruct, if otherwise MVP type, then predict the acquisition motion vector, utilizing the original compensation method of MPEG-4 to carry out motion compensation then according to the method for coding step 4.
3) decoded residual image, and merge to produce decoded picture with the image of compensation.
Above embodiment is the unrestricted technical scheme of the present invention in order to explanation only, those of ordinary skill in the art is to be understood that: can make amendment or be equal to replacement the present invention, and not breaking away from any modification or partial replacement of the spirit and scope of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.
Claims (14)
1, a kind of method of multi-angle video encoding and decoding predictive compensation is characterized in that, described coding step is as follows:
Two look between, use one of them to regard as with reference to looking, another adjacent target of regarding as is looked, and to carrying out the global motion prediction with reference to looking, obtains the global motion Prediction Parameters through the distortion back; Carry out the compensation of each macro block global motion predictive compensation and motion vector prediction then;
Select to calculate the global motion that obtains and predict less predictive compensation in cost and the local motion prediction cost, as the predictive compensation of macro block;
According to selected predictive compensation, carry out the global motion predictive compensation of macro block, perhaps carry out the local motion compensated of macro block, do difference operation with original macro again, thereby obtain the macro block residual image;
Residual image is carried out conversion, quantification, entropy coding, final output macro block encoding code stream;
Described decoding step is as follows:
At first decoding obtains target and looks middle present frame to reference to the global motion Prediction Parameters of looking, and enters each macro-block decoding process then;
Carry out entropy decoding, inverse quantization and inverse transformation, information such as the predictive compensation mode that the acquisition macro block adopts, residual image;
Utilize the global motion Prediction Parameters that obtains to calculate the global motion vector that obtains current decoded macroblock, and preserve;
Macro block adopts global motion predictive compensation mode, perhaps adopts the motion prediction compensation, and the macro block compensating images of acquisition and the residual image of decoding are merged, and obtains the decoded macroblock image.
2, the method for multi-angle video encoding and decoding predictive compensation according to claim 1 is characterized in that, described global motion predictive compensation cataloged procedure comprises:
Utilize the global motion prediction cost of the global motion Prediction Parameters computing macro block that obtains;
Calculate the global motion vector that obtains this macro block,, be equipped with the back and used its preservation.
3, the method for multi-angle video encoding and decoding predictive compensation according to claim 2, it is characterized in that, described global motion prediction cost adopts original macro and the absolute difference that the macro block after encoding is decoded between each pixel of reconstruct macro block that obtains to weigh.
4, the method for multi-angle video encoding and decoding predictive compensation according to claim 1 is characterized in that, described motion vector prediction compensation coding process comprises:
Motion vector prediction acquisition target before and after the global motion vector of this macro block that utilization obtains and the global motion vector that former frame has been preserved and the reference of having encoded are looked between the frame is looked the motion vector between the frame of front and back;
Utilize the local motion prediction cost of predicting the motion vector calculation macro block that obtains.
5, the method for multi-angle video encoding and decoding predictive compensation according to claim 1 is characterized in that, carries out the global motion predictive compensation of macro block in the described decoding, comprising:
To be out of shape according to the global motion Prediction Parameters that obtains with reference to corresponding reference frame in looking;
Utilize the global motion vector that obtains current decoded macroblock to be carried out global motion compensation, obtain the macro block compensating images with reference to the reference frame after being out of shape.
6, the method for multi-angle video encoding and decoding predictive compensation according to claim 1 is characterized in that, carries out the motion prediction compensation of macro block in the described decoding, comprising:
Utilize the decoded global motion vector and the decoded reference of having preserved to look the motion vector of front and back interframe, calculate and obtain the motion vector that target is looked front and back interframe;
The last decoded frame that the motion vector references target of utilizing calculating to obtain is looked carries out motion compensation to current decoded macroblock, obtains the macro block compensating images.
7, a kind of motion vector central point Forecasting Methodology is characterized in that, comprises the steps:
Step 1, in global motion predictive compensation process, obtain global motion vector GMC0 and GMC1, and, obtain local motion vector MV0 in the coded reference apparent time;
The central point of step 2, calculating acquisition current macro (CurXcent, CurYcent);
Step 3, by GMC1 motion vector and position (CurXcent, CurYcent) calculate obtain point (CurXcent, CurYcent) position in the reference-view picture (CurRefX, CurRefY);
Step 4, utilize MV0, calculate obtain point (CurRefX, CurRefY) in its former frame reference-view picture corresponding position (PreRefX, PreRefY);
Step 5, utilize GMC0, by the iteration coupling obtain with reference to look the global motion vector that target looks (GMVX, GMVY);
Step 6, with the global motion vector that obtains (GMVX, GMVY) reverse, and then utilize its calculating location (PreRefX, PreRefY) correspondence position in the former frame image that target is looked (PreXcent, PreYcent);
Step 7, calculating obtain motion vector MV1, MV1X=PreXcent-CurXcent, MV1Y=PreXcent-CurYcent.
8, motion vector central point Forecasting Methodology according to claim 7 is characterized in that, described step 5 obtains to comprise with reference to looking the global motion vector that target is looked by the iteration coupling:
The Minerr initialization makes iteration error Minerr=MAXERROR;
Calculate to obtain point (PreRefX, PreRefY) macro block position at place (PreMBX, PreMBY), by GMC0 and macro block position (PreMBX, PreMBY) obtain global motion vector (GMVX, GMVY);
Point (PreRefX, PreRefY) deduct global motion vector (GMVX, GMVY), obtain new position (PreCurX, PreCurY); Calculate to obtain point (PreCurX, PreCurY) macro block position at place (PreCurMBX, PreCurMBY);
With the macro block position that obtains after the iteration (PreCurMBX, PreCurMBY) with iteration before macro block position (PreMBX PreMBY) relatively, enters iterative process.
9, motion vector central point Forecasting Methodology according to claim 7 is characterized in that, the macro block position before macro block position that obtains after the described iteration and the iteration relatively, if both equate, then expression obtains fully, and iteration error Minerr is 0, directly withdraws from iterative process.
10, motion vector central point Forecasting Methodology according to claim 7, it is characterized in that, macro block position before macro block position that obtains after the described iteration and the iteration relatively, if both are unequal, if the left side of the macro block position before iteration, then obtain the absolute difference Xerr of the directions X position of the correspondence position point after directions X position, the macro block left side and the iteration, otherwise obtain the absolute difference Xerr of the directions X position of the correspondence position point after macro block directions X position, the right and the iteration; If up, then obtain the absolute difference Yerr of the Y direction position of the correspondence position point after Y direction position, macro block top and the iteration, otherwise obtain the absolute difference Yerr of the Y direction position of the correspondence position point after following Y direction position of macro block and the iteration, last Xerr and Yerr's and with Minerr relatively, get its minimum value and give Minerr.
11, motion vector central point Forecasting Methodology according to claim 7 is characterized in that, also comprises the maximum iterations N of control is set, and in the time can not obtaining to mate fully by iteration, the control iteration finishes.
12, a kind of multi-view video coding predictive compensation device, comprise reference-view as coding unit and target view as coding unit, it is characterized in that:
Described reference-view is handled by the local motion estimation module as coding unit, obtains motion vector, and it is carried out entropy coding; Simultaneously motion vector is finished with reference to the apparent motion compensation with the cooperation of former frame reconstructed reference view picture, the reference picture after obtaining to compensate, and this image is tried to achieve residual image looking like to do poor with the original reference view; Residual image is through conversion, quantification and entropy coding module, and final output is with reference to looking code stream;
Described target view is as in the coding unit, overall motion estimation module receiving target look original image and with reference to apparent weight composition picture as input, obtain the global motion parameter, and obtain global motion prediction cost and global motion vector thus, and global motion vector is kept in the global motion vector buffering area;
The motion vector of preserving when utilization is stored in the global motion vector of global motion vector that former frame target view picture when coding in the buffering area obtain and current acquisition and corresponding reference view picture coding, predict by target apparent motion vector forecasting module, obtain the local motion vector that the target view picture looks with reference to former frame reconstruct target and output to the local motion prediction cost that local motion prediction cost module is calculated the target view picture, and select it and the less predictive compensation mode of global motion prediction cost by selection control;
Target image after obtaining to compensate according to selected predictive compensation mode, this image is tried to achieve residual image looking like to do poor with the original object view;
Residual image is through conversion, quantification and entropy coding resume module, and final export target is looked code stream.
13, according to the multi-view video coding predictive compensation device of claim 12, it is characterized in that: the residual image after handling through quantization modules is input in inverse quantization and the inverse transform block again, with with reference to after looking predicted picture and adding up, the reference apparent weight composition that obtains is looked like to store into reference in the apparent weight structure buffering area.
14, according to the multi-view video coding predictive compensation device of claim 12, it is characterized in that:
Be input to again in inverse quantization and the inverse transform block through the residual image after the quantization modules processing, look predicted picture with target and add up, and the target apparent weight composition that will arrive look like to store into target apparent weight structure buffering area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03148031 CN1258925C (en) | 2003-06-27 | 2003-06-27 | Multiple visual-angle video coding-decoding prediction compensation method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03148031 CN1258925C (en) | 2003-06-27 | 2003-06-27 | Multiple visual-angle video coding-decoding prediction compensation method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1568015A true CN1568015A (en) | 2005-01-19 |
| CN1258925C CN1258925C (en) | 2006-06-07 |
Family
ID=34472161
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 03148031 Expired - Fee Related CN1258925C (en) | 2003-06-27 | 2003-06-27 | Multiple visual-angle video coding-decoding prediction compensation method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1258925C (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101690220A (en) * | 2007-04-25 | 2010-03-31 | Lg电子株式会社 | A method and an apparatus for decoding/encoding a video signal |
| CN101222639B (en) * | 2007-01-09 | 2010-04-21 | 华为技术有限公司 | Inter-view prediction method, encoder and decoder of multi-viewpoint video technology |
| CN101175210B (en) * | 2006-10-30 | 2010-08-11 | 中国科学院计算技术研究所 | Entropy decoding method and device used for decoding video estimation residual error coefficient |
| CN101248669B (en) * | 2005-09-21 | 2011-01-12 | 三星电子株式会社 | Apparatus and method for encoding and decoding multi-view video |
| CN102036078A (en) * | 2011-01-21 | 2011-04-27 | 哈尔滨商业大学 | Motion estimation method based on relevance among viewing angles for multiple visual angle video coding and decoding system |
| CN101411181B (en) * | 2006-04-06 | 2011-07-06 | 高通股份有限公司 | Electronic video image stabilization device and method |
| CN101529918B (en) * | 2006-10-30 | 2011-08-03 | 日本电信电话株式会社 | Predictive reference information generation method, dynamic image encoding and decoding method, device thereof, program thereof, and storage medium containing the program |
| CN102236798A (en) * | 2011-08-01 | 2011-11-09 | 清华大学 | Image matching method and device |
| CN102257817A (en) * | 2008-12-19 | 2011-11-23 | 汤姆森特许公司 | Video coding based on global movement compensation |
| CN101366285B (en) * | 2006-01-09 | 2012-04-18 | 汤姆逊许可证公司 | Methods and apparatuses for multi-view video coding |
| CN101743750B (en) * | 2007-05-14 | 2012-06-27 | 三星电子株式会社 | Method and apparatus for encoding and decoding multi-view image |
| CN102572418A (en) * | 2010-12-28 | 2012-07-11 | 深圳市融创天下科技股份有限公司 | Motion prediction method and motion prediction system of interframe encoding |
| CN101390396B (en) * | 2006-03-09 | 2012-07-18 | 三星电子株式会社 | Method and apparatus for encoding and decoding multi-view video to provide uniform picture quality |
| CN102868879A (en) * | 2011-07-05 | 2013-01-09 | 北京大学 | Method and system for converting video frame rate |
| CN104380744A (en) * | 2012-06-19 | 2015-02-25 | Lg电子株式会社 | Method and device for processing video signal |
| CN104871541A (en) * | 2012-11-07 | 2015-08-26 | Lg电子株式会社 | Method and apparatus for processing video signals |
| CN105075256A (en) * | 2013-03-27 | 2015-11-18 | 国立研究开发法人情报通信研究机构 | Method for encoding a plurality of input images and storage medium and device for storing program |
| CN105103546A (en) * | 2013-03-27 | 2015-11-25 | 国立研究开发法人情报通信研究机构 | Method and device for encoding a plurality of input images, and storage medium for storing program |
| CN109076234A (en) * | 2016-05-24 | 2018-12-21 | 华为技术有限公司 | Image prediction method and relevant device |
| CN110024404A (en) * | 2016-09-30 | 2019-07-16 | 交互数字Vc控股公司 | The method for encoding global rotary motion compensation image, equipment and stream |
| CN113271464A (en) * | 2021-05-11 | 2021-08-17 | 北京奇艺世纪科技有限公司 | Video encoding method, decoding method and related devices |
-
2003
- 2003-06-27 CN CN 03148031 patent/CN1258925C/en not_active Expired - Fee Related
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101248669B (en) * | 2005-09-21 | 2011-01-12 | 三星电子株式会社 | Apparatus and method for encoding and decoding multi-view video |
| US9521429B2 (en) | 2006-01-09 | 2016-12-13 | Thomson Licensing | Methods and apparatus for multi-view video coding |
| US8842729B2 (en) | 2006-01-09 | 2014-09-23 | Thomson Licensing | Methods and apparatuses for multi-view video coding |
| CN102685489B (en) * | 2006-01-09 | 2015-04-29 | 汤姆逊许可证公司 | Methods and apparatus for multi-view video coding |
| US10194171B2 (en) | 2006-01-09 | 2019-01-29 | Thomson Licensing | Methods and apparatuses for multi-view video coding |
| CN102685489A (en) * | 2006-01-09 | 2012-09-19 | 汤姆逊许可证公司 | Methods and apparatus for multi-view video coding |
| US9525888B2 (en) | 2006-01-09 | 2016-12-20 | Thomson Licensing | Methods and apparatus for multi-view video coding |
| US9143782B2 (en) | 2006-01-09 | 2015-09-22 | Thomson Licensing | Methods and apparatus for multi-view video coding |
| CN101366285B (en) * | 2006-01-09 | 2012-04-18 | 汤姆逊许可证公司 | Methods and apparatuses for multi-view video coding |
| US8270482B2 (en) | 2006-03-09 | 2012-09-18 | Samsung Electronics Co., Ltd. | Method and apparatus for encoding and decoding multi-view video to provide uniform picture quality |
| CN101390396B (en) * | 2006-03-09 | 2012-07-18 | 三星电子株式会社 | Method and apparatus for encoding and decoding multi-view video to provide uniform picture quality |
| CN101411181B (en) * | 2006-04-06 | 2011-07-06 | 高通股份有限公司 | Electronic video image stabilization device and method |
| CN101529918B (en) * | 2006-10-30 | 2011-08-03 | 日本电信电话株式会社 | Predictive reference information generation method, dynamic image encoding and decoding method, device thereof, program thereof, and storage medium containing the program |
| CN101175210B (en) * | 2006-10-30 | 2010-08-11 | 中国科学院计算技术研究所 | Entropy decoding method and device used for decoding video estimation residual error coefficient |
| CN101222639B (en) * | 2007-01-09 | 2010-04-21 | 华为技术有限公司 | Inter-view prediction method, encoder and decoder of multi-viewpoint video technology |
| CN101690220B (en) * | 2007-04-25 | 2013-09-25 | Lg电子株式会社 | A method and an apparatus for decoding/encoding a video signal |
| CN101690220A (en) * | 2007-04-25 | 2010-03-31 | Lg电子株式会社 | A method and an apparatus for decoding/encoding a video signal |
| CN101743750B (en) * | 2007-05-14 | 2012-06-27 | 三星电子株式会社 | Method and apparatus for encoding and decoding multi-view image |
| CN102257817A (en) * | 2008-12-19 | 2011-11-23 | 汤姆森特许公司 | Video coding based on global movement compensation |
| CN102257817B (en) * | 2008-12-19 | 2014-10-08 | 汤姆森特许公司 | Video coding based on global movement compensation |
| CN102572418B (en) * | 2010-12-28 | 2014-09-03 | 深圳市云宙多媒体技术有限公司 | Motion prediction method and motion prediction system of interframe encoding |
| CN102572418A (en) * | 2010-12-28 | 2012-07-11 | 深圳市融创天下科技股份有限公司 | Motion prediction method and motion prediction system of interframe encoding |
| CN102036078A (en) * | 2011-01-21 | 2011-04-27 | 哈尔滨商业大学 | Motion estimation method based on relevance among viewing angles for multiple visual angle video coding and decoding system |
| CN102036078B (en) * | 2011-01-21 | 2012-07-25 | 哈尔滨商业大学 | Motion estimation method based on relevance among viewing angles for multiple visual angle video coding and decoding system |
| CN102868879A (en) * | 2011-07-05 | 2013-01-09 | 北京大学 | Method and system for converting video frame rate |
| CN102868879B (en) * | 2011-07-05 | 2015-04-29 | 北京大学 | Method and system for converting video frame rate |
| CN102236798A (en) * | 2011-08-01 | 2011-11-09 | 清华大学 | Image matching method and device |
| CN104380744B (en) * | 2012-06-19 | 2018-05-04 | Lg电子株式会社 | The method and apparatus for handling vision signal |
| US10021417B2 (en) | 2012-06-19 | 2018-07-10 | Lg Electronics Inc. | Method and device for processing video signal |
| CN104380744A (en) * | 2012-06-19 | 2015-02-25 | Lg电子株式会社 | Method and device for processing video signal |
| CN104871541A (en) * | 2012-11-07 | 2015-08-26 | Lg电子株式会社 | Method and apparatus for processing video signals |
| CN105075256A (en) * | 2013-03-27 | 2015-11-18 | 国立研究开发法人情报通信研究机构 | Method for encoding a plurality of input images and storage medium and device for storing program |
| CN105103546A (en) * | 2013-03-27 | 2015-11-25 | 国立研究开发法人情报通信研究机构 | Method and device for encoding a plurality of input images, and storage medium for storing program |
| CN109076234A (en) * | 2016-05-24 | 2018-12-21 | 华为技术有限公司 | Image prediction method and relevant device |
| CN110024404A (en) * | 2016-09-30 | 2019-07-16 | 交互数字Vc控股公司 | The method for encoding global rotary motion compensation image, equipment and stream |
| CN110024404B (en) * | 2016-09-30 | 2023-06-20 | 交互数字Vc控股公司 | Method, apparatus and stream for encoding global rotary motion compensated images |
| US11812066B2 (en) | 2016-09-30 | 2023-11-07 | Interdigital Vc Holdings, Inc. | Methods, devices and stream to encode global rotation motion compensated images |
| CN113271464A (en) * | 2021-05-11 | 2021-08-17 | 北京奇艺世纪科技有限公司 | Video encoding method, decoding method and related devices |
| CN113271464B (en) * | 2021-05-11 | 2022-11-18 | 北京奇艺世纪科技有限公司 | Video encoding method, decoding method and related devices |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1258925C (en) | 2006-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1568015A (en) | Multiple visual-angle video coding-decoding prediction compensation method and apparatus | |
| CN1099656C (en) | Method and device for video compression | |
| JP5992070B2 (en) | Image decoding apparatus, image decoding method, image encoding apparatus, image encoding method, and data structure of encoded data | |
| EP1618744B1 (en) | Video transcoding | |
| JP2019149847A (en) | Image decoding device, image decoding method, image coding device, image coding method, and data structure of coded data | |
| CN1719735A (en) | Method or device for coding a sequence of source pictures | |
| JP2014132766A (en) | Image encoder and image decoder | |
| KR20110129805A (en) | Method for predicting motion vectors in a video codec that allows multiple referencing, motion vector encoding/decoding apparatus using the same | |
| EP2283658A2 (en) | Apparatus and method for computationally efficient intra prediction in a video coder | |
| CN1926884A (en) | Video encoding method and apparatus | |
| WO2011088594A1 (en) | Video encoder, video decoder, method for video encoding and method for video decoding, separately for each colour plane | |
| CN101047860A (en) | Vedio layering coding method at interleaving mode | |
| CN1615656A (en) | Picture level adaptive frame/field coding for digital video content | |
| WO2009134641A2 (en) | Apparatus and method for high quality intra mode prediction in a video coder | |
| JP2012089905A (en) | Image encoder and image encoding method, and image decoder and image decoding method | |
| CN1812577A (en) | Method of decoding digital video and digital video decoder system thereof | |
| CN110351552B (en) | Fast coding method in video coding | |
| KR100856392B1 (en) | Video Encoding and Decoding Apparatus and Method referencing Reconstructed Blocks of a Current Frame | |
| US20130128954A1 (en) | Encoding method and apparatus | |
| CN1268136C (en) | Frame field adaptive coding method based on image slice structure | |
| JP2008271127A (en) | Coding apparatus | |
| CN100586185C (en) | Mode selection method for transcoding 264 video to reduce resolving capability | |
| Song et al. | Unified depth intra coding for 3D video extension of HEVC | |
| Ahmmed et al. | Discrete cosine basis oriented motion modeling for fisheye and 360 degree video coding | |
| CN1232933C (en) | Method for converting format of imiage sequence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: UNITED XINYUAN DIGITAL AUDIO-VIDEO TECHNOLOGY (BE Free format text: FORMER OWNER: INST. OF COMPUTING TECHN. ACADEMIA SINICA Effective date: 20080328 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20080328 Address after: Beijing city Haidian District East Road No. 1 Yingchuang power building block A room 701 Patentee after: Lianhe Xinyuan Digit Audio-Frequency Technology Co., Ltd., Beijing Address before: Digital room (Institute of Physics), Institute of computing, Chinese Academy of Sciences, South Road, Zhongguancun, Haidian District, Beijing 6, China Patentee before: Institute of Computing Technology, Chinese Academy of Sciences |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20060607 Termination date: 20210627 |