CN101198061A - Solid video stream encoding method based on sight point image mapping - Google Patents
Solid video stream encoding method based on sight point image mapping Download PDFInfo
- Publication number
- CN101198061A CN101198061A CN 200810050226 CN200810050226A CN101198061A CN 101198061 A CN101198061 A CN 101198061A CN 200810050226 CN200810050226 CN 200810050226 CN 200810050226 A CN200810050226 A CN 200810050226A CN 101198061 A CN101198061 A CN 101198061A
- Authority
- CN
- China
- Prior art keywords
- image
- viewpoint
- video
- visual point
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a stereo video stream coding method on the basis of viewpoint image mapping and belongs to the video coding technical field, wherein, a mapping image is obtained at first through mapping image pixel information of a first viewpoint to a second viewpoint, and then stereo video stream coding of the mapping image and a second viewpoint image is performed. The invention overcomes the defects of low coding efficiency of the stereo video coding method on the basis of parallax estimation and compensation, and large residual values and lowered compressibility after parallax estimation of a forecasting algorithm is combined, etc.The invention has the advantages of capability of making the best of correspondence which is formed between binocular viewpoint image data through mutual mapping of a three-dimensional space and improvement of coding performances of stereo video streams.
Description
Technical field
The invention belongs to technical field of video coding, relate to the coding method of stereo video streaming, relate in particular to a kind of based on the visual point image mapping method for encoding stereo video of (also claiming the three-dimensional video-frequency coupling).
Background technology
The notion of parallax compensation is proposed since Lukacs, use parallax information set up three-dimensional video-frequency between the correlation of similar area, method for encoding stereo video based on disparity estimation and compensation grows up thereupon, and becomes a kind of classical technology of stereo scopic video coding.Be that the method for encoding stereo video that unit carries out disparity estimation and compensation is used in many viewpoints class of Moving Picture Experts Group-2 with the piece.In this method, encoder is at first used algorithm for single-pass video encoding compression algorithm left side passage video.For right passage video, the frame of each frame left side passage video sequence correspondence is predicted based on the optical parallax field of estimating, and optical parallax field and prediction residual image is encoded.Disparity estimation and compensation process all adopt block-based method; That is to say that for each image block in the right view, they seek a piece the most similar in left view.One preferably method be which provides less predicated error according to, between the motion compensated prediction between the different frame of the parallax compensation between the different views of identical time prediction and identical view, switch.The temporal scalability sexual norm of its available Moving Picture Experts Group-2 is realized, with the temporal scalability of MPEG-2, sequence is divided into the collection of two sub-sampling frames.Basic layer (such as, form by even frame) encode with the one-way movement compensation prediction, wherein reference frame is the even frame from preceding face code.The enhancement layer of being made up of remaining strange frame is encoded with the bi directional motion compensation prediction, and reference frame is from the strange frame of preceding face code, and another is from close even frame in the basic layer.Left view is encoded as basic layer, and right view is encoded as enhancement layer.For left view, image is encoded with I, B or P pattern.For B and P image, be used for predictive reference frames and only come from left view.For right view, image is encoded with P or B pattern, and in the P pattern, image carries out predictive coding with frame corresponding in the left view as the reference frame.In the B frame pattern, reference frame is from a frame of front in the right view, and another reference frame is a frame corresponding in the left view.The used motion vector of latter event is actually difference vector, and forecasting process is the parallax compensation prediction.The advantage of this method is to use the method and the technology of conventional motion compensation prediction in many algorithm for single-pass video encoding, but the efficient of its coding is not high.
In conjunction with the associated prediction algorithm of Forecasting Methodologies such as motion compensation, further improved performance based on the method for encoding stereo video of disparity estimation and compensation.Disparity estimation is similar with estimation, the both is the correlation of seeking between two width of cloth images, different is, estimation is to go up two adjacent width of cloth images at the time to carry out, because the correlation that the time goes up between the two adjacent width of cloth images is very strong, in estimation, have only a spot of pixel to be moved usually, so the residual values after the motion compensation is all less, even near null value.Yet in disparity estimation, two width of cloth correspondence image of left and right sides video channel are synchronous in time, and they are taken by two different at one time visual fields of camera and obtain, so the residual values after the disparity estimation is all bigger, thereby compression performance is descended.
Summary of the invention
The object of the present invention is to provide a kind ofly to have overcome based on the efficient of the method for encoding stereo video of disparity estimation and compensation coding lowly, and the residual values after the disparity estimation of associated prediction algorithm is big, the degradation drawback based on the visual point image mapping three-dimensional video-frequency stream encoding method of (also claiming the three-dimensional video-frequency coupling) under the compression performance.
The present invention is on the basis of conventional three-dimensional video-frequency stream encoding method, before encoding, the binocular vision dot image of stereoscopic video stream increases a mapping process, utilize the characteristics of binocular tri-dimensional video stream, the first visual point image Pixel Information is mapped to second viewpoint, obtain map image, then the map image and second visual point image are encoded with conventional three-dimensional video-frequency volume stream code method.
Three-dimensional video-frequency stream encoding method based on the visual point image mapping of the present invention is made up of the following step:
1. the stereo video streaming first visual point image Pixel Information is mapped to second viewpoint, obtains map image;
2. the map image and second visual point image are carried out the three-dimensional video-frequency stream encryption.
Above-described the stereo video streaming first visual point image Pixel Information is mapped to second viewpoint, obtains map image and realize by the following step:
A. establish p
1(u
1, v
1) flow the image coordinate of the point in first visual point image, p for binocular tri-dimensional video
2(u
2, v
2) be p
1Be mapped to the image coordinate of the point in second visual point image, P (X
w, Y
w, Z
w) be the two corresponding spatial point (same point in the world coordinate system), according to the pin-hole imaging model, the some P in the world coordinate system and its be the subpoint p in the binocular vision dot image respectively
1And p
2The relational expression of representing with homogeneous coordinates is:
With
In formula (1) and (2), s
1And s
2Be respectively two image coordinate of stereo video streaming and the camera coordinate system scale factor when transforming mutually, M
1And M
2Be respectively the projection matrix of two views, projection matrix can be determined by the inner parameter and the external parameter of video camera.
B. with 3 * 4 projection matrix M
1And M
23 * 3 parts on the left side are remembered respectively and are made M in the matrix
11And M
21, 3 * 1 parts on the right are remembered respectively and are made m
1And m
2If will
Note is done
Then formula (1) and (2) can be written as respectively:
With
With in formula (3) and (4)
Cancellation gets:
That is:
C. according to the characteristics of binocular tri-dimensional video stream, establish s
1=s
2=s, thus, formula (6) can be reduced to:
In the formula (7), both sides are three-dimensional vectors, have promptly comprised three equatioies, behind the cancellation s wherein, just can obtain u
2And v
2
The present invention since the characteristics of at first utilizing three-dimensional video-frequency with the image mapped of stereo video streaming first viewpoint to second viewpoint, form map image, then its image with second viewpoint is carried out the three-dimensional video-frequency stream encryption together, thereby can utilize the corresponding relation that shines upon formation between the binocular vision dot image data by three dimensions mutually more fully, improve the coding efficiency of stereo video streaming.
Embodiment
Core content of the present invention is binocular tri-dimensional video to be flowed first visual point image be mapped to second viewpoint, obtains map image, and the image with second viewpoint carries out the three-dimensional video-frequency stream encryption together then.
Concrete grammar is: at first binocular tri-dimensional video is flowed the first visual point image Pixel Information and be mapped to second viewpoint, obtain map image, then this map image image of second viewpoint is together carried out the three-dimensional video-frequency stream encryption together.
The described stereo video streaming first visual point image Pixel Information is mapped to second viewpoint, and the concrete grammar that obtains map image step (unit) is:
If p
1(u
1, v
1) flow the image coordinate of the point in first view, p for binocular tri-dimensional video
2(u
2, v
2) be p
1Be mapped to the image coordinate of the point in second view, P (X
w, Y
w, Z
w) be the two corresponding spatial point (same point in the world coordinate system), according to the pin-hole imaging model, and the characteristics of utilizing binocular tri-dimensional video to flow, can determine the subpoint p of some P in the binocular vision dot image in the world coordinate system roughly
1And p
2Relation, promptly be mapped to second viewpoint by the first visual point image Pixel Information, obtain map image.
Concrete implementation step is:
1) establishes p
1(u
1, v
1) flow the image coordinate of the point in first view, p for binocular tri-dimensional video
2(u
2, v
2) be p
1Be mapped to the image coordinate of the point in second view, P (X
w, Y
w, Z
w) be the two corresponding spatial point (same point in the world coordinate system), according to the pin-hole imaging model, the some P in the world coordinate system and its be the subpoint p in the binocular vision dot image respectively
1And p
2The pass of representing with homogeneous coordinates is:
With
Wherein, s
1And s
2Be respectively two image coordinate of stereo video streaming and the camera coordinate system scale factor when transforming mutually, M
1And M
2Be respectively the projection matrix of two visual point images.Projection matrix can be determined by the inner parameter and the external parameter of video camera.
2) with 3 * 4 projection matrix M
1And M
23 * 3 parts on the left side are remembered respectively and are made M in the matrix
11And M
21, 3 * 1 parts on the right are remembered respectively and are made m
1And m
2If will
Note is done
Then formula (1) and (2) can be written as respectively:
With
With in formula (3) and (4)
Cancellation gets:
That is:
3) according to the characteristics of binocular tri-dimensional video stream, can establish s
1=s
2=s, thus, formula (6) can be reduced to:
In the formula (7), both sides are three-dimensional vectors, have promptly comprised three equatioies, behind the cancellation s wherein, just can obtain u
2And v
2
Claims (2)
1. three-dimensional video-frequency stream encoding method based on visual point image mapping is characterized in that it is made up of the following step:
A. the stereo video streaming first visual point image Pixel Information is mapped to second viewpoint, obtains map image;
B. the map image and second visual point image are carried out the three-dimensional video-frequency stream encryption.
2. three-dimensional video-frequency stream encoding method according to claim 1 is characterized in that described the stereo video streaming first visual point image Pixel Information being mapped to second viewpoint, obtains map image and is realized by the following step:
A. establish p
1(u
1, v
1) flow the image coordinate of the point in first visual point image, p for binocular tri-dimensional video
2(u
2, v
2) be p
1Be mapped to the image coordinate of the point in second visual point image, P (X
w, Y
w, Z
w) be the two corresponding spatial point (same point in the world coordinate system), according to the pin-hole imaging model, the some P in the world coordinate system and its be the subpoint p in the binocular vision dot image respectively
1And p
2The relational expression of representing with homogeneous coordinates is:
With
In formula (1) and (2), s
1And s
2Be respectively two image coordinate of stereo video streaming and the camera coordinate system scale factor when transforming mutually, M
1And M
2Be respectively the projection matrix of two views, projection matrix can be determined by the inner parameter and the external parameter of video camera.
B. with 3 * 4 projection matrix M
1And M
23 * 3 parts on the left side are remembered respectively and are made M in the matrix
11And M
21, 3 * 1 parts on the right are remembered respectively and are made m
1And m
2If will
Note is done
Then formula (1) and (2) can be written as respectively:
With in formula (3) and (4)
Cancellation gets:
That is:
C. according to the characteristics of binocular tri-dimensional video stream, establish s
1=s
2=s, thus, formula (6) can be reduced to:
In the formula (7), both sides are three-dimensional vectors, have promptly comprised three equatioies, behind the cancellation s wherein, just can obtain u
2And v
2
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200810050226 CN101198061A (en) | 2008-01-08 | 2008-01-08 | Solid video stream encoding method based on sight point image mapping |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200810050226 CN101198061A (en) | 2008-01-08 | 2008-01-08 | Solid video stream encoding method based on sight point image mapping |
Publications (1)
Publication Number | Publication Date |
---|---|
CN101198061A true CN101198061A (en) | 2008-06-11 |
Family
ID=39548154
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 200810050226 Pending CN101198061A (en) | 2008-01-08 | 2008-01-08 | Solid video stream encoding method based on sight point image mapping |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101198061A (en) |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101841726A (en) * | 2010-05-24 | 2010-09-22 | 宁波大学 | Three-dimensional video asymmetrical coding method |
CN102006468A (en) * | 2010-08-24 | 2011-04-06 | 吉林大学 | Multi-grade hybrid predictive coding structure of stereo element images |
CN102055982A (en) * | 2011-01-13 | 2011-05-11 | 浙江大学 | Coding and decoding methods and devices for three-dimensional video |
CN102917233A (en) * | 2012-11-14 | 2013-02-06 | 中航华东光电有限公司 | Stereoscopic video coding optimization method in space teleoperation environment |
CN103544717A (en) * | 2013-10-22 | 2014-01-29 | 北京交通大学长三角研究院 | Two-phase three-dimensional image compression encoding method based on SIFT feature |
CN105308969A (en) * | 2013-01-10 | 2016-02-03 | 高通股份有限公司 | View synthesis in 3d video |
CN105551033A (en) * | 2015-12-09 | 2016-05-04 | 广州视源电子科技股份有限公司 | Component marking method, system and device |
CN105791851A (en) * | 2016-04-07 | 2016-07-20 | 贵阳动视云科技有限公司 | Video compression method and apparatus, and video decoding method and apparatus |
CN106507117A (en) * | 2008-07-20 | 2017-03-15 | 杜比实验室特许公司 | The Encoder Optimization of stereoscopic video delivery |
CN114697632A (en) * | 2022-03-28 | 2022-07-01 | 天津大学 | End-to-end stereo image compression method and device based on bidirectional condition coding |
-
2008
- 2008-01-08 CN CN 200810050226 patent/CN101198061A/en active Pending
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106507117A (en) * | 2008-07-20 | 2017-03-15 | 杜比实验室特许公司 | The Encoder Optimization of stereoscopic video delivery |
CN106507117B (en) * | 2008-07-20 | 2020-01-21 | 杜比实验室特许公司 | Method and apparatus for encoder optimization for stereoscopic video delivery systems |
CN101841726B (en) * | 2010-05-24 | 2011-10-05 | 宁波大学 | Three-dimensional video asymmetrical coding method |
CN101841726A (en) * | 2010-05-24 | 2010-09-22 | 宁波大学 | Three-dimensional video asymmetrical coding method |
CN102006468A (en) * | 2010-08-24 | 2011-04-06 | 吉林大学 | Multi-grade hybrid predictive coding structure of stereo element images |
CN102006468B (en) * | 2010-08-24 | 2012-11-14 | 吉林大学 | Multi-grade hybrid predictive coding structure of stereo element images |
CN102055982B (en) * | 2011-01-13 | 2012-06-27 | 浙江大学 | Coding and decoding methods and devices for three-dimensional video |
CN102055982A (en) * | 2011-01-13 | 2011-05-11 | 浙江大学 | Coding and decoding methods and devices for three-dimensional video |
CN102917233A (en) * | 2012-11-14 | 2013-02-06 | 中航华东光电有限公司 | Stereoscopic video coding optimization method in space teleoperation environment |
CN105308969A (en) * | 2013-01-10 | 2016-02-03 | 高通股份有限公司 | View synthesis in 3d video |
CN105308969B (en) * | 2013-01-10 | 2018-09-14 | 高通股份有限公司 | View synthesis in 3 D video |
US10136119B2 (en) | 2013-01-10 | 2018-11-20 | Qualcomm Incoporated | View synthesis in 3D video |
CN103544717A (en) * | 2013-10-22 | 2014-01-29 | 北京交通大学长三角研究院 | Two-phase three-dimensional image compression encoding method based on SIFT feature |
CN103544717B (en) * | 2013-10-22 | 2016-09-28 | 北京交通大学长三角研究院 | A kind of two-stage three dimensional image processing coded method based on SIFT feature |
CN105551033A (en) * | 2015-12-09 | 2016-05-04 | 广州视源电子科技股份有限公司 | Component marking method, system and device |
CN105551033B (en) * | 2015-12-09 | 2019-11-26 | 广州视源电子科技股份有限公司 | Component marking method, system and device |
CN105791851A (en) * | 2016-04-07 | 2016-07-20 | 贵阳动视云科技有限公司 | Video compression method and apparatus, and video decoding method and apparatus |
CN114697632A (en) * | 2022-03-28 | 2022-07-01 | 天津大学 | End-to-end stereo image compression method and device based on bidirectional condition coding |
CN114697632B (en) * | 2022-03-28 | 2023-12-26 | 天津大学 | End-to-end stereoscopic image compression method and device based on bidirectional conditional coding |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101198061A (en) | Solid video stream encoding method based on sight point image mapping | |
CN101668205B (en) | Self-adapting down-sampling stereo video compressed coding method based on residual error macro block | |
JP5052134B2 (en) | Image encoding method, image decoding method, image encoding device, image decoding device, image encoding program, and image decoding program | |
CN102055982B (en) | Coding and decoding methods and devices for three-dimensional video | |
CN102685532B (en) | Coding method for free view point four-dimensional space video coding system | |
CN102905150B (en) | Novel multi-view video fractal coding, compressing and decompressing method | |
KR101747434B1 (en) | Apparatus and method for encoding and decoding motion information and disparity information | |
CN102801995B (en) | A kind of multi-view video motion based on template matching and disparity vector prediction method | |
CN104995916B (en) | Video data decoding method and video data decoding device | |
CN101980538B (en) | Fractal-based binocular stereoscopic video compression coding/decoding method | |
CN102291579B (en) | Rapid fractal compression and decompression method for multi-cast stereo video | |
CN102438167B (en) | Three-dimensional video encoding method based on depth image rendering | |
CN101990103B (en) | Method and device for multi-view video coding | |
CN101986716A (en) | Quick depth video coding method | |
CN104469336B (en) | Coding method for multi-view depth video signals | |
CN102970529A (en) | Multi-viewpoint video fractal coding compressing and uncompressing method based on objects | |
CN100581265C (en) | Processing method for multi-view point video | |
CN101222647B (en) | Scene global depth estimation method for multi-vision angle video image | |
CN103475884B (en) | Three-dimensional video-frequency B frame entire frame loss error concealing method towards HBP coded format | |
CN102611893B (en) | DMVC (distributed multi-view video coding) side-information integration method on basis of histogram matching and SAD (security association database) judgment | |
CN101873484B (en) | Method and device for selecting coding mode in layered video coding | |
CN102316323B (en) | Rapid binocular stereo-video fractal compressing and uncompressing method | |
CN102790895B (en) | Multi-viewpoint video encoding and viewpoint synthesis predication method based on least square | |
CN103873867A (en) | Free viewpoint video depth map distortion prediction method and free viewpoint video depth map coding method | |
CN102917233A (en) | Stereoscopic video coding optimization method in space teleoperation environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20080611 |