CN110708548A - Method for bit allocation in panoramic video frame - Google Patents
Method for bit allocation in panoramic video frame Download PDFInfo
- Publication number
- CN110708548A CN110708548A CN201910974363.8A CN201910974363A CN110708548A CN 110708548 A CN110708548 A CN 110708548A CN 201910974363 A CN201910974363 A CN 201910974363A CN 110708548 A CN110708548 A CN 110708548A
- Authority
- CN
- China
- Prior art keywords
- strip
- frame
- utility function
- panoramic video
- slice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a method for bit allocation in a panoramic video frame, which comprises the following steps: step S1, carrying out projective transformation on the panoramic video, wherein the code rate of an encoder is controlled in such a way that when the video is compressed and encoded, the minimum processing unit is a unit block of 64 multiplied by 64, and the width for splitting a strip in a panoramic video frame is set to 64 and the length is kept as the length of a video image during the projective transformation; step S2, establishing an internal utility function of the strip in the panoramic video frame, namely setting a strip utility function, a strip minimum utility function and an overall utility function; step S3, carrying out bit distribution on the strips, namely solving the bit distribution of the strips by utilizing a Lagrange optimization algorithm; step S4, completing corresponding bit distribution in each strip in the panoramic video frame, setting the bit number of each maximum coding unit in the strip by equally dividing the bit number of the strip, and then performing actual coding operation by the coder to complete the coding flow in the frame; the invention can better improve the video quality.
Description
Technical Field
The invention relates to the technical field of video coding, in particular to a bit allocation method in a panoramic video frame.
Background
The panoramic video is a spherical video formed by splicing video content information obtained by synchronous motion shooting of a plurality of cameras. Most Of intra-frame optimization Of the existing panoramic video is performed by means Of subjective experimental data, eye movement rules Of a subject when the subject watches the panoramic video are collected through VR equipment, a Region Of Interest (ROI) Of a user is analyzed, corresponding bit allocation optimization is performed on the basis, the improvement is subjective coding evaluation indexes, different ROI Of different panoramic videos are different, and therefore the intra-frame optimization is difficult to popularize and apply in a coder.
When the existing encoder HEVC/h.265 processes panoramic video, a bit allocation scheme suitable for normal 2D video is adopted, and when the bit allocation in the panoramic video frame is processed, the bit allocation estimation is made only according to MAD (Mean Absolute Difference), so that a further optimization space exists.
Disclosure of Invention
To overcome the above problems, an object of the present invention is to provide a method for bit allocation in a panoramic video frame, which can better improve video quality.
The invention is realized by adopting the following scheme: a method of bit allocation within a panoramic video frame, the method comprising the steps of:
step S1, carrying out projective transformation on the panoramic video, and then entering an encoder for encoding after the projective transformation, wherein the code rate of the encoder is controlled in such a way that when the video is compressed and encoded, the minimum processing unit is a unit block of 64 multiplied by 64, the width of splitting a strip in a panoramic video frame is set to 64 during the projective transformation, and the length is kept as the length of a video image;
step S2, establishing a strip utility function in the panoramic video frame, namely setting the strip utility function, the strip minimum utility function and the overall utility function;
step S3, carrying out bit distribution on the strips, namely solving the bit distribution of the strips by utilizing a Lagrange optimization algorithm;
step S4, after the corresponding bit allocation is completed for each slice in the panoramic video frame, the encoder performs the actual encoding operation, thereby completing the encoding process in the frame.
Further, the projection transformation adopts an equidistant rectangular projection ERP mode.
Further, the step S2 is further specifically: setting a strip utility function, specifically reflecting the requirement of strip contrast number for the strip utility function, and selecting each strip in the frameAs a function of the utility of each band to reflect the requirement of different bands for the number of bits, where Di,jThe MSE value of the jth strip of the ith frame; the MSE value represents the mean square error value for the jth strip of the ith frame,
to obtain ui,jAnd Di,jIn relation to (1), adopt
Wherein m isi,jMean absolute difference MAD, k representing jth stripe of ith framei,jIs a model parameter one; qstepRepresenting a quantization step;
Ri,ja bit allocation number representing a slice; and Di,j＝γi,j·Qstep；
Wherein r isi,jThe model parameter II is set;
simultaneous formula (1) and formula (2) to obtain Di,jThe value of (A) is as follows:
setting a minimum utility function of the strip, specifically:
to reflect the minimum bit requirement of each slice in the frame, the minimum utility function of the jth slice in the ith frame is defined as:
where M is the number of bands contained in the current frame, R0For the remaining number of bits of the current frame, ei,jThe average MSE value of the jth strip of the ith frame is represented, and c is a scale factor and represents the proportion of the residual bit number so as to ensure the minimum bit number of the jth strip;
setting an overall utility function, specifically:
according to the setting of the strip utility function and the strip minimum utility function, the overall utility function can be constructed as follows:
wherein the subject of this function is
The last two rows
And Ri，j，0＜Ri，jFor the constraint, the body content of the s.t. representation function is to satisfy the constraints of the last two rows.
Further, the Lagrange optimization algorithm is utilized to obtain the bit distribution number R of the stripi,jI.e. by
Wherein p is the p-th band, p<j。
Further, between the step S3 and the step S4, the method further includes: step S30, performing bit allocation on the intra-slice coding units, that is, after the bit allocation of each intra-slice is completed, because the largest coding units LCUs in the intra-slice belong to the same latitude, the largest coding units LCUs in the intra-slice coding units have the same latitude, which ensures the same latitudeViewing experience, bit allocation to the xth largest coding unit LCU inside the stripe:
the encoder encoding is performed after the respective intra-slice and intra-slice maximum coding unit LCU have completed the corresponding bit allocation.
The invention has the beneficial effects that: the analysis object of the bit allocation method is each strip in a frame, each strip comprises a plurality of strips in one frame, each strip is taken as an independent individual, the analysis is carried out by utilizing the game theory, the bit allocation values of the other strips are improved aiming at the allocated bit condition, the self-adaptive bit allocation can be carried out on panoramic videos with different contents, and the bit allocation method has higher universality; the scheme of intra-frame bit allocation in code rate control is improved, and the coding quality of the panoramic video is improved under the condition of the same code rate.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a schematic isometric view of a rectangular projection of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1 and 2, a method for bit allocation in a panoramic video frame according to the present invention includes the following steps:
step S1, carrying out projective transformation on the panoramic video, and then entering an encoder for encoding after the projective transformation, wherein the code rate of the encoder is controlled in such a way that when the video is compressed and encoded, the minimum processing unit is a unit block of 64 multiplied by 64, the width of splitting a strip in a panoramic video frame is set to 64 during the projective transformation, and the length is kept as the length of a video image;
step S2, establishing a strip utility function in the panoramic video frame, namely setting the strip utility function, the strip minimum utility function and the overall utility function;
step S3, carrying out bit distribution on the strips, namely solving the bit distribution of the strips by utilizing a Lagrange optimization algorithm;
step S4, after the corresponding bit allocation is completed for each slice in the panoramic video frame, the encoder performs the actual encoding operation, thereby completing the encoding process in the frame.
Wherein, 1, intra-frame stripe splitting
Before a panoramic video enters an encoder to be encoded, projection transformation is needed, projection quality of the projected video on different dimensions is inconsistent due to the fact that the projection mode processes the panoramic spherical video, and due to the fact that the projection modes are numerous and different in applicability, the scheme analyzes the most common Equidistant Rectangular Projection (ERP) mode at present and is shown in figure 2.
Rate control when compression-encoding a video, the smallest processing unit is a 64 × 64 unit block, and therefore the division width of the slice is also set to 64, and the length is maintained as the length of the video picture.
2. Establishment of strip internal utility function
(1) Setting of band utility function
The utility function reflects the requirement of the strip to the number of the bits, and selects each strip in the frame
to obtain ui,jAnd Di,jIn relation to (1), adopt
Wherein m isi,jMean absolute difference MAD, k representing jth stripe of ith framei,jIs a model parameter one; qstepRepresenting a quantization step; the model parameter is a mathematical expression, and the only uncertain quantity is obtained on the basis of expressing the other variable values of the known function expression. Where k is this uncertainty.
Ri,jA bit allocation number representing a slice; and Di,j＝γi,j·Qstep；
Wherein r isi,jThe model parameter II is set;
simultaneous equations (1) and (2) (these equations are functions commonly used in the field of video coding, see reference 1: author: J.Katto, M.Ohta; subject matter: physical analysis of MPEG compression capability and its application to rate control; year: 1995), available as Di,jIs taken (the reciprocal is u)i,j)：
(2) Setting a minimum utility function of the strip, specifically:
to reflect the minimum bit requirement of each slice in the frame, the minimum utility function of the jth slice in the ith frame is defined as:
where M is the number of bands contained in the current frame, R0For the remaining number of bits of the current frame, ei,jThe average MSE value of the jth strip of the ith frame is represented, and c is a scale factor and represents the proportion of the residual bit number so as to ensure the minimum bit number of the jth strip; i.e. the band utility function is ui,jAnd the minimum utility function is ui,j，0Wherein 0 is the minimum;
(3) setting the overall utility function, specifically:
according to the setting of the strip utility function and the strip minimum utility function, the overall utility function can be constructed as follows:
3. stripe optimal bit allocation
Obtaining bit allocation R of strip by Lagrange optimization algorithmi,jI.e. by
Wherein p is the p-th band, p<j。
The method further comprises the following steps between the step S3 and the step S4: step S30, performing bit allocation on the intra-slice internal coding unit, that is, after the bit allocation of each slice in the frame is completed, since the intra-slice maximum coding unit LCU belongs to the same latitude, to ensure that the same latitude has a consistent viewing experience, allocating the bit of the xth maximum coding unit LCU inside the slice:
After the LCUs in each strip and the strip in the frame complete the corresponding bit allocation, the actual coding operation is carried out through the quantization parameter calculation of the encoder, and the coding process in the frame is completed.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (5)
1. A method for bit allocation within a panoramic video frame, comprising: the method comprises the following steps:
step S1, carrying out projective transformation on the panoramic video, and then entering an encoder for encoding after the projective transformation, wherein the code rate of the encoder is controlled in such a way that when the video is compressed and encoded, the minimum processing unit is a unit block of 64 multiplied by 64, the width of splitting a strip in a panoramic video frame is set to 64 during the projective transformation, and the length is kept as the length of a video image;
step S2, establishing a strip utility function in the panoramic video frame, namely setting the strip utility function, the strip minimum utility function and the overall utility function;
step S3, carrying out bit distribution on the strips, namely solving the bit distribution of the strips by utilizing a Lagrange optimization algorithm;
step S4, after the corresponding bit allocation is completed for each slice in the panoramic video frame, the encoder performs the actual encoding operation, thereby completing the encoding process in the frame.
2. The method of claim 1, wherein the bit allocation in the panoramic video frame is: the projection transformation adopts an equidistant rectangular projection ERP mode.
3. The method of claim 1, wherein the bit allocation in the panoramic video frame is: the step S2 further includes: setting a strip utility function, specifically reflecting the requirement of strip contrast number for the strip utility function, and selecting each strip in the frameAs a function of the utility of each band to reflect the requirement of different bands for the number of bits, where Di,jThe MSE value of the jth strip of the ith frame; the MSE value represents the mean square error value for the jth strip of the ith frame,
to obtain ui,jAnd Di,jIn relation to (1), adopt
Wherein m isi,jRepresenting jth slice of ith frameMean absolute difference MAD, ki,jIs a model parameter one; qstepRepresenting a quantization step;
Ri,ja bit allocation number representing a slice; and Di,j＝γi,j·Qstep；
Wherein r isi,jThe model parameter II is set;
simultaneous formula (1) and formula (2) to obtain Di,jThe value of (A) is as follows:
setting a minimum utility function of the strip, specifically:
to reflect the minimum bit requirement of each slice in the frame, the minimum utility function of the jth slice in the ith frame is defined as:
where M is the number of bands contained in the current frame, R0For the remaining number of bits of the current frame, ei,jThe average MSE value of the jth strip of the ith frame is represented, and c is a scale factor and represents the proportion of the residual bit number so as to ensure the minimum bit number of the jth strip;
setting an overall utility function, specifically:
according to the setting of the strip utility function and the strip minimum utility function, the overall utility function can be constructed as follows:
wherein the subject of this function is
The last two rows
And Ri，j，0＜Ri，jFor the constraint, the body content of the s.t. representation function is to satisfy the last two rowsThe conditions are defined.
4. A method of bit allocation within a panoramic video frame as claimed in claim 3, characterized in that: the Lagrange optimization algorithm is utilized to obtain the bit distribution number R of the stripi,jI.e. by
Wherein p is the p-th band, p<j。
5. The method of claim 4, wherein the bit allocation in the panoramic video frame is: the method further comprises the following steps between the step S3 and the step S4: step S30, performing bit allocation on the intra-slice internal coding unit, that is, after the bit allocation of each slice in the frame is completed, since the intra-slice maximum coding unit LCU belongs to the same latitude, to ensure that the same latitude has a consistent viewing experience, allocating the bit of the xth maximum coding unit LCU inside the slice:
the encoder encoding is performed after the respective intra-slice and intra-slice maximum coding unit LCU have completed the corresponding bit allocation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910974363.8A CN110708548B (en) | 2019-10-14 | 2019-10-14 | Method for bit allocation in panoramic video frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910974363.8A CN110708548B (en) | 2019-10-14 | 2019-10-14 | Method for bit allocation in panoramic video frame |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110708548A true CN110708548A (en) | 2020-01-17 |
| CN110708548B CN110708548B (en) | 2022-03-08 |
Family
ID=69199401
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910974363.8A Active CN110708548B (en) | 2019-10-14 | 2019-10-14 | Method for bit allocation in panoramic video frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110708548B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112616014A (en) * | 2020-12-09 | 2021-04-06 | 福州大学 | GAN-based panoramic video adaptive streaming transmission method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207601A (en) * | 2007-05-24 | 2008-06-25 | 北京交通大学 | Method and system of OFDM bit power distribution based on game theory |
| CN105245919A (en) * | 2015-10-08 | 2016-01-13 | 清华大学 | Energy-consumption-optimization adaptive streaming media distribution method for intelligent terminal |
| US20180020238A1 (en) * | 2016-07-15 | 2018-01-18 | Mediatek Inc. | Method and apparatus for video coding |
| CN108200431A (en) * | 2017-12-08 | 2018-06-22 | 重庆邮电大学 | A kind of video frequency coding rate controls frame-layer Bit distribution method |
| CN109451309A (en) * | 2018-12-04 | 2019-03-08 | 南京邮电大学 | The full I frame of HEVC encodes the CTU layer bit rate distribution method based on conspicuousness |
-
2019
- 2019-10-14 CN CN201910974363.8A patent/CN110708548B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207601A (en) * | 2007-05-24 | 2008-06-25 | 北京交通大学 | Method and system of OFDM bit power distribution based on game theory |
| CN105245919A (en) * | 2015-10-08 | 2016-01-13 | 清华大学 | Energy-consumption-optimization adaptive streaming media distribution method for intelligent terminal |
| US20180020238A1 (en) * | 2016-07-15 | 2018-01-18 | Mediatek Inc. | Method and apparatus for video coding |
| CN108200431A (en) * | 2017-12-08 | 2018-06-22 | 重庆邮电大学 | A kind of video frequency coding rate controls frame-layer Bit distribution method |
| CN109451309A (en) * | 2018-12-04 | 2019-03-08 | 南京邮电大学 | The full I frame of HEVC encodes the CTU layer bit rate distribution method based on conspicuousness |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112616014A (en) * | 2020-12-09 | 2021-04-06 | 福州大学 | GAN-based panoramic video adaptive streaming transmission method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110708548B (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111432207B (en) | Perceptual high-definition video coding method based on salient target detection and salient guidance | |
| US9762917B2 (en) | Quantization method and apparatus in encoding/decoding | |
| CN104954791B (en) | Key frame real-time selection method in the wireless distributed Video coding of mine | |
| CN101466035B (en) | Method for distributing video image set bit based on H.264 | |
| US6963610B2 (en) | Moving image coding device, moving image coding method and program thereof employing pre-analysis | |
| CN108924554B (en) | Panoramic video coding rate distortion optimization method based on spherical weighting structure similarity | |
| CN108200431B (en) | Bit allocation method for video coding code rate control frame layer | |
| AU2016337193B2 (en) | Data compression | |
| CN110662044A (en) | Video coding method, video coding device and computer storage medium | |
| CN103533365B (en) | Bit-rate control method and Bit-Rate Control Algorithm system | |
| CN111741300B (en) | Video processing method | |
| US9185419B2 (en) | Method and device for coding a sequence of images | |
| WO2021129007A1 (en) | Method and device for determining video bitrate, computer apparatus, and storage medium | |
| CN110876060A (en) | Code rate adjusting method and device in coding process | |
| CN102938840A (en) | Key frame quantization parameter selecting method applied to multi-viewpoint video coding system | |
| EP3545677A1 (en) | Methods and apparatuses for encoding and decoding video based on perceptual metric classification | |
| CN108810530A (en) | A kind of AVC bit rate control methods based on human visual system | |
| CN1498501A (en) | Method for compressing and decompressing video data | |
| CN112437301A (en) | Code rate control method and device for visual analysis, storage medium and terminal | |
| CN110708548B (en) | Method for bit allocation in panoramic video frame | |
| TWI510050B (en) | Quantization control method and apparatus, and quantization control program | |
| CN114143537B (en) | All-zero block prediction method based on possibility size | |
| CN107592535B (en) | H.265/HEVC image layer code rate control method | |
| CN109547781B (en) | Compression method and device based on image prediction | |
| CN102271247B (en) | Method and device for adjusting coding code rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |