CN101529915A - Video coding rate adaptation to reduce packetization overhead - Google Patents

Abstract

The present invention discloses techniques for video coding rate adaptation to reduce packetization overhead. The video coding rate controls the number of coding bits allocated to a segment of encoded video, and hence the length of the encoded video segment. Differences between the length of the encoded video segment and the cumulative length of a series of packets used to encode the video segment result in unused packet space within the last packet in the series. This unused packet space is typically filled with padding bits. In accordance with the disclosure, the video coding rate is adjusted for a segment of digital video so that the encoded video more closely fits within the series of packets, thereby reducing the number of padding bits required by the last packet.

Description

In order to reduce the video coding rate adaptation of packetize overhead

Technical field

The present invention relates to digital video coding, and more particularly relate to the technology that is used for the control of video code rate.

Background technology

Digital video capabilities can be incorporated in the device of broad range, comprises Digital Television, digital live broadcast system, radio communication device, PDA(Personal Digital Assistant), laptop computer, desktop computer, video game console, digital camera, digital recorder, honeycomb fashion or satellite radio telephone and analog.Digital video apparatus can be in the remarkable improvement that provides aspect processing and the transmission of video sequence than the conventional simulation video system.

Set up the different video coding standard and be used for the encoded digital video sequence.Motion Picture Experts Group (MPEG) has developed and has comprised for example numerous standards of MPEG-1, MPEG-2 and MPEG-4.Other example comprises H.263 H.264 standard and homologue ISO/IEC MPEG-4 thereof of standard and emerging ITU-T of International Telecommunication Union-T, and Part 10, that is, and and advanced video coding (AVC).These video encoding standards are by supporting the improved efficiency of transmission of video sequence with the compress mode coded data.

Rate control techniques is assigned to the number of the bits of coded of each frame of video, i.e. code rate in order to adjustment.Code rate can be through adjusting to guarantee that encoded video sequence conforms to quality requirements and/or bandwidth constraints.Some rate control techniques through design producing constant code speed, and other rate control techniques through design to produce constant-quality.Other rate control techniques can make code rate and quality level balance, and in response to video frame content.

In packet network (wired or wireless), encoded video through packetize be used for the transmission.Procotol applicatory stipulates to wrap size requirements usually.For instance, transmission control protocol (TCP) the regulation maximum transfer unit (MTU) that is used for Internet transmission.Supposing the bag size of defined, can be a plurality of bags with the burst segmentation of encoded video with in transmission over networks.In general, the size of burst may accurately not mated with the size of bag.For this reason, bag will comprise at least some fillers usually at last.

Summary of the invention

The present invention describes and is used for video coding rate adaptation to reduce the technology of packetize overhead.The video coding rate Control Allocation is to the number of the bits of coded of the fragment of encoded video, and therefore controls the length of encoded video segment.The length of encoded video segment and cause the untapped bag space in the last bag in the described series in order to the difference between the cumulative length of a series of bags of encoded video fragment.This untapped bag space is filled up with filler usually.According to the present invention, adjust video coding rate at the fragment of digital video, so that encoded video more relevantly is matched with in the bag of described series, reduce the number of the required filler of last bag whereby.

In one aspect, the invention provides a kind of method for video coding, it comprises the code rate of determining to select in order to the size of the bag of the encoded fragment of packetize digital of digital video data with based on described bag size to be used for the described fragment of digital of digital video data.

On the other hand, the invention provides a digital video coding equipment, it comprises the rate controlled unit, the size in order to the bag of the encoded fragment of packetize digital of digital video data is determined in described rate controlled unit, and selection is used for the code rate of the described fragment of digital of digital video data based on described bag is big or small.

In an additional aspect, the invention provides a kind of processor that is used for encoding digital video data, described processor is configured to determine the size in order to the bag of the encoded fragment of packetize digital of digital video data, and selects to be used for the code rate of the described fragment of digital of digital video data based on described bag size.

Can in hardware, software, firmware or its any combination, in digital-video equipment, implement technology described in the present invention.If in software, implement, so can be in the machine of for example processor executive software.According to the present invention, software can be stored in the machine-readable medium and by described machine as instruction at first and carry out to support video coding rate adaptation to reduce the packetize overhead.

The additional detail of the various aspects of statement in the following drawings and the description content.From describe content and graphic and accessory rights claim, will understand other features, objects and advantages.

Description of drawings

Fig. 1 illustrates to adopt the block diagram of video coding rate adaptation with the digital video processing apparatus of minimizing packetize overhead according to an aspect of the present invention.

Fig. 2 is the figure that the packetize of the video segment with the code rate that causes a large amount of packetize overheads is described.

Fig. 3 is explanation has the packetize of the video segment through adjusting packet rate that causes the packetize overhead that reduces according to the present invention figure.

Fig. 4 is an explanation video segment residue size at the curve chart by the distribution on the video sequence of some dissimilar content association.

Fig. 5 and 6 is that explanation is used to control the curve chart of code rate with the function of minimizing packetize overhead.

Fig. 7 illustrates to be used for the flow chart of video coding rate adaptation with the method for minimizing packetize overhead according to an aspect of the present invention.

Fig. 8 is that explanation uses historical data to adjust the flow chart of video coding rate in the method for Fig. 7.

Embodiment

The present invention describes and is used for video coding rate adaptation to reduce the technology of packetize overhead.The number of the position of the frame of video coding rate Control Allocation in the fragment of encoded video, and therefore control the length of encoded video segment.The length of encoded video segment and cause the untapped bag space in the last bag in the described series in order to the difference between the cumulative length of a series of bags of encoded video fragment.This untapped bag space is filled up with filler usually, thereby causes waste bandwidth.

According to the present invention, adjust video coding rate so that encoded video more relevantly is matched with in the bag of described series at the frame in the digital video fragment, reduce the number of the required filler of last bag whereby.For instance, maximizing or the part that falls into last bag (that is, residue) that increases fragment at least to be more relevantly mating with the size of last bag, thereby stay the less white space that is used for filler.

In general, in some respects, can be based on the historical data of the mean value of the residue of the previous encoded fragment of the estimated variance of the rate control algorithm of the code rate that is used for fragment in order to control and designation number video data and select code rate.The video coding rate adaptation technology is suitable for the different video content, can need low computational complexity and can be feature with a plurality of parameters that are used for the different rates control algolithm through fine tuning.

Can with H.263 or H.264 for example MPEG-1, MPEG-2 or MPEG-4 standard, ITU standard or ISO/IECMPEG-4, Part 10 standards (promptly, advanced video coding (AVC), standard is identical substantially with H.264 for it) multiple prediction video encoding standard in any one use described technology together.For instance, but as described in the present inventionly be used for the technology combined standard rate control algorithm of video coding rate adaptation and use, so that adjust the packetize overhead efficient that speed obtains to strengthen.In some respects, being used for the technology of video coding rate adaptation can be in order to adjust the code rate that is produced by the standard speed control algolithm.Described standard speed control algolithm can be constant rate of speed or variable bit rate algorithm.

Fig. 1 is the block diagram of illustrated example digital video processing apparatus 10.In the example of Fig. 1, video processing equipment 10 comprises video source 12, video encoder 14, video packetizer 16 and transmitter 18.Video processing equipment 10 can reside at can encode and any device of transmitting video data in, for example radio communication device, PDA(Personal Digital Assistant), laptop computer, desktop computer, video game console or the analog of video camera, digital live broadcast system, for example honeycomb fashion or satellite radio telephone.

Video source 12 can be for example video capture device of video camera, or stores the video archive (archive) of the digital video of before having captured.Video source 12 also can be live or through the interface of file video feed-in.Video encoder 14 comprises video encoding module 20, the video that it is encoded and obtain from video source 12 according to any one (for example H.264) in the various video coding standard as mentioned above.In addition, video encoder 14 comprises rate controlled module 22, and it is controlled by the code rate of video encoding module 20 in order to the frame in the encoded video fragment.The code rate regulation is assigned to the number of the bits of coded of the frame in the video segment.

Video packetizer 16 receives encoded video segment and encoded video segment is divided into a series of bags to transmit via transmitter 16 from video encoding module 20.Gained Bao Kecong application layer is delivered to other layer (for example transport layer and physical layer) and further handles to be used for for example multiplexed, extra packetize and other operation etc.

Each bag that is produced by video packetizer 16 can comprise the part of the fragment of encoded video data, and any header data applicatory.In particular, but each bag carrying from one or more frames of video segment.In order to " residue " of the last bag in the bag of the described series of encoded video fragment with the carrying video segment, that is, mismatch the remainder of the previous bag in described series, add the white space that filler occupies.

Under some situations, the encoded video data that can before the video packetize video encoding module 20 be produced is archived in the memory or data storage device in (for example) video processing equipment 10.Perhaps, can file but not transmit immediately by video packetizer 16 produce through the packetize video.Under arbitrary situation, transmitter 18 can be and can go up any suitable transmitter of transmission by the bag of video packetizer 16 generations at wired or wireless communication medium (for example, packet network).

Video encoding module 20 produces the fragment of encoded video data with burst form.Owing to its " sudden " character, variable bandwidth when compressing video frequency flow has.Under the situation of standard H.264, for example, each video data fragment of being handled by video encoding module 20 can be so-called super frame (sf), and it constitutes burst in a second of video data usually.But a plurality of frames of described SF carrying video data.For instance, in some applications, but SF carrying 30 frames roughly.Described frame is the consecutive image in the video sequence, and can intra-encodedly be the I frame, is the P frame through interframe encode, or is two-way (B) frame through interframe encode.

Frame can be different sizes, but and one or more frames of bag carrying.For this reason, each fragment of video data (for example, each SF) according to the video data fragment in content be associated the position number can have different sizes.The number that is assigned to the bits of coded of each frame also can be different.In addition, the number (that is the code rate that, is used for fragment) of crossing the bits of coded of the frame distribution in the fragment is adjusted technology and difference according to rate controlled described in the present invention.

The size of fragment that contains the video data of relative high complexity scene usually will be greater than the size of the fragment of the video data that contains relative low-complexity scene.In addition, the big I of the individual frame in the fragment changes according to complexity.For instance, some frames can comprise than other frame than motion or complicated texture.In any case the size of each frame and the big young pathbreaker of containing the fragment of a plurality of frames are changing between the frame and between fragment.For those reasons, the variable bit rate control algolithm can be with the different coding rate-allocation to different fragments.

Each bag that is produced by video packetizer 16 can have fixed size, or stands the variable-size of some constraints.For instance, procotol applicatory is set maximum bag size requirements, for example MTU that stipulates for TCP usually.Suppose the bag size of defined, the fragment by video packetizer 16 video that video encoder 14 is coded is divided into a plurality of bags with transmission on network.In general, the big young pathbreaker of the encoded video that is produced by video encoding module 20 is not accurately mated with the size of the bag that is produced by packetizer 16, and as mentioned above, will for the time become.

Owing to the mismatch between the cumulative size of the size of encoded video segment and the needed some bags of the encoded video segment of carrying, the last bag that is produced by packetizer 16 will comprise at least some fillers usually.Filler is filled up the white space that is produced by the mismatch between the cumulative size of the size of encoded video and bag.Comprising of filler is invalid, causes can be used for the consumption of the bandwidth of other purpose.According to the present invention, rate controlled module 22 adjust video encoding module 20 applied code rates come with through planning with the frame in the mode encoded video fragment of the number that reduces the required filler of packetizer 16.In this way, can make based on the bag coding and video data communicate by letter more effective.

According to the present invention, rate controlled module 22 can be used through skew to reduce the standard speed control algolithm of packetize overhead.For instance, rate controlled module 22 is determined the size of the bag of the encoded digital of digital video data that produced in order to packetize video encoding module 20.As shown in fig. 1, rate controlled module 22 can receive the bag size from video packetizer 22.The bag size can be fixing or variable, and is gone up and stipulate in (packet-by-packet), periodicity or intermittent basis packet-by-packet by video packetizer 22.Perhaps, bag size can be fixing and for rate controlled module 22 known.Under arbitrary situation, the size in order to the video packets of the encoded video segment of packetize is treated in 22 acquisitions of rate controlled module.

In addition, rate controlled module 22 can receive the historical data of average residual thing of the previous encoded fragment of designation number video data.Based on bag size and/or historical information, rate controlled module 22 selects to treat to be used for by video encoding module 20 code rate of the current fragment of digital of digital video data to be encoded.Code rate can change between fragment.In particular, the code rate of being set by rate controlled module 22 can change according to the size of current fragment to be encoded, so that rate controlled module 22 is adjusted video coding rate to reduce the packetize overhead on the segment by segment basis.In this way, rate controlled module 22 can be suitable for the change of video content between fragment.

Use selected code rate, video encoding module 20 coding video frequency data fragments are more relevantly to mate with the cumulative size in order to a series of bags of the encoded video data fragment of carrying.In this way, rate controlled module 22 reduces the packetize overhead and promotes bandwidth efficiency.Bandwidth efficiency may be important for any communication medium, but is even more important for having band-limited wireless communication media.In addition, bandwidth efficiency for relate to wireless channel on the application of real-time Transmission of video sequence may be the problem of significantly paying close attention to.

Fig. 2 is the figure that the packetize of the video segment with the code rate that causes a large amount of packetize overheads is described.As shown in Figure 2, the fragment 23 of coding module 20 encoding digital video datas.(but unrestricted) for purposes of illustration, fragment 23 can be known as super frame (sf).Technology described herein can be applicable to the fragment of any size.Once more, SF is often referred to the roughly fragment of 30 successive frames with video sequence, but the number of frame will change between SF.

In the example of Fig. 2, coding module 20 (for example) uses the standard speed control algolithm to come with given code rate encode fragment 23, and does not consider in order to the size of the bag of packetize fragment how encoded fragment 23 with respect to.Packetizer 16 is segmentation 23 between an integer number bag 24A-24N (being referred to as bag 24).Each wraps the part of the encoded video segment 23 of 24 carryings, but and an amount of header information or other management data of carrying also.Video segment 23 is encoded to comprise a plurality of frames 25 of video data.But a plurality of encoded frames 25 of each bag 24 carrying.

Because the cumulative size of the bag 24A-24N of described series has a large amount of white spaces 26 greater than the size of encoded video segment 23 so wrap 24N at last, packetizer 16 is filled up described white space with filler.In other words, the last remainder of encoded video segment 23 is only filled up the part 28 of last bag 24N, stays white space 26, and it is wasted and fills up with filler.The amount of white space 26 changes and changes between fragment with the size of current video fragment.Yet, under each situation, will have the white space 26 of a certain amount usually, thereby cause invalid bandwidth usage.

Fig. 3 is explanation has the packetize of the video segment through adjusting packet rate that causes the packetize overhead that reduces according to the present invention figure.The figure of Fig. 3 meets the figure of Fig. 2 substantially.Yet in the example of Fig. 3, the code rate that rate controlled module 22 is adjusted the frame 25 of institute's carrying in the video segment 23 based on the size of each bag 24A-24N is to produce the encoded video segment 23 that more relevantly mates with the cumulative size of bag 24A-24N.Under this situation, coding module 20 is with through selecting more effectively to utilize the frame 25 in the wide code rate encode fragment 23 of band.For instance, rate controlled module 22 can be configured to revise the standard speed control algolithm so that the packetize overhead can be reduced.

Selecting when adjusting code rate, rate controlled module 22 is considered the size and (according to circumstances) amount by each bag of header or any other management information consumption of each bag 24A-24N.The code rate that rate controlled module 22 does not need a large amount of fillers through design to select to impel encoded video segment to be fit to an integer number bag 24.The number of the bag that the encoded video segment of carrying 23 is required is particular importance not.In fact, the feature of being paid close attention to is the size of fragment residue in the last bag.Under some situations, the minimizing of filler can directly cause the slight quality improvement when the frame of coding in the given fragment.Under other situation, can reduce quality slightly and be fit to bag and need not a large amount of fillings with the fragment of guaranteeing frame.

In general, rate controlled module 22 can be through design with the slight deficiency of supporting an integer number bag but not slightly surmount, so that rate controlled module 22 drives code rates with the big relatively residue of generation.Big relatively residue produces the white space of relatively small amount in the last bag again, thereby promotes the bandwidth usage of enhancing.As being described, in order to be offset (bias) rate control algorithm to produce big residue, rate controlled module 22 can (for example) considers that according to its accuracy the estimated variance of rate control algorithm, described accuracy are according to through minute coordination and be used for poor between the actual bit of previous encoded fragment.In addition, rate controlled module 22 can be considered the historical data of mean value of residue of the previous encoded fragment of designation number video data.

When considering the cumulative size of bag 24, rate controlled module 22 can be supposed the fixed size or the variant pack size of each bag.To suppose fixed size for purposes of illustration in the present invention, but not have restriction as the rate control techniques of extensive embodiment and description.Once more, the present invention is applicable on basis packet-by-packet or periodically or the variant pack size that changes on the intermittent basis.It is fixing to it should be noted that the number that wraps in a series of bags in order to the packetize video segment is not required to be, and will not fix usually, but can be according to the code rate of fragment to be encoded and complexity and change.Therefore, rate controlled module 22 mode that can reduce the packetize overhead of variable order the bag with fixed size is selected code rate.

Rate controlled module 22 can be used algorithm as described below substantially.For instance, for the example rate control algolithm that can be implemented by rate controlled module 22 is described, suppose digital video with each second as burst transfer, described burst can be known as super frame (sf).The number of k SF meta b is b (k).For packetize, suppose that further k the individual position of the b (k) among the SF need be split into bag, each bag has u position.In other words, each bag comprises the space (not comprising header and other management information) of u position, with the part that comprises one or more encoded frames of k SF of carrying.

Fill to form u in order to the last Bao Kejing in the bag of the described series of k SF of carrying.Therefore, the actual number that is used for the position B (comprise encoded video bit and through filler) of k SF through transmission is:

Wherein ceiling represents the ceiling function, and it produces the smallest positive integral more than or equal to variable.Therefore, when being applied to b (k)/u, the ceiling function produces the number of the needed bag of k SF of carrying, and B (k) is the sum of the bag meta of whole series, comprises video bit and filler.In order to reduce the packetize overhead, should use less filler.

Fig. 4 is the curve chart of the distribution of the encoded video segment size of explanation on the long video test sequence of getting in touch by some dissimilar contents (for example animation, music video, news and motion).Data among Fig. 4 are can be through the example of assessment with the historical data of the average residual thing of the fragment of estimation previous coding.Each fragment can be known as super frame (sf), and can suppose that it comprises burst in a second of video data.In the example of Fig. 4, with the nominal rate encoded video fragment of 256 kbps (Kbps).

In Fig. 4, vertical bar 30 is showed fragment size distribution, and modulus u=12 kilobit.Therefore,, suppose that each bag has 12 kilobits to hold at least a portion of SF for the purpose of Fig. 4, for example, one or more frames.X axle (residue) expression among Fig. 4 surpasses the number of the cumulative number purpose SF position of position available in an integer number bag.In other words, the x axle is represented in the last bag number of the residue position that will fill up by SF.Therefore, the x axle is also indicated indirectly and is added need to the SF position so that fill up the number of the filler of described bag fully, and the indication of packetize overhead is provided.

The number of video segment in y axle (freq) the expression theme video test sequence, the theme video test sequence has some positions of the residue level shown in producing on the x axle.For instance, the chart among Fig. 4 is illustrated in and has roughly 42 SF ' that are produced as zero residue in the cycle tests, because the number of available position accurately mates in the number of the position in each among those SF ' and the integer number bag.By contrast, existence is produced as roughly roughly 112 SF ' of the residue of 6000 positions in cycle tests.Because each bag provides 12 kilobits to hold SF, so remain half that fill up last bag in 6000 positions.Therefore, bag needs 6000 extra fillers to fill up the white space in the bag at last.Similarly, there are roughly 108 SF ' of the residue level with 9000 positions, so that last bag needs 3000 fillers to fill up whole 12 kilobits in the bag.

As can be seen from Figure 4, the residue between the fragment distributes near evenly distributing.Yet,, need adjust rate control algorithm applicatory to change above distribution in order to reduce the packetize overhead.In particular, need to change distribution so that it more closely meets the curve 32 among Fig. 4.By suitable rate controlled, curve 32 be provided as zero or the much bigger distribution of the SF residue that closely mates with the size of last bag (for example, 12Kbps).In other words, consistent with curve 32, need SF to have zero residue so that last bag is filled up fully and do not needed extra packet, or need SF to have very large residue so that the last filler that bag is almost filled up and needs are considerably less.

In addition, according to curve 32, the distribution with little SF ' to medium sized residue is reduced substantially.The leftmost side of curve is represented slightly to surmount, and slight deficiency is represented in the rightmost side of curve.Accurately coupling does not need filler.Slight not enough need considerably less filler, it is that the purpose of bandwidth efficiency is required.By contrast, slightly surmount a large amount of fillers of needs, and produce the remarkable waste of bandwidth.As an example, suppose the space that 12 kilobits are provided by each bag, the slight deficiency that is produced by the residue of 11000 video bits will only need 1000 fillers.By contrast, slightly surmount and to produce the very little residue of the undesirable a large amount of fillers of needs.For instance, slightly surmounting of 1000 video bits comprises 11000 fillers at last with needs.

By controlling code rate based on bag size, estimated variance and average residue, rate controlled module 22 (Fig. 1) can be adjusted SF and distribute with more residues of size greatly of the not enough a little bag size of generation, and the bag space of reducing the waste of being filled up by filler whereby.Can once set up one group of test data of the historical average residual thing data on a series of previous encoded fragments of expression as shown in Figure 4, and (for example) before using video processing equipment, assess described group of test data and be used for rate controlled to reduce the function of adjusting of packetize overhead with definition.Under this situation, the average that is used for the adaptability rate controlled can be treated one group of static historical data of the video segment disposed by video encoder 14 based on prediction.Perhaps, can upgrade the actual video fragment of historical data along with the time, so that the adaptability rate controlled dynamically changes according to the average residual thing on the previous encoded video segment sequence to be used for disposing by video encoder.

As an example, historical data can be used for the individual video treatment facility or be used for the video processing equipment of a classification or kind through foundation through foundation.Under arbitrary situation, can (for example) be written in the video processing equipment at the function of adjusting that " factory " will produce from the analysis of data.Perhaps or in addition, function can obtain one group of test data and periodically analyze described group of test data from actual video data, so that can periodically be upgraded or be calibrated to the actual video content of being disposed by video processing equipment in the operating period of video processing equipment 10.As another alternative method, as mentioned above, can (for example) on the sliding window of encoded video segment periodically or analyze average residual thing value substantially continuously produce real surplus thing value so that rate controlled module 22 is suitable for previous encoded video segment.

Historical data can be provided as the input of rate controlled module 22, for example, and as one group of data of indication residue value or as the pretreatment values of indication mean value.For described purpose, dynamic analysis functional that is used for average residual thing value is provided in the independent assembly of video encoder 14 or is integrated in the rate controlled module 22.Under arbitrary situation, video packetizer module 16 can be through the number of equipment with the required filler of indication packetize each encoded video segment, and therefore indicate the residue value of each video segment.

To further analysis and the processing that is used to define the historical data of adjusting function be described referring to the example of Fig. 4.For for example data set shown in Fig. 4, the probability distribution of curve 32 can following equation be a feature:

$p\left(x\right)=1-A+A\underset{n=-\∞}{\overset{\∞}{\mathrm{\Σ}}}{N}_x(\mathrm{\μ}-n,{\mathrm{\σ}}^2),0\≤\mathrm{\μ}\≤1,0\≤x\≤1,---\left(2\right)$

Wherein x is the SF residue on the x axle, and A is the model parameter that can select based on simulation, and:

$N_x(\mathrm{\μ},{\mathrm{\σ}}^2)=\frac{1}{\mathrm{\σ}\sqrt{2\mathrm{\π}}}\exp (-\frac{{(x-\mathrm{\μ})}^2}{2{\mathrm{\σ}}^2})---\left(3\right)$

Be average and the σ that has at the μ place ²The normal distribution N of residue x of variance _x(μ, σ ²), variances sigma wherein ²Indication is by the standard speed control algolithm of the rate controlled module 22 of encoder 14 and the therefore variance of the bits of coded of the accuracy decision (shape) of rate control algorithm.Can be based on being used for the real data of rate control algorithm and selecting variance through obtaining, but or estimate variance.It should be noted that under situation about being without loss of generality, above probability function is normalized to [0,1] from [0, u].

For the normal distribution in the example of Fig. 4,68.3% probability is in [σ, the σ] about μ, and 95.4% probability is in [2 σ, 2 σ] about μ, and 99.7% probability is in [3 σ, 3 σ] about μ.Therefore, if supposition σ≤0.5, so above distribution can be approximately:

p(x)≈1-A+AN _x(μ，σ ²)+AN _x(μ-1，σ ²) 0≤μ≤1 0≤x≤1。(4)

In order to realize minimum packetize overhead, the average μ of following maximization SF residue x:

$E\left(x\right)\&ap;\frac{1-\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="A20078004018600191.png"\; state="\{\{state\}\}">A</figure-callout>}}{2}+A\underset{x=0}{\overset{1}{\&Integral;}}x{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}+A\underset{x=0}{\overset{1}{\&Integral;}}x{N}_x(\mathrm{\&mu;}-1,{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$\&ap;\frac{1-\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="A20078004018600191.png"\; state="\{\{state\}\}">A</figure-callout>}}{2}+A\underset{x=-\&infin;}{\overset{1}{\&Integral;}}x{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}+A\underset{x=0}{\overset{\&infin;}{\&Integral;}}x{N}_x(\mathrm{\&mu;}-1,{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$\&ap;\frac{1-\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="A20078004018600191.png"\; state="\{\{state\}\}">A</figure-callout>}}{2}+A\underset{x=-\&infin;}{\overset{1}{\&Integral;}}x{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}+A\underset{x=1}{\overset{\&infin;}{\&Integral;}}(x-1)N_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$\&ap;\frac{1-\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="A20078004018600191.png"\; state="\{\{state\}\}">A</figure-callout>}}{2}+A\underset{x=-\&infin;}{\overset{1}{\&Integral;}}x{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}+A\underset{x=1}{\overset{\&infin;}{\&Integral;}}x{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}-A\underset{x=1}{\overset{\&infin;}{\&Integral;}}{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$=\frac{1-\mathrm{<figure-callout\; id="2"\; label="A"\; filenames="A20078004018600191.png"\; state="\{\{state\}\}">A</figure-callout>}}{2}+A(\mathrm{\&mu;}-\underset{x=1}{\overset{\&infin;}{\&Integral;}}{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx})---\left(5\right)$

Therefore, in order to maximize E (x), maximize with minor function:

$f\left(\mathrm{\&mu;}\right)=\mathrm{\&mu;}-\underset{x=1}{\overset{\&infin;}{\&Integral;}}{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}---\left(6\right)$

Foundation can be from the standard deviation of the rate control algorithm the use of simulated estimation, and the value of μ can be in order to further fine tuning rate controlled target in the equation (6), and (for example) arrives as shown in (12) as following equation (9).

Fig. 5 marks and draws the curve chart of above function f (μ) under σ=0,0.1,0.2,0.3,0.4 and 0.5 situation for the purpose that further specifies.The curve chart of Fig. 5 is showed for by σ ²Difference in the different variance functions of expression.Therefore, in order to realize the required distribution of SF residue, suppose known σ applicable to specific encoder 14 ², can select different f (μ) curve to use for rate controlled module 22.For σ 〉=0.39894228, realize the maximum of f (μ) at μ=1 place, and, when satisfying following criterion, realize the maximum of f (μ) for less σ:

$0=\frac{d}{\mathrm{d\&mu;}}f\left(\mathrm{\&mu;}\right)=1-\frac{d}{\mathrm{d\&mu;}}\underset{x=1}{\overset{\&infin;}{\&Integral;}}{N}_x(\mathrm{\&mu;},{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$=1-\frac{d}{\mathrm{d\&mu;}}\underset{x=1-\mathrm{\&mu;}}{\overset{\&infin;}{\&Integral;}}{N}_x(0,{\mathrm{\&sigma;}}^2)\mathrm{dx}$

$=1-N_x(0,{\mathrm{\&sigma;}}^2){|}_{x=1-\mathrm{\&mu;}}---\left(7\right)$

Given above content then has following formula:

Above expression formula produces should be through selecting the best μ with the required distribution of the SF residue of realizing reducing best the packetize overhead.Therefore, can be continuously or periodically deflection rate control module 22 selected code rates are so that best μ, or a certain μ in the predetermined tolerance limit of best μ can be maintained substantially.Once more, can or pass by in time through periodically or upgrade continuously based on the static historical data that characterizes before the video segment by video encoding module 20 codings to be used in fact obtaining μ by the dynamic history data of the video segment of video encoding module 20 codings.

Fig. 6 marks and draws μ in the above expression formula (8) for the purpose that further specifies _OptAnd the curve chart of the relation between the σ.The accuracy of the rate control algorithm in being used by rate controlled module 22 deviations that settles the standard.If rate control algorithm can be adjusted SF size modulus u histogram so that 0.2≤σ≤0.25 should select the working point of μ to be roughly 0.77 so.

For the frame level of rate control algorithm, intended target frame sign before coded frame usually.Be F if suppose this target frame size _t, after the coding of frame, the actual frame size is F so _aAt F _tWith F _aBetween to have mismatch and ratio therebetween usually be the variable that slowly changes.Can following expression F _tWith F _aBetween ratio:

$\mathrm{\&gamma;}=\frac{F_a}{F_t}.---\left(9\right)$

Can estimate ratio γ by following use linear weighted function function:

$\mathrm{\&gamma;}\&LeftArrow;(1-\mathrm{\&alpha;})\mathrm{\&gamma;}+\mathrm{\&alpha;}\frac{F_a}{F_t}.---\left(10\right)$

Wherein α is the weighting factor with persistent value of expression current video content.The rate control algorithm of being implemented by rate controlled module 22 (Fig. 1) can be configured to following with next frame sign target execution frame horizontal rate control:

S wherein _SFBe the SF size of being estimated by rate controlled module 22, rounding off is round-off function (rounding function), and μ is the kurtosis estimation of from rate adaptation after SF size modulus u.By using above rate control algorithm, with the rate control algorithm contrast that does not have rate adaptation, rate controlled module 22 can realize the low overhead of filling.

In above algorithm, rate controlled module 22 based target frame sign F _tAnd adjustment frame level code speed.Rate controlled module 22 is again based on estimated SF size and the big or small u of bag, and the ratio γ of actual frame size and target frame size and average μ and determine the target frame size.In this way, poor between rate controlled module 22 compensation target frame code rates and the actual frame code rate.In operation, the rate control algorithm of being implemented by rate controlled module 22 is set the rate controlled target.For the height of keeping last bag takies, can be in fine tuning rate controlled target periodically or on the continuous foundation on the modulus meaning, to keep best μ substantially.Therefore, can be by " insertion " any existing rate control algorithm with fine tuning rate controlled target so that the algorithm that last bag reaches average maximum richness is realized this fine tuning of rate controlled target.

It should be noted that by the SF horizontal rate of rate controlled module 22 application and adjust and with picture group (GOP) or to cut the error resilient (error resilience) of level (slice level) rate control algorithm cooperation so that interpolation to be provided.For instance, if last frame to be encoded has significantly little codec complexity and have big remaining bandwidth in the modulus size of u=12 kilobit among the SF, so coded present frame can be error resilient and adjusts and cut size.For instance, mode decision also can be through adjusting to utilize remaining bit to improve error resilient.For instance, more macro zone blocks can be intra-encoded but not interframe encode with from may channel impairments recovering.Under this situation, extra bits of coded replaces filler.In other words, the number of the relative filler of number of the bits of coded in the last bag of this techniques permit increases to some extent.

Fig. 7 is the flow chart that the method for video coding that is used to reduce the packetize overhead according to an aspect of the present invention is described.Video processing equipment 10 that can be by Fig. 1, and the coding module 20 of (in particular) video encoder 14 and rate controlled module 22 and packetizer 16 are implemented the method for Fig. 7.As shown in Figure 7, coding module 20 is from the fragment (70) of video source 12 receiving video datas.The size (72) of the bag that the predetermined bag size that coding module 20 is used by packetizer based on information that packetizer provided or based on supposition and determining is produced by packetizer 16.

In case determine the bag size, rate controlled module 22 is just selected code rate (74) based on the bag size.Coding module 20 is used selected code rate and is come coding video frequency data fragment (76), and packetizer 16 packetizes are by the encoded video data fragment (78) of coding module generation.Process proceeds to next fragment (90) subsequently and repeats.Therefore, rate controlled module 22 is selected to be used for the code rate of each new video fragment to be encoded based on the bag size adaptively, and reduces the packetize overhead whereby.

Fig. 8 is that explanation uses historical data to adjust the flow chart of video coding rate in the method for Fig. 7.In general, be used to select the additional detail of code rate (74) in the example of Fig. 8 key diagram 7.As shown in Figure 8, for the rate controlled in response to the actual video data of being handled by video processing equipment 10 during operation, rate controlled module 22 can obtain or access historical data (82).The mean value of the residue of the previous encoded fragment of historical data designation number video data, and can be similar to the data of being marked and drawed in the curve chart of Fig. 4.Once more, the mean value of previous encoded video segment can obtain on the sliding window of encoded video segment, and can be analyzed and be calculated by the rate controlled module 22 of another assembly in the video encoder 14.

In case estimate the variance (84) of rate controlled module 22 employed rate control algorithms, rate controlled module 22 just obtains average residual thing value (86) and is offset code rate to use above equation (6) to increase the mean size (88) of the residue of following fragment with (for example) from historical data.As previously mentioned, can select variance, maybe can estimate or suppose variance based on obtaining to be used for the real data of rate control algorithm.In case adjust code rate with maximization or optimization average residual thing value, just it is process illustrated in fig. 8 can be carried out repetition at continuous videos fragment to be encoded, indicated as loop 89.

Can in hardware, software, firmware or its any combination, implement technology described herein.Can be in video encoder-decoder (CODEC) the various assemblies of embodiment such as video encoding module 20 and rate controlled module 22.If be implemented in the software, technology can be at the machine-readable medium that comprises program code or instruction so, and it carries out one or more in the method mentioned above when carrying out in the machine in encoded video sequence when described program code or instruction.Under described situation, computer-readable media can comprise for example random-access memory (ram), read-only memory (ROM), nonvolatile RAM (NVRAM), Electrically Erasable Read Only Memory (EEPROM), flash memory and the analog of Synchronous Dynamic Random Access Memory (SDRAM).

Program code or instruction can computer-readable instruction form be stored on the memory.Under described situation, for example the processor of DSP can be carried out and be stored in the instruction in the memory so that carry out one or more in the technology described herein.Under some situations, can carry out described technology with the DSP of speech coding process by calling various nextport hardware component NextPorts.Under other situation, video encoder can be embodied as microprocessor, one or more application-specific integrated circuit (ASIC)s (ASIC), one or more field programmable gate arrays (FPGA) or a certain other hardware-combination of software.

Various aspects have been described.These and other aspect within the scope of the appended claims.

Claims (23)

1. method for video coding, it comprises:

Determine size in order to the bag of the encoded fragment of packetize digital of digital video data; And

Selection is used for the code rate of the described fragment of digital of digital video data based on described bag is big or small.

2. method according to claim 1, it further comprises:

Use described selected code rate to come the described fragment of encoding digital video data; And

The described encoded fragment of coming the packetize digital of digital video data via a series of bags,

Wherein the described encoded fragment of digital of digital video data comprises the residue of the part of the last bag in the bag of filling up described series.

3. method according to claim 2, it further comprises based in order to the historical data of the mean value of the described residue of the previous encoded fragment of the estimated variance of the rate control algorithm of the described fragment of encoding digital video data and designation number video data and select described code rate.

4. method according to claim 3 is wherein selected described code rate to comprise and is selected described code rate to increase the size of described residue, reduces the number of filling up the required filler of described last bag whereby.

5. method according to claim 1, each in the wherein said bag has the fixed packet size.

6. method according to claim 1, wherein said fragment comprises a plurality of frames, and selects described code rate to comprise based on the variable of the difference between the actual size of the described previous encoded frame of the target sizes of the previous encoded frame of designation number video data and digital of digital video data and adjust described code rate.

7. method according to claim 1, it further comprises each code rate of a plurality of extra fragments of selecting to be used for digital of digital video data based on described bag size, use described selected code rate to come the described extra fragments of encoding digital video data and the described extra encoded fragment of packetize digital of digital video data.

8. digital video coding equipment, it comprises the rate controlled unit, the size in order to the bag of the encoded fragment of packetize digital of digital video data is determined in described rate controlled unit, and selection is used for the code rate of the described fragment of digital of digital video data based on described bag is big or small.

9. equipment according to claim 8, it further comprises:

Coding module, it uses described selected code rate to come the described fragment of encoding digital video data; And

The packetize module, the described encoded fragment that it comes the packetize digital of digital video data via a series of bags,

Wherein the described encoded fragment of digital of digital video data comprises the residue of the part of the last bag in the bag of filling up described series.

10. equipment according to claim 8, wherein said rate controlled unit is based in order to the historical data of the mean value of the described residue of the previous encoded fragment of the estimated variance of the rate control algorithm of the described fragment of encoding digital video data and designation number video data and select described code rate.

11. equipment according to claim 10, wherein said rate controlled unit select described code rate to increase the size of described residue, reduce the number of filling up the required filler of described last bag whereby.

12. equipment according to claim 8, each in the wherein said bag has the fixed packet size.

13. equipment according to claim 10, wherein said fragment comprises a plurality of frames, and described rate controlled unit is based on the variable of the difference between the actual size of the described previous encoded frame of the target sizes of the previous encoded frame of designation number video data and digital of digital video data and adjust described code rate.

14. equipment according to claim 8, each code rate of a plurality of extra fragments of digital of digital video data is selected to be used in wherein said rate controlled unit based on described bag size, described equipment further comprises and uses described selected code rate to come the packetize module of the described extra encoded fragment of the coding module of the described extra fragments of encoding digital video data, packetize digital of digital video data.

15. processor that is used for encoding digital video data, described processor is configured to determine the size in order to the bag of the encoded fragment of packetize digital of digital video data, and selects to be used for the code rate of the described fragment of digital of digital video data based on described bag size.

16. a video encoder, it comprises:

Be used for determining device in order to the size of the bag of the encoded fragment of packetize digital of digital video data; And

Be used for selecting being used for the device of code rate of the described fragment of digital of digital video data based on described bag size.

17. equipment according to claim 16, it further comprises:

Be used to use described selected code rate to come the device of the described fragment of encoding digital video data; And

Be used for coming the device of the described encoded fragment of packetize digital of digital video data via a series of bags,

Wherein the described encoded fragment of digital of digital video data comprises the residue of the part of the last bag in the bag of filling up described series.

18. equipment according to claim 17, it further comprises and being used for based in order to the historical data of the mean value of the described residue of the previous encoded fragment of the estimated variance of the rate control algorithm of the described fragment of encoding digital video data and designation number video data and select the device of described code rate.

19. equipment according to claim 18, it further comprises and is used to select described code rate to reduce the device of the number of filling up the required filler of described last bag whereby with the size that increases described residue.

20. equipment according to claim 16, each in the wherein said bag has the fixed packet size.

21. equipment according to claim 16, wherein said fragment comprises a plurality of frames, and described equipment further comprises and is used for based on the variable of the difference between the actual size of the described previous encoded frame of the target sizes of the previous encoded frame of designation number video data and digital of digital video data and adjusts the device of described code rate.

22. equipment according to claim 16, its further comprise a plurality of extra fragments of being used for selecting being used for digital of digital video data based on described bag size each code rate device, the described extra fragments that is used to use described selected code rate to come encoding digital video data device and be used for the device of the described extra encoded fragment of packetize digital of digital video data.

23. a machine-readable medium, it comprises the instruction that is used for video coding, and wherein said instruction impels machine when carrying out:

Determine size in order to the bag of the encoded fragment of packetize digital of digital video data; And

Selection is used for the code rate of the described fragment of digital of digital video data based on described bag is big or small.

Images