AU2021203777B2

AU2021203777B2 - Coding systems

Info

Publication number: AU2021203777B2
Application number: AU2021203777A
Authority: AU
Inventors: Jiancong Luo; Jiheng Yang; Peng Yin; Lihua Zhu
Original assignee: Dolby International AB
Current assignee: Dolby International AB
Priority date: 2007-04-18
Filing date: 2021-06-08
Publication date: 2023-07-06
Anticipated expiration: 2028-04-07
Also published as: AU2021203777A1; AU2020203130B2; AU2020203130A1; AU2015203559A1; AU2015203559B2; AU2017258902A1; AU2017258902B2

Abstract

In an implementation, a supplemental sequence parameter set ("SPS") structure is provided that has its own network abstraction layer ("NA) unit typeand allows transmission of layer-dependent parameters for non-base layers in a SVC environment. The supplemental SPS structure also may be used for view information in an MVC environment. In a general aspect, a structure is provided that includes (1) information (1410) from an SPS NAL unit, the information describing a parameter for use in decoding a first-layer encoding of a sequence of images, and (2) information (1420) from a supplemental SPS NAL unit having a different structure than the SPS NAL unit, and the information from the supplemental SPS NAL unit describing a parameter for use in decoding a second-layer encoding for the sequence of images. Associated methods and apparatuses are provided on the encoder and decoder sides, as well as for the signal.

Description

CODING SYSTEMS CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a divisional application of Australian Patent Application No 2020203130, which is a Divisional of Australian Patent No 2017258902, which is a divisional of Australian Patent No. 2015203559 which is a divisional of Australian Patent No 2012238298, which is a divisional of Australian Patent No 2008241568, the entire disclosure of each of which is incorporated herein by reference. TECHNICAL FIELD

At least one implementation relates to encoding and decoding video data in a scalable manner.

BACKGROUND

Coding video data according to several layers can be useful Iwhen terminals for which data are intended have different capacities and therefore do not decode a full data stream but only part of a full data stream. When the video data are coded according to several layers in a scalable manner, the receiving terminal can extract from the received bit-stream a portion of the data according to the terminal's profile. A full data stream may also transmit overhead information for each supported layer, to facilitate decoding of each of the layers at a terminal.

SUMMARY

According to one aspect, there is provided herein a method comprising:

accessing information from a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first-layer encoding of an image in a sequence of images;

accessing supplemental information from a supplemental SPS NAL unit having an available NAL unit type code that is a different NAL unit type code from that of the SPS NAL unit, and having a different syntax structure than the SPS NAL unit, and the supplemental information from the supplemental SPS NAL unit describing a parameter for use in decoding a second-layer encoding of the image in the sequence of images; and

1B

decoding the first-layer encoding, and the second-layer encoding, based on, respectively, the accessed information from the SPS NAL unit, and the accessed supplemental information from the supplemental SPS NAL unit.

According to a another aspect, the present invention provides a decoder comprising: a parsing unit to receive

information in a first parameter set contained in a first network abstraction layer unit, the first parameter set being a syntax structure which contains syntax elements that apply to zero or more entire coded video sequences, and the information describing a parameter for use in decoding multiple layers of the video sequences, and

supplemental information contained in a second network abstraction layer unit, the second NAL unit having a different syntax structure than the first network abstraction layer unit and corresponding to one layer of said multiple layers; and

a decoding unit to decode said one layer of said multiple layers based on, the accessed information from the first NAL unit and the accessed supplemental information from the second NAL unit.

According to a another aspect, the invention provides a method comprising:

receiving information in a first parameter set contained in a first network abstraction layer unit, the first parameter set being a syntax structure which contains syntax elements that apply to zero or more entire coded video sequences, and the information describing a parameter for use in decoding multiple layers of the video sequences;

receiving supplemental information contained in a second network abstraction layer unit, the second NAL unit having a different syntax structure than the first network abstraction layer unit and corresponding to one layer of said multiple layers; and

decoding said one layer of said multiple layers based on, the accessed information from the first NAL unit and the accessed supplemental information from the second NAL unit.

1C

According to a yet another aspect aspect, the present invention provides an encoder comprising:

a generation unit to generate

supplemental information in a second network abstraction layer unit, the second NAL unit having a different syntax structure than the first network abstraction layer unit, the second NAL unit corresponding to one layer of said multiple layers; and

an encoding unit to encode said one layer of said multiple layers based on the generated information for the first NAL unit and the generated supplemental information for the second NAL unit.

According to a further aspect, the present invention provides a method comprising:

generating information in a first parameter set contained in a first network abstraction layer unit, the first parameter set being a syntax structure which contains syntax elements that apply to zero or more entire coded video sequences, and the information describing a parameter for use in decoding multiple layers of the video sequences;

generating supplemental information in a second network abstraction layer unit, the second NAL unit having a different syntax structure than the first network abstraction layer unit, the second NAL unit corresponding to one layer of said multiple layers; and

encoding said one layer of said multiple layers based on the generated information for the first NAL unit and the generated supplemental information for the second NAL unit.

According to a yet another aspect, the present invention provides a signal having

ID

decodin lalaral eters, the silal for:] atted to co lise:

information fro: a first laraleter set contained in a first network abstraction la-r unit, the first laraleter set Jein a s-hta structure which contains s-htal elements that a:]I to zero or :ore entire coded video sequences, and the information descriinl a laraleter at least for use in decoding aultile la-ers of the video seluencesa

sulielental information fro: a second NAL unit havinla different s-htaa

structure than the first networlalstraction la-ir unit, the second NAL unit

corres-bndin ato one la-er of said : ultile lai-rsland

data relresentin a said one la-er of said : ultile la-rsa

.0 Accordina to one aslIect, there is :ovided herein an i ale decoding :ethod col lisina accessin: information from a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first]]a-Or encoding of an i ale in a sequence of il aleslaccessinl sulielental information fro: a sulielental SPS NAL unit having an availal]e NAL unit t-e code that is a different NAL .5 unit t-e code fro: that of the SPS NAL unit, and having a different s-htalstructure than the SPS NAL unit, and the sulielental information fro: the sulielental SPS NAL unit descriilnl a llaraleter for use in decoding a second]]a-r encoding of the ilale in the selluence of i alesland decoding the first]]a-r encodin, and the second]]a-r encodin], leasedd on, reslllectivelal the accessed information fro: the SPS NAL unit, and the accessed '0 sulielental information fro: the sulielental SPS NAL unit, wherein the sulielental SPS NAL unit comprises video usability information ("VUl") parameter and hypothetical reference decoder ("HRD") parameters.

According to another asllect, there is providedd herein an i alL :Irocessina slatel co lisin ll larsina unit to access information fro: a seluence llaraleter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first]a-er encoding of an i ale in a sequence of i ales and to access sulielental information fro: a sulielental SPS NAL unit having an availa]e NAL unit t-e code that is a different NAL unit t-e code fro: that of the SPS NAL unit, and having a different s-htalstructure than the SPS NAL unit, and the sulielental information fro: the sulielental SPS NAL unit descrilnl a llaraleter for use in decoding a second]]a-or encoding of the i ale in the seluence of i alesland a decoder to decode the first]]a-or encodin:, and the second]]a-or encodin, lased on, reslectivela the accessed information fro: the SPS NAL unit, and the accessed sulielental information fro: the sulielental

IE

SPS NAL unit wherein the suil]elental SPS NAL unit collises video usalitlinforlation ("VUl") parameter and hypothetical reference decoder ("HRD") parameters.

Accordin: to a further aslect, there is :Iovided herein a nonransitora co:luter]readale ]ediu: containin: instructions, which if elcuted a collutina slste], causes the collutina slstel to lerfor: a :ethod colu isinaccessina information fro] a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a zIaraleter for use in decodin: a firstalr encodin: of an ilale in a seluence of ilales] accessing suillelental information fro] a sulelental SPS NAL unit havin: an availa]e .o NAL unit t-e code that is a different NAL unit tle code fro] that of the SPS NAL unit, and havin: a different s-htalstructure than the SPS NAL unit, and the suillelental information fro] the sulmental SPS NAL unit descrinl a laraleter for use in decodin: a second] lazr encodin: of the ilaL in the seluence of ilaLesland decodin: the firstler encodin, and the secondlaer encodin, llased on, resectivela the accessed information fro] the SPS NAL unit, and the accessed suillelental information fro] the suil]elental SPS NAL unit, wherein the suil]elental SPS NAL unit collises video usa]Ilit] information ("VUl") parameter and hypothetical reference decoder ("HRD") parameters.

Accordin: to a generall aslct, information is accessed fro] a seluence laraleter set ("SPS") network abstraction layer ("NAL") unit. The information describes a laraleter for use in decodin: a firstalr encodin: of a sequence of ilales] Inforlation is also accessedfro a sulelental SPS NAL unit havin]a different structure than theSPS NAL unit:]The information fm thesupplemntalSPS NAL unit describs a parameter for use in decoding a secondlayer encoding of the sequence of images, A decoding of the sequence of images is generated based on the firstlayer encoding,t esecond-layer encoding.the accessedinformation from the SPS NAL unit, and the accesednformation from the supplementa SPSNALnt. According to another genera aspects a syntax structure is used that provides for decoding a sequence ofimsges In multiple layers The synta structure includes:syntax foran SPS NALunitincluding information describing a parameter for use in decoding a first-layer encoding of a sequence of images, The syntax stucture also includes syntax for asuppiemrental SPS NAL unit having a different sticture than the SPS NAL unit The supplemental SPS NAL unit includesinformation describing a parameter for use in deeding aseondlayerencoding of thesequence ofimages. A decoding of the sequence of images may be generated based on the first layer encoding, the seond4ayer encoding, the information from the.SP NAL unitandthe infonnationfronm the supplemental $PS NAL.unit. According to another general aspect, asignal is formatted to Include information from an$PSNAL unit. The information describes a parameter for use in decoding a firstlayer encoding of a sequence of images. The signals further formatted to include information from a supplemental SPS NAL unit having different structure thai the SPS NAL unit. The informationfom the supplemental SPS NAL unit describes a parameter for use in decoding a second-layer encodingo thesequence ofimages Accordingto another general aspect, aSPS NAL unit is generatedthat includes information descibing aparameterforuse indecoding a first-layer encoding of a sequence of images. A supplemental SPS NAL unit is generated that has a different structure than the PS. NAL unit. The supplementalSPS NALunit includes information that describes a parameter for use in decoding:a second-layer encoding of the sequence of images, A set of data is provided that includes the first-1ayer encoding of the sequence of Imagesthe sepond-layerencodingofthesequenceofimagesetheSPSNAL unit, and the spplementalSPSNALunit

According to another general aspect, a syntax structure is used that provides for encoding a sequence images in multiple layers. The syntax structure includes syntax for anSPS NAL unit. TheSPSNALunincludes iIforrmation that describes.a parameter for use in decoding a first-layer encoding of a sequence of images., The syntax structure includes syntaxfor a supplemental SPS NAL unit. The supplemental SPS NAL unit hasa different structure than the SPS NAL unit. Thesupplemental SPS NALunit includes information that describes a parameter for use indecodingasecondayer encoding of the sequence of Images, A set of data ay be provided that includes the first-layer encoding of the sequen of imagesthe seondayer encoding of the sequence ofimagesthe: SPS NAL unit, and thesupplemental $PS NAL unit Accordingtoanother general aspectfirstlaye-dependent information is accessed in a first normative parameter set The accessed first layer dependent information is for use in decoding a first-layerencoding of a sequence of images, .Second layer-dependentinformations accessed in a second normative parameter set The second nomati ve parameter set has a different structure than the first normative parameter set The accessed second layerependent information for use in deoding a second'layer encoding of the sequence of images, The sequence ofImages is decoded basedon one or more of theaccessed first.ayerdependent information or the accessed secondlayerependent information. According to another general aspect, a first normative parameter set is geherated that includes firstlayer-dependent information. The first layer dependent.informatioi foruse i decoding a first-layer encoding of a sequence of images. A second normtive parametersetis generated having different structure than the first normative parameteset, The second normative parameter:set includessecondlayer-deeident information for use in decoding a second-layer encoding of the seqpncef images. A set of data is provided that includes the firstnoativeparameter set and the second normativeparameter set.

The details of one or more.implementations are set forth in the accompanying dawings and the desciptionbelow, Even ifdescribed in one partiular manner, it should be lear thatimplementations may be configured or embodied in various manners, For example, an implementainmaybe S performed as amethod,or embodied asanaapparatus,seuch as for examplean apparatus configred to perform a set of operations or an apparatus storing instructions for performing a set of operadons, or embodied in signal. Other aspects and features wil become apparent from the followingdetailed description considered in conjunction with the accompanying drawings and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS FIG .is ablock diagramforan implementationofan encoder. FIGlaisbockdiagramo r anotherimpementation of an encoder. FIG 2 is ablock diagm foran implementation of decode FIG.a is a block diagram foranotherireentation ofa# decoder FIG. 3 is aStructure of an implementation of a SingeLayerSeuence Parameter Set("SPS") Network AbstactionLaer (NAL")unit. FIG. 4 is a block view of an oxmple of portions of a data stream illustratinguseof'an SPSNALunitL FIG, is a structure of animpementation of a supplemental SPS ("SUP SPS¶)NAL unit. FIG 6 is an implementaon of an organizational hierarchy among an SPS unit and multiple SUP SPSunits, FIG.7 is structure ofanother implementation ofia SUP SPSNAL unit FG. 8 is a functional view of an implementation of a scalable video oder that generates SUP SPS units. FIG. 9 is ahierarhicalviewofanimplementation ofthe generation of a datastream that contains SUPSPS units. FIG. 10 is a block view of an example of a data stream generated by the implementation of FIG, 9 FIG 11 is:a block diagrarnofaimplemetationof aneneoder

FIG 12 is a block diagram of another implementation f arenoder. FIG. 1 isa flow hart ofanimplementationofan encodingprocess used by the encoders of FIGS 11or 12 FIG. 14isa block view of sanxample of a data steam generated by the pocess.of FIG. 13 FIG. 151s a block diagramof implementation of a decoder. FIG. 16 is a blockdagram of another implementation of a decoder. HG17 is a flow chart ofan implementation of a decoding process used by the decoders ofFiGS. 15or16:

DETAILED DESQRIPTIQN Several video coding standardsexist today that cancodevyideo data according to different layers andor profiles. Among them, one can.cite: t{264/MPEG4 AVC (the "AVG standard"), also referenced as the International Organization for StandardizationInterationa Electrotechnical Commission (ISO/IEO) Moving PictureExperts Group4 (MPEG-) Part 10 Advanced Video Coding (AVC) standard/Intemtional Telecommunication Union, Telecommunication Sector (ITUT) H.264 recommendation. Additionally,extensions tothe AVC standard exist. A first such extension is a stable video coding (!SVC') extpsion (Annex G) referred to as HI264/MPEGA4AVC scalable videocoding extAension(the "SVC extensionm, A second such extension is-amulti-viewvideocoding(MVO) extension (Annex H) referred to asH.264/MPEG4 AVC, MVC extension (the "MVC extension"). At least one implementation described l this disclosure may be used with the AVC standard aswellasthe SVC and MVC extesions, The implementation provides a.supplermental("SUP") sequence parameter set ("SPS') network abstraction layer("NAL") unit having a different NAL unit type than SPS NAL units An SPS unit typically:includesbut need noinformation for at least a single layer Further, the SUP SPS'NAL. unit includeslayer dependent information for atleast one additional layer. Thus byaccessing

SPS and SUP SPS units,a deoder hasavailable ertaiandtypill 011) layeependent informtionneededtodecode abit stream Usingthis implementation in an AV system,the SUP SPS NALu iits need notbe transmitted, and a single-layer SPS NAL unit (as described below)may be transmitted. Using this implementation in an SVC(Or MVC) system, the $VP SPS NAL unt(s) may be transmitted for the desired additional layers (or viewsl, in addition toan SPS NAL unit Using this: implementation in a system intdig both AVC- compatible decoders and SVC-compatible (or MVC-compatible) decoders, the AVC-compatible decoders may ignorethe SUp SPS NAL urits by detecting the NAL t type In each case, efficiency and compatibility area chieved. The above implementation also provides benefits for systems (standards or otherwise) thatimpose a requirement that certain layers share header informationsuch as for example, an SPSor particular information typically carried in an SP$ For exaMple, If a baselayer and its composite temporal layers need to share an SPS, then thelayer-dependent information cannot be transmitted with the shared SPS, howeverthe SUP SPS provides mechanism for transmitting the layerAdepedet information. The SUP SP4 of various implementations also provides aneficienc advantage in'that the SUP SPS need not include,and therefore repeat, all of the parameters in the SP$ The SUP SPS Will typically be focused on the layer-dependent parameters. However, various implementations included SUPSSstructure that includes non-layer-dependent parameters, or even repeatsall of an SPSstaructure. Various implementatiorns tel1te to the SVC extension. The SVC extension proposes the transmission of video data according to several spatial levels, temporallevels,and quality levels, For one spatiallevel, one can code according to several temporal levels, and for each temporallevel according to several qualitylevels, Thereforejwhen there are defined m spatial levels n temporallevels,and Oquality levels,thevideo data can be codedaCording to mnO different cominations-: These combinations are referred to as layers, or as interoperbility points (IOPsw) According to the decoder (also referred to as the receiver or the lien)capabilities, different layers may be transmitted up to a certain layer corresponding to the maximum of the client capabilities: As used heren"layerWdependentinformation referstoinformation that 5 relates specifically to a singlelayer. That:is as the name suggests, the informationis dependent upon the specific layer. Such information need not necessarilyvaryfromlayer to layer, but would typically be provided sepaately for each layer. Asused herein "hIgh level syntax refers tosyntax presentinthe bittreamthatresidesierarchically above the macroblock layer. For exampehighlevelsyntaxas used herein, may refer to, but is not limitedto syhtaX at the slice headerlevI Supplemental Enhancement information (SED leve Picture Parameter Set (PPS)levelSequence Parameter St(SP) level.andNetwork AbstractionLayer (NAL) unit headerlevel Referring to F exemplary SVC encoder is indicatedgenerally by the reference humeral 100. The SVC encoder 100 may also be usedfor AVC encoding, that is, for a single layer (for eXample,baselayer). Further, the SVC encoder 100 may be used forMVC encoding as-oneof ordinary skill in heart will appreciate. Forexample, various componentsofthe$VC .20 encoder 10, or variations of these components, may be used In encoding multiple views. A first output of a temporal decomposition module 142.isc onnected in signal communication with a first input of an intraprediction for intra block module 146. A second output of the temporal decomposition module 142 is connected in sial communication with first input of a motion coding module 144, An output of the intra prediction for intra block module 146 is connectedA signal communication withan input ofatransformientropy coder (signalto noise ratio (SNR) scalable) 149 A firstoutput of the transformlentropy coder 149 is connected signal communicationwithafirst inputof a multiplexer170. First output of atemporal decomposition module 132 is connected in signal-communication with a first input. of an intra predOtion for intra block module 136. A second output of the temporal decomposition module 132 is connected in signal communicationvith a first input of a:motion doding module 134. An output of the intraprediction for intra block module 136 is connected inesignalcommunication with an inputof a transform/entropy coder 5 (signal to noise ratio (SNR) scalable) 139. A firstoutput of the transformlentropyoder 139 is connectedin signalcommunicationWith a first inputofamultiplexer 170., A::second output of the tratforientropy coder149 is connected in signal communication with an input of a 20Dspatialinterpolation module 138. An output of 20 spatial interpolatin module 138 is connected insignal communication with a second input ofthe intra predicion for intra block module 136 A second output ofthe motioncoding module 144 isconnected in signal communication with aninputof the motion coding module(IT34 A first output of a temporal decomposition module 122 is connected in signal communicationwith firstinput of an intra predictor126.Asecond output of the temporal decomposition module 22 is connected in signal communication:ith astinput of a motion coding module 124. Anoutput-of the intra predictor 126 Is connected in signal communication with an input of a transformentropy coder(signal to noise ratio (SNR) scalable) 129,. An output oftheransformlentropy coder129 connectedIn signal conmunication withl a first input;ofamultiplexer 170, A second output of the transformlentropy coder 139 is conneted in signal communication with an input of a 20 spatia interpolation module 12 An output of 20 spatial interpolation module 128, is connected in signal communication ith a second input of the intrapredictor module 126 A second output of the motion coding module 134 is connected in signal communication withan inptof themotion coding modu ,124, First out uof the motion coding module 124, a first output ofthe motion coding module 134, and a first ottputof the motion coding module 144 3 are each connected insignal communicatin with a second input of the multiplexer170.

A first output of a:20spatial decimation module 104 is connected in signal communicationwith arninput of the temporal decomposition module 132,Asecondoutput ofithe 20 spatial decimation module 04-sconnected insignalcommunication wAh an inputof the temporaldecomposition module 142. An inputofthe temporal decomposition module 122 and aninput of the 20spailt decimation module 104 are avaiabl as inputs of the encoder 100, forreceiving inputvideo 102. An output of the mutpexer 170is available as an out put of the encoder 100, for providing a itstream180 The temporaldecomposition module 122the tempral decomposition module 132, the temporal decomposition module 142, the motion coding module124,themotion coding module 134 themotioncoding module 144, the intralpredictor 126, the itra predictor 136, the intra predictor 146, the transformtentropy coder 1291 the transformtentropy coder 139, the transformletropy coder 149,the 2Dspatial interpolation module 128, and the 2D spatial interpolaton module 138 are included in a core encoder portion 187 of the encoder 100 FIG I includes three core encoders 187. In the implementation shown the bottommost core encoder 187 May enoddea base layer,with the middle and upper core encoders 187 encodinghigher layers. Turning to FIG, 2 artexemplary SVC decoder is indicated generally by the reference numeral 200, The SVC decoder 200 mayalso be used for AVC decoding,that is forasingleview Furthe ,the SVC deoder 200 may be used for MVC decodingas one of ordinary skill in'the art will. appreciate. For example, various components of the SVC decoder 200,r variationsof these components, may be used indecodingmultiple views. Notethat encoder 100 and decoder 200,as wells otherencoders and decoders discussed in this disclosure,can configured to perform various methods shown throughoutthis disclosure. In addition to performingencoding operations, the encodersdescribed inthis disclosure may perform various decoding operationsduring a reconstruction process in order to mirror the expected actions of a decoder. Foexample, an encoder may decode SUP SP$unitsto decode encoded video data inorder to produce a reconstruction of the encoded video data for use in predicting additional video data Consequently anencoder may performsubstantily all of theoperations that areperformed bya decoder. An input of a demultiplexer 2021is available as an input to thescalable video decoder 200, for recengascalablebitstream. First output of te demutiplexer 202 is connected in signal communication with an input ofa spatialinverse transformSNR scalable entropy decoder 204. Afirst output of thespatial inverse transform SNR scalable entropy decoder 204 isconneted in-signal communication with a first input of a predition module 206. An outputofthe prediction module 206 is cnnected in signal communicatior With a first inputsof acombner 230. Asecond outpUtof thepatial iversetransform SNR calabte entropy decoder 204 is connected in'sighal communication with a first input of a motion vector(MV) decoder 210. An output of the MV decoder 210 is: connected in signal commuication with an inputof a motion compensator 232. An output of the motion compensator32 is connected in signal communication with a secondinputof the combiner'230. A second output of the demultiplexer 202 is connectedfrsignal communication with an input of a spatial inversetransform SNR.scalable entropy decoder 212, A firstoutput of the spatial inverse transform SNR scalable entropy decoder 212isconnetedinsignal communicaon with firstinput of a prediction module 214. A firstoutput of the prediction module 214 is connected in signal communication with an input of anntpolatiot module 216. An output ofthe interpolation module 2.16is connected in signal communication witha second input of the prediction module 20. A second output ofthe prediction module 214 is connected in signal communication with a first input of a combiner 240. A secondoutput of the spatial inverse transform SNR scalable entropy decoder 2112 is connected in signal communication with a first inputofaMV decoder 220. A first output of the MV decoder 220 is connected in signal communication with:asecond inputofthe MV decoder 210. Asecondoutput of the MV decoder:220 is connected in signal communication with an input of a motion compensator 242. An output of the motion compensator 242 is connected insignal communication with second input of the combiner 240,. Atirdoutput of the dernultiplexer 202 is connected in signal communicationwith an input of a spatial invers transform SNK sclable entropydecoder222; Afirstoutputofthespatialinversetransform SNR scalable entropy decoder 222 is connected insignal communication within input of aprediction.module 224. A first output of the prediction module 224 is connected in signal communication with aninputof a interpolatiomodule 226. An outputof theinterpolation module 226 is connected in signal communication with a second input af-the predictionmodule214 A second output of the prediction module 224 is connected in sign communication with a first input of a combiner250.. A second outputofthe spatial inverse transform-SNR scOlable entropy decoder 222 is connected in signal commuication with an input ofan MV decoder230. A first output of the MV decoder 230 is connected in signalcommuication with a second input ofthe MV decoder 220. A second output of the MV decoder 230 is conncted in signal communication with an input ofemotion compensator 252. An output of the motion compensator 252 is connected in.signal communication with Asecond input of the combiner 256. An output of the combiner 250is available asanoutput of the decoder 200 for outputting alayer 0signal, An output ofthecombiner240isavilable as an output of the decoder 200, for outputting alayer I signal. An output of the combier 230is available as an output ofthedecoder 200,foroutputtinga layer.2 signal. Referring toFIG.apn exemplary AVC encoder isindicated generally by the reference numeral 2100. The AVC encoder.2100 may be used, for example. for encoding asingle layer (forexample,baselayer Thevideo encoder 2100 includes af ameordering buffer 2110havirig an output in signal communication with a non-inverting input of a combined 2185 An output of the combiner 2186 is connectedin signal communication with a first inut of a transfOrmer and quantizer 2125. An output of the transformer and quantizer:2425 isconnected lsignalcommuniAation with a first input of an entropycode r2145 anda ftrst inputofaninversetransformer and inverse quantizer 2150. An output of the entropy coder 2145 is connected in signal communication with a first noinverting input of a combiner 2190. An output of the combined 2190 is connected insignal communicationwith first input ofan output buffer2135. First output of an encoder controller 2105 is connected in sinal communication ith a second input of the frame orderingbuffer2110,a second input of the inverse transformerand inverse quantzer2150, an input Of a picture-type decision module 2115 aninput of amacroblok-type (MB type) decision module 2120, a second input of an intra prediction module 2160, a second input of a deblocking filer2165, first input ofamonto compensator 2170 a first input of-a motion estimator 217, and second input of a reference picture buffer 2180 Asecond output of the encoder controller 2105 is conniected in signal communication with a first input of a Supplemental EnhancementInformation ("SElfinserter 2130a second input of the transformer andquantizer2125,a second input of the entropy coder 2140,a seopnd input of the output buffer 2135, and an input of the Sequene Parameter Set (PS) and Picture Parameter Set (PPS) inserter:2140. A first output of the plcturetype decision module 2145is connected in signal communication ith a third input of a frame ordering buffer 2110. A second output of the picture-type decision module 215 is connected in signal communication with a second inputofa amacrobIOcktype decision module 2120. An output of the Sequence Parameter Set (5PS) and Picture Parameter Set ("PPS) inserter 2140 is connected in signacommunication with athird non-inveting input of the combiner 2190,AnoutputoftheSEI Inserter 2130 is connected in signa communication with a second non invertinginputof thecombiner 2190,

An output of the inverse quantizer and inverse transformer 2160 is connected in signal communication with first non-inverting input of a combiner 2127. An output of the combiner 2127 is connected in signal communicationwith first input of the intra predictionmodule 2160andairst input of the deblocking filter265.Anoutput of thedeblockingfilter2165is connected in signal communication with a first input of a reference picture buffer 2180. An output:of (he reference picture buffer 210 is connected in signal com nation with a second input of the motion estimator215and with first inputof a motion compensatory 2170Afirstoutputofthe motion estimator 2175 isclnectedin signal communication with asecond input of the motion compensator 2170. A second output of the motion estimnator 2175 i connected in signal cominuication with third input of the entropy coder 2145. An output of the motion corpensatof 2170 is connected insignal communication th a first inputofaswitch 2197 An output of the intra prediction module2160 is connected in signal comrunication with second input ofthe switch 2197. Anoutptof the madroblcktype decisionmodule 2120 isconnected in signal communication with a third input of the sitch: 2197in order to provide a control input to the switch2197, An output of the switch 2197 isconnectedinsignlcommuication with aasodd honinveting Input Ofthe combiner 2127 andwith anInvertingInputof the combiner2185,. Inputs of the frame ordering buffer 2110 and the encoder controller 2105 are available as input of the encoder 2100,for receiving an iputpieture 2101 Moreover, aninput of the Ei Inserter 20is avaiabe as an input of the encoder 2100, for receiving metadta. Anoutputoftheoutput buffer2135 is availableas an output of the encoder2100 for outputting a bitstream. Referring to FG. 2a, a video decoder capable of performing video decoding in accordancewith theMPG-4 AVC standard is indicatedgenerally by the reference numeral 2200. The-videq decoder 2200 includes an input buffer 2210 having an output connected in signal comunication with fist input of an entropy decoder 2245 A first output of the entropy decoder 2245 is connected in signal

14. commnication with a first input of an inverse transformer and inverse quantizer 2250. An utput of the inverse transformer andinverse quantizer 2250 is connected in signa(communication with a secondnon-inverting input of acombiner 2225. An output of thecombiner 2225 is connected in signal S communicationwith second inputof-adeblocking filter 22 and a firstnput of an intra prediction module 2260. A second output t thedeblocking filter 2265 is connected in signal ommunication with;'a first inputof a reference picture buffer.2280. An output of the reference picture buffer 2280 is connected in signal communication with a second input of a motion compensator 2270. A second output of the entropy decoder:2245 is connected in signal communication with athird input of the motion compensator 2270rand a first iput of the.debloekingflter 2265. A thirdoutput of theqntropy decoder 2245 is connected in signal communication with an input of a decoder controller 2205 A first output of the decodercontrolier 2205 is connected'in signal communication with a second input of the entropy decoder 2245. A second output ofthe decoder controller 2205 is connected in signalormunication with a second.input oftheinverse transformerad inverse quantizer 2250.A third output of the decoder controller 2205 is connected in signal communication with third inut of thedeblocking filter 2265. A fourth output of the.decoder controller 2205 is connected in signal coimunication with a second input of the Inta prdiction module 2260, with a fitstinput of the motion compensator 2270 andwith as econd input of the reference picture buffer-2280. An output of the motion compensator 2270 is connected in signa communication with a first input of a switch 2297. An output of the intra prediction module 2260 is connected in signal communication with a second input of the switch 2297. An output of the switch 2297 isconnected in signal communication with a first non invertinginputof the combiner 2225, 3D Aninputof the input buffer 2210isavailable as an.input ofthe decoder 2200, for receiving an input bitstream. A firstputput of the deblocking filter

2265 is available as ar utpt ofthe decoder 220D,feroutputting an output picture. Referring to FIG. 3, astucture for a single-layer SPS 300 is shown SP5 is a syntax stcturethat generally contains syntaxelements that apply to zero or more entire codedvideo sequences. in the SVC extensionthe valuesof some syntax elements conveyed in the SPS are:layer dependent. Theselayer-dependent syntax elements include but arenot limited to, the timing information RD (standing for "Hypothetical Reference Decoder) parameters, and bitstream restriction information. HRD parameters may include,for example, indicators of buffersizemaximum bitrate andinitial delay.HRDparameters may allow a receiving system, for exmplej toverify thetegrity of a received bit stream andr to determine ifthereeiving system(forexample a decoder)can decode the bit stream. Therefore system ay provide for the Oarsmissi Of the aforementioned syntax elements for each layer. Thesingle-ayer SPS 300 includes an:SIS4ID310that provides an identifier for the SPS. The single-layer SPS 300 also includes the VUI (standing for Video liabilityy Information) parameters320 for a single layer, ThO VU Iparametersinlude the HRD parameters 330 for single layer, such as, for example, thebase layer. The singledayerS6PS 30 also may include additional parameters 340, although implementations need not include any additional parameters 340. ReferdngAo FIG. 4, a blockview ofa data stream 400 shows a typical use of the singlelayer SPS 300, in the AVC standard, for example, a typical data stream mayinclude, among other components, an SPS unit, multiple PPS (picture parameter sequenceunits providing parameters for a particular picture. and multiple units for encoded picture data. Such general framrework is followed in FIG.4, whidh includestheSPS 300, a PP- 410, one or more units 420 including encoded pictu re-data aPPS-2430,and one or more nits 440including encoded picture2 data. The PPS-1 410 includes parameters for the encoded picture-1 data 420,and the PPS-2 430 includesparameters for the encoded picture-2 data 440

Theencoded picture- data 420,and the enodedpiture-2 data 440, are each associated with a particular SPS (the 9PS 3001iniheimplementation of FIG 4) This is achieved through the use ofpointers, as now explained. The encoded picture- da A420inldes aPPS-D (not shown) identifying the PPS4 10, as shown by an arrow 450. The PPSID may be stored in, for example a slice header. The encoded piture- data 440includes a PP$-ID (not shown):ideritifyingthe PP$-2430,as shown by'an arrow460, ThePPS8 1 410 and the PPS-2 430 each include anSPSD notshown) identifying the SPS 300,as shown by aows 470 and 48 respectively. Referring toFIG,. 5, a structure for a SUPSPS 500 is shown SUP SPS 500 includes anPS1 D 510 a VU 520 that ncludes HRD parameters 530 fora single additional layer referred toby (D22,Q2f and optional additional parameters 540, "D2, T2jQ2" refers to a second layer having spatial (D) level2, temporal (T) level 2andqality (O)level.2. Note that variousnubring schemes may be used torefeto layers In one numbering scheme,base layers have a D$ %of0,x0,meaninga ipatial levl of zero,any temporal level, and a quality level of zero. In that numbering scheme, enhancement layers have a D, T. Q Ai which Dor O are greater than zero. The use of SUP SPS500 allows, for example,a system to use an SPS structure that only includes parameters for a single layer, or that does not include any Jayer-dependent information. Such a system may create a separate SUP SPS foreach additional layer beyond the base layer, The additional layers can identifytheSSwithwhich theyaeassociated through these of the SPS10 0.Clearly severallayers can share single SPSby using a common SPS ID in their respective-SUP SPSunits. Referringto FIG. 6, an organizational hierarchy 600is shown among an 8.3 unit 605 and multiple-SUP SPS units610 and:620. TheSUP SPS units 610 and 620 are shown as being single-layer$UP SPS units, but.other implementations may use one or more multipleayerSPSSunitsi addition to, or in lieu of, single-layer SUP SP8 units. The hierarchy 600 illustrates that, in a typical scenario, multiple SUP PS units may be associated with a single SPS'uniti implementators.may ofcoursenclude multiple SPS units,andeach of the SPS uits may have associated SUP SPS units Referring to FG 7, a structure for another SUP SPS 700 is shown. SUPSP670 includes parameters for multipelayers whereas SUP SIRS500 includes parametersora single layer. SUPSPS 700includes anSPS ID 710, a-W 720, and optional additia parameters 740. The VU 720 includes HRD parameters 730for a first additionallayer(D2.T2, 02), and for otheradditional layers upiolayer(Dn, Tn, Qn). Referring again to:FIG. 6, thehierarchy 600 may be modified to use a multiple layer .UP SPS. For example. the combinationof theSUP SPS 610 and 620.may be replaced with the SUP SPS 700 if both the SUP SPS610 and 620 include the same SPS1D: Additionallythe SUP SPS 700 may be usedj for examplewith an SPS that includes parameters for a single layer, or that includesparametersfor multiplelayers, or that does not include layer4ependent paamete for any layers. The SUP SPS 700 allows a system to provide parametesform Iultiple layerswth little overhead. Other impeentationsmay be based forexample, on an PS that Includes all theneeded parameters foray possible layers That is, theS'S of such an implementation includes all thecorresponding spatial (D),termporal (Ti), andquality (i levels that are available to be transmitted;whotherallthe layers are transmitted ornot. Even with sucha system, however a SUP SS may be used toprovide an abilityto change theparameters for one ormre layers without transmitting theerntire SPS again. ReferingtoTable1.syntaxisprovided fr a specific implementation of a-single-layer SUP SPS. The syntaxincludes sequenceparameter_ejd to identify the associated S.PS, and the identifiers of temporatieve, dependency, andqualityjevel to identify a scalablelayer TheVUI parameters are included through the use ofsvp vui~parameters( (see Table 2), which includes HRD parameters throughthe use of hrd-parameters0. The syntax below allows each layer to specify its own layer-dependent parameters, such as, for exampleRD parameters.

supsaameefersesvc () { _ Des___r___r wqaenepatametei setJd

epeudeziy RC qualitylevel vupramttr preitv~jag (ittimterspresentsaajag)_____ svcvijararetesO)_____

6 Table 1

The semantics fortheo ytax of supSeqparammterseLtVC() is as follows.

sequetteeparameter_setd identifies the sequence parameter set which the current:SUP$PS maps to for the currentlayer; = temporallevel, dependency id, and quality level specify the temporal level, dependency identifier, and quality level for the current layer. Dependencyjd generally indicates spatial level. However,dependencyjd also is used to indicate theCoarse Grain Scaability ("CGS") hierarchy which includes both spatial and SNR scalability, with SNR scaability being a traditional quality scalability Accordingly: qlityjeveland dependency_id may both be used to distinguish quality levels. vui_parameerspresent svcjflag equals to 1 specifies that sVc_vuiparameters() syntx structure as defined below is present. vui_parameters_present svcaflag equals to 0 specifies that swouiparameterso syntaxstructure is rotpresent.

Table 2 gives thesynaxforsc. yuparaeters(, The VUI parameters are thereforeseparated for eachlayer and put into individual SUP SPS units. Other implementations, however, group the VUI parameters for multiplelayers intoa single'SUP SPS.

Timing 11OjPM~ntflg) _____

nunaunisInick0 4i32).

jthXC4ifr o-rtuuflag

R. ra ~rjMe terse r't=ent..a 0J14)

If1 *hrd amte Present 110j;U()

hrdparameterq( jI~ ~ vtd a~~e~tig _______-v

low dielay hrd fjag 0 U(t) pitr t-presuitjlag U(I) b~t~rv~m4~trdiiifl6 0 (I) lI ft eam rcstlicionfi8){ motion Vtctors iwerpIjkboundrej 0 UfI) malt-bytesPtericatn 0: Ue(v) max its-pr-Mb 4nQM 0 ue(v) ........... en~. .. y ri a c

5 Th' fiels of the 6vrvuia ametrs() syntax dfTable2 are deflnedin the: oft, oof theVC exa otaexistd inApril 200 uder N.T-12 annex EE. 1. In paticular rdramters() isa06defined for the AVC: stohdard. Note also that svq, yWparaneters()inclidesafriOUS layer dependent information,including HRDrelated parameters. TheHRD-related' parametersinclude rnmunitsintick, tme scaefixedjframejate la, nahrpaaetrs.presen~fiag, vclhrd~paramet.rspresentlAg,, hrd..paramneters ,fowvelayhrjag ndk sjrcptpresent jlag. Further, the syntax element Inthe bitsram_restrictionJlag if100p elayer deperident even thoughtrot HRD-related As mentioned above the SUP SPSis defined as a new type of NAL unit Table 3 listss$ome of the NA.L Unit codes as defied by the standard S JT20 but modified to assig type24totheSUSP$Theelipsis between NALunit types1and 16.and between 18ard24, indicate thatthose types are unchanged, The ellipsi between NAL qnittypes 25and 31 meant that those types arealnspecified, The implementation ofTable below changes typeof the standard from 24 "unspecified" to "supseqjarameterjset.sc(f. "Unspecifed" isgenerallyreserved for user applications4 Reservedrt onthe other hand, is generally reserved for future standard modifications. Accordingly,another implementation changes one of the "reserved" types (for example, type 16, 17? or 18) to "supseqparameter_set svc(f" Changing an "unspecified" type results ian implementation for a given user, whereas changing a "reserved"type results inan implementation that changes the standardfor al users., n unitjype Contentof NALunit and RBSP syta structure C

Q Unspecfied 1 Coded slice of a non-IDR picture 2,,4 slice laye wIthoutpar tlonig bsp(

10-18 Reserved

24 sup seqparameter set. svoc( 25 31 Unspecified

Table

FIG shows a functional view of an implementation of a scalable video coder 800 that generates SUP SPS units. A video is receivedat the input of the scalable video coder I. The videois odedaccording to different spatialtevels. Spatiallevels mainly refer todifferent levels of resolution of the same video. For example, as the input of a scalable video coder, on' can haveaIsequence (352per 28) or a QCF sequence 6 per 144) which represent each one spatial level, Each ofthe spatial levels is sent to aencoder. The spatial leve1 is sent to an encoder 2, the spatiat level 2 is sent to an encoder 2' and the spatial level m dissent to an encoder 2 The spatial levels are coded with 3 bits, sing thedependencyid, Therefore..themaximuirnumber of spatiat levels inthis implementaionis,8 The encoders 2, 2' and 2" enode one or more layers.having the indicated spatal level. Theerncoder 2,2,and 2 may be designed to have particular quality levels and temporallevelsor the qualitylevels and tempora level may be configurables canbeseen from.FIG. .the encoders 2,2, and 2" are hierarchically arranged. That is, the encoder 2" feeds the encoder 2'which in turn feeds the encoder 2. The hierarchicalarrangement indicates the typicalscenario in whichNgher layers use aower players) asa reference After the coding, the headers are preparedfor each of the layers. In the implementation shown,for each spatIallevelan SPS message, a PPS message, and multiple SUP SPS messages are created. SUP SPS messages (or units) may be created, for example, for layers corresponding to the vadous differentquality andrtemporallevels For spatiall level 1, SSandP 5" are create and a set of SUP _SPS SUP .SPN. SUPSPS,~.areais created For spatial level 2,SPS and PPS 5' are created a a set of SUP-SPS1 SUP -SPS, -SUP _SPSy are also created For patialevel M PS and PMS 5 recreated and a set of SUPSP$,SU SuPaS~' -SUP _SPSareasO created. The bitstrearns 7 7,and ! encodedabythie encoders ,2'and 2" typically follow the plurality of SPS, PPSand SUP_SPS aso referred to as leaders, unit. ormessages)In the global bitstream.

A bitstream "includes SPS and PPS 5", SUSPS SUP -sP SUP SPS 6anh enoded video bitstream 7", which onstituteal the encoded data associated with spatialivel I. Abitstrearr'includes:SPS and PPS S SUPSPS, SUP SPn" SUP SPS 6' and encoded Video bttreaM T whichconStitteall the

encoded data associated with spatial level.2. 0 Abitstream 8 includesSPS and PPS 5 SUP $SF 1 SUP SPSI" SUP SPS 6, and encoded video bitstream 7,which consttute all the encoded data associated withspatilevelm, The different SUP.SPS headers are compliant with the headers described in Tables 13. The encoder800 depicted. in FIG8 generates one SP forceach spatial levet.However, other implementations may generate multipleSPS for eachspatialevel ormay generate an SPS that servemultiple spatial levels. The bitstreams 8 and . arecombined in amultiplexer 9 which produces an SVO bitstream, asshown in FIGA Referring to FIG. 9, a hierarchical view 900 lustrates the generation of a data: stream that contains SUP SP$ units The view 900 may be used to illustrated possibe bitstreams generatedby the scalablevideoencoder800 of FIG.8, The view 00 provides an SVC bitstream to a transmission interface 17. The $V bitstream may be generated, for example, aording to the implementation of FIG.8,and comprisesone SPS fore ach of the spatial levels. When m spatial levels-areencoded, the SVC bitstream comprises SPS1,SPS2and SP$m representedby 10,10'and10"inFIG-9. Inthe SVC bitstream. each SPS codethe general information relative tothespatiallevel. TheSPSisfollowedbyaheader1IIiti1311 15 15'.and 15" of SUPSPS type. The SUP-SP is followed by the corresponding enoded video dat 12,1 12", 14,14,44", 16, 16 and 16" Which eachcortespond to one temporallevel (n) and one quality level (0).

Therefore, when one layer is not transmitted, the corresponding SP.PS is also not transmitted Thisbecausethere is typicallyone SUPSPS heder correapondigto each layer. Typical impleentatirs usea numbering scheme for layers in which Sthe base layer hasa.Dand Q ofzer Ifsuchanubering scheme is usedfor theview9000, then theview 900does notexplicitshowabaselayer That does not preclude these ofabase layer. Additional however, the view 900 may be augmented to explicitly show abitstreantfora base layer; as well as,for example, a separate Pfohra base layer.Fudhr, theview 900 may use an alternate numbering scheme for baselayers, in whih one or more of the bitstreams(1, 1, through (m n, 0) refersto a baselayer. Referringto FIG. 10,a block view is provided of a data stream10 generated by the implementation of FIGS.8and 9. FIG illustrates the transmission of thefollowing layers: Layer (1, 1 partiall level 1, temporallevel1, quality level 1; which includes transmission of bloks 10, 1t, and 12k Layer (1 1):sPatial level teniporal level 2,quality level 1 whi includes the addtionaltransmission of blocks 11' and 12'; Layer (2,1 # 1) spatia level2t, em1quality lel1;which includes the additional transmission of blocks 10 13, and 14 o Layer (3, 1, 1) spatiallevel 3, temporal level 1 qulty level ; which includes the additionaltransmissionofblocks10",15,and15; o Layer (3,2. 1): spatial level 3temporal level2qualitylevel 1 which includes the additiopaltransmission of blocks 15'and16; Layer (3 3 1): spatialleve temporalevee3,:quality level1;which includes the additionaftransmission ofrblocks 5 and 16.

The block view of the data strearn 1000 illustrates thatSPS 10 isonly sent once and is used byboth Layer ( 1, 1) and Layer (1,2 1,and that SPS10"ionly sent once isused each of Layer (3 1), Layer (32, 1 ,and Layer(3, 3, 1). Fuerthe data stream 1000 ilustrates that the parameters for all f the layers aren't transmitted, but rather only the parameters

24:

cotresponding to the transmitted layers, For example, the parameters for layer (2 2, ),corresponding to SUP JPS9 arenottansmittedbecausethat layerisnot transmitted. This providesanfficiency for thisimplementtion Referring toFIGE,11,an encoder100 includes an SPSFgeneration unit 1110, a video encoder 1120 and formatter 1130. Thevideo encoder 1120 receives input video, encodes the inputvideo and provides the encodedirput video to the formatter 1130. The encoded inputi video may include, for example, mUltipl layers uch as, for example, an encoded base layer and an encodede nhancement layer The SPS generation unit 1110 geneates to headerinformationsuch as, for example SPS units and SUP S units arid provides theheaderinformation tol the formatter 1130. TheSPSgenertion unit 1110 also. communicates with the video encoder 1120 to provide parametersused by the video encoder 120 in encodingthe input video The PS generation unit 1110 may be configured for example, to generate an SPs NAL unit. The SPS NALunitmay include information that describes a. parameter for use in decoding a firstlayer encoding of a sequence of images, The SPSgeneration unit10alsomay be configured, for exampleto generate a SUP SPS NAL unit having a different structure than the SPS NAL unit. The:SUP SPS NALunit may inudeinformation that decribs o parameter for use in decoding a seond4ayer encoding of the sequenceOfimages. The firstayer encoding andthesecond-layer encoding may be produced by the video enoder1120 The formatter 1130 multiplexes the encoded video from the video encoder 1120, and the header information from the SPS generation unit 1110, toproduceanoutput encoded bitstream The encodedbitstream may bea set of data that includes the first-layer encoding of the sequence of image the second-fayer encoding of the sequence ofmages the SPS NAL unit, and the:SUP SPSNAL unit The components 110,11 20and 130ofthe encoder 1100 maytake many forms. One or more of the components 1 ,1120and1130 may include hardware, software, firmware, ora combination, and may be operated

2b from a variety of platforms, such as, for example, a dediated encoder or a general prcessorconfigured throughsoftware to function as an encoder. FIGS.8 and 11 may be compared. TheSPSgeneration unit 1110 may generate the SPS and the vaous SUP $PS hoWn in F G. .The video encoder 1120 may generate the bitstreams 7,7'. and7" (which re the endings of thinputvideo).sho in FIG.8 .The video encode 1120may correspondfor example to one or more of the encoders2,or:2". The formatter 1130 may generate the hierarchically artarged data shownby referenceumerals 8, 8% as well as perform the operation of the multiplexer 9 to generate the SVC bitstream of FIGAL FIGS. 1 and 11also may be cmpred. The video encoder 1120 May correspond, for example, to blocks104 and187 of FG. 1. Theformater 1130 may corresporld, for example,:to theMultiplexer 170. The SPS generation unit-1110 is not explicitly shownnFIG. Although the functionality of the SPS generationunit 1110 may be performed, for example, bythe multiplexer 170. Other implementations of enoder 110 do not include the video encoder 1120 becauseforexample the data is preencoded. The encoder 1100 also may provide additional outputs and provide additional communication between the components. The encoder 1100 also maybe modified to provide additional componhtswhich may fp rexamplebe located between existing components Referring to FIG. 12, an encoder1200 is shown that operates in the same manner as the encoder1100. The encoder 1200 includes a memory 1210 ih communication with a processor"1220, The memory 1210 may be used, for example to store theinput video,to store ending or decoding parameters, to ste intermediate or final results during the encoding process, or to storeinstructions for performing an encodingmethod. Such storagemay betemporary orpermanent The processor 1220.receives input video and encodes the inputideo. The processor 1220 also generates header information, and formats an encoded bitstream that includes header information and encoded input video. As in the encoder 1100, theheader information provided bythe processor

1220 may include separatestructures forconveying header information for mltiple layers. The processor 1220 thay operate according to instructions storedop... rotherwise resident on or part of,for example the processor 1220 or the memory 1210. ReferringtoFIG.:13,aprocess1300 sshowforrencoding input video. The process 1300 may be perfornedby, forexample, either of the encoders 1100or 1200. The process 1300 includes generating an SPS NAL unit (1310) The SPs NAL unit includes informationthat desribes a parameter for usein decoding the first-layerenoding of the sequence ofimages. The SPS NAL unit may be defined by a coding standard or not. if the SPS NAL units defined by a codig standard, thenthe coding standardmay require a decoder to operate in accordance with received 5PS NAL units,. Such a requirement is generallyreferredto by stating that the SPS NAL unfit is 15. normatiel". 8PS for example, are normative in the AVC standard, whereas supplemental enhancement information ("SEI messages, for exampleare not normative Accordingly, AVC-compatible decoders mayignore received SE1messages butmust perateintaccodancewith receivedSP1S The SPS NAL unit includes information describing one or more parameters for decoding a first layer The parameter may be forexample, information thatisayer-dependent or is not layerdependent Examples of parameters that aretypically laerdependent include a VUI parameter oran HRD parameter. Operation Oa may be:perfomedforxample, bytheSPgeneration unit 1110,:the processor 1220, or the SPS and PPSnlserter 2140. The operation1310alsonaycorrespond to the generation of SPS in any ofblocks 5,5'5" inFIG. B. Accordingly; a means for performing the operation 1310, that is generating an SPS NAL unit; may include various components. For example, such means may include a modue for generating SP$ 5, 5or 5",an entire encoder system of FIG. 1,B,11, or 12, an SPS generation unit 1110,a processor 1220, or an SPS and PP Inserter 2140. or their equivaents

:27

including known and future-divelopedencoders The process 1300 includes generating A supplemental (SUP' $PS NAL unit having a different structure than the SPS NALunit (1320). The:SUP SPS NAL unit includes information that describes a parameter for use in decoding the second-ayer encodingof the sequence of images. The SUP SPS.NAL unit may be defiried bya coding standard or not. ifthe SUP $PS NAL unit is defined by a coding standard, then the codingstandard may require a decoder to operate in acordance with received SUP SP NAL units. As discussed above with respect to operation 1310, such requirement is generally referred to bystating that the UP SP NAL unit is "nomnative". Various implementations include normative SUP-SPS messages. For example, U PS messages may be normative for dcoders that decode more than one layer (for exampleSVC-compatible decoders Suchmutti layer decoders(for exampleSVC-compatibledecoders)wouldb required to operate in accordance with the information conveyed in.UP SPSmessages However, single-layer decoders (for example, AVC-compatible decoders) could ignore SUP SPS messages. As another example, SUP SPS messages may be normative for all decders including single-layer and multidayer decoders. It is not surprising that manyimplementations include normative SUP PS messages, given that 0P SPSmessages are based in large part on $PS messages, and that SFSamessagesare normative in the AVC standard and the SVC and MVC extensions, That is, SUP SPS: messages carry similar data as SPS messages serve a similar purpose asSPS messages, and may be considered tobeatpe ofPS message itshould be clear that implementations having normative'SUP SPS messages may provide compatibility advantages, for ampl allowing AVO and SVC decodersto receive a common data stream. The'SUP SPS NAL uhit,(also referred to as the SUP SPS message) includes one or more parameters for decoding a second layer. The parameter $0 may be, for example, information that is layer-dependent, or is not layer dependent. Specific examples include a VUI parameter ot an HRD parameter.T The SUP SPS may also be used for decoding the first layer in addition to being used for deoding the second layer. Operation 1320 may be performed, for example, by the SPSIgeneration unit 111.,the processor 1220,or a module analogoustotheSPSandPPS Inserter2140. The operation1320-alsomycorrespond tthegenerationof $PSPS in any of blocks 6,V, 6"in FIG. 8. Accordingly, a means for performing the operation 1320, that is; generating a SUP SPS NAL unit, may include various components For example, such means mayinclude a module forigenerating SUP_SPS 6 6% or 6"anentireencodersystemofFIGA.1,i tor12,anSPSgenerationunit 1110,a processor 1220,or a module analogous to the SPSand PPS Inserter 2140, or their equivalents including known and future-developed encoders, The process 1300 includes endoding'a first-layer encoding, such as, for example, the base layer; for a sequence ofimages, and encoding secod leyerencodingfor the sequence ofimages (1330) Theseencodingsofthe sequene of images produce the first4ayer encoding and the secondlayer encoding. The first-layer encodingmay be formatted intoa series ofuits referred to as first-layer encodingunits, and the secondlayer encodingimay be formtted into a series of nits referred toas send-layer encoding units. The operation 1330 may be performed, for example, by the video encoder 26 112,the poessor 1220,the encoders2, 2 or 2" of FIG. 8, or the implementation of FIG. Accordinglya means for perfoming the operation 1330, may include variouscomponents. For example, such-means may includeian encoder 24, or2",anentire encoder system of FIG. t8,1,or 12, a video encoder 112, A processor 1220, or one ormore core enoders 187 (possibly including decimation module 104), or their equivalents including known and future developedencoders. The process 1300 includes providing a set of data (1340). The set of data includes ihe fitst-ayer encoding ofthe sequence of images.the second layer encoding of the sequence of images, the SPS NAL unit, and the SUP SPNALunitThe set of data may be,forexample, a bitstream encoded accordingto a know standard to be stored in memory or transmitted to one or more decoders. Operation 1340 may be performed, for example, bythe formatter f13,theprocessor 120, or the muliplexer 170 of FI. Operation 1340 may alsobe performed in FIG. 8 bythe generation of any of the itstreams 8,8, and8 as. well as the generation of the multiplexed SVC bitstream. Accordingly, a means for perforingthe operation 1340, that is providing a set of data may include various components, For examplesuch means: ay include amodule for generating the bitstream 8, V, or8l a multplexer9, an entire encoder system f FIG, 1, 8.1 ot, 12 a formatter 1130 a processor 122, or aMltiplexer or their equivalents including known and future-developed encodrs. The process300 may be modified in various ways, For example. operation 1330 may be removed from theproces 1300 inplementationsin which, forexample,t data is pre-encoded. Further, in addition to removing operation 330, operation 1340 may be removed to provide a prcess directed toward generating description units for multiple layers. Referring to FIG 14, a data stream 1400 is :shown that may be generated, for example, by theprocess 1300 The data stream 1400includes a portion 1410 for anSPS NAL unit, a portion1420for aSUPSPS NAL unit a portion 1430 for the fst-layer encodeddataandaportion 1440for the second layer encoded data, The first4yer ended data 1430 isthefirst layer encoding, whichmay be formattedasfirstlayer encodingunits The secondayer encoded data1440 is the second-ayer encoding, which may be formatted as second4ayer encoding units The data stream1400 May include additional portions which may be appended after the portion 1440 or interspersed between the portions 1410-1440. Additionally, other implementations may modify one or more of the portions 1410-1440. The datastream 1400maybecomparedtoFIGS.9fand10. The SPS NAL unit 1410 may be forexamplenyoftheSPSI 10,theSPS210',orthe SPSm 10" The.SUP PSNAL rnit 1420 may before example, any of the SUP SPS headers 11, 11,11", 1313 13"; 15, 15' or15' The firstlayer encoded data 1430 and the second layerencoded data 1440 may be any of the bitstreams for the individual layers shownas Bitstrearn of Layer (1, 1,1) 12 through h (ri, 0) 16',and, icluding the bitstreams 12, 12 12", 14, 14f 14, 161, 16',a-d16" Itispossible forth firstlayer encoded data 1430 to be abitstream with a higher set of levels than thesecond-layer encoded data 1440. For example, the fistlyer encoded data 1430 may be the Bitstream of Layer (2, 1) 14% and thesecond-layer encoded data 1440:may be the Bitstream f Layer (1, 1,1) 12. An implementtionofthedatastream 1400ruay also correspond to the data stream 1000. The SPS NAL unit 1410 may correspond to the SPS module 10 of tho data stream 1000., The SP SPS NALunit 1420 may correspond to the SUP.SPS module 11 of the data stream 1000. The first layer encoded data1 430 may correspond to the Bitstream of Layer 1,1) 12 of the data stream 1000. Thesecond-layer encoded data 1440 may correspond to the:Bitstream of Layer.(1, 2,) 12'of the data stream 1000. The SUPSPS module 1' of the data stream 1000 may be interspersed between the first-ayer encoded data1430 and the secondalayerencoded data 1440. The remaining blocks (10-'A") shown in the data stream1000 maybe appended tothedatastream 1400 in the same order shown in the data stream 100. FIGS .and 10may suggest thatthe SPSmodules do not Includeany layer-specific parameters. Various implementations dooperate in this manner, and typically require a SUP SPS for eab layer. However, other implementations allow the SPS to include layer-specific parameters foroneor more lyers, thus along e or more layers to be rnsmitted without requiring a SUP_SP& FIGS. 9 and 10:suggest that each spatiallevelhas its own SPS Other implementation varythis feature. For ekamp e, other implementations provide a separate SPS for each temporal level orfor eachqality level. Still other implementationsprovide *a separate SPS for each layer, and other implementations provide a single SPS that serves allJayers. Referring to:FIG. 15, a decoder1 500 includes a parsing unit 1510 that receives anrencded bitstream, such as, for example, the encoded bitstream provided by the encoder 1100, the encoder 1200, the process 1300, or the datasWeam1400. The parsing uit 1510 iscoupled to a decodet 1520. The parsingunit 1510 is configured to access information from an.SPS AL. unit. The Information from the SPS NAL unit describes aparameterrfor use in decoding a first-layer encoding of a sequence ofirmages, The parsing unit 1510 is further configured to access information from a SUP SPS NAL unit having a different structure than the SPSNAL unit. The information from the SlUP SPS NALunit describes a parameter for se inidecodinga second layer encoding of the sequence of images As desribedabove in conjunction with FIG, 13, the parameters may be. ayer-dependent or nonayer dependent, The parsing unit 1510 provides parsed header data as an output, The header data includes the inftMation accessed from the SNS NAL unitand also includesthe information. atessed from the SUP SPSNALunrt. The parsing unit 1510altorovide pased encoded video data as an output.The encoded video data includes the first-layer encoding and the second-ayer encoding. Both the header data and the encoded video data are provided to the decoder 1520. The decoder 1520 decodes the firstlayer encoding using the information accessed from the SPS NAL unit The decoder 1520 also decodes the secondlayer encoding using the information accessed from the SUP SPS NAL unit The decoder 1520 further generates areconstructionof the sequence of images based on the decoded firstayer and/or the decoded second-tayer The decoder1520 provides a reconstructed video as an output. The reconstructed video may be. for example, a reconstruction of thefirst layer encoding or a reconstruction of the second-4ayerencoding. Comparing FIGS. 15. 2, and 2athe parsing unit1510 may correspond, for example t the demultipler 202, and/or one or moreof the entropy decoders 204, 212, 222,or 2245, i someimplementations Thedeboder 1520 may correspond, forexample, to the remaining blockinI. The decoder 1500 also may provide additional outputs and provide additional communication between the components. The decoder 1500 also may be modified to provided dditonalcomponents which mayfor example, be located between existingcomponets. The components 1510 and 1520of the decoder 1500mray take many forms. One or more of the components 1510 and 1520 may include hardware software, firmwareora combinations and may be operated from a variety of platforms, such as, forexample, dedicated deo deorageneral processor configured through software function as a decoder., Referring to FIG. 16 decoder4600. isshown that operates in the same manner as the decoder 500. The decoder 1600 includes amemory 1610 in communication With a processor 1620. The rehory 1610 may be used, for example, tostore the input ended bitstream, tdtoredecodingor encoding parameters to store intermediate orfialresults during the decoding process, or to store instruction for efrming adecodingmrnthod. Such storage may be temporary or percent. The processor 1620 receives an encoded bitstrearm and decodes the encoded bitstream into a reconstructed video. The encoded bitstream includes,for example,(1) a firstlayerencoding of a sequence of images, (2) a secondayerending of the sequene of irnages, (3) an $PS NALunit having information that describes a. parameter for use Indecoding theirst layer encoding,and:(4) a SUP:SPS NAL unit having a different structure than the SPS NAL unit, and having information that describes a parameter for use in decoding the second4ayer-encoding. The processor 1620 produces the reconstructed videobased on at least the first-layer encoding, the secondayerencoding, the informationfrom the.SPS NAL unit, and theinformation from the SUP SPS NIAL unit. The reconstructed video may be, for example,a reconstruction of the first-layer encoding or a reconstction of the second-layer encoding. The processor 1620 may operate according to instructionsstored on, or otherwise resident on or part of, forexample the processor 1620 or themmory 1610. Referring to FIG, 17,a process 1700 is shown for decodigan encoded bitstream. The process 1700 may be performed by, for example either of the decoders 1500 or 1600.

The process 1700 includesaccessinginformation froman SPS NAL unit(1710). The accessed iformationdescribes aparameterforusein decodigafirstlayerending of sequence ofImaget The:SPS NAL unitmaybe as describedealierwith respect to FIG. 13. 6 Furtherthe accessedinforMation may be,forexample, an HR parameter Operation 1710 may be performed, for example,by the paring unit 1510.the processor 1620, an entropy decoder 204, 212, 222, or 2245, or decoder control 2205. Operation 1710 also may be performed in a reconstruction process at an encoder byTone or morecomponents of an encoder. Accordingly, a means for performing the operation 1710, that is, accessing information fromanSPS NAL unit, may include vari qs components. For example such meansamay include a parsing unit 1510,a processor 1620, asingle-layer decoder, anentire decoder system of FIG. 2, 15, or 16, or one or more components of a decoder, or one or more components of encoders 800 1100 or 1200,or their equivalents including known and future-developed decoders and encoders. The: process 1700 includes accessing informaonfromaSUPSPS NAL unit having a different structure than the SPS NAL unit (1720). The information accessedfrom the SUP SPS NAL unit descnbes a parameter-for use in decoding a second-layer encoding of the sequence of images The SUP SPS NAL unit nay be asdescribed earlier with respect to FG.13. Furth, the accessed information maybe, for example :an HRD paaneter. Operation 1720 may be performed, for example, by the parsing unit: 15,the processor 1620, an entropy decoder 204, 212, 222or:2245 or 26 decoder control 2205. Operation 1720 also may be performed In a reconstruction proess atan encoder by one or more components of an encoder Accordingly, :a means for performing the operation 1720, that is, accessing iformationfrom a SUP SPS NAL unit, may include various components. Forexample, such means may include a parsing unit 1510, a processor 1620, a demultiplexr202,an entropy decoder 204212or222, a singlelayer decoder or an entire decoder system 2001500, or 1600, or one or more components of decoder, or one or more componentsaufencoders 800,1100,or 1200 ortheir equivalents includingknown and futue-developed decoders and encoders. The process 1700 includes accessing a first-layer entodinganda second-layerencoding for the sequence of images (1730). The firstlayer encoding may have been formatted into first-layer encoding units, and the second-layer encoding may have been formatted into second-layer encoding units, Operaion1730 may be performed ,for exampleby the prsing unit 1510, the decoder 1520, the processor 1620, an entropy decoder 2041212, 222, or 2245.,or various other blocks downstream of theientrpydecoders. Operation 1730 also may be performed ina reconstruction proves at-an encoder by one ormor components an encoder According means for performingthe operation 1730mayinclude various components, For examples uchmeans ray included parsing Unt 1510, adecoder 1520,a professor 1620 A demultiplexer 202, an entropy decodr 204, 212, or 222, a sirglelayer decoder, a bitstream receiver, a receiving devie oran entire decoder system 200, 1600, or 100, or one or morecomponents of adecoder or oneormore components of encoders 800, 1100, or 1200, or their equivalent including known and future-devloped decoders and encoders. Theprocess 1700 includes generating a decoding ofthesequence of images (1740), The decoding ofthe sequence of images maybe based on thefirst-layer encoding, the second-layer encoding, the accessed information from the SPS NAL unit, and the accessedinformation from theSUP SPS NAL unit. Operation 1740.may be performed, forexample, bythedecoder1520 the processor 1020, or various blocks downstream of demultiplexer 202 and input buffer 2210. Operation 1740 also may be performed in a reconstruction process at an encoder by-one ormore components of an encoder. Accordingly,a means for performing the operation 1740 may include various components, For exariple, such means may indludea decoder 1530, a processor 1620,a single-layer dedoderi an entire decoder system 200, 1500, or 1600, or one or more components of a decoder an encoder performing reconstruction,or oneor more components of encoders800, 110 or 1200or their equivalents including Inownand future-devloped decoders or encoders. Another implementation performsan encoding method that includes accessing first layerdependentinformation in a first normative parameter set Theaccessedfirstlayer-dependentinfrmationisforuse indecoding a first layer encoding of a s equece of images. The first normative parameterset aybe, for examplean SPS that incdesHRD-related parameters or other layerdependent informati6h. However, the first normative parameter set need not be anSP$ and need notbe related to an 264 standard In addition to the first parameter set being normativewhich requires a decoder to operatein accordance with the first parameter set if such a parameter set is received, te first parameter set may also be required to be received in an implementation. That is, an implementation may further require that the first parameter set be provided toa decoder. The encoding method of this implementation further includes cessg secondlayer-dependent information in a secondnomative parameter set. The second normative parameter set has a different structure than the fist normative parameter set. Also, the accessed second layer-dependent information isfor use in decoding a second-layer encoding of the seuenceof images. The second normtive parameter setmay be, for example a supplemental SPS& The supplementalSPS has a structure that is different from, for example on. SPS. The supplemental SPS also includeHHRD parameters or other layer-dependent information for a second layer (different from the first laye). The encoding method of this implementation further includes decoding the sequence of imagesbasdononeormoreoftheaccessed first layer dependent information or the accessed second layer4ependent information. Thismay include, for example, decoding a base layer or an enhancement layer. Corresponding apparatuses are also provided in other implementations, for implementing the, encoding method of this implementation. Such apparatus include for example, pogrammed encoders, programmed processors, hardware implementations, or processor readable Media having instructions for performing the encoding method The systems I100 and 1200,for examplemay implement the encoding method of thisimplementation. Correspondingsignals, and mediastoring suchsignals or the data of such signalsare also provided. Such signalsareproduced,forexampleby an encoder that performs the encodingmethod of this impemehtatin. Another implementation performs decoding methodnaboustothe above encoding method. Thedecodingmnethodincudesgeneratinga first nornativeparameter set that includes first layer-dependent information. The first layer-dependent information isfor use in decoding afirst-layer encoding of a sequence of images The decoding method alo includes generating second norrative parameter set having a different structure than the first nominative parameter set. The second normative parameter set includes second layer-dependent :information foruse in decoding a secondayer encoding of thesequence ofimages The decdingmethod.further includes providing a set of data includingthe first normative parameter set .and the secondnormative parameter set. Corresponding apparatuses are also provided in other implementations for mplementing the above decoding method of this implementation. $uch apparatuses include, for example, programmed decoders, programmedprocessors hardware implementations, or processor readable media having instructions for performing the decoding method, The systems1500 and 1600, for example, may implement the decoding method of this implementation. Note that the term "suppismentari as used above, for example, in referring to"supplemental SPS" isla descriptive term. Assch supplemental SPS" does not preclude units that donot include the term *supplenental" in the unit name. Accordingly and by Way of example, a current draft of the SVC extension defines a "subset SPS" syntaxstructure and the"subset SI'S syntax structure is fully encompassed'by the descripive term:supplemental".

$37

So that the "subset SPS' of the urrentVCextension is one implementation of a:SUPSPS asdescribed inthisdisctsure, Implementationmay use other types of messages in addition to, or as a replacement for, the SPS NAL units and/or the SOP SPS NAL units. For example, at least one, implementations generates, transmits, receives, accesses, and parses other parameter sets having layer-dependent information. Further although SPS and supplermental SPS have been: discussed largely in the contextof H264standards other standards also may incude SPS, supplemental SPS or variations of SPS or supplemental SP. Accordiy, other standards (existing or future-developed) may include structures refrred to as SPS orsupplemental 8PS andsuch structures may beidentical to or be variations ofthe SPSansupplementabPS described herein. Such otherStandards mayfor example berelatedto current .264 standards (for exampeanamendment to an existing H.264 standard) orbhe completely new standards. Alternatively, other standards (existing or future developed) may include structures that are not referred to as SPS or supplemental PS, but suchstructures may be identicalto, analogous to, or variations of theSPS or supplementalIPS-described herein. Note that a parameter set is a set of data including parameters. For example,an SPS, a PPSor a supplemental $P. i vaeiousimplementations, data is saidto be. accessed "Accessing! data may include, for example, reeiving, storing, transmitting, or proceing data. 2' Various implementations are, provided and described,. These implementations can be used to solve a variety of problems. One such problem arises when multiple interoperability points(lPs)(alsoreferredtoa layers) need different values forparameters that are typicallycarried in the SPS. There isno adequate method to transmit the layer dependent syntax elements in the SPS for different:layers havingthesmePS identifier. It is problematic to sendseparate SPS data for each such layer For example, in many existing systems baselayer and its composite temporal layers share the:same SPS identifier. Several implementations prode a different NAL unitype for supplemental SPS data, Thus, multiple NAL units may be sent, and each NAL unit may include supplementalSPS information for a different S0laye but each NAL unit may be identified by the same NAL unittype. The supplementalS PSInformation may, in oneimplementation, be provided in the "subset SPS"NAL unit type of the current SVC extension. Itshould be clearthat theimplementations describedin this disclosure are not restricted to the SVC extension or to any other standard. The concepts and features, of the disclosed implementations may be used with otherstandards that exist now or are developed in the futureor may be used in systems that do not adhere to any standard. As one example the concepts and features disclosed herein may be usedfor implementations that work in the environment of the MVC extension. For exampleMVG views may need different SPS information, or'SVC layers supported within the MVCextension may need different SPS:information Additionay, features and aspects of described implementations may also be adapted for yet other implementations. Accordingly although implementations described herein may be described in the cntext SPS for SVC layers,such desriptionsshould in no way be taken as imitih the features and concepts to such implementations or contexts. The,.implementations described herein may be implemented in, or example, a method or process, an apparatusor a software program, Even if only discussed inthe context of a single form of implementation (for example, discussed only as a method) the implementation of features discussed may also be implemented in other forms(for example, an apparatus or program) An apparatus may beimplemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented i, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including ,for example acomputer,a miroprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/pesonaldigitaj assistants ("PDAs"), and other devices.that facilitate communication of informaton between adusers4 Implementation of the various processes and features described herein may be embodied in a variety of different equipment or applications, S particular, for example, equipment or applications associated With data encoding and decoding Examples ofequipment include video coders, video decodes, video codes, web servers, settop boxes, laptops, personal computers, cell phones, PDAsand other communication devices. As should be clear the equipm entmaybe mobile and even installed ina mobile vehicle, Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions may be stored on a processorreadable medium such as, for example, an integrated circuit? a softwararrierorother storage device such asforexampe, a hard disk, a compact diskette, a random access memory (RAM") or a read-only memory (ROMI) The instructins may form an application program tangibly embodied on a: processor-readable medium. instructions may be, for example, in hardwarefirmware software,oracombination. Instructionsmay befoundin, for exampiean operating system,a separate application, or a combination of the two. A processor may be charaderized, therefore, as, for example, both a device configured to carry out-aprocess and a device that includes a computer readable medium having instructions for carryingout a process, As will be evident to oneofskill intheart,implementationsmay produce a variety of signals formatted to carry information that maybefor example, stored or transmitted. The information may include,for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading thesyntaxofa describederbodimert ortoarryas data the actual syntaxvalues written by a described embodiment Such signal may be formatted.for example, as an electrognetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal The formatting May include, for example, encoding a data stream and modulating acarrierwiththeencoded data stream. The information that the signal carries may be, for example, analog digital information. The signal may be transmitted over a variety of differentwired or wireless linksas is known. A number of implementations have been described. Nevertheless, itwill be understoodthat various modifications maybe made. For example, elementsof different implementationsmay be combined,.supplemented, modifiedor removed to produce other implementations. Additionally, one of ordinary skill will undrstand that other structures and processes may besubstituted for those disclosed and the resulting implementations will perform at leastsubstantially the same function(s) ,in at least substantially the same way(s), to achieve at least substantially the same resut(s) as the implementations disclosed. Accordingly these and other implementations are contemplatedby tfis application and are within the scope of the following claims. Where'the terms"comprise", "comprises","comprised" or "comprising"are used in this specification(inpcluding the claims) they're tpbe interpreted as specifyingthe presence ofthe stated features, intgers, teps orcomponents, but notprecluding the presence of one ormore other features, integers, steps or components, or group thereof

Claims

1. An image decoding method comprising: accessing information from a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first layer encoding of an image in a sequence of images; accessing supplemental information from a supplemental SPS NAL unit having an available NAL unit type code that is a different NAL unit type code from that of the SPS NAL unit, and having a different syntax structure than the SPS NAL unit, and the supplemental information from the supplemental SPS NAL unit describing a parameter for use in decoding a second-layer encoding of the image in the sequence of images; and decoding the first-layer encoding, and the second-layer encoding, based on, respectively, the accessed information from the SPS NAL unit, and the accessed supplemental information from the supplemental SPS NAL unit wherein the supplemental SPS NAL unit comprises video usability information ("VU") parameter and hypothetical reference decoder ("HRD") parameters.

2. The method of claim 1, wherein the supplemental SPS NAL unit comprises additional parameters for a third layer.

3. The method of claim 2, wherein the additional parameters are optional.

4. An image processing system comprising: a parsing unit to access information from a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first-layer encoding of an image in a sequence of images and to access supplemental information from a supplemental SPS NAL unit having an available NAL unit type code that is a different NAL unit type code from that of the SPS NAL unit, and having a different syntax structure than the SPS NAL unit, and the supplemental information from the supplemental SPS NAL unit describing a parameter for use in decoding a second-layer encoding of the image in the sequence of images; and a decoder to decode the first-layer encoding, and the second-layer encoding, based on, respectively, the accessed information from the SPS NAL unit, and the accessed supplemental information from the supplemental SPS NAL unit wherein the supplemental SPS NAL unit comprises video usability information ("VUl") parameter and hypothetical reference decoder ("HRD") parameters.

5. The image processing system of claim 4, wherein the supplemental SPS NAL unit comprises additional parameters for a third layer.

6. The image processing system of claim 5, wherein the additional parameters are optional.

7. A non-transitory computer-readable medium containing instructions, which if executed by a computing system, causes the computing system to perform a method comprising: accessing information from a sequence parameter set ("SPS") network abstraction layer ("NAL") unit, the information describing a parameter for use in decoding a first layer encoding of an image in a sequence of images; accessing supplemental information from a supplemental SPS NAL unit having an available NAL unit type code that is a different NAL unit type code from that of the SPS NAL unit, and having a different syntax structure than the SPS NAL unit, and the supplemental information from the supplemental SPS NAL unit describing a parameter for use in decoding a second-layer encoding of the image in the sequence of images; and decoding the first-layer encoding, and the second-layer encoding, based on, respectively, the accessed information from the SPS NAL unit, and the accessed supplemental information from the supplemental SPS NAL unit wherein the supplemental SPS NAL unit comprises video usability information ("VUl") parameter and hypothetical reference decoder ("HRD") parameters.

8. The non-transitory computer-readable medium of claim 7, wherein the supplemental SPS NAL unit comprises additional parameters for a third layer.

9. The non-transitory computer-readable medium of claim 8, wherein the additional parameters are optional.