WO2015137432A1 - 画像復号装置、及び画像復号方法 - Google Patents
画像復号装置、及び画像復号方法 Download PDFInfo
- Publication number
- WO2015137432A1 WO2015137432A1 PCT/JP2015/057251 JP2015057251W WO2015137432A1 WO 2015137432 A1 WO2015137432 A1 WO 2015137432A1 JP 2015057251 W JP2015057251 W JP 2015057251W WO 2015137432 A1 WO2015137432 A1 WO 2015137432A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- layer
- output
- decoding
- target
- picture
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
Definitions
- the present invention relates to an image decoding apparatus and an image decoding method for decoding hierarchically encoded data obtained by hierarchically encoding an image.
- One of information transmitted in a communication system or information recorded in a storage device is an image or a moving image. 2. Description of the Related Art Conventionally, a technique for encoding an image for transmitting and storing these images (hereinafter including moving images) is known.
- Non-patent Document 1 As video encoding methods, AVC (H.264 / MPEG-4 Advanced Video Coding) and HEVC (High-Efficiency Video Coding), which is a successor codec, are known (Non-patent Document 1).
- a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and obtained by subtracting the predicted image from the input image (original image).
- Prediction residuals (sometimes referred to as “difference images” or “residual images”) are encoded.
- examples of the method for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).
- SHVC Scalable HEVC
- MV-HEVC MultiView HEVC
- SHVC supports spatial scalability, temporal scalability, and SNR scalability.
- spatial scalability an image downsampled from an original image to a desired resolution is encoded as a lower layer.
- inter-layer prediction is performed in order to remove redundancy between layers (Non-patent Document 2).
- MV-HEVC supports viewpoint scalability (view scalability). For example, when encoding three viewpoint images, that is, viewpoint image 0 (layer 0), viewpoint image 1 (layer 1), and viewpoint image 2 (layer 2), the viewpoint image that is the upper layer from the lower layer (layer 0) 1. Redundancy between layers can be removed by predicting the viewpoint image 2 by inter-layer prediction (Non-patent Document 3).
- each layer belonging to a specified target output layer set is decoded from input hierarchical encoded data, and a decoded picture of a layer specified as an output layer is output.
- a layer set indicating a set of layers, an output layer flag specifying which layer of the layer set is an output layer, and profile / level information corresponding to each layer set (hereinafter, PTL information) , HRD information, DPB information, etc. are decoded / encoded.
- the output layer sets OLS # 0 to OLS # have layer sets LS # 0 to the output layer set subscripts (also called output layer set identifiers).
- the output layer of each output layer set is determined by the value of default output layer identifier (default_target_ouput_layer_idc). For example, when the value of the default output layer identifier is 0, all layers of the output layer set are output layers.
- An output layer is specified by an output layer flag (output_layer_flag) that is explicitly notified.
- Non-Patent Document 4 shows that, as a stereo profile constraint (profile constraint) of MV-HEVC, a sub-bitstream extracted with a stereo profile does not include an auxiliary picture layer.
- the decoding process is executed with all layers included in the output layer set as decoding targets. Therefore, there is a problem that it is necessary to execute a decoding process for a layer that is not necessary for decoding an output layer.
- the layer L # 1 and the layer L # 0 are independent (not referred to).
- decoding of the output layer L # 1 and the layer L # 0 that is a non-output / non-reference layer is also executed.
- the present invention has been made in view of the above problems, and an object of the present invention is to eliminate the non-output / non-reference layer decoding process in the output layer set, thereby eliminating the process required for decoding the non-output / non-reference layer.
- An object is to realize an image decoding apparatus capable of reducing the amount and the memory size. Further, by reducing the redundancy of DPB information and PTL information related to an output layer set that refers to the same layer set, an image decoding device capable of decoding / encoding DPB information and PTL information with a smaller code amount, and An image encoding device is to be realized.
- Non-Patent Document 4 requires that the sub-bitstream does not include an auxiliary picture layer in order to omit decoding of the auxiliary picture, which is not essential, but there is an auxiliary picture layer in the output layer set. There is a problem that the decoding process of the auxiliary picture layer cannot be omitted.
- the present invention has been made in view of the above problems, and an object of the present invention is to decode the auxiliary picture layer by omitting the decoding process of the auxiliary picture layer when the auxiliary picture layer is included in the output layer set.
- An object of the present invention is to realize an image decoding apparatus capable of reducing the processing amount and the memory size.
- an image decoding apparatus is an image decoding apparatus that decodes hierarchical image encoded data, First flag decoding means for decoding a first flag indicating whether or not each layer is included in the layer set in units of layer sets; Layer set information decoding means for deriving a layer ID list of the layer set based on the first flag; For each output layer set, a) Layer set identifier, and b) output layer set information decoding means for decoding output layer set information including an output layer flag indicating whether each layer included in the output layer set is an output layer; Dependency flag deriving means for deriving a dependency flag indicating whether or not the first layer is a reference layer of the second layer; Based on the layer ID list indicating the configuration of the layer set corresponding to the output layer set, the output layer flag of the output layer set, and the dependency flag, a decoding layer ID list indicating a layer to be decoded in the output layer set Derived decoding layer ID list derivation means;
- An image decoding method is an image decoding method for decoding hierarchical image encoded data, A first flag decoding step of decoding a first flag indicating whether or not each layer is included in the layer set in units of layer sets; A layer set information decoding step for deriving a layer ID list of the layer set based on the first flag; For each output layer set, a) Layer set identifier, and b) an output layer set information decoding step for decoding output layer set information including an output layer flag indicating whether or not each layer included in the output layer set is an output layer; A dependency flag deriving step for deriving a dependency flag indicating whether or not the first layer is a reference layer of the second layer; Based on the layer ID list indicating the configuration of the layer set corresponding to the output layer set, the output layer flag of the output layer set, and the dependency flag, a decoding layer ID list indicating a layer to be decoded in the output layer set A deriving decoding layer ID list deriving step; A picture decoding step of
- the processing amount required for decoding the non-output / non-reference layer and the memory size can be reduced. Is possible.
- the present invention by omitting the decoding process of the auxiliary picture layer in the output layer set, it is possible to reduce the processing amount and the memory size required for decoding the auxiliary picture layer.
- FIG. 1 It is a figure which shows the example of the syntax contained in a NAL unit layer.
- (A) shows an example of syntax constituting the NAL unit layer
- (b) is a diagram showing an example of syntax of the NAL unit header.
- (a) is a sequence layer which prescribes
- (b) is a picture layer which prescribes
- (c) is a slice layer that defines a slice S
- (d) is a slice data layer that defines slice data
- (e) is a coding tree layer that defines a coding tree unit included in the slice data
- (f) ) Is a diagram illustrating a coding unit layer that defines a coding unit (Coding Unit; CU) included in a coding tree. It is a figure for demonstrating the reference relationship of the parameter set which concerns on this embodiment.
- (A) shows an example of a reference picture list
- (b) is a conceptual diagram showing an example of a reference picture.
- (A) is a figure which shows the correspondence table of a scalable identifier and a scalability type
- (b) is a pseudo code which shows an example of the derivation
- (c) is an example of the syntax table which concerns on a scalable identifier It is. It is an example of the syntax table of DPB information which concerns on embodiment of this invention.
- (A) is an example of DPB information of the output layer set OLS # 0
- (A) is an example of an SPS syntax table
- (b) is an example of a PPS syntax table
- (c) is a syntax table of slice headers and slice data included in a slice layer.
- (D) shows an example of a slice header syntax table
- (e) shows an example of a slice data syntax table.
- (A) shows a transmission device equipped with a hierarchical video encoding device, and (b) shows a reception device equipped with a hierarchical video decoding device. It is the figure shown about the structure of the recording device carrying the said hierarchy moving image encoder, and the reproducing
- (A) shows a recording device equipped with a hierarchical video encoding device, and (b) shows a playback device equipped with a hierarchical video decoding device.
- the hierarchical moving picture decoding apparatus 1 and the hierarchical moving picture encoding apparatus 2 according to an embodiment of the present invention will be described based on FIGS. 2 to 29 as follows.
- a hierarchical video decoding device (image decoding device) 1 decodes encoded data that has been hierarchically encoded by a hierarchical video encoding device (image encoding device) 2.
- Hierarchical coding is a coding scheme that hierarchically encodes moving images from low quality to high quality.
- Hierarchical coding is standardized in SVC and SHVC, for example.
- the quality of a moving image here widely means an element that affects the appearance of a subjective and objective moving image.
- the quality of the moving image includes, for example, “resolution”, “frame rate”, “image quality”, and “pixel representation accuracy”.
- the quality of the moving image is different, it means that, for example, “resolution” is different, but it is not limited thereto.
- the quality of moving images is different from each other.
- the hierarchical coding technique is (1) spatial scalability, (2) temporal scalability, (3) SNR (Signal to Noise Ratio) scalability, and (4) view scalability from the viewpoint of the type of information to be hierarchized. May be classified. Spatial scalability is a technique for hierarchizing resolution and image size. Time scalability is a technique for layering at a frame rate (number of frames per unit time). SNR scalability is a technique for layering in coding noise. Also, view scalability is a technique for hierarchizing at the viewpoint position associated with each image.
- the hierarchical video encoding device 2 Prior to detailed description of the hierarchical video encoding device 2 and the hierarchical video decoding device 1 according to the present embodiment, first, (1) the hierarchical video encoding device 2 generates and the hierarchical video decoding device 1 performs decoding.
- the layer structure of the hierarchically encoded data to be performed will be described, and then (2) a specific example of the data structure that can be adopted in each layer will be described.
- FIG. 2 is a diagram schematically illustrating a case where a moving image is hierarchically encoded / decoded by three layers of a lower layer L3, a middle layer L2, and an upper layer L1. That is, in the example shown in FIGS. 2A and 2B, of the three layers, the upper layer L1 is the highest layer and the lower layer L3 is the lowest layer.
- a decoded image corresponding to a specific quality that can be decoded from hierarchically encoded data is referred to as a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, in the upper hierarchy L1).
- Decoded image POUT # A a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, in the upper hierarchy L1).
- FIG. 2A shows a hierarchical moving image encoding apparatus 2 # A to 2 # C that generates encoded data DATA # A to DATA # C by hierarchically encoding input images PIN # A to PIN # C, respectively. Is shown.
- FIG. 2B shows a hierarchical moving picture decoding apparatus 1 # A ⁇ that generates decoded images POUT # A ⁇ POUT # C by decoding the encoded data DATA # A ⁇ DATA # C, which are encoded hierarchically. 1 # C is shown.
- the input images PIN # A, PIN # B, and PIN # C that are input on the encoding device side have the same original image but different image quality (resolution, frame rate, image quality, and the like).
- the image quality decreases in the order of the input images PIN # A, PIN # B, and PIN # C.
- the hierarchical video encoding device 2 # C of the lower hierarchy L3 encodes the input image PIN # C of the lower hierarchy L3 to generate encoded data DATA # C of the lower hierarchy L3.
- Basic information necessary for decoding the decoded image POUT # C of the lower layer L3 is included (indicated by “C” in FIG. 2). Since the lower layer L3 is the lowest layer, the encoded data DATA # C of the lower layer L3 is also referred to as basic encoded data.
- the hierarchical video encoding apparatus 2 # B of the middle hierarchy L2 encodes the input image PIN # B of the middle hierarchy L2 with reference to the encoded data DATA # C of the lower hierarchy, and performs the middle hierarchy L2 Encoded data DATA # B is generated.
- additional data necessary for decoding the decoded image POUT # B of the intermediate hierarchy is added to the encoded data DATA # B of the intermediate hierarchy L2.
- Information (indicated by “B” in FIG. 2) is included.
- the hierarchical video encoding apparatus 2 # A of the upper hierarchy L1 encodes the input image PIN # A of the upper hierarchy L1 with reference to the encoded data DATA # B of the intermediate hierarchy L2 to Encoded data DATA # A is generated.
- the encoded data DATA # A of the upper layer L1 is used to decode the basic information “C” necessary for decoding the decoded image POUT # C of the lower layer L3 and the decoded image POUT # B of the middle layer L2.
- additional information indicated by “A” in FIG. 2 necessary for decoding the decoded image POUT # A of the upper layer is included.
- the encoded data DATA # A of the upper layer L1 includes information related to decoded images of different qualities.
- the decoding device side will be described with reference to FIG.
- the decoding devices 1 # A, 1 # B, and 1 # C corresponding to the layers of the upper layer L1, the middle layer L2, and the lower layer L3 are encoded data DATA # A and DATA # B, respectively.
- And DATA # C are decoded to output decoded images POUT # A, POUT # B, and POUT # C.
- a part of information of the higher layer encoded data is extracted (also referred to as bitstream extraction), and a specific quality moving image is obtained by decoding the extracted information in a lower specific decoding device. It can also be played.
- the hierarchy decoding apparatus 1 # B of the middle hierarchy L2 receives information necessary for decoding the decoded image POUT # B from the hierarchy encoded data DATA # A of the upper hierarchy L1 (that is, the hierarchy encoded data DATA # A decoded image POUT # B may be decoded by extracting “B” and “C”) included in A.
- the decoded images POUT # A, POUT # B, and POUT # C can be decoded based on information included in the hierarchically encoded data DATA # A of the upper hierarchy L1.
- the hierarchical encoded data is not limited to the above three-layer hierarchical encoded data, and the hierarchical encoded data may be hierarchically encoded with two layers or may be hierarchically encoded with a number of layers larger than three. Good.
- Hierarchically encoded data may be configured as described above. For example, in the example described above with reference to FIGS. 2A and 2B, it has been described that “C” and “B” are referred to for decoding the decoded image POUT # B, but the present invention is not limited thereto. It is also possible to configure the hierarchically encoded data so that the decoded image POUT # B can be decoded using only “B”. For example, it is possible to configure a hierarchical video decoding apparatus that receives the hierarchically encoded data composed only of “B” and the decoded image POUT # C for decoding the decoded image POUT # B.
- Hierarchically encoded data can also be generated so that In that case, the lower layer hierarchical video encoding device generates hierarchical encoded data by quantizing the prediction residual using a larger quantization width than the upper layer hierarchical video encoding device. To do.
- a profile assumes a specific application and prescribes the processing functions that a standard-compliant decoder should have, and is defined by a combination or set of encoding tools (elemental technologies). By defining the profile, it is only necessary to mount a suitable profile in each application, not the entire standard, and there is an advantage that the complexity of the decoder / encoder can be reduced.
- Level The level specifies the upper limit of the processing capacity of the decoder and the range of the circuit scale. The maximum number of processing pixels per unit time, the maximum resolution of the image, the maximum bit rate, the maximum reference image buffer size, the minimum It defines parameter restrictions such as the compression ratio. That is, the level defines the processing capability of the decoder and the complexity of the bit stream. In addition, the level also specifies to what extent the tools specified in each profile are supported. Therefore, the upper level needs to support the lower level.
- the level-limited parameters include the maximum luminance picture size (Max (luma picture size), the maximum bit rate (Max bitrate), the maximum CPB size (Max CPB size), and the maximum number of slice segments per picture unit (Max slice) segments per picture), the maximum number of tile rows per picture unit (Max number of tile rows), the maximum number of tile columns per picture unit (Max number of tile columns), and the like.
- the various level-restricted parameters that apply to a specific profile include the maximum luminance sample rate (Max luma sample rate), maximum bitrate (Max bit rate), and minimum compression ratio (Mincompression Ratio). Can be mentioned.
- HRD is a virtual model of the decoder centered on the operation of the buffer, also called the buffer model.
- the HRD includes (1) CPB (Coded Picture Buffer), which is a bitstream transmission buffer, (2) a decoding processing unit that performs an instantaneous decoding operation, and (3) a DPB that stores a decoded picture ( Decoded Picture ⁇ Buffer (decoded picture buffer), and (4) a cropping processing unit (a process for cutting out only an effective area of an image) for performing a cutting process.
- CPB Coded Picture Buffer
- Decoding processing unit that performs an instantaneous decoding operation
- DPB that stores a decoded picture
- Decoded Picture ⁇ Buffer decoded picture buffer
- HRD The basic operation of HRD is as follows.
- SA01 store the input bitstream in the CPB;
- SA02 Instantaneously decode AU stored in CPB;
- SA03 Save the decoded picture obtained by the instantaneous decoding process to the DPB;
- SA04 Crop the decoded picture stored in DPB and output it;
- HRD parameter is a parameter indicating a buffer model used for verifying whether the input bitstream satisfies the conformance condition in the HRD.
- Bitstream conformance is a condition that a bitstream to be decoded by the hierarchical video decoding device (here, the hierarchical video decoding device according to the embodiment of the present invention) needs to be satisfied.
- the bitstream generated by the hierarchical video encoding device (here, the hierarchical video encoding device according to the embodiment of the present invention) is also a bitstream that can be decoded by the hierarchical video decoding device. In order to guarantee, it is necessary to satisfy the bit stream conformance.
- VCL NAL unit VCL (Video Coding Layer) ⁇ NAL unit is a NAL unit that includes encoded data of moving images (video signals).
- the VCL NAL unit includes slice data (CTU encoded data) and header information (slice header) commonly used through decoding of the slice.
- Non-VCL NAL unit Non-VCL (non-Video Coding ⁇ Layer, non-video coding layer, non-VCL) NAL unit is a sequence or picture of video parameter set VPS, sequence parameter set SPS, picture parameter set PPS, etc. Is a NAL unit including encoded data such as header information and auxiliary information SEI, which is a set of encoding parameters used when decoding.
- a layer identifier (also referred to as a layer ID) is for identifying a layer (layer), and corresponds to the layer one-to-one.
- the hierarchically encoded data includes an identifier used for selecting partial encoded data necessary for decoding a decoded image of a specific hierarchy.
- a subset of hierarchically encoded data associated with a layer identifier corresponding to a specific layer is also referred to as a layer representation.
- a layer representation of the layer and / or a layer representation corresponding to a lower layer of the layer is used. That is, in decoding the decoded image of the target layer, layer representation of the target layer and / or layer representation of one or more layers included in a lower layer of the target layer are used.
- Layer A set of VCL NAL UNIT with a layer identifier value (nuh_layer_id, nuhLayerId) of a specific layer (layer) and a non-VCL NAL UNIT associated with the VCL ⁇ ⁇ NAL unit, or a syntax having a hierarchical relationship One of the set of structures.
- Upper layer A layer located above a certain layer is referred to as an upper layer.
- the upper layers of the lower layer L3 are the middle layer L2 and the upper layer L1.
- the decoded image of the upper layer means a decoded image with higher quality (for example, high resolution, high frame rate, high image quality, etc.).
- Lower layer A layer located below a certain layer is referred to as a lower layer.
- the lower layers of the upper layer L1 are the middle layer L2 and the lower layer L3.
- the decoded image of the lower layer refers to a decoded image with lower quality.
- Target layer A layer that is the target of decoding or encoding.
- a decoded image corresponding to the target layer is referred to as a target layer picture.
- pixels constituting the target layer picture are referred to as target layer pixels.
- Reference layer A specific lower layer referred to for decoding a decoded image corresponding to the target layer is referred to as a reference layer.
- a decoded image corresponding to the reference layer is referred to as a reference layer picture.
- pixels constituting the reference layer are referred to as reference layer pixels.
- the reference layers of the upper hierarchy L1 are the middle hierarchy L2 and the lower hierarchy L3.
- the hierarchically encoded data can be configured so that it is not necessary to refer to all of the lower layers in decoding of the specific layer.
- the hierarchical encoded data can be configured such that the reference layer of the upper hierarchy L1 is either the middle hierarchy L2 or the lower hierarchy L3.
- the reference layer can also be expressed as a layer different from the target layer that is used (referenced) when predicting an encoding parameter or the like used for decoding the target layer.
- a reference layer that is directly referred to in inter-layer prediction of the target layer is referred to as a direct reference layer.
- the direct reference layer B referred to in the inter-layer prediction of the direct reference layer A of the target layer is also called an indirect reference layer of the target layer because the target layer depends indirectly.
- layer i depends indirectly on layer j via one or more layers k (i ⁇ k ⁇ j)
- layer j is an indirect reference layer for layer i .
- the direct reference layer and the indirect reference layer for the target layer are collectively referred to as a dependency layer.
- the layer located at the lowest layer is called the base layer.
- the decoded image of the base layer is the lowest quality decoded image that can be decoded from the encoded data, and is referred to as a basic decoded image.
- the basic decoded image is a decoded image corresponding to the lowest layer.
- the partially encoded data of the hierarchically encoded data necessary for decoding the basic decoded image is referred to as basic encoded data.
- the basic information “C” included in the hierarchically encoded data DATA # A of the upper hierarchy L1 is the basic encoded data.
- the base layer is a layer composed of one or a plurality of VCL NAL units having at least the same layer identifier and the value of the layer identifier (nuh_layer_id) being zero.
- Extension layer (non-base layer): The upper layer of the base layer is called the extension layer.
- the enhancement layer is a layer including at least one VCL ⁇ ⁇ ⁇ NAL unit having at least the same layer identifier and having a value of the layer identifier (nuh_layer_id) larger than zero.
- Inter-layer prediction is based on the syntax element value, the value derived from the syntax element value included in the layer expression of the layer (reference layer) different from the layer expression of the target layer, and the decoded image. It is to predict the syntax element value of the target layer, the encoding parameter used for decoding of the target layer, and the like. Inter-layer prediction in which information related to motion prediction is predicted from reference layer information is sometimes referred to as inter-layer motion information prediction. In addition, inter-layer prediction predicted from a lower layer decoded image may be referred to as inter-layer image prediction (or inter-layer texture prediction). Note that the hierarchy used for inter-layer prediction is, for example, a lower layer of the target layer. In addition, performing prediction within a target layer without using a reference layer may be referred to as intra-layer prediction.
- Temporal identifier is an identifier for identifying a layer related to temporal scalability (hereinafter, sub-layer).
- the temporal identifier is for identifying the sublayer, and corresponds to the sublayer on a one-to-one basis.
- the encoded data includes a temporal identifier used for selecting partial encoded data necessary for decoding a decoded image of a specific sublayer.
- the temporal (highest) sublayer temporal identifier is referred to as the highest (highest) temporal identifier (highest TemporalId, highestTid).
- a sublayer is a layer related to temporal scalability specified by a temporal identifier. In order to distinguish from other scalability such as spatial scalability, SNR scalability, and the like, they are hereinafter referred to as sub-layers (also referred to as temporal layers). In the following description, temporal scalability is assumed to be realized by sublayers included in encoded data of the base layer or hierarchically encoded data necessary for decoding a certain layer.
- a layer set is a set of layers composed of one or more layers.
- Output layer set is a set of layers that are designated as output layers for the layers included in the layer set.
- the output layer set is also expressed as a set expressed by a combination of a layer set and an output layer flag that specifies the output layer.
- the output layer set identified by the identifier i is referred to as OLS # i.
- Output layer is a layer in the above output layer set that is designated to output a decoded picture of the layer as an output picture among the layers to be decoded or encoded.
- Alternative output layer In the above output layer set, if the decoded image of the layer specified as the output layer cannot be decoded for some reason, the alternative output layer is output with the decoded image of a layer different from the output layer as an alternative. Layer.
- Bitstream extraction processing is a layer ID list LayerIdList that represents a layer included in the target layer set from the target highest temporal identifier (highestTid) from a certain bitstream (hierarchical encoded data, encoded data).
- highTid target highest temporal identifier
- Bitstream extraction is also called sub-bitstream extraction.
- TargetHighestTid the target layer set
- TargetLayerSet the target layer set
- TargetLayerIdList the layer ID list (target layer ID list) of the target layer set
- TargetDecLayerIdList the layer ID list to be decoded
- a bit stream composed of NAL units included in the target set TargetSet generated by bit stream extraction is also referred to as encoded data BitstreamToDecode.
- FIG. 3 shows three layers (L # 0, L # 1, L # 2), and each layer consists of a layer set A consisting of three sublayers (TID1, TID2, TID3), and a layer set B that is a subset thereof.
- code L # N indicates a certain layer N
- each box in FIG. 3 represents a picture
- the numbers in the boxes represent an example of decoding order.
- the number N in the picture is denoted as P # N.
- the arrows between the pictures indicate the dependency direction (reference relationship) between the pictures.
- An arrow in the same layer indicates a reference picture used for inter prediction.
- An arrow between layers indicates a reference picture (also referred to as a reference layer picture) used for inter-layer prediction.
- AU in FIG. 3 represents an access unit
- symbol #N represents an access unit number
- AU # N represents the (N ⁇ 1) th access unit if the AU at a certain starting point (for example, random access start point) is AU # 0, and represents the order of AUs included in the bitstream. . That is, in the example of FIG. 3, on the bitstream, the access units are arranged in the order of AU # 0, AU # 1, AU # 2, AU # 3, AU # 4.
- the access unit represents a set of NAL units aggregated according to a specific classification rule.
- AU # 0 in FIG. 3 can be regarded as a set of VCL NAL including encoded data of pictures P # 1, P # 1, and P # 3. Details of the access unit will be described later. In this specification, when the Xth element is described, the first element is the 0th element, and counting is performed from the 0th element (the same applies hereinafter).
- the bit stream including the layer set A is changed to the target set TargetSet.
- the dotted box represents the discarded picture
- the dotted arrow indicates the dependency direction between the discarded picture and the reference picture. It should be noted that the dependency relationship has already been cut off because the NAL units constituting the sub-layer pictures of layer L # 3 and TID3 have been discarded.
- SHVC and MV-HEVC introduce the concept of layers and sub-layers in order to realize SNR scalability, spatial scalability, temporal scalability, and so on.
- the encoded data of the picture highest temporal ID (TID3)
- TID3 the encoded data of the picture
- the bitstream extraction process To do.
- by discarding the encoded data of pictures (10, 13, 11, 14, 12, 15), encoded data with a frame rate of 1 ⁇ 2 is generated.
- the granularity of each scalability can be changed by discarding the encoded data of the layer that is not included in the target set TargetSet by bitstream extraction. .
- the encoded data of the pictures 3, 6, 9, 12, 15
- encoded data with a coarse scalability granularity is generated.
- the lower layer and the upper layer may be encoded by different encoding methods.
- the encoded data of each layer may be supplied to the hierarchical video decoding device 1 via different transmission paths, or may be supplied to the hierarchical video decoding device 1 via the same transmission path. .
- the base layer when transmitting ultra-high-definition video (moving image, 4K video data) with a base layer and one extended layer in a scalable encoding, the base layer downscales 4K video data, and interlaced video data. It may be encoded by MPEG-2 or H.264 / AVC and transmitted over a television broadcast network, and the enhancement layer may encode 4K video (progressive) with HEVC and transmit over the Internet.
- FIG. 4 is a diagram showing a hierarchical structure of data in the hierarchically encoded data DATA.
- the hierarchically encoded data DATA is encoded in units called NAL (Network Abstraction Layer) units.
- NAL is a layer provided to abstract communication between VCL (Video Coding Layer), which is a layer that performs video coding processing, and a lower system that transmits and stores encoded data.
- VCL Video Coding Layer
- VCL is a layer that performs image encoding processing, and encoding is performed in VCL.
- the lower system here corresponds to the H.264 / AVC and HEVC file formats and the MPEG-2 system.
- the lower system corresponds to the decoding process in the target layer and the reference layer.
- NAL a bit stream generated by VCL is divided into units called NAL units and transmitted to a lower system as a destination.
- Fig. 5 (a) shows the syntax table of the NAL unit.
- the NAL unit includes encoded data encoded by the VCL and a header (NAL unit header: nal_unit_header ()) for appropriately delivering the encoded data to a destination lower system.
- the NAL unit header is represented, for example, by the syntax shown in FIG.
- the NAL unit header includes “nal_unit_type” indicating the type of encoded data stored in the NAL unit, “nuh_temporal_id_plus1” indicating the identifier (temporal identifier) of the sublayer to which the stored encoded data belongs, and stored encoding “Nuh_layer_id” (or nuh_reserved_zero_6bits) representing the identifier (layer identifier) of the layer to which the data belongs is described.
- the NAL unit data includes a parameter set, SEI, slice and the like which will be described later.
- FIG. 6 is a diagram showing the relationship between the value of the NAL unit type and the type of the NAL unit.
- the NAL unit having a NAL unit type of 0 to 15 indicated by SYNA 101 is a slice of non-RAP (random access picture).
- a NAL unit having a NAL unit type of 16 to 21 indicated by SYNA 102 is a slice of RAP (random access picture, IRAP picture).
- RAP pictures are broadly classified into BLA pictures, IDR pictures, and CRA pictures.
- BLA pictures are further classified into BLA_W_LP, BLA_W_DLP, and BLA_N_LP.
- IDR pictures are further classified into IDR_W_DLP and IDR_N_LP.
- Pictures other than the RAP picture include a leading picture (LP picture), a temporal access picture (TSA picture, STSA picture), and a trailing picture (TRAIL picture).
- LP picture leading picture
- TSA picture temporal access picture
- TRAIL picture trailing picture
- the encoded data in each layer is stored in the NAL unit, is NAL-multiplexed, and is transmitted to the hierarchical moving image decoding apparatus 1.
- each NAL unit is classified into data (VCL data) constituting a picture and other data (non-VCL) according to the NAL unit type.
- Pictures are all classified into VCL NAL units, regardless of picture types such as random access pictures, leading pictures, and trailing pictures, and parameter sets that are data necessary for decoding pictures, SEI that is auxiliary information of pictures, Access unit delimiters (AUD), end-of-sequence (EOS), end-of-bit stream (EOB), etc., representing sequence delimiters are classified as non-VCL NAL units.
- Access unit A set of NAL units aggregated according to a specific classification rule is called an access unit.
- the access unit is a set of NAL units constituting one picture.
- the access unit is a set of NAL units constituting pictures of a plurality of layers at the same time (same output timing).
- the encoded data may include a NAL unit called an access unit delimiter (AUD).
- the access unit delimiter is included between a set of NAL units constituting the access unit in the encoded data and a set of NAL units constituting another access unit.
- FIG. 7 is a diagram showing an example of the configuration of the NAL unit included in the access unit.
- AU has an access unit delimiter (AUD) indicating the head of AU, various parameter sets (VPS, SPS, PPS), various SEI (Prefix SEI, Suffix SEI), and the number of layers is 1.
- VCL silice
- PPS parameter sets
- SEI Prefix SEI
- Suffix SEI SEI
- the number of layers is 1.
- the code L # K (K Nmin ...
- Nmax Nmax after VPS, SPS, SEI, VCL represents a layer ID (or an index indicating the order of layers defined on the VPS).
- the SPS, PPS, SEI, VCL of each layer L # Nmin to layer L # Nmax indicates the layer ID (or the order of the layers defined on the VPS) except for the VPS in the AU. Exists in ascending order.
- the VPS is transmitted only with the lowest layer ID.
- an arrow indicates whether the specific NAL unit exists in the AU or repeatedly exists.
- a specific NAL unit exists in the AU, it is indicated by an arrow passing through the NAL unit, and if a specific NAL unit does not exist in the AU, it is indicated by an arrow skipping the NAL unit.
- an arrow heading to the VPS without passing through the AUD indicates a case where the AUD does not exist in the AU.
- An arrow that passes through the VCL and returns to the VCL again indicates a case where one or more VCLs exist.
- VPS having an upper layer ID other than the lowest order may be included in the AU, but the image decoding apparatus ignores a VPS having a layer ID other than the lowest order.
- various parameter sets (VPS, SPS, PPS) and SEI as auxiliary information may be included as part of the access unit as shown in FIG. 7, or transmitted to the decoder by means other than the bit stream. May be. Note that FIG. 7 is only one embodiment of the configuration of the NAL unit included in the access unit, and the configuration of the NAL unit included in the access unit can be changed within a range where the bitstream can be decoded. .
- An IRAP access unit that performs initialization of decoding processing of all layers included in the target set is referred to as an initialization IRAP access unit.
- an initialized IRAP access unit is followed by zero or more non-initialized IRAP access units (access units other than the initialized IRAP access unit), and a set of access units up to the next initialized IRAP access unit ( However, the next initialization IRAP access unit is excluded) is also referred to as CVS (Coded Video Sequence; hereinafter also referred to as sequence SEQ).
- FIG. 8 is a diagram showing a hierarchical structure of data in the hierarchically encoded data DATA.
- Hierarchically encoded data DATA illustratively includes a sequence and a plurality of pictures constituting the sequence.
- (A) to (f) of FIG. 8 respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, a slice data layer that defines slice data, and a slice data.
- sequence layer a set of data referred to by the image decoding device 1 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined.
- the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an additional extension.
- Information SEI Supplemental Enhancement Information
- # indicates the layer ID.
- FIG. 8 shows an example in which encoded data with # 0 and # 1, that is, layer ID 0 and layer ID 1 exists, the type of layer and the number of layers are not limited to this.
- FIG. 11 is an example of a syntax table of the video parameter set VPS
- FIG. 12 is an example of an extension data syntax table of the video parameter set VPS.
- a set of encoding parameters referred to by the image decoding apparatus 1 in order to decode encoded data composed of one or more layers is defined.
- a VPS identifier (video_parameter_set_id) (SYNVPS01 in FIG. 11) used to identify a VPS referred to by a sequence parameter set and other syntax elements described later, and the number of layers (vps_max_layers_minus1) included in the encoded data (FIG.
- output layer set information defining the output layer set constituting the output layer set, PTL information, etc.
- VPS used for decoding a specific sequence belonging to a certain layer
- VPS means an active VPS for a target sequence belonging to a certain layer.
- FIG. 17A is an example of a syntax table of the sequence parameter set SPS.
- sequence parameter set SPS a set of encoding parameters referred to by the image decoding apparatus 1 for decoding the target sequence is defined.
- an active VPS identifier sps_video_parameter_set_id
- SYNSPS01 in FIG. 17 (a) representing an active VPS referred to by the target SPS
- An identifier (sps_seq_parameter_set_id) SYNSPS02 in FIG. 17 (a)) and the width and height of a picture are defined.
- a plurality of SPSs may exist in the encoded data. In this case, an SPS used for decoding is selected from a plurality of candidates for each target sequence.
- SPS used for decoding a specific sequence belonging to a certain layer is also called an active SPS.
- SPS means an active SPS for a target sequence belonging to a certain layer.
- FIG. 17B is an example of a syntax table of the picture parameter set PPS.
- a set of encoding parameters referred to by the image decoding apparatus 1 in order to decode each picture in the target sequence is defined.
- an active SPS identifier (pps_seq_parameter_set_id) (SYNPPS01 in FIG. 17B) representing an active SPS referred to by the target PPS
- a PPS identifier used to identify a PPS referred to by a slice header or other syntax elements described later ( pps_pic_parameter_set_id) (SYNPPS02 in FIG.
- quantization width reference value (pic_init_qp_minus26) used for picture decoding
- flag (weighted_pred_flag) indicating application of weighted prediction
- scaling list (quantization matrix) It is. Note that a plurality of PPSs may exist, and in this case, one of a plurality of PPSs is selected from each picture in the target sequence.
- PPS used for decoding specific pictures belonging to a certain layer is called active PPS.
- PPS means an active PPS for a target picture belonging to a certain layer.
- the active SPS and the active PPS may be set to different SPSs and PPSs for each layer. That is, the decoding process can be executed with reference to different SPSs and PPSs for each layer.
- Picture layer In the picture layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a picture PICT to be processed (hereinafter also referred to as a target picture) is defined.
- the picture PICT includes slices S0 to SNS-1 as shown in FIG. 8B (NS is the total number of slices included in the picture PICT).
- NS is the total number of slices included in the picture PICT.
- the reference numerals may be omitted.
- data included in hierarchically encoded data DATA which will be described below, and other subscripted data.
- slice layer In the slice layer, a set of data that the hierarchical video decoding device 1 refers to in order to decode the slice S to be processed (slice segment, also referred to as a target slice) is defined. As shown in FIG. 8C, the slice S includes a slice header SH and slice data SDATA.
- the slice header SH includes a group of encoding parameters that the hierarchical video decoding device 1 refers to in order to determine a method for decoding the target slice.
- FIG. 17D is an example of a syntax table of the slice header.
- an active PPS identifier (slice_pic_parameter_set_id) (SYNSH02 in FIG. 17 (d)) that specifies a PPS (active PPS) to be referred to for decoding the target slice is included.
- the SPS referred to by the active PPS is specified by an active SPS identifier (pps_seq_parameter_set_id) included in the active PPS.
- the VPS (active VPS) referred to by the active SPS is specified by an active VPS identifier (sps_video_parameter_set_id) included in the active SPS.
- a PPS (active PPS) used for decoding is designated (also called activation) by the identifier. Note that the identifiers of the PPS, SPS, and VPS referenced by slices in the same picture must be the same.
- the activated PPS includes an active SPS identifier that designates an SPS (active SPS) to be referred to in the decryption process, and an SPS (active SPS) used for decryption is designated by the identifier.
- the activated SPS includes an active VPS identifier that designates a VPS (active VPS) to be referred to in the decoding process of the sequence belonging to each layer, and the VPS (active VPS).
- slice type designation information for designating a slice type is an example of an encoding parameter included in the slice header SH.
- I slice using only intra prediction at the time of encoding (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
- the slice data layer a set of data referred to by the hierarchical video decoding device 1 for decoding the slice data SDATA to be processed is defined.
- the slice data SDATA includes a coded tree block (CTB).
- the CTB is a fixed-size block (for example, 64 ⁇ 64) constituting a slice, and may be called a maximum coding unit (LCU).
- the coding tree layer defines a set of data that the hierarchical video decoding device 1 refers to in order to decode the coding tree block to be processed.
- the coding tree unit is divided by recursive quadtree division.
- a tree-structured node obtained by recursive quadtree partitioning is called a coding tree.
- An intermediate node of the quadtree is a coded tree unit (CTU), and the coded tree block itself is defined as the highest CTU.
- the CTU includes a split flag (split_flag). When the split_flag is 1, the CTU is split into four coding tree units CTU.
- the coding tree unit CTU is divided into four coding units (CU: Coded Unit).
- the coding unit CU is a terminal node of the coding tree layer and is not further divided in this layer.
- the encoding unit CU is a basic unit of the encoding process.
- the partial area on the target picture decoded by the coding tree unit is called a coding tree block (CTB: “Coding” Tree ”block).
- the CTB corresponding to the luminance picture that is the luminance component of the target picture is called luminance CTB.
- luminance CTB the partial area on the luminance picture decoded from the CTU.
- color difference CTB the partial area corresponding to the color difference picture decoded from the CTU.
- the luminance CTB size and the color difference CTB size can be converted into each other. For example, when the color format is 4: 2: 2, the color difference CTB size is half of the luminance CTB size.
- the CTB size means the luminance CTB size.
- the CTU size is a luminance CTB size corresponding to the CTU.
- the encoding unit layer defines a set of data that the hierarchical video decoding device 1 refers to in order to decode the processing target encoding unit.
- the coding unit CU (coding unit) includes a CU header CUH, a prediction tree, and a conversion tree.
- the CU header CUH it is defined whether the coding unit is a unit using intra prediction or a unit using inter prediction.
- the encoding unit is the root of a prediction tree (PT) and a transform tree (TT).
- PT prediction tree
- TT transform tree
- CB coding block
- CB on the luminance picture is called luminance CB
- CB on the color difference picture is called color difference CB.
- the CU size (encoding node size) means the luminance CB size.
- the encoding unit CU is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined.
- the transform block is one or a plurality of non-overlapping areas constituting the encoding unit CU.
- the conversion tree includes one or a plurality of conversion blocks obtained by the above division. Note that information regarding the conversion tree included in the CU and information included in the conversion tree are referred to as TT information.
- the division in the transformation tree includes the one in which an area having the same size as that of the encoding unit is assigned as the transformation block, and the one in the recursive quadtree division like the above-described division in the tree block.
- the conversion process is performed for each conversion block.
- the transform block that is a unit of transform is also referred to as a transform unit (TU).
- the transformation tree TT includes TT division information SP_TT that specifies a division pattern for each transformation block of the target CU, and quantized prediction residuals QD 1 to QD NT (NT is the total number of transformation units TU included in the target CU). Is included.
- TT division information SP_TT is information for determining the shape of each conversion block included in the target CU and the position in the target CU.
- the TT division information SP_TT can be realized from information (split_transform_unit_flag) indicating whether or not the target node is divided and information (trafoDepth) indicating the depth of the division.
- Each quantization prediction residual QD is encoded data generated by the hierarchical video encoding device 2 performing the following processes 1 to 3 on a target block that is a conversion block to be processed.
- Process 1 Frequency conversion (for example, DCT (Discrete Cosine Transform) and DST (Discrete Sine Transform)) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image;
- Process 2 Quantize the transform coefficient obtained in Process 1;
- Process 3 Variable length coding is performed on the transform coefficient quantized in Process 2;
- the encoding unit CU is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined.
- the prediction block is one or a plurality of non-overlapping areas constituting the encoding unit CU.
- the prediction tree includes one or a plurality of prediction blocks obtained by the above division. Note that information regarding the prediction tree included in the CU and information included in the prediction tree are referred to as PT information.
- Prediction processing is performed for each prediction block.
- a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).
- Intra prediction is prediction within the same picture
- inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images). That is, in inter prediction, decoding on a reference picture is performed using either a reference picture (in-layer reference picture) of the same layer as the target layer or a reference picture (inter-layer reference picture) on the reference layer of the target layer as a reference picture. A predicted image is generated from the image.
- the division method is encoded by part_mode of encoded data, and 2N ⁇ 2N (the same size as the encoding unit), 2N ⁇ N, 2N ⁇ nU, 2N ⁇ nD, N ⁇ 2N, nL X2N, nRx2N, and NxN.
- the prediction image of the prediction unit is derived by a prediction parameter associated with the prediction unit.
- the prediction parameters include a prediction parameter for intra prediction or a prediction parameter for inter prediction.
- the intra prediction parameter is a parameter for restoring intra prediction (prediction mode) for each intra PU.
- the parameters for restoring the prediction mode include mpm_flag which is a flag related to MPM (Most Probable Mode, the same applies hereinafter), mpm_idx which is an index for selecting an MPM, and an index for designating a prediction mode other than MPM. Rem_idx is included.
- the inter prediction parameter includes prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and vectors mvL0 and mvL1.
- the prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 reference list and L1 reference list are used, respectively, and a reference picture list corresponding to a value of 1 is used.
- predFlagL0, predFlagL1 (1, 0 )
- FIG. 10A is a conceptual diagram illustrating an example of a reference picture list.
- RPL0 the five rectangles arranged in a line on the left and right indicate reference pictures, respectively.
- Reference signs P1, P2, Q0, P3, and P4 shown in order from the left end to the right are signs indicating respective reference pictures.
- codes P4, P3, R0, P2, and P1 shown in order from the left end to the right are codes indicating respective reference pictures.
- P such as P1 indicates the target layer P
- Q of Q0 indicates a layer Q different from the target layer P
- R of R0 indicates a layer R different from the target layer P and the layer Q.
- the subscripts P, Q, and R indicate a picture order number POC (Picture Ordering Count).
- a downward arrow directly below refIdxL0 indicates that the reference picture index refIdxL0 is an index that refers to the reference picture Q0 from the reference picture list RPL0 in the decoded picture buffer.
- a downward arrow directly below refIdxL1 indicates that the reference picture index refIdxL1 is an index that refers to the reference picture P3 from the reference picture list RPL1 in the decoded picture buffer.
- FIG. 10B is a conceptual diagram illustrating an example of a reference picture.
- the horizontal axis indicates the display time
- the vertical axis indicates the number of layers.
- the illustrated rectangles of three rows and three columns (total of nine) each indicate a picture.
- the rectangle in the second column from the left in the lower row indicates a picture to be decoded (target picture), and the remaining eight rectangles indicate reference pictures.
- Reference pictures Q2 and R2 indicated by downward arrows from the target picture are pictures having the same display time and different layers as the target picture.
- the reference picture Q2 or R2 is used.
- a reference picture P1 indicated by a left-pointing arrow from the target picture is the same layer as the target picture and is a past picture.
- a reference picture P3 indicated by a rightward arrow from the target picture is the same layer as the target picture and is a future picture.
- motion prediction based on the target picture the reference picture P1 or P3 is used.
- the vector mvLX includes a motion vector and a displacement vector (disparity vector).
- a motion vector is a positional shift between the position of a block in a picture at a certain display time of a layer and the position of the corresponding block in a picture of the same layer at a different display time (for example, an adjacent discrete time). It is a vector which shows.
- the displacement vector is a vector indicating a positional shift between the position of a block in a picture at a certain display time of a certain layer and the position of a corresponding block in a picture of a different layer at the same display time.
- the pictures of different layers may be pictures with the same resolution and different quality, pictures with different viewpoints, or pictures with different resolutions.
- a displacement vector corresponding to pictures of different viewpoints is called a disparity vector.
- FIG. 18 is a schematic diagram illustrating a configuration of the hierarchical video decoding device 1 according to the present embodiment.
- the hierarchical moving picture decoding apparatus 1 decodes the hierarchical encoded data DATA supplied from the hierarchical moving picture encoding apparatus 2, and decodes pictures of each layer included in the target set TargetSet determined by output designation information supplied from the outside. And the decoded picture of the output layer is output as the output picture POUT # T.
- the hierarchical video decoding device 1 uses the element TargetDecLayerIdList [0] .. TargetDecLayerIdList [N-1] of the target decoding layer ID list TargetDecLayerIdList indicating the layer configuration required for decoding the target output layer set TargetOptLayerSet indicated by the output designation information. ] (N is the number of layers included in the target set) in this order, the encoded data of the picture of layer i is decoded and the decoded picture is generated, and the output layer information OutputLayerFlag [i] of the layer i is set to “output layer ”Indicates that the decoded picture of layer i is output at a predetermined timing.
- the hierarchical video decoding device 1 includes a NAL demultiplexing unit 11 and a target set picture decoding unit 10. Further, the target set picture decoding unit 10 includes a non-VCL decoding unit 12, a parameter memory 13, a picture decoding unit 14, a decoded picture management unit 15, and an output control unit 16.
- the NAL demultiplexing unit 11 further includes a bit stream extraction unit 17.
- Hierarchical encoded data DATA includes NALU including parameter sets (VPS, SPS, PPS) and SEI in addition to NALU (NAL unit) generated by VCL.
- NALU NAL unit
- Those NALs are called non-VCL NALU (non-VCL NALU) versus VCL ⁇ ⁇ ⁇ ⁇ NALU.
- the output control unit 16 derives the output control information based on the output designation information supplied from the outside, the syntax of the active VPS held in the parameter memory 13, and the parameters derived from the syntax. More specifically, the output control unit 16 outputs the output layer set identifier TargetOLSIdx that identifies the output layer set included in the output designation information, the layer set information (layer set) of the active VPS held in the parameter memory 13, and Based on the output layer set information (layer set identifier and output layer flag), a target output layer ID list TargetOptLayerIdList indicating the layer configuration of the output layer of the target output layer set TargetOptLayerSet is derived, and decoded picture management is performed as part of the output control information To the unit 15.
- the output control unit 16 determines the dependency derived from the output layer set identifier TargetOLSIdx included in the output designation information, the layer set information of the active VPS held in the parameter memory 13, the output layer set information, and the inter-layer dependency information.
- Target decoding layer ID indicating the configuration of the layers required for decoding the target output layer set, excluding the non-output layer and the non-dependent layer, based on the flag and the target output layer ID list TargetOptLayerIdList derived by the output control unit 16
- a list TargetDecLayerIdList is derived and supplied to the bitstream extraction unit 17 and the target set picture unit 10 as part of the output control information. Details of the derivation process of the target output layer ID list and the target decoding layer ID list in the output control unit 16 will be described later.
- the bit stream extraction unit 17 included in the NAL demultiplexing unit 11 roughly performs a bit stream extraction process, and decodes the target decoding layer ID list supplied from the output control unit 16 from the hierarchically encoded data DATA, and the decoding Target set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the target TargetSet, a set determined by the target highest-order sublayer identifier TargetHighestTid, is extracted. Details of processing highly relevant to the present invention in the bitstream extraction unit 17 will be described later.
- the NAL demultiplexing unit 11 demultiplexes the target set encoded data DATA # T (BitstreamToDecode) extracted by the bitstream extraction unit 17 and includes a NAL unit type and a layer identifier (layer ID) included in the NAL unit. ), Referring to the temporal identifier (temporal ID), the NAL unit included in the target set is supplied to the target set picture decoding unit 10.
- the target set picture decoding unit 10 supplies non-VCL NALU to the non-VCL decoding unit 12 and VCL NALU to the picture decoding unit 14 among the NALUs included in the supplied target set encoded data DATA # T. . That is, the target set picture decoding unit 10 decodes the supplied NAL unit header (NAL unit header), and based on the NAL unit type, the layer identifier, and the temporal identifier included in the decoded NAL unit header, The NALU encoded data is supplied to the non-VCL decoding unit 12 and the VCL NALU encoded data is supplied to the picture decoding unit 14 together with the decoded NAL unit type, layer identifier, and temporal identifier.
- NAL unit header the supplied NAL unit header
- the NALU encoded data is supplied to the non-VCL decoding unit 12 and the VCL NALU encoded data is supplied to the picture decoding unit 14 together with the decoded NAL unit type, layer identifier, and temporal identifier.
- the non-VCL decoding unit 12 decodes the parameter set, that is, VPS, SPS, and PPS, from the input non-VCL NALU and supplies them to the parameter memory 13. Details of processing highly relevant to the present invention in the non-VCL decoding unit 12 will be described later.
- the parameter memory 13 holds the decoded parameter set for each parameter set identifier and the encoding parameter of the parameter set. Specifically, in the case of a VPS, a VPS encoding parameter is held for each VPS identifier (video_parameter_set_id). In the case of SPS, SPS encoding parameters are held for each SPS identifier (sps_seq_parameter_set_id). In the case of PPS, the PPS encoding parameter is held for each PPS identifier (pps_pic_parameter_set_id). Note that the encoding parameter held in the parameter memory 13 may include a layer identifier and a temporal identifier of each parameter set.
- the parameter memory 13 supplies the picture decoding unit 14 with encoding parameters of a parameter set (active parameter set) that is referred to by the picture decoding unit 14 described later for decoding the picture.
- the active PPS is specified by the active PPS identifier (slice_pic_parameter_set_id) included in the slice header SH decoded by the picture decoding unit 14.
- the active SPS is specified by the active SPS identifier (pps_seq_parameter_set_id) included in the specified active PPS.
- the active VPS is specified by the active VPS identifier (sps_video_parameter_set_id) included in the active SPS.
- the parameter memory 13 supplies the output control unit 16 with the encoding parameters of the active parameter set that the output control unit 16 refers to in order to derive the output control information.
- the picture decoding unit 14 generates a decoded picture based on the input VCL NALU, the active parameter set (active PPS, active SPS, active VPS), and the reference picture, and supplies the decoded picture to the decoded picture management unit 15.
- the supplied decoded picture is recorded in a buffer in the decoded picture management unit 15. Detailed description of the picture decoding unit 14 will be described later.
- the decoded picture management unit 15 records an input decoded picture in an internal decoded picture buffer (DPB: “Decoded” Picture ”Buffer), and generates a reference picture list and determines an output picture. Also, the decoded picture management unit 15 uses the decoded picture of the output layer included in the target output layer ID list TargetOptLayerIdList derived by the output control unit 16 among the decoded pictures recorded in the DPB as the output picture POUT # T. Output to the outside at a predetermined timing.
- DPB internal decoded picture buffer
- Non-VCL decoding unit 12 decodes a parameter set (VPS, SPS, PPS) used for decoding the target set from the input target set encoded data.
- the encoded parameters of the decoded parameter set are supplied to the parameter memory 13 and recorded for each identifier of each parameter set.
- the parameter set is decoded based on a predetermined syntax table. That is, a bit string is read from the encoded data according to the procedure defined by the syntax table, and the syntax included in the syntax table is decoded. Further, if necessary, variables may be derived based on the decoded syntax and included in the output parameter set. Therefore, the parameter set output from the non-VCL decoding unit 12 is expressed as a syntax relating to the parameter set (VPS, SPS, PPS) included in the encoded data and a set of variables derived from the syntax. You can also
- the non-VCL decoding unit 12 includes a parameter set decoding unit that decodes a parameter set (VPS / SPS / PPS) based on a prescribed syntax table (not shown). Further, the parameter set decoding unit further includes a layer set decoding unit that decodes layer set information, an inter-layer dependency information decoding unit that decodes inter-layer dependency information, and output layer set information that decodes output layer set information.
- the layer set information is a list (hereinafter referred to as a layer ID list LayerIdList) representing a set of layers constituting the layer set included in the hierarchically encoded data, and is decoded from the VPS by the layer set information decoding unit.
- the layer set information includes a syntax (vps_num_layer_sets_minus1) (SYNPVS06 in FIG. 11) indicating the number of layer sets defined on the VPS, and the jth layer (layer j) in the order of layer definition on the VPS.
- the syntax “layer_id_included_flag [i] [j]” (SYNVPS07) indicating whether or not it is included in the i-th layer set (layer set i) is included.
- the layer set number VpsNumLayerSets is set to (vps_num_layer_sets_minus1 + 1).
- the layer set i includes a layer j having a syntax “layer_id_included_flag [i] [j]” value of 1. That is, the layer j constituting the layer set i is included in the layer ID list LayerIdList [i].
- the number of layers NumLayersInIdList [i] included in the layer set i is derived from the number of flags whose syntax value is 1 for the layer set i in the syntax “layer_id_included_flag [i] [j]”.
- the layer set information decoding means derives the layer ID list LayerIdList [i] of each layer set i and the number of layers NumLayersInIdList [i] included in the layer set i by the following pseudo code.
- SA01 This is the starting point of the loop related to the derivation of the layer ID list of layer set i.
- the variable i is initialized to zero.
- the loop variable of the following iterative process is the variable i, and the processes shown in SA0A2 to SA0A are executed for the variable i from 0 to (NumLayerSets-1).
- SA03 This is the starting point of the loop related to the addition of the element of the mth layer (layer m) to the layer ID list of layer set i.
- the variable m is initialized to zero.
- the loop variable of the following iterative process is the variable m, and the processes shown in SA04 to SA06 are executed for the variable m from 0 to the maximum layer identifier “vps_max_layer_id”.
- SA04 It is determined whether or not layer m is included in layer set i (layer_id_included_flag [i] [m]). If layer_id_included_flag [i] [m] is 1, the process proceeds to step SA05. If layer_id_included_flag [i] [m] is 0, the processes in steps SA05 to SA06 are omitted, and the process proceeds to SA0A.
- the layer ID list LayerIdList [i] of each layer set i can be derived.
- the layer which is the mth element in the layer set i knows which layer is the layer among all layers (layers defined by VPS) be able to.
- the number of layers included in the layer set i can be grasped by referring to a variable NumLayersInIdList [i] that represents the number of layers in the layer set i. Note that the derivation procedure is not limited to the above steps, and may be changed within a practicable range.
- the inter-layer dependency information includes a direct dependency flag “direct_dependency_flag [i] [j]” (SYNVPS0C in FIG. 12).
- the inter-layer dependency information is decoded from, for example, VPS extension data by the inter-layer dependency information decoding unit.
- the direct dependency flag direct_dependency_flag [i] [j] indicates whether or not the i-th layer (hereinafter referred to as layer i) directly depends on the j-th layer (hereinafter referred to as layer j). It takes a value of 1 when it is dependent, and a value of 0 when it is not directly dependent.
- the parameter set, decoded picture, and related decoded syntax related to layer j are the target. It means that it may be directly referenced by the layer.
- the parameter set, decoded picture, and related decoded syntax related to layer j are directly Means not to be referenced.
- the direct dependency flag direct_dependency_flag [i] [j] for layer i of layer i is 1, layer j is a direct reference layer of layer i.
- the flag is 0, layer j is a non-direct reference layer of layer i.
- the layer-dependent information decoding unit is configured to refer to a layer i direct reference layer list (also referred to as a reference layer ID list) RefLayerId [] [] and a layer i
- the number of direct reference layers NumDirectRefLayers [] is derived.
- the reference layer ID list RefLayerId [] [] is a two-dimensional array
- the first dimension index is the layer identifier (layer_id_in_nuhnu [i]) of the target layer (layer i)
- the second dimension index Is an index of an element in the reference layer ID list of the target layer (layer i).
- layer_id_in_nuh [] is an array for deriving the layer identifier nuh_layer_id of layer i (the same applies hereinafter).
- the pseudo code can be expressed in steps as follows.
- (SL01) Reference layer ID list for layer i and the starting point of the loop related to the derivation of the number of direct reference layers.
- the variable i is initialized to zero.
- the process in the loop is executed when the variable i is less than the number of layers VpsMaxLayers, and the variable i is incremented by “1” every time the process in the loop is executed once.
- step SL04 Whether layer j is a direct reference layer of layer i is determined based on a direct dependency flag (direct_dependency_flag [i] [j]). If the direct dependency flag is 1 (direct reference layer), the process proceeds to step SL05 in order to execute the processes of steps SL05 to SL07. If the direct dependency flag is 0 (is a non-direct reference layer), the processing in steps SL05 to SL07 is omitted, and the process proceeds to SL0A.
- direct_dependency_flag [i] [j] direct reference layer
- reference layer ID list and the procedure for deriving the number of direct reference layers are not limited to the above steps, and may be changed within a practicable range.
- the layer-dependent information decoding means is configured such that the layer j is a dependent layer (direct reference layer or indirect reference layer) based on the derived reference layer ID list RefLayerId [] [] and the number of direct reference layers NumDirectRefLayers [].
- the dependency flag recursiveRefLayerFlag [] [] indicating whether or not is derived.
- the dependency flag is derived by the following pseudo code.
- the starting point of the loop for the direct reference layer j of layer i The variable j is initialized to 0 before the loop starts.
- the processing in the loop is executed when the variable j (direct reference layer j) is less than the direct reference layer number NumDirectRefLayers [currLayerId] (j ⁇ NumDirectRefLayers [currLayerId]), and every time the processing in the loop is executed once The variable j is incremented by “1”.
- (S006) This is a loop start point for searching whether layer k is a dependency layer of layer i.
- the variable k is initialized to 0 before the start of the loop.
- the process in the loop is executed when the variable k (layer k) is less than the number of layers VpsMaxLayers (j ⁇ VpsMaxLayers), and the variable k is incremented by “1” every time the process in the loop is executed once. .
- step S007 It is determined by the dependency flag recursiveRefLayerFlag [refLayerId] [k] whether or not the layer k is a dependency layer of the direct reference layer j of the layer i.
- the process proceeds to step S008.
- the process proceeds to step S009.
- dependency flag derivation procedure is not limited to the above steps, and may be changed within a practicable range.
- the PTL information is information indicating a profile and a level required for decoding the output layer set, and is decoded from the VPS or SPS by the PTL information decoding unit.
- the PTL information corresponding to the output layer set OLS # 0 is notified in SYNVPS04 shown in FIG. 11 on the VPS or in FIG. 17 (a) on the SPS.
- SYNVPS0D), i-th (i 1 ... num_profile_tier_level_minus1) profile presence / absence flag “vps_profile_present_flag [i]” (SYNVPS0E in FIG. 12) and i-th PTL information “profile_tier_level ()” It consists of “” (SYNVPS0F in FIG. 12).
- Each PTL information is associated with the output layer set OLS # i by a PTL designation identifier (profile_level_tier_idx [i]) (SYNVPS0J on FIG. 12) included in the output layer set OLS # i described later.
- a PTL designation identifier profile_level_tier_idx [i]
- SYNVPS0J on FIG. 12 the PTL designation identifier of the output layer set OLS # 3
- the 10th PTL information from the top in the list of PTL information on the SYNVPS0F in FIG. 12 is the output layer set OLS # 3 PTL information applied to
- the PTL information includes a syntax group (SYNPTL01, SYNPTL02, SYNPTL03, SYNPTL04, SYNPTL05, SYNPTL06) regarding the profile and level, and is decoded by the PTL information decoding means.
- the syntax group SYNPTL01 includes the following syntax. ⁇ Profile space general_profile_space ⁇ Tier flag general_tier_flag -Profile identifier general_profile_idc -Profile compatibility flag general_profile_compatibility_flag [i] Profile reservation syntax general_reserved_zero_44bits
- the syntax group SYNPTL02 includes a level identifier general_level_idc.
- the syntax group SYNPTL03 includes a sublayer profile presence / absence flag of a sublayer and a sublayer level presence / absence flag.
- the syntax group SYNPTL04 is byte aligned data (reserved_zero_2bits [i]) for the number of bits determined based on the number of sublayers (MaxNumSbuLayersMinus1, or MaxNumSubLayers-1).
- the syntax group SYNPTL05 includes the following syntax. ⁇ Sublayer profile space sub_layer_profile_space [i] -Sublayer tier flag sub_layer_tier_flag [i] -Sublayer profile identifier sub_layer_profile_idc [i] Sublayer profile compatibility flag sub_layer_profile_compatibility_flag [i] [j] Sublayer profile reservation syntax sub_layer_reserved_zero_44bits [i] The syntax group SYNPTL05 includes a sublayer level identifier sub_layer_level_idc [i] as sublayer level information of the sublayer.
- a scalable identifier decoding unit decodes a scalable identifier (ScalabilityId) assigned to each layer from input target layer encoded data.
- the scalable identifier ScalabilityId is an ID for distinguishing layer properties between layers, and is also called a scalable ID.
- a single layer can have a multi-dimensional scalable ID.
- the following j-dimensional scalable ID of layer i is derived from dimension_id [i] [j] of encoded data.
- the index j ranges from 0 to 15.
- FIG. 14C is an example of a syntax table showing the configuration of VPS extension data.
- the scalable identifier decoding means decodes the separation flag splitting_flag, the scalable mask flag scalability_mask_flag, the dimension ID length dimension_id_len_minus1, and the dimension ID dimension_id from the encoded data.
- ⁇ Splitting_flag> is a syntax element indicating the encoding position of dimension_id.
- dimension_id is not explicitly encoded in the VPS and is derived from the layer identifier (“layer_id_in_nuh [i]”) corresponding to each layer i.
- layer_id_in_nuh [i] the layer identifier
- Scalability_mask_flag [j] indicates whether or not the dimension ID indicated by the index j is used.
- the scalable identifier decoding means derives the number NumScalabilityTypes of the dimension whose scalability_mask_flag [j] is 1 based on scalability_mask_flag []. When scalability_mask_flag [j] is 0, dimension_id [i] [j] of the corresponding index j is not decoded.
- Dimension_id_len_minus1 indicates (dimension_id [i] [j] bit length-1) of index j.
- the scalable identifier decoding means decodes the dimension ID (dimension_id [i] [j]) of the jth dimension of layer i when splitting_flag is 0.
- FIG. 14B is a pseudo code showing a method for deriving the scalable identifier ScalabilityId.
- the scalable identifier decoding means derives the scalable identifier ScalabilityId [i] [smIdx] from the dimension ID (dimension_id [i] [j]) for the index i from 0 to the maximum number of layers ⁇ 1 (MaxLayersMinus1).
- the scalable identifier decoding means for each layer index i (layer i) in SPEP2 of FIG. 14B, the scalable identifier scalabilityId [i] [0], ScalabilityId [] derived in SPTEP1 of FIG. i] [1], ScalabilityId [i] [2], ScalabilityId [i] [3], depth IDDepthId [lId], view order IDViewOrderIdx [lId], dependency ID DependencyId [lId], auxiliary picture layer ID AuxId [lId] Set to and derive. That is, the auxiliary picture layer ID (AuxId []) is derived from ScalabilityId [i] [3].
- the relationship between the dimension ID and the scalable ID type is not limited to the above-described FIG. 14B, and may be another correspondence relationship.
- ScalabilityId [i] [0], ScalabilityId [i] [1], ScalabilityId [i] [2], ScalabilityId [i] [3], ViewOrderIdx [lId], DependencyId [lId], AuxId [lId] , DepthId [lId] may be mapped.
- AuxId is derived from ScalabilityI [i] [2] instead of ScalabilityI [i] [3].
- Depth IDDepthId [lld] indicates texture or depth, 0 corresponds to texture and 1 corresponds to depth.
- the view order ID ViewOrderIdx [lld] indicates the order of viewpoints.
- the order of viewpoints does not have to correspond to the position of the camera.
- the view ID can be determined separately from the view order ID.
- the dependency ID DependencyId [0] is an ID indicating a hierarchy of SNR scalability and spatial scalability. For example, when a layer is configured from a base layer, an enhancement layer 1 that refers to the base layer, and an enhancement 2 that refers to the enhancement layer 1 Take values of 0, 1, 2 respectively.
- the auxiliary picture layer ID AuxId [lld] is used for identifying the primary picture layer and the auxiliary picture layer, and identifying the type of the auxiliary picture layer.
- 0 corresponds to the primary picture layer
- non-zero corresponds to the auxiliary picture layer.
- Reference numerals 1 and 2 indicate an alpha picture (layer) and a depth picture (layer), respectively. Note that two or more values can be used as the auxiliary picture layer ID.
- the output layer set information is defined by a combination of a set of layers to be output (output layer information) and a set of layers (layer set information), and is decoded by an output layer set information decoding unit (not shown) included in the hierarchical video decoding device.
- the hierarchical video decoding apparatus sets a layer included in an output layer set decoded by the output layer set information decoding means (a layer set associated with the output layer set) as a decoding target, and decodes the layer A picture is decoded and recorded in a buffer, and a decoded picture of a specific layer recorded in the buffer is selected and output for output layer information included in the output layer set.
- the output layer set information includes the following syntax elements (E1 to E7).
- E1 Number of additional output layer sets (num_add_output_layer_sets) (SYNVPS0G on FIG. 12)
- E2 Default output layer identifier (default_target_output_layer_idc) (SYNVPS0H on FIG. 12)
- E3 Layer set identifier (output_layer_set_idx_minus1) (SYNVPS0I on FIG. 12)
- E4 Output layer information (output_layer_flag) (SYNVPS0J on FIG. 12)
- E5 Alternate output layer flag (alt_output_layer_flag) (SYNVPS0K on FIG.
- E6 PTL / DPB information presence / absence flag (ptl_dpb_info_present_flag) (SYNVPS0L in FIG. 12)
- E7 PTL designation identifier (profile_level_tier_idx) (SYNVPS0M in FIG. 12)
- the output layer set information decoding means of this embodiment decodes at least the layer set identifier and output layer flag of the output layer set.
- the output layer set is information obtained by combining the corresponding layer set and the designation of the output layer in the layer set.
- a layer set corresponding to the output layer set a layer set specified by a layer set identifier can be used.
- Output layer information can be used to specify an output layer.
- each output layer set has one associated layer set.
- Output layer sets can be classified into basic output layer sets and additional output layer sets.
- output layer sets When output layer sets are associated with the same layer set, one of them is a basic output layer set, and an output layer set other than the basic output layer set with which the same layer set is associated is an extended output layer set.
- the basic output layer set is also an output layer set derived based on the layer set decoded by VPS. In the present embodiment, one output layer set corresponding to each layer set decoded by VPS is derived as a basic output layer set.
- the output layer sets with identifiers 0 to VpsNumLayerSets-1 correspond one-to-one with the layer sets with identifiers 0 to VpsNumLayerSets-1.
- the output layer set information decoding means of this embodiment decodes the number of layer sets (VpsNumLayerSets), and decodes the number of layer sets equal to the number of layer sets from the VPS. Then, from the decoded layer set of identifier 0 (VpsNumLayerSets)-1), the output layer set of (VpsNumLayerSets-1) is derived from identifier 0, respectively, and the basic output layer set is derived.
- the output layer set of identifier i output layer set identifier i
- the layer set identifier i associated with the layer set of identifier i (layer set identifier i) is referred to as a basic output layer set corresponding to the layer set of layer set identifier i. Call.
- the layer set corresponding to the basic output layer set with the output layer set identifier i is the layer set with the layer set identifier i.
- the additional layer set is an output layer set that is defined in addition to the basic output layer set.
- the number of additional output layer sets (num_add_output_layer_sets) is decoded from the VPS extension, and the number of output layer sets corresponding to the number of additional output layer sets is decoded based on the layer set identifier and output layer information. Derived.
- the basic output layer set and the additional output layer set can be defined as follows. That is, the basic output layer set is an output layer set in which the layer set identifier representing the corresponding layer set is not explicitly decoded, and the additional output layer set is explicitly decoded in the layer set identifier representing the corresponding layer set. The output layer set to be output.
- the number of output layer sets NumOutputLayerSets is derived from the number of layer sets VpsNumlayerSets + the number of additional output layer sets num_add_output_layer_sets.
- output layer sets with identifiers from 0 to (VpsNumLayerSets-1) are basic output layer sets.
- An output layer set whose identifier is VpsNumLayerSets to (NumOutputLayerSet-1) is an additional output layer set.
- the default output layer identifier default_target_output_layer_idc is a syntax element that specifies the derivation process of the output layer set (output layer information).
- the output layer set information decoding unit of the present embodiment decodes the default output layer identifier, and executes decoding control or derivation of the output layer information by processing according to the value of the default output layer identifier.
- a value of 3 or more is a reserved value for future standard expansion.
- the layer set identifier is a value that identifies the layer set associated with the output layer set.
- the output layer set information decoding means of this embodiment decodes the syntax element output_layer_set_idx_minus1 [i], and uses a value obtained by adding 1 to the syntax element value as the layer set identifier for the output layer set of the identifier i.
- the layer set (LS # (output_layer_set_idx_minus1 [i] +1)) indicated by the layer set identifier is associated with the output layer set (OLS # i) having the identifier i.
- the output layer set information decoding means may estimate when the layer set identifier of the output layer set OLS # i does not exist in the encoded data (when omitted). For example, in the case of a basic output layer set whose output layer set identifier is i, the layer set identifier is estimated as (i ⁇ 1).
- the syntax element related to the layer set identifier is expressed by “layer set identifier value ⁇ 1”, but is not limited thereto, and may be “layer set identifier value” itself.
- the output layer information is a set of flags (OutputLayerFlag [i] [j]) indicating whether each layer included in the layer set associated with the output layer set is an output target layer.
- the output layer set information decoding means of the present embodiment sets output layer information OutputLayerFlag [i] [j] from the decoded syntax element output_layer_flag [i] [j].
- output_layer_flag [i] [j] is a flag indicating whether or not the jth layer included in the output layer set i is an output target layer, and when the value is true (1), If the value is false (0), it indicates that the layer is not an output target layer.
- the output layer set information decoding means may omit decoding of some or all of the output layer information and determine the output layer information by estimation or a derivation process based on the value of another syntax element. . For example, based on the default output layer identifier (default_target_output_layer_idc), one of the following derivation processes (1) to (3) may be selected to determine the output layer information of the basic output layer set.
- the variable LayerSetIdx [i] represents a layer set identifier indicating the layer set associated with the output layer set OLS # i, and is set to (output_layer_set_idx_minus1 [i] + 1), and the variable NumLayersInIdList [LayerSetIdx [i ]] Is the number of layers included in the layer set LS # (LayerSetIdx [i]) (the same applies hereinafter).
- the output layer information (OutputLayerFlag) is derived by the following pseudo code.
- the auxiliary picture layer is a layer for notifying a depth map for a picture belonging to the primary picture layer and an auxiliary picture such as an alpha channel.
- the details of the scalable identifier and the auxiliary picture layer ID have already been described in (Scalable identifier and auxiliary picture layer ID).
- OutputLayerFlag [i] [j] output_layer_flag [i] [j]; ⁇
- the number of output layers NumOptLayersInOLS [i] of the output layer set OLS # i is the number of flags in which the output layer flag OutputLayerFlag [i] [j] of the layer j indicates “output layer”.
- the layer identifier of the highest order output layer is the highest order layer whose OuputLayerFlag [i] [] is 1 (true) in the layer ID list LayerIdList [LayerSetIdx [i]] [] of the output layer set OLS # i Layer identifier.
- the alternative output layer flag (alt_output_layer_flag [i]) (SYNVPS0K in FIG. 12) is information indicating whether or not an alternative layer decoded picture output can be applied.
- an alternative layer decoded picture output is applied, if there is no decoded picture of the layer specified by the output layer information, an alternative layer is set and the decoded picture of the alternative layer is output instead.
- the syntax element value alt_output_layer_flag [i] is alternative output layer information for the output layer set i.
- the output layer set information decoding means decodes the syntax element alt_output_layer_flag [i] from the encoded data when both of the following conditions (A1) to (A2) are satisfied, and converts to the alternative output layer flag AltOutputLayerFlag [i] Set the value of alt_output_layer_flag [i].
- the output layer set information decoding means estimates that the value of the syntax element is 0, and outputs the alternative layer decoded picture output to the alternative layer output flag AltOutputLayerFlag [i]. Set the value corresponding to not applying. In the present embodiment, the value of AltOutputLayerFlag [i] is set to 0.
- the PTL / DPB information presence / absence flag (ptl_dpb_present_flag [i]) (SYNVPS0L in FIG. 12) is a flag indicating whether or not the PTL designation identifier applied to the output layer set and the DPB information are present in the encoded data. .
- the output layer set information decoding means having the above configuration, decoding related to the PTL / DPB information presence / absence flag relating to the basic output layer set can be omitted. That is, there is an effect that the PTL / DPB information presence / absence flag relating to the basic output layer set and the additional output layer set can be decoded / encoded with a smaller code amount.
- ptl_dpb_info_present_flag a flag for controlling the PTL identifier and DPB information
- ptl_info_present_flag a flag for controlling the PTL identifier
- DPB information presence / absence flag dpb_info_present_flag a flag for controlling the DPB information
- dpb_info_present_flag a flag for controlling the DPB information
- the output layer set information decoding means decodes the PTL information presence / absence flag ptl_info_present_flag or the DPB information presence / absence flag dpb_info_present_flag by the same processing instead of the PTL / DPB information presence / absence flag ptl_dpb_info_present_flag. Further, the output layer set information decoding means may decode the PTL information presence / absence flag ptl_info_present_flag and the DPB information presence / absence flag dpb_info_present_flag by the same processing.
- the output layer set information decoding means may decode one PTL / DPB information presence / absence flag as ptl_dpb_info_present_flag without decoding ptl_dpb_info_present_flag [i] for each output layer set i.
- the PTL specification identifier (profile_level_tier_idx) (SYNVPS0M in FIG. 12) is a syntax element for specifying PTL information to be applied to the output layer set.
- the PTL information specified by the PTL specification identifier (profile_level_tier_idx [i]) is applied to the output layer set OLS # i.
- the output layer set information decoding means uses the PTL designation identifier (profile_level_tier_idx [i] ).
- the output layer set information decoding unit of the present embodiment decodes the PTL designation identifier of one output layer set (basic output layer set) from the encoded data. . PTL designation identifiers of other output layer sets (additional output layer sets) do not exist in the encoded data, and the output layer set information decoding means associates PTL designation identifiers of nonexistent output layer sets with the same layer set. Already decoded PTL designation identifier of the output layer set is assigned and derived.
- the output layer set information decoding means omits decoding of the PTL designation identifier when the value of the PTL / DPB information presence / absence flag (ptl_dpb_info_present_flag [i]) of the output layer set OLS # i is 0 (false).
- the output layer set information decoding means applies the PTL information specified by the decoded or estimated PTL specification identifier (profile_level_tier_idx [i]) to the output layer set OLS # i.
- the output layer set information decoding means having the above configuration, when the PTL / DPB information presence / absence flag of the output layer set OLS # i is 0, decoding / encoding of the PTL designation identifier (profile_level_tier_idx [i]) may be omitted. it can. That is, it is possible to decode / encode the PTL designation identifier relating to the basic output layer set and the additional output layer set with a smaller code amount.
- the PTL designation identifier and DPB information are explicitly decoded. If the PTL / DPB information presence / absence flag is 1 (true) for the additional output layer set OLS # X that is other output layers associated with the same layer set, the PTL designation identifier of OLS # X and If the DPB information is explicitly decoded and the PTL / DPB information presence flag of the additional output layer set OLS # Y is 0 (false), the basic output layer set OLS associated with the same layer set as the additional output layer set Inferred from #A PTL specified identifier and DPB information. Therefore, it is possible to decode / encode the PTL designation identifier and DPB information of the output layer set with a smaller code amount.
- the output layer set information decoding unit performs the above processing. Then, the PTL / DPB information presence / absence flag ptl_dpb_info_present_flag is replaced with the PTL information presence / absence flag dpb_info_present_flag for execution. In this case as well, the above effect on the PTL designation identifier can be obtained.
- the output layer set information decoding means decodes or estimates the PTL designation identifier based on the PTL / DPB information presence / absence flag, but is not limited thereto. For example, the output layer set information decoding means does not decode the PTL / DPB information presence / absence flag, but decodes the PTL designation identifier based on whether the output layer set is a basic output layer set or an additional output layer set. You may go.
- the DPB information is information indicating the maximum size of the decoded picture held by the decoder in the buffer (DPB) in order to decode the output layer set, and is decoded from the VPS or SPS by the DPB information decoding means.
- the DPB information decoding means converts DPB information corresponding to the output layer set OLS # 0 into the syntax SYNDPB01 to SYNDPB04 (vps_sub_layer_ordering_info_present_flag, vps_max_dec_pic_buffering_minus1 [], vps_max_order [replay] , Vps_max_latency_increase_plus1 []), or SPS is decoded from the syntax in which “vps” in the syntax SYNDPB01 to SYNDPB04 is replaced with “sps”.
- the meaning of each syntax is as follows. In the following syntax, the leading “x” means “vps” or “sps”.
- X_sub_layer_ordering_info_present_flag When the flag is 1, it indicates that DPB information (x_dec_pic_buffering_minus1 [], x_max_num_reorder_pics [], x_max_latency_increase_plus1 []) exists in all sublayers of the output layer set OLS # 0. When the flag is 0, the first value of vps_max_sub_layers_minus of the three types of syntax arrays is applied to all sublayers.
- X_max_dec_pic_buffering_minus1 [] Indicates the “maximum number of requests minus 1” of the number of pictures stored in the buffer (DPB).
- X_max_num_reorder_pics []: Indicates the maximum allowable number of pictures that can precede the picture in the decoding order and follow the picture in the display order when the decoding order and the display order are different in a hierarchical structure such as a B picture.
- sub_layer_flag_info_present_flag [i] (SYNDPB05): When the sublayer information presence / absence flag [i] (same flag) is 1, the sublayer DPB information presence / absence flag (sub_layer_dpb_info_present_flag [i] [j]) of the output layer set OLS # i is encoded data Indicates that it exists. When the flag is 0, the sublayer DPB information presence flag does not exist in the encoded data, and the value of the sublayer DPB information presence flag is estimated to be 0.
- sub_layer_dpb_info_prenset_flag [i] [j] (SYNDPB06): When array [i] [j] (same flag) is 1, max_vps_dec_pic_buffering_minus1 [i] [k] [j], max_vps_num_reorder_pics [i] [k] [j ], Max_vps_latency_increase_plus1 [i] [k] [j]). When the flag is 0, the three types of syntax are estimated to be equal to the value of the syntax array of the sublayer (j-1).
- Max_vps_dec_pic_buffering_minus1 [i] [k] [j] (SYNDPB07): In the output layer set OLS # i, this indicates the “maximum number of requests minus 1” of the number of pictures stored in the kth subbuffer (sub-DPB).
- max_vps_num_reorder_pic [i] [j] (SYNDPB09): In the output layer set OLS # i, when the decoding order and the display order are different in the hierarchical structure such as a B picture in the k-th layer k, precede the picture in the decoding order. And the maximum allowable number of pictures that can follow the picture in the display order.
- the output layer set information decoding unit of the present embodiment decodes the PTL designation identifier of one output layer set (basic output layer set) from the encoded data. . PTL designation identifiers of other output layer sets (additional output layer sets) do not exist in the encoded data, and the output layer set information decoding means associates PTL designation identifiers of nonexistent output layer sets with the same layer set. Already decoded PTL designation identifier of the output layer set is assigned and derived.
- the DPB information decoding means decodes the syntax SYNDPB05 to SYNDPB10 shown in FIG.
- the DPB information decoding means applies the decoded or estimated DPB information DPB_INFO # i to the output layer set OLS # i. Therefore, when the PTL / DPB information presence / absence flag of the output layer set OLS # i is 0, decoding / encoding of the DPB information DPB_INFO # i (syntax SYNDPB05 to SYNDPB10 shown in FIG. 15B) can be omitted. . That is, there is an effect that DPB information DPB_INFO # i of the basic output layer set and the additional output layer set can be decoded / encoded with a smaller code amount.
- DPB information and a PTL designation identifier are explicitly decoded for the basic output layer set OLS # A, which is one of the output layers associated with the same layer set.
- OLS # A which is one of the output layers associated with the same layer set.
- OLS # X which is the other output layer associated with the same layer set.
- the PTL / DPB information presence / absence flag is 1 (true)
- the DPB information of OLS # X If the PTL / DPB information presence flag of the additional output layer set OLS # Y is 0 (false), the basic output layer associated with the same layer set as the additional output layer set is explicitly decoded.
- the output layer set information decoding means The PTL / DPB information presence / absence flag ptl_dpb_info_present_flag is replaced with the DPB information presence / absence flag dpb_info_present_flag for execution. Also in this case, the above effect on the DPB information can be obtained.
- an output layer set of i ⁇ vps_num_layer_sets_minus1 (basic output) among the output layers of index i DPB information is decoded for layer sets), and DPB information is decoded when ptl_dpb_info_present_flag is 1 for other output layer sets (extended output layer sets) of i> vps_num_layer_sets_minus1.
- the DPB information of the output layer set with the identifier i that does not exist is derived by the DPB with the identifier output_layer_set_idx_minus1 [i].
- the DPB information decoding means decodes or estimates the DPB information based on the PTL / DPB information presence / absence flag, but is not limited thereto.
- the DPB information decoding means may decode the DPB information based on whether the output layer set is a basic output layer set or an additional output layer set regardless of the PTL / DPB information presence / absence flag.
- the DPB information decoding means corresponds to the output layer set OLS # i from the encoded data. DPB information DPB_INFO # i is decoded.
- the DPB information decoding means outputs DPB information DPB_INOF # i corresponding to the output layer set OLS # i.
- DPB information DPB_INFO # i related to the output layer set can be decoded / encoded.
- the output control unit 16 derives the target output layer ID list TargetOptLayerIdList [] and the decoded layer ID list, and outputs them to the decoded picture management unit 15.
- the output control unit 16 outputs a target output layer ID list TargetOptLayerIdList based on an output layer set identifier TargetOLSIdx) that is output designation information supplied from the outside, a layer set LayerIdList [] [], and an output layer flag OutputLayerFlag [] []. [] Is derived as output control information.
- active VPS active parameter set
- variables derived from the syntax are already decoded and stored in the parameter memory 13.
- an active VPS identifier may be included in the output designation information.
- the output control unit 16 selects an output layer set OLS # TargetOLSIdx designated by the output layer set identifier TargetOLSIdx included in the output designation information as a processing target. Further, the output control unit 16 derives the target output layer ID list TargetOptLayerIdList [] by the following pseudo code (output layer ID list derivation means).
- the pseudo code can be expressed in steps as follows.
- SA01 This is the starting point of the loop related to the derivation of the target output layer ID list TargetOptLayerIdList [].
- variables k and j Prior to the start of the loop, variables k and j are initialized to zero.
- the loop variable for the following iterative process is the variable j, and the output control unit 16 performs the processes shown in SA02 to SA04 on the variable j from 0 to (NumLayersInIdList [LayerSetIdx [TargetOLSIdx]] -1).
- LayerSetIdx [TargetOLSIdx] is the layer set identifier indicated by TargetOLSIdx
- NumLayersInIdList [x] is the number of layers of the layer set indicated by the layer set identifier x
- NumLayersInIdList [LayerSetIdx [TargetOLSIdx]] is the target output This is the number of layers included in the layer set LS # (LayerSetIdx [TargetOLSIdx]) associated with the layer set OLS # (TargetOLSIdx).
- SA02 It is determined whether each layer included in the target output layer set is an output layer. Specifically, in the target output layer set, when the output layer flag OutputLayerFlag [TargetOLSIdx] [j] of the layer indicated by the variable j is 1 (true) (is an output layer), the process proceeds to step SA04. When the output layer flag OutputLayerFlag [TargetOLSIdx] [j] is 0 (false) (not an output layer), the process proceeds to step SA0A.
- a layer whose output layer flag is 1 is derived as an output layer ID list TargetOptLayerIdList [].
- the jth element of the layer set LS # (LayerSetIdx [TargetOLSIdx]) associated with the output layer set OLS # (TargetOLSIdx) is used as the output layer ID list TargetOptLayerIdList [] of the output layer set OLS # (TargetOLSIdx).
- TargetOptLayerIdList [k] LayerIdList [LayerSetIdx [TargetOLSIdx]] [j];
- a decoding layer ID list derivation means included in the output control unit 16 is derived from the target output layer ID list TargetOptLayerIdList, the active VPS layer set LayerIdList [] [] held in the parameter memory 13, and inter-layer dependency information. Based on the dependency flag, a target decoding layer ID list TargetDecLayerIdList [] indicating a layer configuration necessary for decoding the target output layer set is derived. The derived TargetDecLayerIdList [] is supplied to the bit stream extraction unit 17 and the target set picture unit 10 as a part of the output control information.
- the decoding layer ID list derivation means derives the target decoding layer ID list by, for example, the following pseudo code.
- Decoding layer ID list deriving means derives a layer identifier of a layer (hereinafter, target layer j) identified by a variable j included in the output layer set. Specifically, the j-th element (target layer j) (LayerIdList [LayerSetIdx [TargetOLSIdx]] [j]) of the layer set LS # (LayerSetIdx [TargetOLSIdx]) associated with the output layer set OLS # (TargetOLSIdx) Set the layer identifier to the variable iNuhLId.
- the decoding layer ID list derivation unit is configured such that the layer (target layer j) of the layer set associated with the output layer set is a dependency layer (directly) of the target output layer TargetOptLayerIdList [k] whose output layer flag is 1 A flag refLayerFlag indicating whether or not it is a reference layer or an indirect reference layer is derived from SB03 to SB05.
- the decoding layer ID list derivation means for each layer belonging to the target output layer TargetOptLayerIdList [k] (hereinafter, output layer k), a dependency flag indicating whether the target layer j depends on the output layer k recursiveRefLayerFlag [ If the layer ID of the output layer k] [layer ID of the target layer j] is determined and there is at least one dependency flag recursiveRefLayerFlag [] [], the target layer j is the dependency layer of the output layer k.
- the target layer dependence flag refLayerFlag indicating whether or not is set to 1.
- SB03 initializes variable k and flag refLayerFlag to 0 before starting the loop.
- the process in the loop is executed when the variable k is less than the number of output layers “NumOptLayerIdList [TargetOptLayerIdx]”, and the variable k is incremented by “1” every time the process in the loop is executed once.
- the decoding layer ID list deriving unit determines whether the target layer j is an output layer or a dependency layer of an output layer in the target output layer set TargetOptLayerSet.
- the output layer flag OutputLayerFlag [TargetOLSIdx] [j] of the target layer j is 1 (true) or the target layer dependence flag refLayerFlag of the target layer j is 1 (true)
- steps SB08 to SB09 are executed.
- the decoding layer ID list deriving means derives the target layer j as an element of the target decoding layer ID list TargetDecLayerIdList [] when the target layer j is an output layer or a dependency layer of the output layer. Specifically, the j-th element of the layer set LayerSetIdx [TargetOLSIdx] associated with the target output layer set TargetOptLayerSet is added to the i-th element of the target decoding layer ID list TargetDecLayerIdList [].
- non-output output layer flag OutputLayerFlag [TargetOLSIdx] [j] is 0
- independent layers (refLayerFlag is 0) are excluded. That is, the decoding layer ID list derivation means includes all layers (output layer or dependent layer) except for a non-output and non-reference layer in the target decoding layer ID list in the output layer set TargetOptLayerSet.
- the dependency flag derivation procedure is not limited to the above steps, and may be changed within a practicable range.
- the value of the flag refLayerFlag may be the sum operator '+' instead of the logical sum operator '
- the target output layer ID list TargetOptLayerIdList is information derived from the output layer flag OutputLayerFlag [] [] in the output control unit 16. Therefore, as a whole, the output control unit 16 derives a target decoding layer ID list from the output layer set identifier TargetOLSIdx, the layer set LayerIdList [] [], the output layer flag OutputLayerFlag [] [], and the dependency flag recursiveRefLayerFlag.
- the output control unit 16 configured as described above is a layer to be decoded depending on whether each layer of the layer set associated with the target output layer set TargetOptLayerSet is an output layer of the target output layer set or a dependency layer of the output layer.
- a certain target decoding layer ID list TargetDecLayerIdList [] is derived. That is, the output control unit 16 does not include in the target decoding layer ID list TargetDecLayerIdList [] layers that are not essential for decoding the output layer of the target output layer set (non-output and non-reference layers). Thereby, the target set picture decoding unit 10 can omit decoding of non-output and non-reference layers.
- the output control unit 16 configured as described above includes, in the target decoding layer ID list TargetDecLayerIdList, NAL units having a layer identifier of a non-output and non-reference layer that are not essential for decoding the output layer of the target output layer set. Therefore, the bitstream extraction unit 17 discards these layers.
- the output control unit may be the output control unit 16a configured to include the layer having the layer identifier of the specific layer in the target decoding layer ID list TargetDecLayerIdList regardless of the output layer or the dependency layer of the output layer.
- the conditional expression in step SB07 of the pseudo code indicating the derivation of the target decoding layer ID list TargetDecLayerIdList is changed to the following conditional expression (A1) or (A2).
- the output control unit 16a determines whether the target layer j is an output layer, or is a dependency layer for the output layer in the target output layer set TargetOptLayerSet, It is determined whether the layer identifier is 0.
- the output control unit 16a further sets the output layer flag OutputLayerFlag [TargetOLSIdx] [j] to 1 (true), the flag refLayerFlag to 1 (true), or the target layer j to the base layer (the layer identifier of the layer j is 0). If there is, steps SB08-SB09 are executed.
- the output control unit 16a having the above-described configuration, with respect to the target output layer set TargetOptLayerSet, sets the output layer of the target output layer set, the dependency layer of the output layer, the layer (base layer) designated as essential in the profile, and the like. Then, the target decoding layer ID list TargetDecLayerIdList [] is derived by setting the layer as the decoding target. That is, the output control unit 16a does not include the non-output, non-reference layer, and non-base layer layers that are not essential for decoding the output layer of the target output layer set in the target decoding layer ID list TargetDecLayerIdList [].
- the target set picture decoding unit 10 can omit the non-output and non-reference layers that are not essential for decoding of the output layer if they are not layers that are designated as essential in the profile (here, the base layer). It is.
- the output control unit 16 configured as described above designates a NAL unit having a layer identifier of a non-output and non-reference layer that is not essential for decoding the output layer of the target output layer set as essential in the profile.
- the bitstream extraction unit 17 discards these layers.
- the output control unit may be the output control unit 16b in which the primary picture layer is included in the target decoding layer ID list TargetDecLayerIdList in the target output layer set.
- a decoding layer ID list derivation means (not shown) included in the output control unit 16b is an auxiliary picture layer ID (AuxId []) derived from the active VPS layer set LayerIdList [] [] held in the parameter memory 13 and the scalable identifier.
- auxiliary picture layer ID AuxId []
- TargetDecLayerIdList [] indicating a layer configuration required for decoding the target output layer set.
- the derived TargetDecLayerIdList [] is supplied to the bit stream extraction unit 17 and the target set picture unit 10 as a part of the output control information.
- the target output layer ID list means included in the output control unit 16b is the same as the target output layer ID list derivation means included in the output control unit 16, and thus description thereof is omitted.
- the decoding layer ID list derivation means derives the target decoding layer ID list by, for example, the following pseudo code.
- SC01 This is the starting point of the loop related to the derivation of the target decoding layer ID list TargetDecLayerIdList [].
- Variables i and j are initialized to 0.
- the loop variable of the following iterative process is the variable j, and the decoding layer ID list deriving means performs the processes shown in SC02 to SC06 on the variable j from 0 to (NumLayersInIdList [LayerSetIdx [TargetOLSIdx]] -1). .
- Decoding layer ID list deriving means derives a layer identifier of a layer (hereinafter, target layer j) identified by a variable j included in the output layer set. Specifically, the j-th element (target layer j) (LayerIdList [LayerSetIdx [TargetOLSIdx]] [j]) of the layer set LS # (LayerSetIdx [TargetOLSIdx]) associated with the output layer set OLS # (TargetOLSIdx) Set the layer identifier to the variable iNuhLId.
- the decoding layer ID list deriving unit determines whether the target layer j is a primary picture layer. When the auxiliary picture layer ID (AuxId [iNuhLId]) of the target layer j is 0, it is determined that the target layer j is a primary picture layer, and steps SC04 to SC05 are executed.
- the decoding layer ID list deriving means derives the target layer j as an element of the target decoding layer ID list TargetDecLayerIdList [] when the target layer j is the primary picture layer. Specifically, the j-th element of the layer set LayerSetIdx [TargetOLSIdx] associated with the target output layer set TargetOptLayerSet is added to the i-th element of the target decoding layer ID list TargetDecLayerIdList [].
- the decoding layer ID list derivation means includes all primary picture layers except the auxiliary picture layer in the target decoding layer ID list in the output layer set TargetOptLayerSet.
- the procedure for deriving the target decoding layer ID list is not limited to the above steps, and may be changed within a practicable range.
- the output control unit 16b having the above-described configuration is a target decoding layer that is a layer to be decoded depending on whether each layer of the layer set associated with the target output layer set TargetOptLayerSet is a primary picture layer (not an auxiliary picture layer)
- the ID list TargetDecLayerIdList [] is derived. That is, the output control unit 16b does not include the auxiliary picture layer (AuxId []> 0) that is not essential for decoding the primary picture layer of the target output layer set in the target decoding layer ID list TargetDecLayerIdList []. Thereby, the target set picture decoding unit 10 can omit decoding of the auxiliary picture layer.
- the output control unit 16b having the above configuration does not include, in the target decoding layer ID list TargetDecLayerIdList, a NAL unit having a layer identifier of the auxiliary picture layer that is not essential for decoding the primary picture layer of the target output layer set.
- the NAL unit having the layer identifier of the auxiliary picture layer is discarded.
- the output control unit 16 may be the output control unit 16c in which the primary picture layer and the auxiliary picture layer that is the output layer are included in the target decoding layer ID list TargetDecLayerIdList in the target output layer set.
- the decoding layer ID list derivation means (not shown) included in the output control unit 16c is configured to output the output layer flag OutputLayerFlag [TargetOLSIdx] [] of the target output layer set and the layer set LayerIdList [] [] of the active VPS held in the parameter memory 13 Then, based on the auxiliary picture layer ID (AuxId []) derived from the scalable identifier, a target decoding layer ID list TargetDecLayerIdList [] indicating the configuration of layers required for decoding the target output layer set is derived. The derived TargetDecLayerIdList [] is supplied to the bit stream extraction unit 17 and the target set picture unit 10 as a part of the output control information.
- the target output layer ID list means included in the output control unit 16c is the same as the target output layer ID list derivation means included in the output control unit 16, and thus the description thereof is omitted.
- the decoding layer ID list derivation means derives the target decoding layer ID list by, for example, the following pseudo code.
- SD01 This is the starting point of the loop related to the derivation of the target decoding layer ID list TargetDecLayerIdList [].
- Variables k and j are initialized to 0.
- the loop variable of the following iterative process is the variable j, and the decoding layer ID list deriving means performs the processes shown in SD02 to SD06 on the variable j from 0 to (NumLayersInIdList [LayerSetIdx [TargetOLSIdx]] -1). Execute.
- Decoding layer ID list deriving means derives a layer identifier of a layer (hereinafter, layer j) identified by a variable j included in the output layer set. Specifically, the j-th element (target layer j) (LayerIdList [LayerSetIdx [TargetOLSIdx]] [j]) of the layer set LS # (LayerSetIdx [TargetOLSIdx]) associated with the output layer set OLS # (TargetOLSIdx) Set the layer identifier to the variable iNuhLId.
- the decoding layer ID list deriving unit determines whether the target layer j is a primary picture layer or an auxiliary picture layer that is an output layer. If the auxiliary picture layer ID (AuxId [iNuhLId]) of the target layer j is 0, or the auxiliary picture layer ID of the target layer j is greater than 0, and the output layer flag of the target layer j is 1, step SD04- Run SD05.
- the decoding layer ID list derivation means derives the target layer j as an element of the target decoding layer ID list TargetDecLayerIdList [] when the target layer j is a primary picture layer or an auxiliary picture layer that is an output layer. Specifically, the j-th element of the layer set LayerSetIdx [TargetOLSIdx] associated with the target output layer set TargetOptLayerSet is added to the i-th element of the target decoding layer ID list TargetDecLayerIdList [].
- the decoding layer ID list derivation means includes all layers (primary picture layer or auxiliary picture layer that is the output layer) except for the auxiliary picture layer that is not the output layer in the target decoding layer ID list in the output layer set TargetOptLayerSet. .
- the procedure for deriving the target decoding layer ID list is not limited to the above steps, and may be changed within a practicable range.
- the output control unit 16c configured as described above depends on whether each layer of the layer set associated with the target output layer set TargetOptLayerSet is a primary picture layer (not an auxiliary picture layer) or an auxiliary picture that is an output layer. Then, a target decoding layer ID list TargetDecLayerIdList [] that is a layer to be decoded is derived. That is, the output control unit 16c selects an auxiliary picture layer (AuxId []> 0) whose output layer flag is 0, which is not essential for decoding the primary picture layer of the target output layer set, and the target decoding layer ID list TargetDecLayerIdList []. Not included.
- the target set picture decoding unit 10 can omit decoding of the auxiliary picture layer whose output layer flag is 0.
- the output control unit 16c having the above-described configuration is configured so that the NAL unit including the layer identifier of the auxiliary picture layer whose output layer flag is 0, which is not essential for decoding the primary picture layer of the target output layer set, Since it is not included in the ID list TargetDecLayerIdList, the bit stream extraction unit 17 discards the NAL unit having the layer identifier of the auxiliary picture layer that is not the output layer.
- the designated output layer set OLS # (TargetOLSIdx) does not have an output layer
- at least one layer included in the output layer set is included. It is preferable to designate it as an output layer. For example, all layers included in the output layer set or a primary picture layer having the highest layer identifier may be designated as the output layer.
- the output control unit 16 may be an output control unit 16d that changes the operation depending on whether or not the decoding is for the conformance test. Whether or not the decoding is for the conformance test is given from the outside of the hierarchical video decoding device. Decoding for conformance testing is decoding for testing whether or not it operates according to specified parameters (for example, according to DPB parameters). Otherwise, in order to actually watch a moving image Normal decryption used. The output control unit 16d changes the operation depending on whether or not the decoding is for the conformance test.
- the decoding layer ID list deriving unit derives the target decoding layer ID list using, for example, the following pseudo code.
- the output control unit 16d derives the target decoding layer ID list TargetDecLayerIdList by any of the output control unit 16, the output control unit 16b, and the output control unit 16c described above. To do. That is, a non-output / non-reference layer not related to an output layer is not added (output control unit 16), an auxiliary picture layer is not added (output control unit 16b), and a non-output auxiliary picture layer is not added (output control unit) 16c), the target decoding layer ID list TargetDecLayerIdList is derived.
- the DPB parameter added to the output layer set added so as to satisfy the conformance test is a value corresponding to the case where all layers including the auxiliary picture layer are decoded. Therefore, according to the DPB parameter added to the output layer set, the hierarchical moving picture decoding apparatus can perform decoding availability and prepare the decoding memory according to the DPB parameter when decoding a layer including the auxiliary picture layer. There is an effect. Furthermore, in cases other than the decoding for the conformance test (normal playback), as already described, the decoding of the layer not related to the output and the decoding of the auxiliary layer are omitted, so that the processing is simplified. effective.
- the picture decoding unit 14 generates and outputs a decoded picture based on the input VCL NAL unit and the active parameter set.
- FIG. 20 is a functional block diagram illustrating a schematic configuration of the picture decoding unit 14.
- the picture decoding unit 14 includes a slice header decoding unit 141 and a CTU decoding unit 142.
- the CTU decoding unit 142 further includes a prediction residual restoration unit 1421, a predicted image generation unit 1422, and a CTU decoded image generation unit 1423.
- the slice header decoding unit 141 decodes the slice header based on the input VCL NAL unit and the active parameter set.
- the decoded slice header is output to the CTU decoding unit 142 together with the input VCL NAL unit.
- the CTU decoding unit 142 roughly corresponds to each CTU included in a slice constituting a picture based on the slice segment (slice header and slice data) included in the input VCL NAL unit and the active parameter set.
- a decoded image of the slice is generated by decoding the decoded image of the area to be processed.
- the decoded image of the CTU is generated by the prediction residual restoration unit 1421, the prediction image generation unit 1422, and the CTU decoded image generation unit 1423 inside the CTU decoding unit 142.
- the prediction residual restoration unit 1421 decodes prediction residual information (TT information) included in the input slice data, generates a prediction residual of the target CTU, and outputs it.
- TT information prediction residual information
- the predicted image generation unit 1422 generates and outputs a predicted image based on the prediction method and the prediction parameter indicated by the prediction information (PT information) included in the input slice data. At that time, a decoded image of the reference picture and an encoding parameter are used as necessary. For example, when using inter prediction or inter-layer image prediction, a corresponding reference picture is read from the decoded picture management unit 15.
- the CTU decoded image generation unit 1423 adds the input predicted image and the prediction residual to generate and output a decoded image of the target CTU.
- FIG. 21 is a flowchart showing a decoding process in units of slices constituting a picture of the target layer i in the picture decoding unit 14.
- the first slice flag (first_slice_segment_in_pic_flag) (SYNSH01 in FIG. 17D) of the decoding target slice is decoded.
- the decoding target slice is the first slice in the decoding order (hereinafter, processing order) in the picture, and the position (hereinafter, the first CTU of the decoding target slice in the raster scan order in the picture).
- CTU address is set to 0.
- the counter numCtu hereinafter, the number of processed CTUs numCtu
- the head CTU address of the decoding target slice is set based on a slice address decoded in SD106 described later.
- SD102 The active PPS identifier (slice_pic_paramter_set_id) (SYNSH02 in FIG. 17D) that specifies the active PPS to be referred to when decoding the decoding target slice is decoded.
- the active parameter set is fetched from the parameter memory 13. That is, the PPS having the same PPS identifier (pps_pic_parameter_set_id) as the active PPS identifier (slice_pic_parameter_set_id) referred to by the decoding target slice is set as the active PPS, and the encoding parameter of the active PPS is fetched (read) from the parameter memory 13.
- the SPS having the same SPS identifier (sps_seq_parameter_set_id) as the active SPS identifier (pps_seq_parameter_set_id) in the active PPS is set as the active SPS, and the encoding parameter of the active SPS is fetched from the parameter memory 13.
- the VPS having the same VPS identifier (vps_video_parameter_set_id) as the active VPS identifier (sps_video_parameter_set_id) in the active SPS is set as the active VPS, and the encoding parameter of the active VPS is fetched from the parameter memory 13.
- step SD105 Whether the decoding target slice is the first slice in the processing order in the picture is determined based on the first slice flag. If the first slice flag is 0 (Yes in SD105), the process proceeds to step SD106. In other cases (No in SD105), the process of step SD106 is skipped. When the head slice flag is 1, the slice address of the decoding target slice is 0.
- the slice address (slice_segment_address) of the decoding target slice (SYNSH03 in FIG. 17 (d)) is decoded, and the first CTU address of the decoding target slice is set.
- the head slice CTU address slice_segment_address.
- the CTU decoding unit 142 configures a picture based on the input slice header, active parameter set, and each CTU information (SYNSD01 in FIG. 17 (e)) in the slice data included in the VCL NAL unit. A CTU decoded image of an area corresponding to each CTU included in the slice is generated. Furthermore, after each CTU information, a slice end flag (end_of_slice_segment_flag) (SYNSD2 in FIG. 17 (e)) indicating whether the CTU is the end of the decoding target slice. Further, after decoding each CTU, the value of the number of processed CTUs numCtu is incremented by 1 (numCtu ++).
- SD10B It is determined based on the slice end flag whether or not the CTU is the end of the decoding target slice.
- the slice end flag is 1 (Yes in SD10B)
- the process proceeds to step SD10C.
- the process proceeds to step SD10A in order to decode subsequent CTU information.
- numCtu is equal to PicSizeInCtbsY (Yes in SD10C)
- the decoding process in units of slices constituting the decoding target picture ends.
- numberCtu ⁇ PicSizeInCtbsY No in SD10C
- the process proceeds to step SD101 in order to continue the decoding process in units of slices constituting the decoding target picture.
- Bitstream extraction unit 17 The bitstream extraction unit 17 is based on the output control information supplied from the output control unit 16 (target decoding layer ID list TargetDecLayerIdList indicating the configuration of the decoding target layer in the output layer set and the target highest temporal identifier TargetHighestTid. Then, a bit stream extraction process is performed, and NAL units that are not included in a set (referred to as a target set TargetSet) determined by a target highest temporal identifier TargetHighestTid and a target decoding layer ID list TargetDecLayerIdList are input from the hierarchically encoded data DATA that is input.
- the target layer set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the target set TargetSet is extracted and output.
- bit stream extraction unit 17 includes NAL unit decoding means for decoding a NAL unit header (not shown).
- FIG. 22 is a flowchart showing bit stream extraction processing in units of access units in the bit stream extraction unit 17.
- the bit stream extraction unit 17 decodes the NAL unit header of the supplied target NAL unit according to the syntax table shown in FIG. That is, the NAL unit type (nal_unit_type), the layer identifier (nuh_layer_id), and the temporal identifier (nuh_temporal_id_plus1) are decoded. Note that the layer identifier nuhLayerId of the target NAL unit is set to “nuh_layer_id”, and the temporal identifier temporalId of the target NAL unit is set to “nuh_temporal_id_plus1 ⁇ 1”.
- step SG102 Whether or not the layer identifier and temporal identifier of the target NAL unit are included in the target set TargetSet is determined based on the target decoding layer ID list TargetDecLayerIdList and the target highest temporal identifier. More specifically, when at least one of the following conditions (C1) to (C2) is false (No in SG102), the process proceeds to step SG103. In other cases (both (C1) and (C2) are true) (Yes in SG102), step SG103 is omitted.
- (SG103) Discard target NAL unit. That is, since the target NAL unit is not included in the target set TargetSet, the bitstream extraction unit 17 removes the target NAL unit from the input hierarchical encoded data DATA.
- step SG10A Judge whether there is an unprocessed NAL unit in the same access unit. If there is an unprocessed NAL unit (No in SG10A), the process proceeds to step SG101 in order to continue extracting the bit stream in units of NAL units constituting the target access unit. In other cases (Yes in SG10A), the process proceeds to step SG10B.
- SG10B It is determined whether or not there is an access unit next to the target access unit in the input hierarchical encoded data DATA. If there is a next access unit (Yes in SG10B), the process proceeds to step SG101 in order to continue processing the next access unit. If there is no next access unit (No in SG10B), the bitstream extraction process is terminated.
- bitstream extraction unit 17 has been described above. However, the operation is not limited to the above step, and the step may be changed within a feasible range.
- the bitstream extraction processing is performed based on the layer ID list LayerIdListTarget of the layers constituting the target layer set LayerSetTarget supplied from the outside and the target highest temporal identifier HighestTidTarget.
- NAL units not included in the target set TargetSet determined by the target highest temporal identifier HighestTidTarget and the layer ID list LayerIdListTarget of the target layer set LayerSetTarget are removed (discarded) from the hierarchically encoded data DATA to be included in the target set TargetSet
- the encoded data BitstreamToDecode composed of NAL units can be extracted and generated.
- the hierarchical moving picture decoding apparatus 1 (hierarchical picture decoding apparatus) according to the present embodiment described above includes an output layer set identifier TargetOLSIdx supplied from the outside, active VPS layer set information held in the parameter memory 13, and output.
- a target output layer ID list indicating the layer configuration of the output layer of the target output layer set TargetOptLayerSet is derived based on the layer set information, and further, the output layer set identifier TargetOLSIdx and the layer set information of the active VPS held in the parameter memory 13 And target layer indicating the layer configuration required for decoding the target output layer set TargetOptLayerSet based on the output layer set information, the dependency flag derived from the inter-layer dependency information, and the derived target output layer ID list TargetOptLayerIdList Deriving the layer ID list TargetDecLayerIdList Comprising a force controller 16 (or the output control unit 16a).
- the output control unit 16 (and the output control unit 16a) excludes non-output layers and non-dependent layers that are not essential for output layer decoding from the target decoding layer ID list. That is, the output control unit 16 can instruct the hierarchical video decoding device 1 to omit the decoding of the non-output / non-reference layer, which is not essential for decoding the output layer of the target output layer set. Therefore, the hierarchical video decoding device 1 that decodes the layers included in the target decoding layer ID list TargetDecLayerIdList outputs the encoded data of the output layer that is essential for decoding and the dependency layer of the output layer, among the target output layer set TargetOptLayerSet. It is possible to perform decoding and omit the decoding process of the non-output layer and the non-dependent layer.
- the output control unit 16 may instruct the bitstream extraction unit 17 to discard the NAL unit having the layer identifier of the non-output / non-reference layer that is not essential for decoding the output layer of the target output layer set.
- the bitstream extraction unit 17 included in the hierarchical video decoding device 1 includes the target decoding layer ID list TargetDecLayerIdList supplied from the output control unit 16 and the highest-order sublayer associated with the decoding target layer supplied from the outside.
- the target set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the target set TargetSet is removed (destroyed) from NAL units not included in the target set TargetSet determined by the target highest temporal identifier TargetHighestTid. Can be extracted.
- the hierarchical video decoding device 1 (hierarchical image decoding device) according to the present embodiment described above includes an output control unit 16b to an output control unit 16c instead of the output control unit 16 (or the output control unit 16a). May be.
- the output control unit 16b excludes, from the target decoding layer ID list, auxiliary picture layers that are not essential for decoding the primary picture layer in the target output layer set. That is, a target decoding layer ID list that does not include an auxiliary picture layer is constructed. Therefore, the output control unit 16b can instruct the hierarchical video decoding device 1 to omit decoding of the auxiliary picture layer, which is not essential for decoding of the primary picture layer of the target output layer set. Accordingly, the hierarchical video decoding device 1 that decodes the layers included in the target decoding layer ID list TargetDecLayerIdList decodes the encoded data of the primary picture layer in the target output layer set TargetOptLayerSet and omits the decoding process of the auxiliary picture layer. can do.
- the output control unit 16b can instruct the bitstream extraction unit 17 to discard the NAL unit having the layer identifier of the auxiliary picture layer that is not essential for decoding the primary picture layer of the target output layer set. is there. That is, the bitstream extraction unit 17 included in the hierarchical video decoding device 1 includes the target decoding layer ID list TargetDecLayerIdList supplied from the output control unit 16b and the highest-order sublayer associated with the decoding target layer supplied from the outside.
- the target set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the target set TargetSet is removed (destroyed) from NAL units not included in the target set TargetSet determined by the target highest temporal identifier TargetHighestTid. Can be extracted.
- the output control unit 16c excludes the auxiliary picture layer that is not the output layer from the target decoding layer ID list in the target output layer set. That is, a target decoding layer ID list that does not include the auxiliary picture layer that is a non-output layer is constructed. Therefore, the output control unit 16c can instruct the hierarchical video decoding device 1 to omit decoding of the auxiliary picture layer whose output layer flag of the target output layer set is 0. Therefore, the hierarchical video decoding device 1 that decodes the layers included in the target decoding layer ID list TargetDecLayerIdList decodes the encoded data of the primary picture layer and the auxiliary picture layer that is the output layer in the target output layer set TargetOptLayerSet. The decoding process of the auxiliary picture layer that is not the output layer can be omitted.
- the output control unit 16c can instruct the bitstream extraction unit 17 to discard the NAL unit having the layer identifier of the auxiliary picture layer that is not the output layer. That is, the bitstream extraction unit 17 included in the hierarchical video decoding device 1 includes the target decoding layer ID list TargetDecLayerIdList supplied from the output control unit 16c and the highest-order sublayer associated with the decoding target layer supplied from the outside.
- the target set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the target set TargetSet is removed (destroyed) from NAL units not included in the target set TargetSet determined by the target highest temporal identifier TargetHighestTid. Can be extracted.
- Modification Example 1 Hierarchical Video Decoding Device 1A of Hierarchical Video Decoding Device 1
- the hierarchical moving picture decoding apparatus 1A decodes the hierarchical encoded data DATA supplied from the hierarchical moving picture encoding apparatus 2, and decodes pictures of each layer included in the target set TargetSet determined by output designation information supplied from the outside And the decoded picture of the output layer is output as the output picture POUT # T.
- the hierarchical video decoding device 1A the element TargetDecLayerIdList [0] .. TargetDecLayerIdList [N-1] of the target decoding layer ID list TargetDecLayerIdList indicating the layer configuration required for decoding the target output layer set TargetOptLayerSet indicated by the output designation information ] (N is the number of layers included in the target set) in this order, the encoded data of the picture of layer i is decoded and the decoded picture is generated, and the output layer information OutputLayerFlag [i] of the layer i is set to “output layer ”Indicates that the decoded picture of layer i is output at a predetermined timing.
- the hierarchical moving picture decoding apparatus 1A includes a NAL demultiplexing unit 11 and a target set picture decoding unit 10. Further, the target set picture decoding unit 10 includes a non-VCL decoding unit 12, a parameter memory 13, a picture decoding unit 14, a decoded picture management unit 15, and an output control unit 16A.
- the NAL demultiplexing unit 11 further includes a bit stream extraction unit 17A. Note that the same elements as those in the hierarchical video decoding apparatus 1 are denoted by the same reference numerals and description thereof is omitted.
- the output control unit 16A basically has the same function as the output control unit 16. That is, the output control unit 16A selects the output layer set OLS # TargetOLSIdx designated by the output layer set identifier TargetOLSIdx included in the output designation information as a processing target. Subsequently, the output control unit 16A derives the output layer ID list TargetOptLayerIdList by the same process as the derivation of the output layer ID list in the output control unit 16.
- a decoding layer ID list derivation means (not shown) provided in the output control unit 16A is based on the output layer set identifier TargetOLSIdx included in the output designation information, the layer set information of the active VPS held in the parameter memory 13, and the output layer set information.
- a target decoding layer ID list TargetDecLayerIdList indicating a layer configuration necessary for decoding the target output layer set is derived, and supplied to the bitstream extraction unit 17A and the target set picture unit 10 as part of output control information.
- the target decoding layer ID list is derived by, for example, the following pseudo code.
- the decoding layer ID list derivation unit sets the layer ID list LayerIdList [LayerSetIdx [TargetOLSIdx]] of the layer set associated with the target output layer set TargetOptLayerSet as the target decoding layer ID list TargetDecLayerIdList.
- Target extraction unit 17A Among the output control information (output layer set) supplied from the bitstream extraction unit 17A and the output control unit 16A, the target decoding layer ID list TargetDecLayerIdList indicating the configuration of the decoding target layer, and the target highest temporal identifier TargetHighestTid NAL not included in the set (referred to as target set TargetSet) determined by the target hierarchical temporal identifier TargetHighestTid and the target decoding layer ID list TargetDecLayerIdList from the input hierarchical encoded data DATA Remove (destroy) the unit.
- target set TargetSet the target highest temporal identifier TargetHighestTid NAL not included in the set
- the bitstream extraction unit 17A includes a target decoding layer ID list TargetDecLayerIdList indicating a configuration of a decoding target layer, a target output layer ID list TargetOptLayerIdList [], and an active VPS layer set LayerIdList [] held in the parameter memory 13
- the dependency flag recursiveRefLayerFlag [] [] derived from [] and inter-layer dependency information the NAL unit of the non-output layer and the non-dependent layer of the target output layer set is discarded, and the input hierarchical encoded data DATA
- the NAL unit not included in the target set TargetSet is removed (discarded) by the bitstream extraction process, and the target set encoded data DATA # T (BitstreamToDecode) composed of the NAL units included in the target set TargetSet is extracted.
- Output the target decoding layer ID list TargetDecLayerIdList indicating a configuration of a decoding target layer, a target output layer ID list TargetOptLayerIdList [], and
- Bitstream extraction process 2 Below, with reference to FIG. 23, the operation
- the operations common to the bitstream extraction unit 17 are SG101 to SG103 and SG10A to SG10B, and the same step numbers are given and description thereof is omitted.
- steps SG104 to SG105 added after SG101 to SG103 will be described.
- bitstream extraction unit 17A performs the following condition determinations (C3) to (C4). That is, if all of the conditions (C3) to (C4) are false (No in SG104), the process proceeds to step SG105. Otherwise (either (C3) or (C4) is true) (Yes in SG104), the process proceeds to step SG10A.
- TargetOptLayerIdList [] has the same value as the layer identifier of the target NAL unit (if the layer identifier of the target NAL unit is equal to the layer identifier of the output layer), determine true In other cases (the target output layer ID list TargetOptLayerIdList does not have the same value as the layer identifier of the target NAL unit), it is determined to be false.
- the layer having the layer identifier of the target NAL unit is a dependency layer of one of the output layers included in the target output layer ID list TargetOptLayerIdList []”, it is determined to be true.
- the layer having the layer identifier of the NAL unit is an independent layer of the output layer) and is determined to be false.
- (SG105) Discard target NAL unit. That is, since the target NAL unit is a non-output layer and non-dependent layer NAL unit, the bitstream extraction unit 17A further removes the target NAL unit from the input hierarchical encoded data DATA. Note that only the VCLALNAL unit of the non-output layer and the non-dependent layer may be discarded.
- bitstream extraction unit 17A has been described above. However, the operation is not limited to the above step, and the step may be changed within a feasible range.
- step SG104 for example, the true / false value of the flag refLayerFlag derived by the following pseudo code may be determined.
- the pseudo code can be expressed in steps as follows.
- SC02 This is the starting point of the loop related to the derivation of the flag refLayerFlag indicating whether the layer with the layer identifier nuh_layer_id is a dependent layer (direct reference layer or indirect reference layer) of the output layer TargetOptLayerIdList [k].
- the variable k and the flag refLayerFlag are initialized to 0, and the process shown in SC03..SC04 is executed for the variable k from 0 to (NumOptLayersInOLS [TargetOLSIdx]-1).
- the bit stream extraction unit 17A having the above configuration discards a NAL unit having a non-output and non-reference layer layer identifier from the NAL units included in the target set TargetSet. That is, the bitstream extraction unit 17A has an effect of generating target set encoded data BitstreamToDecode that does not include a layer NAL unit that is not essential for decoding of the output layer of the target output layer set. Thereby, the target set picture decoding unit 10 that decodes the target set encoded data BitstreamToDecode supplied from the bitstream extraction unit 17A can omit the decoding of the non-output and non-reference layers.
- step SG102 of the bitstream extraction unit 17A In addition to the condition determinations (C3) and (C4) of the SGB 104 of the bitstream extraction unit 17A, the following condition (D1) may be added.
- the modified example of the bit stream extraction unit 17A having the above configuration includes a base layer in the target set TargetSet. Accordingly, when decoding encoded data including layer set B, which is a subset of layer set A, generated by bitstream extraction processing from encoded data including a certain layer set A, a layer C ( In a layer identifier> 0), when a parameter set (VPS / SPS / PPS) having a layer identifier in the base layer is referred to as an active parameter set, the base layer is not included in the encoded data including the layer set B, and a certain layer C Can be prevented from being decrypted.
- a parameter set VPN / SPS / PPS
- bitstream Extraction Unit 17A Bitstream Extraction Unit 17A1
- the non-output layer and the non-dependent layer that are not essential for decoding of the output layer are excluded from the target set, but the present invention is not limited to this.
- the output layer set is composed of one or more primary picture layers and one or more auxiliary picture layers
- the auxiliary picture layer that is not essential for decoding the primary picture layer is excluded from the target set, and the auxiliary picture layer
- the bit stream extraction unit 17A1 that discards the NAL unit having the layer identifier may be used.
- the bitstream extraction unit 17A1 includes a target decoding layer ID list TargetDecLayerIdList indicating a configuration of a decoding target layer, a target output layer ID list TargetOptLayerIdList [], and an active VPS layer set LayerIdList [] [] held in the parameter memory 13 And the NAL unit having the layer identifier of the auxiliary picture layer of the target output layer set and the NAL unit not included in the target set TargetSet are removed (discarded) based on the auxiliary picture layer ID derived from the scalable identifier, and the target The target set encoded data DATA # T (BitstreamToDecode) composed of NAL units included in the set TargetSet is extracted and output.
- TargetDecLayerIdList indicating a configuration of a decoding target layer
- TargetOptLayerIdList [] an active VPS layer set LayerIdList [] [] held in the parameter memory 13
- Bitstream extraction process 3 (Bitstream extraction process 3)
- bitstream extraction unit 17A1 the operation of the bitstream extraction unit 17A1 according to the present embodiment will be described.
- the operations common to the bitstream extraction unit 17 are SG101 to SG103 and SG10A to SG10B, and the same step numbers are given and description thereof is omitted. Only steps SG104A to SG105A added after SG101 to SG103 will be described below.
- bit stream extraction unit 17A1 performs the condition determination of (C5) below. That is, if the condition of (C5) is false (No in SG104A), the process proceeds to step SG105A. Otherwise ((C5) is true) (Yes in SG104A), the process proceeds to step SG10A.
- bitstream extraction unit 17A1 has been described above. However, the operation is not limited to the above step, and the step may be changed within a feasible range.
- the bit stream extraction unit 17A1 having the above configuration discards the NAL unit having the layer identifier of the auxiliary picture layer from the NAL units included in the target set TargetSet. That is, the bit stream extraction unit 17A1 has an effect of generating target set encoded data BitstreamToDecode that does not include the NAL unit of the auxiliary picture layer that is not essential for decoding the primary picture layer of the target output layer set. Accordingly, the target set picture decoding unit 10 that decodes the target set encoded data BitstreamToDecode supplied from the bitstream extraction unit 17A1 can omit decoding of the auxiliary picture layer.
- the bit stream extraction 17A may be the bit stream extraction unit 17A2 that discards the NAL unit having the layer identifier of the auxiliary picture layer that is a non-output layer in the output layer set.
- the bitstream extraction unit 17A2 includes a target decoding layer ID list TargetDecLayerIdList indicating the configuration of a layer to be decoded, an active VPS layer set LayerIdList [] [] held in the parameter memory 13, an output layer flag OutputLayerFlag [] [], And the NAL unit having the layer identifier of the auxiliary picture layer that is a non-output layer of the target output layer set and the NAL unit not included in the target set TargetSet are removed (destroyed) based on the auxiliary picture layer ID derived from the scalable identifier
- the target set encoded data DATA # T (BitstreamToDecode) composed of the NAL units included in the target set TargetSet is extracted and output.
- Bitstream extraction process 4 Bitstream extraction process 4
- bitstream extraction unit 17A2 the operation of the bitstream extraction unit 17A2 according to the present embodiment will be described.
- the operations common to the bitstream extraction unit 17 are SG101 to SG103 and SG10A to SG10B, and the same step numbers are given and description thereof is omitted. Only steps SG104B to SG105B added after SG101 to SG103 will be described below.
- the bit stream extraction unit 17A2 performs the following condition determinations (C5) to (C6). That is, if all the conditions (C5) to (C6) are false (No in SG104B), the process proceeds to step SG105B. Otherwise ((C5) or (C6) is true) (Yes in SG104B), the process proceeds to step SG10A. Note that the condition (C5) is the same as the condition (C5) in the bitstream extraction process 3, and thus the description thereof is omitted.
- bitstream extraction unit 17A2 has been described above. However, the operation is not limited to the above step, and the step may be changed within a practicable range.
- the bit stream extraction unit 17A2 having the above configuration discards the NAL unit having the layer identifier of the auxiliary picture layer that is a non-output layer from the NAL units included in the target set TargetSet. That is, the bit stream extraction unit 17A2 has an effect of generating target set encoded data BitstreamToDecode that does not include the NAL unit of the auxiliary picture layer that is a non-output layer of the target output layer set. Accordingly, the target set picture decoding unit 10 that decodes the target set encoded data BitstreamToDecode supplied from the bitstream extraction unit 17A2 can omit decoding of the auxiliary picture layer.
- the bitstream extraction unit 17A included in the hierarchical video decoding device 1A includes an output layer ID list TargetOptLayerIdList, a target decoding layer ID list TargetDecLayerIdList, supplied from the output control unit 16A, And the target highest temporal identifier TargetHighestTId and the dependency flag recursiveRefLayerFlag [] [] derived from the inter-layer dependency information, the NAL included in the target set by the bitstream extraction processing from the encoded data input from the outside Generate target set encoded data BitstreamToDecode composed of units.
- the bitstream extraction unit 17A excludes non-output layers and non-dependent layers that are not essential for decoding of the output layer from the target set. Therefore, the hierarchical video decoding device 1A that decodes the target set encoded data BitstreamToDecode generated by the bitstream extraction unit 17A is not essential for decoding the output layer of the target output layer set, and includes the non-output layer and the non-reference layer. There is an effect that decoding can be omitted.
- bitstream extraction unit 17A1 excludes the auxiliary picture layer from the target set. Therefore, the hierarchical video decoding device 1A that decodes the generated target set encoded data BitstreamToDecode by the bitstream extraction unit 17A1 has an effect that the decoding of the auxiliary picture layer can be omitted.
- bitstream extraction unit 17A1 excludes the auxiliary picture layer that is a non-output layer from the target set. Therefore, the hierarchical video decoding device 1A that decodes the target set encoded data BitstreamToDecode generated by the bitstream extraction unit 17A2 has an effect that the decoding of the auxiliary picture layer that is a non-output layer can be omitted.
- the hierarchical moving picture decoding apparatus 1B includes output designation information supplied from the outside from the hierarchical encoded data DATA supplied from the hierarchical moving picture encoding apparatus 2, and a non-VCL decoding unit 12B included in the hierarchical moving picture decoding apparatus 1B.
- the bit stream extraction unit 17B operates the encoded data extraction process specified by the sub bit stream characteristic information decoded from the above to generate the target set encoded data BitstreamToDecode, and decodes the generated target set encoded data bitstreamToDecode
- a decoded picture of each layer included in the target set TargetSet may be generated, and the decoded picture of the output layer may be output as the output picture POUT # T.
- the hierarchical video decoding device 1B has the element TargetDecLayerIdList [0] .. TargetDecLayerIdList [N-1] of the target decoding layer ID list TargetDecLayerIdList indicating the layer configuration required for decoding the target output layer set TargetOptLayerSet indicated by the output designation information. ] (N is the number of layers included in the target set) in this order, the encoded data of the picture of layer i is decoded and the decoded picture is generated, and the output layer information OutputLayerFlag [i] of the layer i is set to “output layer ”Indicates that the decoded picture of layer i is output at a predetermined timing.
- the hierarchical video decoding device 1B includes a NAL demultiplexing unit 11 and a target set picture decoding unit 10. Further, the target set picture decoding unit 10 includes a non-VCL decoding unit 12B, a parameter memory 13, a picture decoding unit 14, a decoded picture management unit 15, and an output control unit 16A.
- the NAL demultiplexing unit 11 further includes a bit stream extraction unit 17B. The same elements as those of the hierarchical video decoding device 1 or the hierarchical video decoding device 1A are denoted by the same reference numerals and description thereof is omitted.
- Non-VCL decoding unit 12B has the same function as the non-VCL decoding unit 12 included in the hierarchical video decoding device 1, and is further generated by an output layer set unit bitstream extraction process and the bitstream extraction process Sub-bitstream characteristic information decoding means for decoding the sub-bitstream characteristic information indicating the characteristic (bit rate information, etc.) of the sub-bitstream to be performed.
- the sub-bitstream characteristic information is generally a sub-bit generated by discarding a picture (NAL unit) of a layer that does not affect (not mandatory) the output layer of the output layer set defined by the active VPS. Provides stream bit rate information. Also, if sub-bitstream characteristic information exists, it is associated with the initial IRAP access unit and applied to the CVS associated with the initial IRAP.
- the sub-bitstream characteristic information includes syntaxes F1 to F7, and these syntaxes are decoded from the parameter set or SEI by the sub-bitstream characteristic information decoding unit and supplied to the bitstream extraction unit 17B. .
- Active VPS identifier active_vps_id (SYNSBP01 in FIG. 24) is an identifier for specifying the active VPS referred to by the sub-bitstream characteristic information.
- F2 The number of additional sub bitstreams num_additional_sub_stream_minus1 (SYNSBP02 in FIG. 24) is a value of the number of sub bitstreams ⁇ 1 defined in the sub bitstream characteristic information.
- the number of additional sub bitstreams NumAddSubStream is num_additional_sbu_stream_minus1 +1.
- the sub bitstream characteristic information decoding means decodes the syntaxes F3 to F7 from the encoded data for the subbitstream 0 to the subbitstream (NumAddSubStream-1).
- Bitstream extraction mode sub_bitstream_mode [i] (SYNSBP03 in FIG. 24) is a syntax that designates a bitstream extraction process for generating a subbitstream (also referred to as substream i) of index i.
- the bit stream extraction processing corresponding to each bit stream extraction mode will be described in the bit stream extraction unit 17B.
- Output layer set identifier output_layer_set_idx_to_vps [i] (SYNSBP04 in FIG. 24) is an index of the output layer set corresponding to substream i. That is, the substream i corresponds to the output layer set OLS # (output_layer_set_idx_to_vps [i]).
- F5 highest temporal identifier highest_sublayer_id [i] (SYNSBP05 in FIG. 24) is the highest temporal identifier of the output layer set corresponding to sub-bitstream i.
- F6 Average bit rate avg_bit_rate [i] (FIG. 24_SYNSBP06) is the average bit rate (bits / sec) of sub-bitstream i.
- F7 Maximum bit rate max_bit_rate [i] (FIG. 24_SYNSBP07) is the maximum bit rate (bits / sec) of sub-bitstream i.
- Bitstream extraction mode sub_bitstream_mode [i] 0: When the value of the bitstream extraction mode is 0, in the bitstream extraction unit 17B, the layer ID list LayerIdList [output_layer_set_idx_to_vps [i]] and the highest temporal identifier highest_sublayer_id [ i] as an input, the bitstream extraction process 1 described above is executed, and a subbitstream i corresponding to the output layer set OSL # (output_layer_set_idx_to_vps [i]) is generated from the CVS associated with the subbitstream characteristic information Indicates that
- bitstream extraction mode sub_bitstream_mode [i] 1: When the value of bitstream extraction mode is 1, in bitstream extraction unit 17B, layer ID list LayerIdList [LayerSetIdx [LayerSetIdx [output_layer_set_idx_to_vps [i]]] and highest temporal With the identifier highest_sublayer_id [i], the output layer set OLS # output_layer_set_idx_to_vps [i], the output layer ID list TargetOptLayeridList, and the dependency flag recursiveRefLayrFlag [] [] as input, the above bitstream extraction process 2 is executed and the sub bit This indicates that a sub bitstream i corresponding to the output layer set OSL # (output_layer_set_idx_to_vps [i]) is generated from the CVS associated with the stream characteristic information.
- the output layer ID list TargetOptLayerIdList of the output layer set OLS # ouptut_layer_set_idx_to_vps [i] is derived, for example, by pseudo code indicating the derivation of the TargetOptLayerIdList described above.
- bitstream extraction mode sub_bitstream_mode [i] is X (for example, 2)
- the bitstream extraction processing 3 described above is executed, and the output layer set OSL # (output_layer_set_idx_to_vps [i ]) May be generated.
- the bitstream extraction unit 17B When the value of the bitstream stream extraction mode sub_bitstream_mode [i] is Y (for example, 3), the bitstream extraction unit 17B performs the layer ID list LayerIdList [LayerSetIdx [output_layer_set_idx_to_vps [i]]] and the highest temporal identifier.
- the above bitstream extraction process 4 is executed with the highest_sublayer_id [i], the auxiliary picture layer ID AuxID [], and the output layer flag OutputLayerFlag [LayerSetIdx [output_layer_set_idx_to_vps [i]]]]]]] as inputs, and the sub bitstream It may be indicated that the sub bitstream i corresponding to the output layer set OSL # (output_layer_set_idx_to_vps [i]) is generated from the CVS associated with the characteristic information.
- the bit stream extraction unit 17B includes at least a bit stream extraction process 1 in the bit stream extraction unit 17 and a bit stream extraction process 2 in the bit stream extraction unit 17A. Furthermore, the bit stream extraction unit 17B may include a bit stream extraction process 3 in the bit stream extraction unit 17A1 and / or a bit stream extraction process 4 in the bit stream extraction unit 17A2.
- the bitstream extraction unit 17B uses the layer ID list LayerIdList [LayerSetIdx [output_layer_set_idx_to_vps [i]]] and the highest temporal identifier highest_sublayer_id [i] as inputs,
- the stream extraction process 2 is executed to generate a sub bitstream i corresponding to the output layer set OSL # (output_layer_set_idx_to_vps [i]) from the CVS associated with the subbitstream characteristic information.
- the bitstream extraction unit 17B performs the layer ID list LayerIdList [LayerSetIdx [output_layer_set_idx_to_vps [i]]], the highest temporal identifier highest_sublayer_id [i], and the output layer set OLS.
- the bitstream extraction unit 17B When the value of the bitstream extraction mode sub_bitstream_mode [i] is X (for example, 2), the bitstream extraction unit 17B performs the layer ID list LayerIdList [LayerSetIdx [output_layer_set_idx_to_vps [i]]] and the highest temporal identifier highest_sublayer_id [i ], Further, with the auxiliary picture layer IDIDAuxID [] as an input, the bitstream extraction process 3 described above is executed, and the output layer set OSL # (output_layer_set_idx_to_vps [i]) is obtained from the CVS associated with the sub bitstream characteristic information. Sub-bitstream i corresponding to may be generated.
- the bitstream extraction unit 17B sets the layer ID list LayerIdList [LayerSetIdx [output_layer_set_idx_to_vps [i]]] and the highest temporal identifier.
- a sub bitstream i corresponding to the output layer set OSL # (output_layer_set_idx_to_vps [i]) may be generated from the CVS associated with the information.
- the bit stream extraction process corresponding to the bit stream extraction mode sub_bitstream_mode [i] of the sub bit stream characteristic information is executed to generate the sub bit stream i.
- the bitstream extraction mode sub_bitstream_mode [i] 1
- decoding of the output layer of the output layer set OLS # (output_layer_set_to_vps [i]) from the CVS (encoded data) associated with the subbitstream characteristic information A sub-bitstream i in which NAL units of a non-output layer and a non-reference layer (non-dependent layer) are discarded is generated.
- the image decoding apparatus 1B that decodes the sub-bitstream i has an effect that the decoding of the non-output layer and the independent layer that is not essential for the decoding of the output layer set OLS # (output_layer_set_to_vps [i]) can be omitted. .
- the output layer set OLS # (output_layer_set_to_vps [i]) is obtained from the CVS (encoded data) associated with the subbitstream characteristic information.
- a sub-bitstream i in which the NAL unit of the auxiliary picture layer that is not essential for decoding the primary picture is discarded is generated. Therefore, the image decoding device 1B that decodes the sub-bitstream i has an effect that the decoding of the auxiliary picture layer of the output layer set OLS # (output_layer_set_to_vps [i]) can be omitted.
- the output layer set OLS # (output_layer_set_to_vps [i]) is obtained from the CVS (encoded data) associated with the subbitstream characteristic information.
- the sub-bitstream i is generated by discarding the NAL unit of the auxiliary picture layer, which is a non-output layer, that is not essential for decoding the primary picture. Therefore, the image decoding device 1B that decodes the sub-bitstream i has an effect that the decoding of the auxiliary picture layer that is a non-output layer of the output layer set OLS # (output_layer_set_to_vps [i]) can be omitted.
- Hierarchical video decoding device 1 (and its modifications (including hierarchical video decoding device 1A and hierarchical video decoding device 1B)) / hierarchical video encoding device 2 is a layer associated with the output layer set shown below. Decode / generate encoded data that satisfies the conformance condition CC1 for the set.
- condition CC1 can be rephrased as conditions CC2 to CC4.
- the conditions CC1 to CC4 mean that a base layer (a layer having a layer identifier of 0) is always included as a decoding target layer in the output layer set.
- the hierarchical video decoding device 1 that decodes encoded data that satisfies the conformance condition CC (CC is one of CC1 to CC4) with respect to a layer set (that is, all layer sets) associated with an output layer set is always a base. It is guaranteed to decode the layer. Accordingly, when decoding encoded data including layer set B, which is a subset of layer set A, generated by bitstream extraction processing from encoded data including a certain layer set A, the base layer (the layer with layer identifier 0) is decoded.
- the decoding device V1 (for example, performing the decoding process defined in HEVC Main profile) compatible only with the decoding of) can operate without problems for the following reasons.
- the encoded data including the extracted layer set B includes VCL (slice segment) and nonVCL (parameter set (VPS / SPS / PPS) of layer identifier 0.
- the decoding device V1 decodes the slice segment with the layer identifier 0, performs decoding when the SPS profile referred to from the slice segment with the layer identifier 0 indicates that decoding is possible, and performs the slice segment with the layer identifier 0.
- the PTL information such as the SPS profile referred to in FIG. 1 does not indicate that decoding is possible, decoding of the encoded data can be stopped.
- the decryption device V1 can decrypt or stop decryption. That is, the decoding device V1 can perform decoding (can handle) without any problem.
- the encoded data that does not satisfy the conditions CC1 to CC4 is decoded by the layer set decoding device V1. That is, the following problem occurs when the decoding device V1 decodes a layer set that does not include a base layer.
- the decoding device V1 Since there is no slice segment with the layer identifier 0 in the encoded data, the decoding device V1 does not decode the slice segment. ⁇ Slice segment's slice_pic_parameter_set_id is not decrypted, so PPS is not activated (SPS and VPS are not activated as well) Since the decryption device V1 does not decrypt the activated SPS (and VPS), it does not decrypt PTL information such as a profile included in the SPS (VPS), and thus is not decrypted. When the encoded data in the internal buffer is exhausted, the decoding device V1 requests a request for encoded data from the encoding device (or the encoded data transmission device or the encoded data buffering device). Since the requested encoded data also does not have an object to be decoded, there is a possibility that the request and decoding of the encoded data are continued forever to decode the requested output image (for example, one picture).
- layer set A (or layer set B, which is a subset of layer set A generated by bitstream extraction from encoded data including layer set A) This has the effect of ensuring that the encoded data including can be decoded (supported).
- Hierarchical video decoding device 1 (and its modifications (including hierarchical video decoding device 1A and hierarchical video decoding device 1B)) / hierarchical video encoding device 2 is a layer associated with the output layer set shown below. Decode / generate encoded data that satisfies the conformance condition CX1 for the set.
- condition CX1 can be rephrased as the condition CX2.
- the conditions CX1 and CX2 mean that at least one or more primary picture layers are included as a decoding target layer in the output layer set.
- the hierarchical video decoding device 1 decodes the encoded data that satisfies the conformance condition CX (CX is one of CX1 to CX2) with respect to the output layer set. It is guaranteed that one or more primary pictures are always decoded. That is, it is possible to prevent a case where there is no layer (primary picture layer) to be decoded in the target decoding layer ID list derived in the output control unit 16b and the output control 16c.
- the hierarchical video decoding device 1 (and its modifications (including the hierarchical video decoding device 1A and the hierarchical video decoding device 1B) / hierarchical video encoding device 2 are configured to conformance conditions CX (CX is CX1 to CX1). It is preferable to decode / generate encoded data that satisfies any one of CX2 and further satisfies the conformance condition CY1.
- the layer j 0..NumLayersInIdList [LayerSetIdx [i]]-1) is an auxiliary picture layer (AuxID [nuh_layer_id [ LayerIdList [LayerSetIdx]] [j]]> 0), the layer j is a non-output layer of the output layer set.
- condition CY1 can be rephrased as conditions CY2 and CY3.
- FIG. 2 is a functional block diagram showing a schematic configuration of the hierarchical video encoding device 2.
- the hierarchical video encoding device 2 encodes the input image PIN # T (picture) of each layer / sublayer included in the target set to be encoded, and generates hierarchical encoded data DATA of the target set. That is, the video encoding device 2 is configured in the order of the elements TargetLayerIdList [0] ... TargetLayerIdList [N-1] (N is the number of layers included in the target set (target layer set)) of the layer ID list of the target set TargetSet.
- the picture of each layer is encoded, and the encoded data is generated.
- the above-described conformance condition CC (CC is CC1 to CC4) is satisfied. It is preferable to generate target set hierarchically encoded data DATA.
- the above-described conformance is provided in the hierarchical video decoding device 1 (and its modifications) including the output control unit 16b or the output control unit 16c. It is preferable to generate the hierarchical encoded data DATA of the target set so as to satisfy the condition CX (CX is any one of CX1 and CX2).
- the above-described conformance condition CX in addition to (CX is any one of CX1 to CX2), it is preferable to generate the hierarchical encoded data DATA of the target set so as to satisfy the conformance condition CY (CY is any one of CY1 to CY3).
- the hierarchical video encoding device 2 includes a target set picture encoding unit 20 and a NAL multiplexing unit 21. Furthermore, the target set picture coding unit 20 includes a non-VCL coding unit 22, a picture coding unit 24, a decoded picture management unit 15, and a coding parameter determination unit 26.
- the decoded picture management unit 15 is the same component as the decoded picture management unit 15 included in the hierarchical video decoding device 1 already described. However, since the decoded picture management unit 15 included in the hierarchical video encoding device 2 does not need to output a picture recorded in the internal DPB as an output picture, the output can be omitted. Note that the description described as “decoding” in the description of the decoded picture management unit 15 of the hierarchical video decoding device 1 is replaced with “encoding”, so that the decoded picture management unit 15 included in the hierarchical video encoding device 2 also includes Applicable.
- the NAL multiplexing unit 21 stores the VCL and non-VCL of each layer of the input target set in the NAL unit to generate NAL-multiplexed hierarchical moving image encoded data DATA # T, and to the outside Output.
- the NAL multiplexing unit 21 includes the non-VCL encoded data supplied from the target set picture encoding unit 20, the VCL encoded data, and the NAL unit type and layer identifier corresponding to each non-VCL and VCL. Then, the temporal identifier is stored (encoded) in the NAL unit, and NAL-multiplexed hierarchical encoded data DATA # T is generated.
- the encoding parameter determination unit 26 selects one set from among a plurality of sets of encoding parameters.
- the encoding parameters are various parameters related to each parameter set (VPS, SPS, PPS), prediction parameters for encoding a picture, and encoding targets generated in association with the prediction parameters. It is a parameter.
- the encoding parameter determination unit 26 calculates a cost value indicating the amount of information and the encoding error for each of the plurality of sets of the encoding parameters.
- the cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient ⁇ .
- the code amount is an information amount of encoded data of each layer / sublayer of the target set obtained by variable length encoding of the quantization error and the encoding parameter.
- the square error is the sum between pixels regarding the square value of the difference value between the input image PIN # T and the predicted image.
- the coefficient ⁇ is a real number larger than a preset zero.
- the encoding parameter determination unit 26 selects a set of encoding parameters that minimizes the calculated cost value, and supplies the selected set of encoding parameters to the parameter set encoding unit 22 and the picture encoding unit 24. .
- the Non-VCL encoding unit 22 corresponds to the inverse process of the Non-VCL decoding unit 12 in the hierarchical video decoding device 1, and the encoding parameters and input images of each non-VCL input from the encoding parameter determination unit 26 Based on the above, non-VCL is set for the parameter set (VPS, SPS, and SPS) used for encoding the input image and others, and each parameter set and other non-VCL are stored in the non-VCL NAL unit. The data is supplied to the NAL multiplexer 21.
- the non-VCL encoded by the Non-VCL encoding unit 22 includes layer set information, output layer set information, and PTL information described in the description of the non-VCL decoding unit 12 included in the hierarchical video decoding device 1.
- the non-VCL encoding unit 22 includes parameter set encoding means (not shown).
- the parameter set encoding means includes layer set information encoding means for encoding (generating) layer set information, output layer set information encoding means for encoding (generating) output layer set information, PTL information, not shown PTL information encoding means for encoding, DPB information encoding means for encoding DPB information, sub bit stream characteristic information encoding means for encoding sub bit stream characteristic information, and a scalable identifier for encoding a scalable identifier of each layer An identifier encoding unit is further included.
- each encoding unit and encoding unit correspond to the inverse processing of the corresponding decoding unit and decoding unit, and the decoding unit and decoding unit interpret “decoding” as “encoding”. To do.
- the Non-VCL encoding unit 22 also outputs a NAL unit type, a layer identifier, and a temporal identifier corresponding to the non-VCL when supplying non-VCL encoded data to the NAL multiplexing unit 21. To do.
- the parameter set generated by the Non-VCL encoding unit 22 includes an identifier for identifying the parameter set, and a parameter set (active parameter set) referred to by the parameter set referred to for decoding a picture of each layer. )
- an active parameter set identifier for identifying the VPS is included.
- an SPS identifier for identifying the SPS
- an active VPS identifier for specifying a VPS referred to by the SPS or other syntax
- a PPS identifier for identifying the PPS and an active SPS identifier (pps_seq_parameter_set_id) for identifying an SPS to which the PPS or other syntax refers are included.
- the picture coding unit 24 is recorded in the input image PIN # T of each input layer, the Non-VCL (particularly, parameter set) supplied from the coding parameter determination unit 26, and the decoded picture management unit 15. Based on the reference picture, a part of the input image of each layer corresponding to the slice constituting the picture is encoded to generate encoded data of the part, and NAL multiplexing is performed as data stored in the VCL NAL unit. Supply to unit 21. Detailed description of the picture encoding unit 24 will be described later. Note that when the picture coding unit 24 supplies the VCL coded data to the NAL multiplexing unit 21, the picture coding unit 24 also assigns and outputs the NAL unit type, the layer identifier, and the temporal identifier corresponding to the VCL.
- FIG. 26 is a functional block diagram showing a schematic configuration of the picture encoding unit 24.
- the picture encoding unit 24 includes a slice header encoding unit 241 and a CTU encoding unit 242.
- the slice header encoding unit 241 generates a slice header used for encoding the input image of each layer input in units of slices based on the input active parameter set.
- the generated slice header is output as part of the slice encoded data and is supplied to the CTU encoding unit 242 together with the input image.
- the slice header generated by the slice header encoding unit 241 includes an active PPS identifier that designates a picture parameter set PPS (active PPS) to be referred to in order to decode a picture of each layer.
- the CTU encoding unit 242 encodes the input image (target slice portion) in units of CTU based on the input active parameter set and slice header, and generates slice data and a decoded image (decoded picture) related to the target slice. And output. More specifically, the CTU encoding unit 242 divides the input image of the target slice in units of CTBs having a CTB size included in the parameter set, and encodes an image corresponding to each CTB as one CTU. . CTU encoding is performed by the prediction residual encoding unit 2421, the prediction image encoding unit 2422, and the CTU decoded image generation unit 2423.
- the prediction residual encoding unit 2421 converts the quantization residual information (TT information) obtained by transforming and quantizing the difference image between the input image and the prediction image to be input to the slice data included in the slice encoded data. Output as part. Further, the prediction residual is restored by applying inverse transform / inverse quantization to the quantized residual information, and the restored prediction residual is output to the CTU decoded image generation unit 2423.
- TT information quantization residual information
- the prediction image encoding unit 2422 generates a prediction image based on the prediction method and the prediction parameter of the target CTU included in the target slice, which is determined by the encoding parameter determination unit 26, and the prediction residual encoding unit 2421.
- the data is output to the CTU decoded image generation unit 2423.
- the prediction scheme and prediction parameter information are variable-length encoded as prediction information (PT information) and output as a part of slice data included in the slice encoded data.
- PT information prediction information
- a corresponding reference picture is read from the decoded picture management unit 15.
- the CTU decoded image generation unit 2423 is the same component as the CTU decoded image system generation unit 1423 included in the hierarchical video decoding device 1, description thereof is omitted. Note that the decoded image of the target CTU is supplied to the decoded picture management unit 15 and recorded in the internal DPB.
- FIG. 27 is a flowchart showing an encoding process in units of slices constituting a picture of the target layer i in the picture encoding unit 24.
- the first slice flag (first_slice_segment_in_pic_flag) flag (SYNSH01 in FIG. 17 (d)) of the encoding target slice is encoded. That is, if the input image divided into slice units (hereinafter referred to as encoding target slice) is the first slice in the encoding order (decoding order) (hereinafter referred to as processing order) in the picture, the first slice flag (first_slice_segment_in_pic_flag) is set. 1. If the current slice is not the first slice, the first slice flag is 0. When the head slice flag is 1, the head CTU address of the encoding target slice is set to 0. Further, the counter numCtu for the number of processed CTUs in the picture is set to zero. When the head slice flag is 0, the head CTU address of the encoding target slice is set based on a slice address encoded in SE106 described later.
- SE102 An active PPS identifier (slice_pic_paramter_set_id) (SYNSH02 in FIG. 17 (d)) that specifies an active PPS to be referred to when encoding the current slice is encoded.
- the active parameter set determined by the encoding parameter determination unit 26 is fetched. That is, the PPS having the same PPS identifier (pps_pic_parameter_set_id) as the active PPS identifier (slice_pic_parameter_set_id) referred to by the encoding target slice is set as the active PPS, and the encoding parameter determination unit 26 fetches (reads) the encoding parameter of the active PPS. ).
- the SPS having the same SPS identifier (sps_seq_parameter_set_id) as the active SPS identifier (pps_seq_parameter_set_id) in the active PPS is set as the active SPS, and the encoding parameter of the active SPS is fetched from the encoding parameter determination unit 26.
- the VPS having the same VPS identifier (vps_video_parameter_set_id) as the active VPS identifier (sps_video_parameter_set_id) in the active SPS is set as the active VPS, and the encoding parameter of the active VPS is fetched from the encoding parameter determination unit 26.
- the picture encoding unit 24 refers to the layer set information, output layer set information, and PTL information included in the active VPS, the layer identifier of the active parameter set (VPS, SPS, PPS), the layer identifier of the target layer, and the like. However, it may be verified whether the target set satisfies the conformance condition.
- the conformance condition has already been described in the hierarchical video decoding device 1, and therefore will be omitted. If the conformance condition is satisfied, it is ensured that the hierarchical encoded data DATA of the target set to be generated can be decoded by the hierarchical video decoding device 1 corresponding to the hierarchical image encoding device 2.
- SE105 It is determined based on the head slice flag whether or not the coding target slice is the head slice in the processing order in the picture. If the first slice flag is 0 (Yes in SE105), the process proceeds to step SE106. In other cases (No in SE105), the process of step SE106 is skipped. When the head slice flag is 1, the slice address of the encoding target slice is 0.
- the slice address (slice_segment_address) of the encoding target slice (SYNSH03 in FIG. 17 (d)) is encoded.
- the slice address of the encoding target slice (the leading CUT address of the encoding target slice) can be set based on, for example, the counter numCtu of the number of processed CTUs in the picture.
- the slice address slice_segment_adress numCtu. That is, the leading CTU address of the encoding target slice is also numCtu.
- the method for determining the slice address is not limited to this, and can be changed within a practicable range.
- the CTU encoding unit 242 encodes an input image (encoding target slice) in units of CTUs based on the input active parameter set and slice header, and as a part of slice data of the encoding target slice
- the encoded data of the CTU information (SYNSD01 in FIG. 17 (d)) is output.
- the CTU encoding unit 242 generates and outputs a CTU decoded image of a region corresponding to each CTU.
- a slice end flag end_of_slice_segment_flag
- the slice end flag is set to 1, otherwise it is set to 0 and encoding is performed. Further, after encoding each CTU, 1 is added to the value of the number of processed CTUs numCtu (numCtu ++).
- SE10B It is determined based on the slice end flag whether or not the CTU is the end of the encoding target slice.
- the slice end flag is 1 (Yes in SE10B)
- the process proceeds to Step SE10C.
- the process proceeds to step SE10A in order to encode the subsequent CTU.
- numCtu is equal to PicSizeInCtbsY (Yes in SE10C)
- the encoding process in units of slices constituting the encoding target picture is terminated.
- the process proceeds to step SE101 in order to continue the encoding process in units of slices constituting the encoding target picture.
- the hierarchical video encoding device 2 includes a base layer as a layer set in the hierarchical video decoding device 1 (and its modifications (hierarchical video decoding device 1A, hierarchical video decoding device 1B)).
- the hierarchical encoded data DATA of the target set is generated.
- the output layer set decoded from the encoded data is guaranteed to always decode the base layer, and thus is a subset of the layer set A generated by the bitstream extraction process from the encoded data including a certain layer set A.
- the hierarchical video encoding device 2 always decodes at least one primary picture in the output layer set decoded from the encoded data in the hierarchical video decoding device 1 (and its variations).
- the target set hierarchically encoded data DATA is generated so as to satisfy the above-mentioned conformance condition CX (CX is one of CX1 and CX2). Therefore, in the hierarchical moving picture decoding apparatus 1, it is guaranteed that one or more primary picture layers are always decoded in the output layer set decoded from the encoded data. That is, it is possible to prevent a case where there is no layer (primary picture layer) to be decoded in the target decoding layer ID list derived in the output control unit 16b and the output control 16c.
- the hierarchical moving picture encoding device 2 is a hierarchical moving picture decoding device provided with the output control unit 16b or the output control 16c, so that the decoding process of the auxiliary picture layer can be omitted.
- the hierarchical set data DATA of the target set is generated so as to satisfy the conformance condition CY (CY is any one of CY1 to CY3). Therefore, in the hierarchical video decoding device 1 including the output control unit 16b or the output control unit 16c, it is ensured that the decoding process of the auxiliary picture layer can be omitted in the output layer set decoded from the encoded data. be able to.
- the above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images.
- the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.
- FIG. 28 is a block diagram illustrating a configuration of a transmission device PROD_A in which the hierarchical video encoding device 2 is mounted.
- the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, and with the encoded data obtained by the encoding unit PROD_A1.
- a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided.
- the hierarchical moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.
- the transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1.
- An image processing unit A7 that generates or processes an image may be further provided. In FIG. 28A, a configuration in which all of these are provided in the transmission device PROD_A is illustrated, but a part may be omitted.
- the recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.
- FIG. 28 is a block diagram illustrating a configuration of the receiving device PROD_B in which the hierarchical video decoding device 1 is mounted.
- the receiving device PROD_B includes a receiving unit PROD_B1 that receives the modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator.
- a decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2.
- the above-described hierarchical video decoding device 1 is used as the decoding unit PROD_B3.
- the receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3.
- PROD_B6 may be further provided.
- FIG. 28B a configuration in which all of these are provided in the receiving device PROD_B is illustrated, but a part may be omitted.
- the recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
- the transmission medium for transmitting the modulation signal may be wireless or wired.
- the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.
- a terrestrial digital broadcast broadcasting station (broadcasting equipment or the like) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting.
- a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.
- a server workstation etc.
- Client television receiver, personal computer, smart phone etc.
- VOD Video On Demand
- video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication.
- PROD_A / reception device PROD_B usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN.
- the personal computer includes a desktop PC, a laptop PC, and a tablet PC.
- the smartphone also includes a multi-function mobile phone terminal.
- the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.
- FIG. 29A is a block diagram illustrating a configuration of a recording apparatus PROD_C in which the above-described hierarchical video encoding apparatus 2 is mounted.
- the recording apparatus PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M.
- the hierarchical moving image encoding device 2 described above is used as the encoding unit PROD_C1.
- the recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) For example, it may be loaded into a drive device (not shown) built in the recording device PROD_C.
- the recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images.
- the unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided.
- FIG. 29A illustrates a configuration in which the recording apparatus PROD_C includes all of these, but a part of the configuration may be omitted.
- the receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.
- Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images).
- a camcorder in this case, the camera PROD_C3 is a main source of moving images
- a personal computer in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images
- a smartphone in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
- the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
- FIG. 29 is a block showing a configuration of a playback device PROD_D in which the above-described hierarchical video decoding device 1 is mounted.
- the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written on the recording medium PROD_M and a coded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained.
- the hierarchical moving image decoding apparatus 1 described above is used as the decoding unit PROD_D2.
- the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.
- the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2.
- PROD_D5 may be further provided.
- FIG. 29B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but some of the configurations may be omitted.
- the transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.
- Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images).
- a television receiver in this case, the display PROD_D3 is a main supply destination of moving images
- a digital signage also referred to as an electronic signboard or an electronic bulletin board
- the display PROD_D3 or the transmission unit PROD_D5 is a main supply of moving images.
- Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image)
- a smartphone which is a main image supply destination
- a smartphone in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination
- the like are also examples of such a playback device PROD_D.
- each block of the hierarchical video decoding device 1 and the hierarchical video encoding device 2 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central It may be realized by software using a Processing Unit).
- IC chip integrated circuit
- CPU Central It may be realized by software using a Processing Unit
- each of the devices includes a CPU that executes instructions of a control program that realizes each function, a ROM (Read Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and A storage device (recording medium) such as a memory for storing various data is provided.
- An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each of the above devices, which is software that realizes the above-described functions, is recorded in a computer-readable manner This can also be achieved by supplying each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU (Micro Processing Unit)).
- a program code execution format program, intermediate code program, source program
- Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, CD-ROMs (Compact Disc-Read-Only Memory) / MO (Magneto-Optical) / Discs including optical discs such as MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable), cards such as IC cards (including memory cards) / optical cards, mask ROM / EPROM (Erasable) Programmable Read-only Memory / EEPROM (registered trademark) (ElectricallyErasable Programmable Read-only Memory) / Semiconductor memories such as flash ROM, or logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) Etc. can be used.
- tapes such as magnetic tapes and cassette tapes
- magnetic disks such as floppy (registered trademark) disks / hard disks
- each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
- the communication network is not particularly limited as long as it can transmit the program code.
- the Internet intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Area Antenna Television) communication network, Virtual Private Network (Virtual Private Network), A telephone line network, a mobile communication network, a satellite communication network, etc. can be used.
- the transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type.
- IEEE Institute of Electrical and Electronic Engineers 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. wired such as IrDA (Infrared Data Association) or remote control
- IrDA Infrared Data Association
- remote control such as Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance) (registered trademark), mobile phone network, satellite line, terrestrial digital network, etc. It can also be used wirelessly.
- the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.
- the present invention includes at least the image decoding device shown in aspects 1 to 23 and the image encoding device shown in aspects 24 to 33.
- An image decoding device is an image decoding device that decodes hierarchical image encoded data, Layer set information decoding means for decoding the layer set; Output layer set information decoding means for decoding the layer set identifier and output layer flag of the output layer set; Scalable identifier decoding means for decoding the scalable identifier; Output layer set selection means for selecting one of the output layer sets as a target output layer set; An output layer ID list derivation means for deriving an output layer ID list indicating the configuration of the target output layer based on the layer set corresponding to the output layer set and the output layer flag; Decoding layer ID list derivation means for deriving a decoding layer ID list indicating a configuration of a layer to be decoded based on the layer set corresponding to the layer set and the scalable identifier; A picture decoding means for generating a decoded picture of each layer included in the derived decoded layer ID list is provided.
- the image decoding apparatus is the image decoding apparatus according to aspect 1, in which the decoding layer ID list derivation means decodes a layer whose scalable identifier indicates a primary picture layer in the layers included in the output layer set. It is derived as a layer ID list.
- the image decoding apparatus is the image decoding apparatus according to aspects 1 and 2, wherein the decoding layer ID list deriving unit determines whether each of the layers included in the output layer set is a primary picture layer. Judgment, If the layer is a primary picture layer, add it to the element of the decoding layer ID list, When the layer is an auxiliary picture layer, it is not added as an element of the decoding layer ID list.
- An image decoding device is an image decoding device that decodes hierarchical image encoded data, Layer set information decoding means for decoding the layer set; Output layer set information decoding means for decoding the layer set identifier and output layer flag of the output layer set; Scalable identifier decoding means for decoding the scalable identifier; Output layer set selection means for selecting one of the output layer sets as a target output layer set; An output layer ID list derivation means for deriving an output layer ID list indicating the configuration of the target output layer based on the layer set corresponding to the output layer set and the output layer flag; Decoding layer ID list derivation means for deriving a decoding layer ID list indicating a configuration of a layer to be decoded based on the layer set corresponding to the layer set, the output layer flag, and the scalable identifier; A picture decoding means for generating a decoded picture of each layer included in the derived decoded layer ID list is provided.
- the image decoding apparatus is the image decoding apparatus according to aspect 4, wherein the decoding layer ID list derivation means further includes a layer in which the scalable identifier indicates a primary picture layer in a layer included in the output layer set, or The scalable identifier indicates an auxiliary picture layer and the layer whose output layer flag is 1 is derived as a decoded layer ID list.
- the image decoding apparatus is the image decoding apparatus according to aspects 4 to 5, wherein the decoding layer ID list deriving unit is further configured to use a primary picture layer for each layer included in the selected output layer set. Determine whether there is an auxiliary picture layer whose output layer flag is 1, If the layer is a primary picture layer or an auxiliary picture layer whose output layer flag is 1, add it to the element of the decoding layer ID list, When the layer is an auxiliary picture layer whose output layer flag is 0, it is not added to the element of the decoding layer ID list.
- the image decoding apparatus is the image decoding apparatus according to Aspects 1 to 6, wherein the decoding layer ID list derivation means is all included in the layer set corresponding to the output layer set when the decoding layer ID list derivation means is a conformance test.
- a layer is derived as the decoded layer ID list.
- the image decoding apparatus is characterized in that in the above aspects 1 to 7, the output layer set is composed of at least one or more primary picture layers.
- the image decoding apparatus is characterized in that, in the above aspects 1 to 8, when the output layer set includes an auxiliary picture layer, the output layer flag of the auxiliary picture layer is 0. To do.
- An image decoding device is an image decoding device that decodes hierarchical image encoded data, Layer set information decoding means for decoding the layer set; Output layer set information decoding means for decoding the layer set identifier and output layer flag of the output layer set; Inter-layer dependency information decoding means for decoding inter-layer dependency information; Output layer set selection means for selecting one of the output layer sets as a target output layer set; Output layer ID list derivation means for deriving an output layer ID list indicating the configuration of the target output layer based on the layer set corresponding to the output layer set and the output layer set flag; Decoding layer ID list deriving means for deriving a decoding layer ID list indicating a configuration of a layer to be decoded based on the layer set corresponding to the layer set, the output layer flag, and the inter-layer dependency information; A picture decoding means for generating a decoded picture of each layer included in the derived decoded layer ID list is provided.
- the image decoding apparatus is the image decoding apparatus according to aspect 10, wherein the decoding layer ID list derivation unit further decodes an output layer whose output layer flag is 1 and a dependency layer of the output layer as a decoding layer ID. It is derived as a list.
- the image decoding apparatus is characterized in that, in the above aspect 11, the decoding layer ID list derivation means further includes a layer having a layer identifier of 0 in the decoding layer ID list. To do.
- the decoding layer ID list derivation means has an output layer flag of 1 for each layer included in the output layer set. Or whether it is a dependency layer of the output layer, If the layer is an output layer or an output layer dependent layer, add it to the element of the decoding layer ID list, When the layer is a non-output layer and an output layer independent layer, the layer is not added to the element of the decoding layer ID list.
- the decoding layer ID list derivation means is an output layer for each layer included in the selected output layer set. Or whether it is a dependent layer of the output layer or the layer identifier is 0, If the layer is an output layer, or a dependent layer of the output layer, or the layer identifier is 0, add it to the element of the decoding layer ID list, When the layer is a non-output layer and an output layer independent layer, the layer is not added to the element of the decoding layer ID list.
- the image decoding apparatus is the image decoding apparatus according to aspect 10, wherein the output layer set information decoding means further includes: PTL / DPB indicating presence / absence of DPB information of the output layer set or a PTL designation identifier of the output layer set Decrypt the information presence flag When the PTL / DPB information presence / absence flag is true, the PTL designation identifier is decoded from the encoded data, When the PTL / DPB information presence / absence flag is false, the decoding of the PTL designation identifier is omitted, and it is estimated that the layer set identifier of the output layer set is equal to the PTL designation identifier of the corresponding basic output layer set. To do.
- the image decoding apparatus is the image decoding apparatus according to aspect 10, further comprising DPB information decoding means for decoding the DPB information of the output layer set,
- the output layer set information decoding means decodes the DPB information of the output layer set, or the PTL / DPB information presence flag indicating the presence or absence of the PTL designation identifier of the output layer set,
- the DPB information decoding means decodes the DPB information of the output layer set from the encoded data
- the DPB information decoding means does not decode the DPB information of the output layer set, and the DPB information of the basic output layer set corresponding to the layer set identifier of the output layer set It is estimated to be equal.
- the image decoding apparatus is the image decoding apparatus according to aspect 15 or 16, wherein the output layer set information decoding means does not decode the PTL / DPB information presence / absence flag of the basic output layer set.
- the DPB information presence / absence flag is estimated to be 1.
- the image decoding apparatus is the image decoding apparatus according to aspect 10, wherein the output layer set information decoding means further includes the PTL designation identifier from encoded data when the output layer set is a basic output layer set.
- the output layer set is an additional output layer set, it is estimated that the layer set identifier of the output layer set is equal to the PTL designation identifier of the corresponding basic output layer set.
- the image decoding apparatus in the tenth aspect, further comprises DPB information decoding means for decoding the DPB information of the output layer set,
- the DPB information decoding means decodes the DPB information of the output layer set from encoded data, If the output layer set is an additional output layer set, the DPB information of the output layer set is not decoded, and is estimated to be equal to the DPB information of the basic output layer set corresponding to the layer set identifier of the output layer set.
- the image decoding apparatus is the image decoding apparatus according to aspect 10, further comprising sub-bitstream characteristic information decoding means for decoding the sub-bitstream characteristic information; Provided with encoded data extraction means for extracting a bit stream of a target set by performing a bit stream extraction process based on sub-bit stream characteristic information corresponding to the selected output layer set from the input encoded data It is characterized by.
- the image decoding apparatus is the image decoding apparatus according to aspect 20, wherein the encoded data extraction means includes at least a non-output layer and an output layer independent layer of the selected output layer set.
- the NAL unit having a layer identifier is discarded.
- the image decoding apparatus is the image decoding apparatus according to aspect 20, wherein the encoded data extraction means includes at least a NAL unit having a layer identifier of an auxiliary picture layer in the selected output layer set. It is characterized by discarding.
- the image decoding apparatus is the image decoding apparatus according to aspect 20, wherein the encoded data extraction means includes at least a layer identifier of an auxiliary picture layer that is a non-output layer of the selected output layer set. It is characterized by discarding a NAL unit having.
- An image encoding device is an image encoding device that generates hierarchical encoded data, and Layer set information encoding means for encoding the layer set; Inter-layer dependency information encoding means for encoding inter-layer dependency information; Output layer set information encoding means for encoding the layer set identifier and output layer flag of the output layer set; Sub-bitstream characteristic information encoding means for encoding sub-bitstream characteristic information corresponding to the output layer set; DPB information encoding means for encoding DPB information corresponding to the output layer set; Picture encoding means for encoding a picture of each layer included in the layer set corresponding to the output layer set.
- the sub bitstream characteristic information includes at least a layer of a non-output layer and an independent layer of an output layer from at least a bit stream of an output layer set.
- a bit stream extraction mode for designating a bit stream extraction process for discarding the NAL unit of the identifier is included.
- the image encoding apparatus is the image encoding apparatus according to aspect 24 or 25, wherein the output layer set information encoding means further includes DPB information of the output layer set or a PTL designation identifier of the output layer set.
- DPB information of the output layer set or a PTL designation identifier of the output layer set.
- a PTL / DPB information presence / absence flag indicating presence / absence is encoded.
- the image encoding apparatus is the image encoding apparatus according to aspect 26, wherein the output layer set information encoding means encodes the PTL designation identifier when the PTL / DPB information presence / absence flag is true, When the PTL / DPB information presence / absence flag is false, the encoding of the PTL designation identifier is omitted, and it is estimated that the layer set identifier of the output layer set is equal to the PTL designation identifier of the corresponding basic output layer set.
- the image encoding device is the image encoding apparatus according to aspect 26, wherein the DPB information encoding means encodes the DPB information of the output layer set when the PTL / DPB information presence / absence flag is true. , When the PTL / DPB information presence / absence flag is false, the DPB information encoding means omits the encoding of the DPB information of the output layer set and sets the basic output layer set corresponding to the layer set identifier of the output layer set. It is estimated to be equal to DPB information.
- the image encoding apparatus according to aspect 29 of the present invention is the image encoding apparatus according to aspect 25 or 26 described above.
- the output layer set information encoding means estimates the PTL / DPB information presence / absence flag as 1 without encoding the PTL / DPB information presence / absence flag of the basic output layer set.
- the output layer set information encoding unit encodes the PTL designation identifier when the output layer set is a basic output layer set, and When the output layer set is an additional output layer set, it is estimated that the layer set identifier of the output layer set is equal to the PTL designation identifier of the corresponding basic output layer set.
- the DPB information encoding unit encodes the DPB information of the output layer set, If the output layer set is an additional output layer set, the DPB information of the output layer set is not encoded, and is estimated to be equal to the DPB information of the basic output layer set corresponding to the layer set identifier of the output layer set. It is characterized by.
- the sub-bitstream characteristic information includes, as the sub-bitstream characteristic information, bitstream extraction that discards the NAL unit of the layer identifier of the auxiliary picture layer from the bitstream of the output layer set A bit stream extraction mode for designating processing is included.
- the image encoding device is the image encoding apparatus according to aspect 24, wherein the sub-bitstream characteristic information includes a NAL unit of a layer identifier of an auxiliary picture layer that is a non-output layer from a bitstream of an output layer set.
- a bit stream extraction mode for designating a bit stream extraction process to be discarded is included.
- the present invention relates to a hierarchical video decoding device that decodes encoded data in which image data is hierarchically encoded, and a hierarchical video encoding device that generates encoded data in which image data is hierarchically encoded. It can be suitably applied to. Further, the present invention can be suitably applied to the data structure of hierarchically encoded data that is generated by a hierarchical video encoding device and referenced by the hierarchical video decoding device.
- Prediction residual restoration unit 1422 ... Prediction image generation unit 1423 ... CTU decoded image generation unit 15 ... Decoded picture management unit 16 ... Output control unit (output layer set selection means, target) Output layer ID deriving means, target decoding layer ID list deriving means) 17: Bitstream extraction means (encoded data extraction means) 20 ... Target set picture encoding unit 21 ... NAL multiplexing unit (NAL unit encoding means) 22...
- Non-VCL encoding section (parameter set encoding means, layer set information encoding means, output layer set information encoding means, PTL information encoding means, DPB information encoding means, sub-bitstream characteristic information encoding means Inter-layer dependency information encoding means, scalable identifier encoding means) 24: Picture encoding unit (VCL encoding means) 26 ... Coding parameter determination unit 241 ... Slice header coding unit 242 ... CTU coding unit 2421 ... Prediction residual coding unit 2422 ... Predictive image coding unit 2423 ... CTU decoded image generation unit
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
レイヤセット単位で各レイヤが前記レイヤセットに含まれるか否かを示す第1のフラグを復号する第1のフラグ復号手段と、
上記第1のフラグに基づいて、前記レイヤセットのレイヤIDリストを導出するレイヤセット情報復号手段と、
出力レイヤセット単位で、
a)レイヤセット識別子、と
b)前記出力レイヤセットに含まれる各レイヤが出力レイヤであるか否かを示す出力レイヤフラグ
を含む出力レイヤセット情報を復号する出力レイヤセット情報復号手段と、
第1のレイヤが、第2のレイヤの参照レイヤであるか否かを示す依存フラグを導出する依存フラグ導出手段と、
上記出力レイヤセットに対応するレイヤセットの構成を示すレイヤIDリスト、上記出力レイヤセットの出力レイヤフラグ、及び上記依存フラグに基づいて、上記出力レイヤセットにおいて、復号するレイヤを示す復号レイヤIDリストを導出する復号レイヤIDリスト導出手段と、
上記導出された復号レイヤIDリストに含まれる各レイヤのピクチャを復号するピクチャ復号手段を備えることを特徴とする。
レイヤセット単位で各レイヤが前記レイヤセットに含まれるか否かを示す第1のフラグを復号する第1のフラグ復号ステップと、
上記第1のフラグに基づいて、前記レイヤセットのレイヤIDリストを導出するレイヤセット情報復号ステップと、
出力レイヤセット単位で、
a)レイヤセット識別子、と
b)前記出力レイヤセットに含まれる各レイヤが出力レイヤであるか否かを示す出力レイヤフラグ
を含む出力レイヤセット情報を復号する出力レイヤセット情報復号ステップと、
第1のレイヤが、第2のレイヤの参照レイヤであるか否かを示す依存フラグを導出する依存フラグ導出ステップと、
上記出力レイヤセットに対応するレイヤセットの構成を示すレイヤIDリスト、上記出力レイヤセットの出力レイヤフラグ、及び上記依存フラグに基づいて、上記出力レイヤセットにおいて、復号するレイヤを示す復号レイヤIDリストを導出する復号レイヤIDリスト導出ステップと、
上記導出された復号レイヤIDリストに含まれる各レイヤのピクチャを復号するピクチャ復号ステップを備えることを特徴とする。
本実施の形態に係る階層動画像復号装置(画像復号装置)1は、階層動画像符号化装置(画像符号化装置)2によって階層符号化された符号化データを復号する。階層符号化とは、動画像を低品質のものから高品質のものにかけて階層的に符号化する符号化方式のことである。階層符号化は、例えば、SVCやSHVCにおいて標準化されている。なお、ここでいう動画像の品質とは、主観的および客観的な動画像の見栄えに影響する要素のことを広く意味する。動画像の品質には、例えば、“解像度”、“フレームレート”、“画質”、および、“画素の表現精度”が含まれる。よって、以下、動画像の品質が異なるといえば、例示的には、“解像度”等が異なることを指すが、これに限られない。例えば、異なる量子化ステップで量子化された動画像の場合(すなわち、異なる符号化雑音により符号化された動画像の場合)も互いに動画像の品質が異なるといえる。
ここで、図2を用いて、階層符号化データの符号化および復号について説明すると次のとおりである。図2は、動画像を、下位階層L3、中位階層L2、および上位階層L1の3階層により階層的に符号化/復号する場合について模式的に表わす図である。つまり、図2(a)および(b)に示す例では、3階層のうち、上位階層L1が最上位層となり、下位階層L3が最下位層となる。
(SA01)入力ビットストリームをCPBへ蓄積する;
(SA02)CPBに蓄積されたAUを瞬時復号処理する;
(SA03)瞬時復号処理して得られた復号済ピクチャをDPBへ保存する;
(SA04)DPBに保存された復号ピクチャをクロップして出力する;
以下、各階層の符号化データを生成する符号化方式として、HEVCおよびその拡張方式を用いる場合について例示する。しかしながら、これに限られず、各階層の符号化データを、MPEG-2や、H.264/AVCなどの符号化方式により生成してもよい。
本実施形態に係る画像符号化装置2および画像復号装置1の詳細な説明に先立って、画像符号化装置2によって生成され、画像復号装置1によって復号される階層符号化データDATAのデータ構造について説明する。
図4は、階層符号化データDATAにおけるデータの階層構造を示す図である。階層符号化データDATAは、NAL(Network Abstraction Layer)ユニットと呼ばれる単位で符号化される。
特定の分類ルールにより集約されたNALユニットの集合のことをアクセスユニットと呼ぶ。レイヤ数が1の場合には、アクセスユニットは1ピクチャを構成するNALユニットの集合である。レイヤ数が1より大きい場合には、アクセスユニットは同じ時刻(同一出力タイミング)の複数のレイヤのピクチャを構成するNALユニットの集合である。なお、アクセスユニットの区切りを示すために、符号化データはアクセスユニットデリミタ(AUD:Access unit delimiter)と呼ばれるNALユニットを含んでも良い。アクセスユニットデリミタは、符号化データ中にあるアクセスユニットを構成するNALユニットの集合と、別のアクセスユニットを構成するNALユニットの集合の間に含まれる。
シーケンスレイヤでは、処理対象のシーケンスSEQ(以下、対象シーケンスとも称する)を復号するために画像復号装置1が参照するデータの集合が規定されている。シーケンスSEQは、図8の(a)に示すように、ビデオパラメータセット(Video Parameter Set)シーケンスパラメータセットSPS(Sequence Parameter Set)、ピクチャパラメータセットPPS(Picture Parameter Set)、ピクチャPICT、及び、付加拡張情報SEI(Supplemental Enhancement Information)を含んでいる。ここで#の後に示される値はレイヤIDを示す。図8では、#0と#1すなわちレイヤIDが0とレイヤIDが1の符号化データが存在する例を示すが、レイヤの種類およびレイヤの数はこれに限定されない。
図11は、ビデオパラメータセットVPSのシンタックス表の一例であり、図12は、ビデオパラメータセットVPSの拡張データのシンタックス表の一例である。ビデオパラメータセットVPSでは、1以上のレイヤから構成される符号化データを復号するために画像復号装置1が参照する符号化パラメータの集合が規定されている。例えば、後述のシーケンスパラメータセットや他のシンタックス要素が参照するVPSを識別するために用いるVPS識別子(video_parameter_set_id)(図11上のSYNVPS01)や、符号化データに含まれるレイヤ数(vps_max_layers_minus1) (図11上のSYNVPS02)、レイヤに含まれるサブレイヤ数(vps_sub_layers_minus1) (図11上のSYNVPS03)、符号化データ中で表現される1以上のレイヤからなるレイヤの集合を規定するレイヤセットの数(vps_num_layer_sets_minus1) (図11上のSYNVPS06)、レイヤセットを構成するレイヤの集合を規定するレイヤセット情報(レイヤセット、layer_id_included_flag[i][j]) (図11上のSYNVPS07)や、レイヤ間の依存関係(直接依存フラグdirect_dependency_flag[i][j])(図12上のSYNVPS0C)、出力レイヤセットを構成する出力レイヤの集合や、PTL情報等を規定した出力レイヤセット情報 (デフォルト出力レイヤ識別子default_target_output_layer_idc、関連するレイヤセット識別子output_layer_set_idx_minus1、出力レイヤフラグoutput_layer_flag[i][j]、代替出力レイヤフラグalt_output_layer_flag[i]、PTL指定識別子profile_level_tier_idx[i]など)(図12上のSYNVPS0G~SYNVPS0M)などが規定されている。VPSは符号化データ内に複数存在してもよい。その場合、対象シーケンス毎に復号に用いられるVPSが複数の候補から選択される。
図17(a)は、シーケンスパラメータセットSPSのシンタックス表の一例である。シーケンスパラメータセットSPSでは、対象シーケンスを復号するために画像復号装置1が参照する符号化パラメータの集合が規定されている。例えば、対象SPSが参照するアクティブVPSを表わすアクティブVPS識別子(sps_video_parameter_set_id)(図17(a)上のSYNSPS01)、後述のピクチャパラメータセットや他のシンタックス要素が参照するSPSを識別するために用いるSPS識別子(sps_seq_parameter_set_id)(図17(a)上のSYNSPS02)や、ピクチャの幅や高さが規定される。SPSは符号化データ内に複数存在してもよく、その場合には、対象シーケンス毎に復号に用いられるSPSが複数の候補から選択される。
図17(b)は、ピクチャパラメータセットPPSのシンタックス表の一例である。ピクチャパラメータセットPPSでは、対象シーケンス内の各ピクチャを復号するために画像復号装置1が参照する符号化パラメータの集合が規定されている。例えば、対象PPSが参照するアクティブSPSを表わすアクティブSPS識別子(pps_seq_parameter_set_id)(図17(b)のSYNPPS01)、後述のスライスヘッダや他のシンタックス要素が参照するPPSを識別するために用いるPPS識別子(pps_pic_parameter_set_id)(図17(b)のSYNPPS02)や、ピクチャの復号に用いられる量子化幅の基準値(pic_init_qp_minus26)や重み付き予測の適用を示すフラグ(weighted_pred_flag)、スケーリングリスト(量子化マトリックス)が含まれる。なお、PPSは複数存在してもよく、その場合、対象シーケンス内の各ピクチャから複数のPPSの何れかを選択する。
ピクチャレイヤでは、処理対象のピクチャPICT(以下、対象ピクチャとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。ピクチャPICTは、図8(b)に示すように、スライスS0~SNS-1を含んでいる(NSはピクチャPICTに含まれるスライスの総数)。なお、以下、スライスS0~SNS-1のそれぞれを区別する必要が無い場合、符号の添字を省略して記述することがある。また、以下に説明する階層符号化データDATAに含まれるデータであって、添字を付している他のデータについても同様である。
スライスレイヤでは、処理対象のスライスS(対象スライスとも称する、スライスセグメント)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。スライスSは、図8(c)に示すように、スライスヘッダSH、および、スライスデータSDATAを含んでいる。
スライスデータレイヤでは、処理対象のスライスデータSDATAを復号するために階層動画像復号装置1が参照するデータの集合が規定されている。スライスデータSDATAは、図8(d)に示すように、符号化ツリーブロック(CTB:Coded Tree Block)を含んでいる。CTBは、スライスを構成する固定サイズ(例えば64×64)のブロックであり、最大符号化単位(LCU:Largest Cording Unit)と呼ぶこともある。
符号化ツリーレイヤは、図8(e)に示すように、処理対象の符号化ツリーブロックを復号するために階層動画像復号装置1が参照するデータの集合が規定されている。符号化ツリーユニットは、再帰的な4分木分割により分割される。再帰的な4分木分割により得られる木構造のノードのことを符号化ツリー(coding tree)と称する。4分木の中間ノードは、符号化ツリーユニット(CTU:Coded Tree Unit)であり、符号化ツリーブロック自身も最上位のCTUとして規定される。CTUは、分割フラグ(split_flag)を含み、split_flagが1の場合には、4つの符号化ツリーユニットCTUに分割される。split_flagが0の場合には、符号化ツリーユニットCTUは4つの符号化ユニット(CU:Coded Unit)に分割される。符号化ユニットCUは符号化ツリーレイヤの末端ノードであり、このレイヤではこれ以上分割されない。符号化ユニットCUは、符号化処理の基本的な単位となる。
符号化ユニットレイヤは、図8(f)に示すように、処理対象の符号化ユニットを復号するために階層動画像復号装置1が参照するデータの集合が規定されている。具体的には、符号化ユニットCU(coding unit)は、CUヘッダCUH、予測ツリー、変換ツリーから構成される。CUヘッダCUHでは、符号化ユニットが、イントラ予測を用いるユニットであるか、インター予測を用いるユニットであるかなどが規定される。符号化ユニットは、予測ツリー(prediction tree;PT)および変換ツリー(transform tree;TT)のルートとなる。なお、CUに対応するピクチャ上の領域は符号化ブロック(CB:Coding Block)と呼ばれる。輝度ピクチャ上のCBを輝度CB、色差ピクチャ上のCBを色差CBと呼ぶ。CUサイズ(符号化ノードのサイズ)とは、輝度CBサイズを意味する。
変換ツリー(以下、TTと略称する)は、符号化ユニットCUが1または複数の変換ブロックに分割され、各変換ブロックの位置とサイズとが規定される。別の表現でいえば、変換ブロックは、符号化ユニットCUを構成する1または複数の重複しない領域のことである。また、変換ツリーは、上述の分割より得られた1または複数の変換ブロックを含む。なお、CUに含まれる変換ツリーに関する情報、及び変換ツリーに包含される情報を、TT情報と呼ぶ。
処理2:処理1にて得られた変換係数を量子化する;
処理3:処理2にて量子化された変換係数を可変長符号化する;
なお、上述した量子化パラメータqpは、階層動画像符号化装置2が変換係数を量子化する際に用いた量子化ステップQPの大きさを表わす(QP=2qp/6)。
予測ツリー(以下、PTと略称する)は、符号化ユニットCUが1または複数の予測ブロックに分割され、各予測ブロックの位置とサイズとが規定される。別の表現でいえば、予測ブロックは、符号化ユニットCUを構成する1または複数の重複しない領域である。また、予測ツリーは、上述の分割により得られた1または複数の予測ブロックを含む。なお、CUに含まれる予測ツリーに関する情報、及び予測ツリーに包含される情報を、PT情報と呼ぶ。
予測ユニットの予測画像は、予測ユニットに付随する予測パラメータによって導出される。予測パラメータには、イントラ予測の予測パラメータ、もしくはインター予測の予測パラメータがある。
次に、参照ピクチャリストの一例について説明する。参照ピクチャリストとは、復号ピクチャバッファに記憶された参照ピクチャからなる列である。図10 (a)は、参照ピクチャリストの一例を示す概念図である。参照ピクチャリストRPL0において、左右に一列に配列された5個の長方形は、それぞれ参照ピクチャを示す。左端から右へ順に示されている符号P1、P2、Q0、P3、P4は、それぞれの参照ピクチャを示す符号である。同様に、参照ピクチャリストRPL1において、左端から右へ順に示されている符号P4、P3、R0、P2、P1は、それぞれの参照ピクチャを示す符号である。P1等のPとは、対象レイヤPを示し、そしてQ0のQとは、対象レイヤPとは異なるレイヤQを示す。同様に、R0のRとは、対象レイヤP、及びレイヤQとは異なるレイヤRを示す。P、Q及びRの添字は、ピクチャ順序番号POC(Picture Ordering Count)を示す。refIdxL0の真下の下向きの矢印は、参照ピクチャインデックスrefIdxL0が、復号ピクチャバッファにおいて、参照ピクチャリストRPL0より参照ピクチャQ0を参照するインデックスであることを示す。同様に、refIdxL1の真下の下向きの矢印は、参照ピクチャインデックスrefIdxL1が、復号ピクチャバッファにおいて、参照ピクチャリストRPL1より参照ピクチャP3を参照するインデックスであることを示す。
次に、ベクトルを導出する際に用いる参照ピクチャの例について説明する。図10 (b)は、参照ピクチャの例を示す概念図である。図10 (b)において、横軸は表示時刻を示し、縦軸はレイヤ数を示す。図示されている、縦3行、横3列(計9個)の長方形は、それぞれピクチャを示す。9個の長方形のうち、下行の左から2列目の長方形は復号対象のピクチャ(対象ピクチャ)を示し、残りの8個の長方形がそれぞれ参照ピクチャを示す。対象ピクチャから下向きの矢印で示される参照ピクチャQ2、及びR2は対象ピクチャと同じ表示時刻であってレイヤが異なるピクチャである。対象ピクチャcurPic(P2)を基準とするレイヤ間予測においては、参照ピクチャQ2、またはR2が用いられる。対象ピクチャから左向きの矢印で示される参照ピクチャP1は、対象ピクチャと同じレイヤであって、過去のピクチャである。対象ピクチャから右向きの矢印で示される参照ピクチャP3は、対象ピクチャと同じレイヤであって、未来のピクチャである。対象ピクチャを基準とする動き予測においては、参照ピクチャP1又はP3が用いられる。
ベクトルmvLXには、動きベクトルと変位ベクトル(disparity vector、視差ベクトル)がある。動きベクトルとは、あるレイヤのある表示時刻でのピクチャにおけるブロックの位置と、異なる表示時刻(例えば、隣接する離散時刻)における同一のレイヤのピクチャにおける対応するブロックの位置との間の位置のずれを示すベクトルである。
以下では、本実施形態に係る階層動画像復号装置1の構成について、図18~図21を参照して説明する。
本実施形態に係る階層動画像復号装置1の構成について説明する。図18は、本実施形態に係る階層動画復号装置1の構成を示す概略図である。
non-VCL復号部12は、入力されるターゲットセット符号化データから、ターゲットセットの復号に用いられるパラメータセット(VPS, SPS, PPS)を復号する。復号されたパラメータセットの符号化パラメータは、パラメータメモリ13に供給され、各パラメータセットの有する識別子毎に記録される。なお、non-VCL復号部12は、パラメータセットに限定されず、図6において、non-VCLに分類されるNALユニット(nal_unit_type=32.. 63)を復号してもよい。パラメータセットと同様に、復号されたnon-VCLの符号化パラメータは、パラメータメモリ13において、各々記録される。
レイヤセット情報は、階層符号化データに含まれるレイヤセットを構成するレイヤの集合を表わすリスト(以下、レイヤIDリストLayerIdList)であり、レイヤセット情報復号手段によってVPSから復号される。レイヤセット情報には、VPS上に定義されるレイヤセットの数を示すシンタックス(vps_num_layer_sets_minus1) (図11上のSYNPVS06)及び、VPS上のレイヤ定義の順番で、j番目のレイヤ(レイヤj)が、i番目のレイヤセット(レイヤセットi)に含まれるか否かを示すシンタックス“layer_id_included_flag[i][j]”(SYNVPS07)が含まれている。レイヤセット数VpsNumLayerSetsは、(vps_num_layer_sets_minus1 + 1)に設定される。また、レイヤセットiは、シンタックス”layer_id_included_flag[i][j]”の値が1であるレイヤjから構成される。すなわち、レイヤセットiを構成するレイヤjは、レイヤIDリストLayerIdList[i]に含まれる。
for( i = 0; i < VpsNumLayerSets; i++){
NumLayersInIdList[i] = 0;
for( m = 0; m <= vps_max_layer_id; m++ ){
if( layer_id_included_flag[ i ][ m ] ){
LayerIdList[ i ][ NumLayersInIdList[i] ] = m;
NumLayersInIdList[i]++;
}
} // end of loop on for(m=0; m<= vps_max_layer_id; m++)
} // end of loop on for(i=0; i<VpsNumLayerSets; i++)
なお、上記疑似コードをステップで表わせば、次の通りである。
レイヤ間依存情報には、直接依存フラグ“direct_dependency_flag[i][j]”(図12のSYNVPS0C)が含まれる。レイヤ間依存情報は、レイヤ間依存情報復号手段により、例えば、VPS拡張データから復号される。
参照レイヤIDリスト、及び直接参照レイヤ数の導出は以下の疑似コードにより実行される。
iNuhLId = layer_id_in_nuh [i];
NumDirectRefLayers[iNuhLId] = 0;
for(j=0; j<i; j++){
if( direct_dependency_flag[i][j]){
RefLayerId[iNuhLId][NumDirectRefLayers[iNuhLId]] = layer_id_in_nuh[j];
NumDirectRefLayers[iNuhLId]++;
}
} // end of loop on for(j=0; j<i; i++)
} // end of loop on for(i=0; i< VpsMaxLayers ; i++)
なお、上記疑似コードをステップで表わせば、次の通りである。
また、レイヤ依存情報復号手段は、導出した参照レイヤIDリストRefLayerId[][]、直接参照レイヤ数NumDirectRefLayers[]に基づいて、レイヤjがレイヤiの依存レイヤ(直接参照レイヤ、又は間接参照レイヤ)であるかを示す依存フラグrecursiveRefLayerFlag[][]を導出する。例えば、以下に示す疑似コードによって依存フラグを導出する。
for(i=0; i<VpsMaxLayers; i++){
currLayerId = layer_id_in_nuh[i];
for(j=0; j<NumDirectRefLayers[currLayerId]; j++){
refLayerId = RefLayerId[currLayerId][j];
recursiveRefLayerId[currLayerId][refLayerId] = 1;
for(k=0; k<VpsMaxLayers; k++){
if ( recursiveRefLayerFlag[refLayerId][k] ){
recursiveRefLayerFlag[currLayerId][k] |=
(recursiveFlag[refLayerId][k]);
}
} // end of loop on for (k=0; k<VpsMaxLayers; k++)
} // end of loop on for (j=0; j<NumDirectRefLayers[currLayerId]; j++)
} // end of loop on for(i=0; i< VpsMaxLayers ; i++)
なお、上記疑似コードをステップで表わせば、次の通りである。なお、ステップSO01の開始前に、依存フラグrecursiveRefLayerFlag [][]の全ての要素の値は、0で初期化済であるものとする。
PTL情報は、出力レイヤセットを復号するために必要とするプロファイル及びレベルを示す情報であり、PTL情報復号手段によってVPSまたはSPSから復号される。
・プロファイル空間general_profile_space
・ティアフラグgeneral_tier_flag
・プロファイル識別子general_profile_idc
・プロファイル互換フラグgeneral_profile_compatibility_flag[ i ]
・プロファイル予約シンタックスgeneral_reserved_zero_44bits
シンタックス群SYNPTL02は、レベル識別子general_level_idcが含まれる。
・サブレイヤプロファイル空間sub_layer_profile_space[ i ]
・サブレイヤティアフラグsub_layer_tier_flag[ i ]
・サブレイヤプロファイル識別子sub_layer_profile_idc[ i ]
・サブレイヤプロファイル互換フラグsub_layer_profile_compatibility_flag[ i ][ j ]・サブレイヤプロファイル予約シンタックスsub_layer_reserved_zero_44bits[ i ]
シンタックス群SYNPTL05には、サブレイヤのサブレイヤレベル情報として、サブレイヤレベル識別子sub_layer_level_idc[ i ]が含まれる。
図字されない、スケーラブル識別子復号手段は、入力される対象レイヤ符号化データから、レイヤ単位に割り当てられるスケーラブル識別子(ScalabilityId)を復号する。スケーラブル識別子ScalabilityIdは、レイヤ間でレイヤの性質を区別するためのIDであり、スケーラブルIDとも呼ばれる。1つのレイヤに対して複数の次元のスケーラブルIDを有することができる。以下のレイヤiのj次元目のスケーラブルIDは符号化データのdimension_id[i][j]から導出される。インデックスjは、0から15までをとる。
出力レイヤセット情報は、出力するレイヤの集合(出力レイヤ情報)と、レイヤの集合(レイヤセット情報)の組み合わせにより定義され、階層動画像復号装置の備える図示しない出力レイヤセット情報復号手段で復号される。階層動画像復号装置は、出力レイヤセット情報復号手段で復号される出力レイヤセットに含まれるレイヤセット(出力レイヤセットに対応づけらるレイヤセット)に含まれるレイヤを復号対象とし、該レイヤの復号ピクチャを復号してバッファに記録し、出力レイヤセットに含まれる出力レイヤ情報を対象として、バッファに記録された特定のレイヤの復号ピクチャを選択して出力する。
E2:デフォルト出力レイヤ識別子(default_target_output_layer_idc)(図12上のSYNVPS0H)
E3:レイヤセット識別子(output_layer_set_idx_minus1)(図12上のSYNVPS0I)
E4:出力レイヤ情報(output_layer_flag)(図12上のSYNVPS0J)
E5:代替出力レイヤフラグ(alt_output_layer_flag)(図12上のSYNVPS0K)
E6:PTL・DPB情報有無フラグ(ptl_dpb_info_present_flag)(図12上のSYNVPS0L)
E7:PTL指定識別子(profile_level_tier_idx)(図12上のSYNVPS0M)
本実施形態の出力レイヤセット情報復号手段は少なくとも出力レイヤセットのレイヤセット識別子と出力レイヤフラグを復号する。
出力レイヤセットは対応するレイヤセットと、レイヤセット内の出力レイヤの指定を組み合わせた情報である。出力レイヤセットに対応するレイヤセットとして、レイヤセット識別子により特定されるレイヤセットが利用できる。出力レイヤの指定には、出力レイヤ情報が利用できる。そのため、各出力レイヤセットは一つの関連するレイヤセットを有する。
デフォルト出力レイヤ識別子default_target_output_layer_idcは、出力レイヤセット(出力レイヤ情報)の導出処理を指定するシンタックス要素である。本実施形態の出力レイヤセット情報復号手段は、デフォルト出力レイヤ識別子を復号し、デフォルト出力レイヤ識別子の値に応じた処理により、出力レイヤ情報の復号制御または導出を実行する。
レイヤセット識別子は、出力レイヤセットに関連付けられるレイヤセットを特定する値である。本実施形態の出力レイヤセット情報復号手段は、シンタックス要素output_layer_set_idx_minus1[i]を復号し、該シンタックス要素値に1を加えた値を識別子iの出力レイヤセットに対するレイヤセット識別子として用いる。レイヤセット識別子が指すレイヤセット(LS#(output_layer_set_idx_minus1[i] + 1))は識別子iの出力レイヤセット(OLS#i)に関連付けられる。
出力レイヤ情報は、出力レイヤセットに関連付けられるレイヤセットに含まれる各レイヤを出力対象レイヤとするかを示すフラグ(OutputLayerFlag[i][j])の集合である。本実施形態の出力レイヤセット情報復号手段は、復号したシンタックス要素output_layer_flag[i][j]から、出力レイヤ情報OutputLayerFlag[i][j]を設定する。output_layer_flag[i][j]は、出力レイヤセットiに含まれるj番目のレイヤを出力対象レイヤとするか否かを表すフラグであって、値が真(1)の場合には出力対象レイヤとすることを示し、値が偽(0)の場合には出力対象レイヤとしないことを表す。
if( AuxID[ nuh_layer_id[ LayerIdList[LayerSetIdx[i]][j]]]==0)
OutputLayerFlag[i][j] = 1;
else
OuptutLayerFlag[i][j] = 0;
}
if( レイヤjがLayerIdList[LayerSetIdx[i]]中にある最高次レイヤ識別子を有するプライマリピクチャレイヤである){
OutputLayerFlag[i][j] = 1;
} else{
OutputLayerFlag[i][j] = 0;
}
}
なお、レイヤjがプライマリピクチャレイヤであるか否かは、レイヤjに関する次元IDを示すシンタックス”dimension_id[i][j]”から導出されるスケーラブル識別子(スケーラビリティID)(ScalabilityId)と、図14(a)上に示すスケーラブル識別子(スケーラビリティID)とスケーラビリティタイプ(Scalability Diemnsion)との対応表を参照して、同表のうち、“Auxiliary”の項目(補助ピクチャレイヤID AuxId[j] = ScalabilityId[j][3])の値によって決まる。すなわち、同項目の値が0である場合(AuxId[j]==0)、レイヤjがプライマリピクチャレイヤであることを示し、0より大きい場合(AuxId[j]>0)、レイヤjが補助ピクチャレイヤ(あるいは、AUXレイヤ)であることを示す。なお、補助ピクチャレイヤとは、プライマリピクチャレイヤに属するピクチャに対するデプスマップや、アルファチャネルなどの補助ピクチャを通知するためのレイヤである。なお、スケーラブル識別子、及び補助ピクチャレイヤIDの詳細については、(スケーラブル識別子、及び補助ピクチャレイヤID)において、説明済である。
OutputLayerFlag[i][j] = output_layer_flag[i][j];
}
なお、出力レイヤセット情報復号手段は、導出された出力レイヤ情報(OutputLayerFlag)に基づいて、各出力レイヤセットOLS#i(i=0..NumOutputLayerSets-1)の出力レイヤ数NumOptLayersInOLS[i]、及び最高次出力レイヤのレイヤ識別子OlsHighestOutputLayerId[i]を、以下に示す疑似コードによって導出してもよい。すなわち、出力レイヤセットOLS#iの出力レイヤ数NumOptLayersInOLS[i]は、レイヤjの出力レイヤフラグOutputLayerFlag[i][j]が”出力レイヤ”を示すフラグの個数である。また、最高次出力レイヤのレイヤ識別子は、出力レイヤセットOLS#iのレイヤIDリストLayerIdList[LayerSetIdx[i]][]の中で、OuputLayerFlag[i][]が1(真)である最高次レイヤのレイヤ識別子である。
for(j=0; j<NumLayersInIdList[LayerSetIdx[i]]; j++){
NumOptLayersInOLS[i] += OuputLayerFlag[i][j];
if (OuputLayerFlag[i][j]){
OlsHighestOutputLayerId[i] = LayerIdList[ LayerSetIdx[i] ][j];
}
}
代替出力レイヤフラグ(alt_output_layer_flag[i])(図12上のSYNVPS0K)は、代替レイヤ復号ピクチャ出力の適用可否を示す情報である。代替レイヤ復号ピクチャ出力の適用時には、出力レイヤ情報で指定されたレイヤの復号ピクチャが存在しない場合に、代替レイヤが設定されて該代替レイヤの復号ピクチャが代わりに出力される。本実施形態では、シンタックス要素値alt_output_layer_flag[i]が出力レイヤセットiに対する代替出力レイヤ情報である。alt_output_layer_flag[i]の値が真(1)の場合には、出力レイヤセットOLS#iの復号時に代替レイヤ復号ピクチャ出力が適用され、値が偽(0)の場合には、代替レイヤ復号ピクチャ出力が適用されない。
PTL・DPB情報有無フラグ(ptl_dpb_present_flag[i])(図12上のSYNVPS0L)は、出力レイヤセットに適用するPTL指定識別子、及びDPB情報が符号化データ中に存在するか否かを示すフラグである。
PTL指定識別子(profile_level_tier_idx)(図12上のSYNVPS0M)は、出力レイヤセットに適用するPTL情報を指定するためのシンタックス要素である。PTL指定識別子(profile_level_tier_idx[i])で指定されるPTL情報が、出力レイヤセットOLS#iに適用される。
上記出力レイヤセット情報復号手段は、PTL・DPB情報有無フラグに基づいて、PTL指定識別子を復号又は推定するが、これに限定されない。例えば、出力レイヤセット情報復号手段は、PTL・DPB情報有無フラグを復号せずに、出力レイヤセットが、基本出力レイヤセット、又は追加出力レイヤセットであるかに基づいて、PTL指定識別子の復号を行ってもよい。
DPB情報は、出力レイヤセットを復号するためにデコーダがバッファ(DPB)で保持する復号ピクチャの最大サイズ等を示す情報であり、DPB情報復号手段によって、VPSまたはSPSから復号される。
上記DPB情報復号手段は、PTL・DPB情報有無フラグに基づいて、DPB情報を復号又は推定するが、これに限定されない。例えば、DPB情報復号手段は、PTL・DPB情報有無フラグによらず、出力レイヤセットが、基本出力レイヤセット、又は追加出力レイヤセットであるかに基づいて、DPB情報の復号を行ってもよい。
出力制御部16は、対象出力レイヤIDリストTargetOptLayerIdList[]と復号レイヤIDリストを導出し、復号ピクチャ管理部15に出力する。
for(k=0; j=0; j< NumLayersInIdList[ LayerSetIdx[ TargetOLSIdx ]]; j++){ //SA01
if( OutputLayerFlag[ TargetOLSIdx ][j] ){ //SA02
TargetOptLayerIdList[k] = LayerIdList[LayerSetIdx[ TargetOLSIdx ] ][j]; //SA03
k++; //SA04
}
} // end of loop //SA05
なお、上記疑似コードをステップで表わせば、次の通りである。
さらに、出力制御部16の備える図示しない復号レイヤIDリスト導出手段は、対象出力レイヤIDリストTargetOptLayerIdList、パラメータメモリ13に保持されたアクティブVPSのレイヤセットLayerIdList[][]、レイヤ間依存情報より導出される依存フラグに基づいて、対象出力レイヤセットの復号に必要とするレイヤの構成を示す対象復号レイヤIDリストTargetDecLayerIdList[]を導出する。導出されたTargetDecLayerIdList[]は、出力制御情報の一部として、ビットストリーム抽出部17、及びターゲットセットピクチャ部10へ供給する。
for(i=0,j=0; j< NumLayersInIdList[ LayerSetIdx[ TargetOLSIdx ]]; j++){ //SB01
iNuhLId = layer_id_in_nuh[ LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][ j ] ];
//SB02
for(refLayerFlag=0, k=0; k< NumOptLayersInOLS[TargetOLSIdx]; k++){ //SB03
iOptLayerId = layer_id_in_nuh[ TargetOptLayerIdList[k] ]; //SB04
refLayerFlag =
(refLayerFlag | recursiveRefLayerFlag[ iOptLayerId ][ iNuhLId ]); //SB05
} //SB06
if( OutputLayerFlag[ TargetOLSIdx ][j] || refLayerFlag ){ //SB07
TargetDecLayerId[i] = LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][j]; //SB08
i++; //SB09
}
} //SB10
なお、上記疑似コードをステップで表わせば、次の通りである。なお、各ステップ番号SB01.. SB10は、疑似コード、及び図19に示す対象復号レイヤIDリストの導出に係るフロー図のステップ番号SB01.. SB10と対応する。
また、出力制御部は、出力レイヤ、又は出力レイヤの依存レイヤに関わらず、特定レイヤのレイヤ識別子を有するレイヤを、対象復号レイヤIDリストTargetDecLayerIdListに含めるようにした出力制御部16aであってもよい。例えば、特定レイヤとしてレイヤ識別子=0であるレイヤ(ベースレイヤ)を含め、対象復号レイヤIDリストTargetDecLayerIdListを導出しても良い。この場合、対象復号レイヤIDリストTargetDecLayerIdListの導出を示す擬似コードのステップSB07の条件式を次の条件式(A1)又は(A2)へ変更する。
if( OutputLayerFlag[ TargetOLSIdx ][j]
|| refLayerFlag
|| LayerIdList[ LayerSetIdx[TargetOLSIdx]][ j ] == 0 ) ・・(A1)
if( OutputLayerFlag[ TargetOLSIdx ][j]
|| refLayerFlag
|| layer_id_in_nuh[(LayerIdList[ LayerSetIdx[TargetOLSIdx]][ j ]) == 0 ) ・・(A2)
上記式(A1)または(A2)によれば、出力制御部16aは、対象レイヤjが出力レイヤであるか、又は対象出力レイヤセットTargetOptLayerSet内の出力レイヤに対する依存レイヤであるか、対象レイヤjのレイヤ識別子が0であるかを判定する。出力制御部16aは、さらに、出力レイヤフラグOutputLayerFlag[TargetOLSIdx][j]が1(真)、又はフラグrefLayerFlagが1(真)、又は対象レイヤjがベースレイヤ(レイヤjのレイヤ識別子が0)である場合、ステップSB08-SB09を実行する。
また、出力制御部は、対象出力レイヤセットにおいて、プライマリピクチャレイヤを、対象復号レイヤIDリストTargetDecLayerIdListに含めるようにした出力制御部16bであってもよい。
for(i=0,j=0; j< NumLayersInIdList[ LayerSetIdx[ TargetOLSIdx ]]; j++){ //SC01
iNuhLId = layer_id_in_nuh[ LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][ j ] ];
//SC02
if( AuxId[ iNuhLId ] == 0 ){ //SC03
TargetDecLayerId[i] = LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][j]; //SC04
i++; //SC05
}
} //SC06
なお、上記疑似コードをステップで表わせば、次の通りである。なお、各ステップ番号SC01.. SC06は、疑似コードのステップ番号SC01.. SC06と対応する。
また、出力制御部16は、対象出力レイヤセットにおいて、プライマリピクチャレイヤ、及び出力レイヤである補助ピクチャレイヤを、対象復号レイヤIDリストTargetDecLayerIdListに含めるようにした出力制御部16cであってもよい。
for(i=0,j=0; j< NumLayersInIdList[ LayerSetIdx[ TargetOLSIdx ]]; j++){ //SD01
iNuhLId = layer_id_in_nuh[ LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][ j ] ];
//SD02
if( AuxId[ iNuhLId ] == 0 ||
( AuxId[ iNuhLId ] > 0 && OutputLayerFlag[ TargetOLSIdx ][ j ]>0 ) ){ //SD03
TargetDecLayerId[i] = LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][j]; //SD04
i++; //SB05
}
} //SB06
なお、上記疑似コードをステップで表わせば、次の通りである。なお、各ステップ番号SD01.. SD06は、疑似コードのステップ番号SD01.. SD06と対応する。
また、出力制御部16は、コンフォーマンステストのための復号であるか否かに応じて、動作を変更する出力制御部16dであってもよい。コンフォーマンステストのための復号であるか否かは、階層動画像復号装置の外部から与えられる。コンフォーマンステストのための復号とは、指定されたパラメータ(例えば、DPBパラメータ通り)に動作するかどうかをテストするための復号であり、それ以外の場合は、実際に動画像を視聴するために使用される通常の復号である。出力制御部16dは、コンフォーマンステストのための復号であるか否かに応じて、動作を変更する。
iNuhLId = layer_id_in_nuh[ LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][ j ] ];
TargetDecLayerId[i] = LayerIdList[ LayerSetIdx[ TargetOLSIdx ] ][j];
i++;
}
すなわち、復号レイヤIDリスト導出手段は、コンフォーマンステストのための復号である場合には、TargetOLSIdxで示される出力レイヤセットに対応するレイヤセット(LayerSetIdx[ TargetOLSIdx ]で示されるレイヤセット)に含まれる全てのレイヤのレイヤIDを対象復号レイヤIDリストTargetDecLayerIdListに追加する。
ピクチャ復号部14は、入力されるVCL NALユニット、および、アクティブパラメータセットに基づいて復号ピクチャを生成して出力する。
スライスヘッダ復号部141は、入力されるVCL NALユニットとアクティブパラメータセットに基づいてスライスヘッダを復号する。復号したスライスヘッダは、入力されるVCL NALユニットと合わせてCTU復号部142に出力する。
CTU復号部142は、概略的には、入力されるVCL NALユニットに含まれるスライスセグメント(スライスヘッダおよびスライスデータ)、及びアクティブパラメータセットに基づいて、ピクチャを構成するスライスに含まれる各CTUに対応する領域の復号画像を復号することで、スライスの復号画像を生成する。CTUの復号画像は、CTU復号部142内部の予測残差復元部1421、予測画像生成部1422、及びCTU復号画像生成部1423により生成される。
以下、図21を参照して、ピクチャ復号部14における対象レイヤiのピクチャの復号の概略的な動作について説明する。図21は、ピクチャ復号部14における対象レイヤiのピクチャを構成するスライス単位の復号プロセスを示すフロー図である。
ビットストリーム抽出部17は、出力制御部16より供給される出力制御情報(出力レイヤセットのうち、復号対象となるレイヤの構成を示す対象復号レイヤIDリストTargetDecLayerIdList、及び対象最高次テンポラル識別子TargetHighestTidに基づいて、ビットストリーム抽出処理を行い、入力される階層符号化データDATAから、対象最高次テンポラル識別子TargetHighestTid、及び対象復号レイヤIDリストTargetDecLayerIdListによって定まる集合(ターゲットセットTargetSetと呼ぶ)に含まれないNALユニットを除去(破棄)し、ターゲットセットTargetSetに含まれるNALユニットから構成される対象レイヤセット符号化データDATA#T(BitstreamToDecode)を抽出し、出力する。
以下、図22を参照して、本実施例に係るビットストリーム抽出部17の概略的な動作について説明する。図22は、ビットストリーム抽出部17におけるアクセスユニット単位のビットストリーム抽出処理を示すフロー図である。
以上説明した本実施形態に係る階層動画像復号装置1(階層画像復号装置)は、外部より供給される出力レイヤセット識別子TargetOLSIdx、及びパラメータメモリ13に保持されたアクティブVPSのレイヤセット情報、及び出力レイヤセット情報に基づいて対象出力レイヤセットTargetOptLayerSetの出力レイヤのレイヤ構成を示す対象出力レイヤIDリストを導出し、さらに、出力レイヤセット識別子TargetOLSIdx、及びパラメータメモリ13に保持されたアクティブVPSのレイヤセット情報、及び出力レイヤセット情報、レイヤ間依存情報より導出される依存フラグ、及び導出された対象出力レイヤIDリストTargetOptLayerIdListに基づいて、対象出力レイヤセットTargetOptLayerSetの復号に必要とするレイヤの構成を示す対象復号レイヤIDリストTargetDecLayerIdListを導出する出力制御部16(又は出力制御部16a)を備える。
階層動画像復号装置1Aは、階層動画像符号化装置2から供給される階層符号化データDATAを復号して、外部より供給される出力指定情報によって定まるターゲットセットTargetSetに含まれる各レイヤの復号ピクチャを生成し、出力レイヤの復号ピクチャを出力ピクチャPOUT#Tとして出力する。
出力制御部16Aは、基本的には、出力制御部16と機能は同じである。すなわち、出力制御部16Aは、出力指定情報に含まれる出力レイヤセット識別子TargetOLSIdxによって指定される出力レイヤセットOLS#TargetOLSIdxを処理対象として選択する。続いて、出力制御部16Aは、出力制御部16における出力レイヤIDリストの導出と同一の処理によって、出力レイヤIDリストTargetOptLayerIdListを導出する。
for(j=0; j< NumLayersInIdList[ LayerSetIdx[ TargetOLSIdx ]]; j++){ //SC01
TargetDecLayerId[i] = LayerIdList[LayerSetIdx[ TargetOLSIdx ] ][j]; //SC02
} //SC03
なお、上記導出手順は、上記ステップに限定されず、実施可能な範囲で、ステップを変更しても構わない。
ビットストリーム抽出部17A、出力制御部16Aより供給される出力制御情報(出力レイヤセット)のうち、復号対象となるレイヤの構成を示す対象復号レイヤIDリストTargetDecLayerIdList、及び、及び対象最高次テンポラル識別子TargetHighestTidに基づいて、ビットストリーム抽出処理を行い、入力される階層符号化データDATAから、対象最高次テンポラル識別子TargetHighestTid、及び対象復号レイヤIDリストTargetDecLayerIdListによって定まる集合(ターゲットセットTargetSetと呼ぶ)に含まれないNALユニットを除去(破棄)する。
以下では、図23を参照して、本実施例に係るビットストリーム抽出部17Aにおける動作について説明する。なお、ビットストリーム抽出部17と共通の動作は、SG101~SG103、及びSG10A~SG10Bであり、同一のステップ番号を付し、説明を省略する。以下では、SG101~SG103の後へ追加されるステップSG104~SG105についてのみ説明する。
iNuhLId = nuh_layer_id; //SC01
for(refLayerFlag=0, k=0; k< NumOptLayersInOLS[TargetOLSIdx]; k++){ //SC02
iOptLayerId = layer_id_in_nuh[( TargetOptLayerIdList[k] )]; //SC03
refLayerFlag =
(refLayerFlag | recursiveRefLayerFlag[ iOptLayerId ][ iNuhLId ]); //SC04
} //SC05
なお、上記疑似コードをステップで表わせば、次の通りである。
ビットストリーム抽出部17AのSGB104の条件判定(C3)、(C4)に加えて、下記条件(D1)を加えてもよい。
以上説明したビットストリーム抽出17Aでは、出力レイヤの復号に必須でない、非出力レイヤかつ非依存レイヤをターゲットセットから除外したが、これに限定されない。例えば、出力レイヤセットが、1以上のプライマリピクチャレイヤ、及び1以上の補助ピクチャレイヤから構成される場合、プライマリピクチャレイヤの復号に必須でない、補助ピクチャレイヤをターゲットセットより除外し、補助ピクチャレイヤのレイヤ識別子を有するNALユニットを破棄するビットストリーム抽出部17A1としてもよい。
以下では、本実施例に係るビットストリーム抽出部17A1における動作について説明する。なお、ビットストリーム抽出部17と共通の動作は、SG101~SG103、及びSG10A~SG10Bであり、同一のステップ番号を付し、説明を省略する。以下では、SG101~SG103の後へ追加されるステップSG104A~SG105Aについてのみ説明する。
ビットストリーム抽出17Aは、出力レイヤセットにおいて、非出力レイヤである補助ピクチャレイヤのレイヤ識別子を有するNALユニットを破棄するビットストリーム抽出部17A2としてもよい。
以下では、本実施例に係るビットストリーム抽出部17A2における動作について説明する。なお、ビットストリーム抽出部17と共通の動作は、SG101~SG103、及びSG10A~SG10Bであり、同一のステップ番号を付し、説明を省略する。以下では、SG101~SG103の後へ追加されるステップSG104B~SG105Bについてのみ説明する。
以上説明した本実施形態に係る階層動画像復号装置1A(階層画像復号装置)の備えるビットストリーム抽出部17Aは、出力制御部16Aから供給される出力レイヤIDリストTargetOptLayerIdList、対象復号レイヤIDリストTargetDecLayerIdList、及び対象最高次テンポラル識別子TargetHighestTIdと、レイヤ間依存情報より導出される依存フラグrecursiveRefLayerFlag[][]に基づいて、外部より入力された符号化データから、ビットストリーム抽出処理によって、ターゲットセットに含まれるNALユニットから構成されるターゲットセット符号化データBitstreamToDecodeを生成する。
階層動画像復号装置1Bは、階層動画像符号化装置2から供給される階層符号化データDATAから、外部より供給される出力指定情報、及び階層動画像復号装置1Bの備えるnon-VCL復号部12Bから復号されるサブビットストリーム特性情報によって指定される符号化データ抽出処理をビットストリーム抽出部17Bにおいて動作させ、ターゲットセット符号化データBitstreamToDecodeを生成し、生成されたターゲットセット符号化データbitstreamToDecodeを復号して、ターゲットセットTargetSetに含まれる各レイヤの復号ピクチャを生成し、出力レイヤの復号ピクチャを出力ピクチャPOUT#Tとして出力してもよい。
non-VCL復号部12Bは、階層動画像復号装置1が備えるnon-VCL復号部12と同一の機能を有し、さらに、出力レイヤセット単位のビットストリーム抽出処理、及び前記ビットストリーム抽出処理によって生成されるサブビットストリームの特性(ビットレート情報など)を示すサブビットストリーム特性情報を復号するサブビットストリーム特性情報復号手段を備える。
サブビットストリーム特性情報は、概略的には、アクティブVPSで定義される出力レイヤセットの出力レイヤの復号に影響がない(必須でない)レイヤのピクチャ(NAL ユニット)を破棄して生成されるサブビットストリームのビットレート情報を提供する。また、サブビットストリーム特性情報が存在する場合、初期IRAPアクセスユニットに関連付けられ、初期IRAPに関連付けられたCVSに対して適用される。
以下、ビットストリーム抽出モードsub_bitstream_mode[i]が示すビットストリーム抽出処理について説明する。
ビットストリーム抽出部17Bは、少なくともビットストリーム抽出部17におけるビットストリーム抽出処理1、及びビットストリーム抽出部17Aにおけるビットストリーム抽出処理2を備える。さらに、ビットストリーム抽出部17Bは、ビットストリーム抽出部17A1におけるビットストリーム抽出処理3、及び/又は、ビットストリーム抽出部17A2におけるビットストリーム抽出処理4を備えてもよい。
以下、出力レイヤセットに関する制限(ビットストリームコンフォーマンス)を満たす符号化データを符号化する階層動画像符号化装置および復号する階層動画像復号装置について説明する。
・復号装置V1は、レイヤ識別子0のスライスセグメントを復号し、レイヤ識別子0のスライスセグメントから参照されるSPSのプロファイルが復号可能であることを示す場合には復号を行い、レイヤ識別子0のスライスセグメントから参照されるSPSのプロファイルなどのPTL情報が復号可能を示さない場合には、符号化データの復号を停止することができる。
・スライスセグメントのslice_pic_parameter_set_idを復号しないため、PPSがアクティベートされない(同様に、SPS、VPSもアクティベートされない)
・復号装置V1は、アクティベートされるSPS(及びVPS)を復号しないため、SPS(VPS)に含まれるプロファイルなどのPTL情報を復号しないため、復号されない。
・復号装置V1は、内部バッファの符号化データが尽きると、符号化装置(もしくは符号化データ送信装置、符号化データバッファリング装置)に符号化データのリクエストを要求する。要求した符号化データもまた復号する対象を有しないため、要求された出力画像(例えばピクチャ1枚)を復号しようと永久に、符号化データの要求と復号を継続する可能性がある。
以下、出力レイヤセットに関する制限(ビットストリームコンフォーマンス)を満たす符号化データを符号化する階層動画像符号化装置および復号する階層動画像復号装置について説明する。
以下では、本実施形態に係る階層動画像符号化装置2の構成について、図25を参照して説明する。
図25を用いて、階層動画像符号化装置2の概略構成を説明する。同図は、階層動画像符号化装置2の概略的構成を示した機能ブロック図である。階層動画像符号化装置2は、符号化対象とするターゲットセットに含まれる各レイヤ/サブレイヤの入力画像PIN#T(ピクチャ)を符号化して、ターゲットセットの階層符号化データDATAを生成する。すなわち、動画像符号化装置2は、ターゲットセットTargetSetのレイヤIDリストの要素TargetLayerIdList [0]…TargetLayerIdList [N-1](Nはターゲットセット(対象レイヤセット)に含まれるレイヤ数)の順で、各レイヤのピクチャを符号化し、その符号化データを生成する。なお、階層動画像復号装置1(及びその変形例を含む)において、レイヤセットにベースレイヤが含まれることを保障するために、前述のコンフォーマンス条件CC(CCはCC1~CC4)を満たすように、ターゲットセットの階層符号化データDATAを生成することが好ましい。さらに、出力制御部16b又は出力制御部16cを備える階層動画像復号装置1(及びその変形例を含む)において、出力レイヤセットにプライマリピクチャレイヤが含まれることを保障するために、前述のコンフォーマンス条件CX(CXはCX1~CX2のいずれか)を満たすように、ターゲットセットの階層符号化データDATAを生成することが好ましい。さらに、出力制御部16b又は出力制御部16cを備える階層動画像復号装置1(及びその変形例を含む)において、補助ピクチャレイヤの復号処理が省略できることを保障するために、前述のコンフォーマンス条件CX(CXは、CX1~CX2のいずれか)に加えて、コンフォーマンス条件CY(CYは、CY1~CY3のいずれか)を満たすように、ターゲットセットの階層符号化データDATAを生成することが好ましい。
図26を参照して、ピクチャ符号化部24の構成の詳細を説明する。図26は、ピクチャ符号化部24の概略的構成を示した機能ブロック図である。
以下、図27を参照して、ピクチャ符号化部24における対象レイヤiのピクチャの符号化の概略的な動作について説明する。図27は、ピクチャ符号化部24における対象レイヤiのピクチャを構成するスライス単位の符号化プロセスを示すフロー図である。
以上説明した本実施形態に係る階層動画像符号化装置2は、階層動画像復号装置1(及びその変形例(階層動画像復号装置1A、階層動画像復号装置1B)において、レイヤセットにベースレイヤが含まれることを保障するために、前述のコンフォーマンス条件CC1(又はCC2~CC4)を満たすように、ターゲットセットの階層符号化データDATAを生成する。従って、階層画像復号装置1において、上記符号化データから復号される出力レイヤセットは、必ずベースレイヤを復号することが保障される。従って、あるレイヤセットAを含む符号化データからビットストリーム抽出処理によって生成された、レイヤセットAのサブセットであるレイヤセットBを含む符号化データを復号する際に、レイヤセットBのあるレイヤC(レイヤ識別子>0)において、ベースレイヤにレイヤ識別子を有するパラメータセット(VPS/SPS/PPS)をアクティブパラメータセットとして参照する場合、ベースレイヤがレイヤセットBを含む符号化データに含まれず、あるレイヤCを復号できないというケースを防止することができる。すなわち、コンフォーマンス条件CC1(C2C~CC4)を満たすことで、レイヤセットAを含む符号化データから、ビットストリーム抽出により生成されたレイヤセットAのサブセットであるレイヤセットBを含む符号化データが復号可能であることを保障することができる。
上述した階層動画像符号化装置2及び階層動画像復号装置1は、動画像の送信、受信、記録、再生を行う各種装置に搭載して利用できる。なお、動画像は、カメラ等により撮像された自然動画像であってもよいし、コンピュータ等により生成された人工動画像(CGおよびGUIを含む)であってもよい。
最後に、階層動画像復号装置1、階層動画像符号化装置2の各ブロックは、集積回路(ICチップ)上に形成された論理回路によってハードウェア的に実現してもよいし、CPU(Central Processing Unit)を用いてソフトウェア的に実現してもよい。
本発明には、少なくとも態様1から23に示す画像復号装置、及び態様24から33に示す画像符号化装置が含まれている。
レイヤセットを復号するレイヤセット情報復号手段と、
出力レイヤセットのレイヤセット識別子と出力レイヤフラグを復号する出力レイヤセット情報復号手段と、
スケーラブル識別子を復号するスケーラブル識別子復号手段と、
上記出力レイヤセットの一つを対象出力レイヤセットとして選択する出力レイヤセット選択手段と、
上記出力レイヤセットに対応するレイヤセット、及び上記出力レイヤフラグに基づいて上記対象出力レイヤの構成を示す出力レイヤIDリストを導出する出力レイヤIDリスト導出手段と、
上記レイヤセットに対応するレイヤセット、及び上記スケーラブル識別子に基づいて、復号対象とするレイヤの構成を示す復号レイヤIDリストを導出する復号レイヤIDリスト導出手段と、
上記導出された復号レイヤIDリストに含まれる各レイヤの復号ピクチャを生成するピクチャ復号手段を備えることを特徴とする。
上記レイヤがプライマリピクチャレイヤである場合には、上記復号レイヤIDリストの要素に追加し、
上記レイヤが補助ピクチャレイヤである場合には、上記復号レイヤIDリストの要素に追加しないことを特徴とする。
レイヤセットを復号するレイヤセット情報復号手段と、
出力レイヤセットのレイヤセット識別子と出力レイヤフラグを復号する出力レイヤセット情報復号手段と、
スケーラブル識別子を復号するスケーラブル識別子復号手段と、
上記出力レイヤセットの一つを対象出力レイヤセットとして選択する出力レイヤセット選択手段と、
上記出力レイヤセットに対応するレイヤセット、及び上記出力レイヤフラグに基づいて上記対象出力レイヤの構成を示す出力レイヤIDリストを導出する出力レイヤIDリスト導出手段と、
上記レイヤセットに対応するレイヤセット、上記出力レイヤフラグ、及び上記スケーラブル識別子に基づいて、復号対象とするレイヤの構成を示す復号レイヤIDリストを導出する復号レイヤIDリスト導出手段と、
上記導出された復号レイヤIDリストに含まれる各レイヤの復号ピクチャを生成するピクチャ復号手段を備えることを特徴とする。
上記レイヤが、プライマリピクチャレイヤ、又は出力レイヤフラグが1である補助ピクチャレイヤである場合には、上記復号レイヤIDリストの要素に追加し、
上記レイヤが、出力レイヤフラグが0である補助ピクチャレイヤである場合には、上記復号レイヤIDリストの要素に追加しないことを特徴とする。
レイヤセットを復号するレイヤセット情報復号手段と、
出力レイヤセットのレイヤセット識別子と出力レイヤフラグを復号する出力レイヤセット情報復号手段と、
レイヤ間依存情報を復号するレイヤ間依存情報復号手段と、
上記出力レイヤセットの一つを対象出力レイヤセットとして選択する出力レイヤセット選択手段と、
上記出力レイヤセットに対応するレイヤセット、及び上記出力レイヤセットフラグに基づいて上記対象出力レイヤの構成を示す出力レイヤIDリストを導出する出力レイヤIDリスト導出手段と、
上記レイヤセットに対応するレイヤセット、上記出力レイヤフラグ、及び上記レイヤ間依存情報に基づいて、復号対象とするレイヤの構成を示す復号レイヤIDリストを導出する復号レイヤIDリスト導出手段と、
上記導出された復号レイヤIDリストに含まれる各レイヤの復号ピクチャを生成するピクチャ復号手段を備えることを特徴とする。
上記レイヤが出力レイヤ、又は出力レイヤの依存レイヤである場合には、上記復号レイヤIDリストの要素に追加し、
上記レイヤが非出力レイヤかつ出力レイヤの非依存レイヤである場合には、上記復号レイヤIDリストの要素に追加しないことを特徴とする。
上記レイヤが出力レイヤ、又は出力レイヤの依存レイヤ、又はレイヤ識別子が0である場合には、上記復号レイヤIDリストの要素に追加し、
上記レイヤが非出力レイヤかつ出力レイヤの非依存レイヤである場合には、上記復号レイヤIDリストの要素に追加しないことを特徴とする。
上記PTL・DPB情報有無フラグが真の場合、符号化データより、上記PTL指定識別子を復号し、
上記PTL・DPB情報有無フラグが偽の場合、上記PTL指定識別子の復号を省略し、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのPTL指定識別子と等しいと推定することを特徴とする。
上記出力レイヤセット情報復号手段は、出力レイヤセットのDPB情報、又は出力レイヤセットのPTL指定識別子の有無を示すPTL・DPB情報有無フラグを復号し、
上記PTL・DPB情報有無フラグが真の場合、上記DPB情報復号手段は、符号化データより、上記出力レイヤセットのDPB情報を復号し、
上記PTL・DPB情報有無フラグが偽の場合、上記DPB情報復号手段は、上記出力レイヤセットのDPB情報を復号せず、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのDPB情報と等しいと推定することを特徴とする。
上記DPB情報復号手段は、上記出力レイヤセットが基本出力レイヤセットである場合、符号化データより、上記出力レイヤセットのDPB情報を復号し、
上記出力レイヤセットが追加出力レイヤセットである場合、上記出力レイヤセットのDPB情報を復号せず、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのDPB情報と等しいと推定することを特徴とする。
入力された符号化データより、上記選択された出力レイヤセットに対応するサブビットストリーム特性情報に基づいて、ビットストリーム抽出処理をし、ターゲットセットのビットストリームを抽出する符号化データ抽出手段を備えることを特徴とする。
レイヤセットを符号化するレイヤセット情報符号化手段と、
レイヤ間依存情報を符号化するレイヤ間依存情報符号化手段と、
出力レイヤセットのレイヤセット識別子と出力レイヤフラグを符号化する出力レイヤセット情報符号化手段と、
上記出力レイヤセットに対応するサブビットストリーム特性情報を符号化するサブビットストリーム特性情報符号化手段と、
上記出力レイヤセットに対応するDPB情報を符号化するDPB情報符号化手段と、
上記出力レイヤセットに対応するレイヤセットに含まれる各レイヤのピクチャを符号化するピクチャ符号化手段とを備えることを特徴とする。
上記PTL・DPB情報有無フラグが偽の場合、上記PTL指定識別子の符号化を省略し、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのPTL指定識別子と等しいと推定することを特徴とする。
上記PTL・DPB情報有無フラグが偽の場合、上記DPB情報符号化手段は、上記出力レイヤセットのDPB情報の符号化を省略し、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのDPB情報と等しいと推定することを特徴とする。
上記出力レイヤセット情報符号化手段は、基本出力レイヤセットのPTL・DPB情報有無フラグの符号化をせず、前記PTL・DPB情報有無フラグを1と推定することを特徴とする。
上記出力レイヤセットが追加出力レイヤセットである場合、上記出力レイヤセットのDPB情報を符号化せず、上記出力レイヤセットのレイヤセット識別子と対応する基本出力レイヤセットのDPB情報と等しいと推定することを特徴とする。
2…階層動画像符号化装置
10…ターゲットセットピクチャ復号部
11…NAL逆多重化部(NALユニット復号手段、レイヤ識別子復号手段)
12…Non-VCL復号部(パラメータセット復号手段、レイヤセット情報復号手段、出力レイヤセット情報復号手段、PTL情報復号手段、DPB情報復号手段、サブビットストリーム特性情報復号手段、レイヤ間依存情報復号手段、スケーラブル識別子復号手段)
13…パラメータメモリ
14…ピクチャ復号部(VCL復号手段)
141…スライスヘッダ復号部
142…CTU復号部
1421…予測残差復元部
1422…予測画像生成部
1423…CTU復号画像生成部
15…復号ピクチャ管理部
16…出力制御部(出力レイヤセット選択手段、対象出力レイヤID導出手段、対象復号レイヤIDリスト導出手段)
17…ビットストリーム抽出手段(符号化データ抽出手段)
20…ターゲットセットピクチャ符号化部
21…NAL多重化部(NALユニット符号化手段)
22…Non-VCL符号化部(パラメータセット符号化手段、レイヤセット情報符号化手段、出力レイヤセット情報符号化手段、PTL情報符号化手段、DPB情報符号化手段、サブビットストリーム特性情報符号化手段、レイヤ間依存情報符号化手段、スケーラブル識別子符号化手段)
24…ピクチャ符号化部(VCL符号化手段)
26…符号化パラメータ決定部
241…スライスヘッダ符号化部
242…CTU符号化部
2421…予測残差符号化部
2422…予測画像符号化部
2423…CTU復号画像生成部
Claims (8)
- 階層画像符号化データを復号する画像復号装置であって、
レイヤセット単位で各レイヤが前記レイヤセットに含まれるか否かを示す第1のフラグを復号する第1のフラグ復号手段と、
上記第1のフラグに基づいて、前記レイヤセットのレイヤIDリストを導出するレイヤセット情報復号手段と、
出力レイヤセット単位で、
a)レイヤセット識別子、と
b)前記出力レイヤセットに含まれる各レイヤが出力レイヤであるか否かを示す出力レイヤフラグ
を含む出力レイヤセット情報を復号する出力レイヤセット情報復号手段と、
第1のレイヤが、第2のレイヤの参照レイヤであるか否かを示す依存フラグを導出する依存フラグ導出手段と、
上記出力レイヤセットに対応するレイヤセットの構成を示すレイヤIDリスト、上記出力レイヤセットの出力レイヤフラグ、及び上記依存フラグに基づいて、上記出力レイヤセットにおいて、復号するレイヤを示す復号レイヤIDリストを導出する復号レイヤIDリスト導出手段と、
上記導出された復号レイヤIDリストに含まれる各レイヤのピクチャを復号するピクチャ復号手段を備えることを特徴とする画像復号装置。 - 上記参照レイヤは、直接参照レイヤ、または間接参照レイヤであることを特徴とする請求1に記載の画像復号装置。
- 上記復号レイヤIDリスト導出手段により導出される復号レイヤIDリストに含まれるレイヤは、上記出力レイヤフラグが1である出力レイヤ、又は上記出力レイヤの参照レイヤであることを特徴とする請求項1に記載の画像復号装置。
- 上記復号レイヤIDリスト導出手段は、
上記出力レイヤセットに含まれる各レイヤに対して、出力レイヤフラグが1であるか、又は出力レイヤの参照レイヤであるかを判定し、
上記レイヤが出力レイヤ、又は出力レイヤの参照レイヤである場合には、上記復号レイヤIDリストの要素に追加することを特徴とする請求項1に記載に画像復号装置。 - 階層画像符号化データを復号する画像復号方法であって、
レイヤセット単位で各レイヤが前記レイヤセットに含まれるか否かを示す第1のフラグを復号する第1のフラグ復号ステップと、
上記第1のフラグに基づいて、前記レイヤセットのレイヤIDリストを導出するレイヤセット情報復号ステップと、
出力レイヤセット単位で、
a)レイヤセット識別子、と
b)前記出力レイヤセットに含まれる各レイヤが出力レイヤであるか否かを示す出力レイヤフラグ
を含む出力レイヤセット情報を復号する出力レイヤセット情報復号ステップと、
第1のレイヤが、第2のレイヤの参照レイヤであるか否かを示す依存フラグを導出する依存フラグ導出ステップと、
上記出力レイヤセットに対応するレイヤセットの構成を示すレイヤIDリスト、上記出力レイヤセットの出力レイヤフラグ、及び上記依存フラグに基づいて、上記出力レイヤセットにおいて、復号するレイヤを示す復号レイヤIDリストを導出する復号レイヤIDリスト導出ステップと、
上記導出された復号レイヤIDリストに含まれる各レイヤのピクチャを復号するピクチャ復号ステップを備えることを特徴とする画像復号方法。 - 上記参照レイヤは、直接参照レイヤ、または間接参照レイヤであることを特徴とする請求5に記載の画像復号方法。
- 上記復号レイヤIDリスト導出ステップにより導出される復号レイヤIDリストに含まれるレイヤは、上記出力レイヤフラグが1である出力レイヤ、又は上記出力レイヤの参照レイヤであることを特徴とする請求項5に記載の画像復号方法。
- 上記復号レイヤIDリスト導出ステップは、
上記出力レイヤセットに含まれる各レイヤに対して、出力レイヤフラグが1であるか、又は出力レイヤの参照レイヤであるかを判定し、
上記レイヤが出力レイヤ、又は出力レイヤの参照レイヤである場合には、上記復号レイヤIDリストの要素に追加することを特徴とする請求項5に記載に画像復号方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016507815A JP6465863B2 (ja) | 2014-03-14 | 2015-03-12 | 画像復号装置、画像復号方法及び記録媒体 |
US15/124,407 US20170019673A1 (en) | 2014-03-14 | 2015-03-12 | Image decoding device, image decoding method, recoding medium, image coding device, and image coding method |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-051864 | 2014-03-14 | ||
JP2014051864 | 2014-03-14 | ||
JP2014-084519 | 2014-04-16 | ||
JP2014084519 | 2014-04-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015137432A1 true WO2015137432A1 (ja) | 2015-09-17 |
Family
ID=54071871
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/057251 WO2015137432A1 (ja) | 2014-03-14 | 2015-03-12 | 画像復号装置、及び画像復号方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20170019673A1 (ja) |
JP (1) | JP6465863B2 (ja) |
WO (1) | WO2015137432A1 (ja) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021117644A1 (en) * | 2019-12-11 | 2021-06-17 | Sharp Kabushiki Kaisha | Systems and methods for signaling output layer set information in video coding |
JP2021193795A (ja) * | 2014-08-07 | 2021-12-23 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | デコード方法およびデコーダシステム |
JP2022515557A (ja) * | 2019-03-15 | 2022-02-18 | テンセント・アメリカ・エルエルシー | ビデオ符号化のための復号されたピクチャバッファ管理 |
CN114424572A (zh) * | 2019-09-24 | 2022-04-29 | 华为技术有限公司 | 用于多层视频码流的dpb参数的指示 |
US20220210458A1 (en) * | 2019-09-24 | 2022-06-30 | Huawei Technologies Co., Ltd. | OLS For Multiview Scalability |
JP2022549649A (ja) * | 2019-09-24 | 2022-11-28 | 華為技術有限公司 | マルチレイヤービデオビットストリームにおける不要なレイヤーの禁止 |
JP2022549647A (ja) * | 2019-09-24 | 2022-11-28 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | ビデオコーディングにおけるマルチビューのためのサイマルキャストレイヤ |
JP7419507B2 (ja) | 2019-09-24 | 2024-01-22 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | Olsのためのスケーラブルネスティングseiメッセージ |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2875204B1 (en) * | 2012-07-20 | 2020-09-02 | Merlin Technology Inc. | Inground operations, system, communications and associated apparatus |
EP3806471A4 (en) * | 2018-06-01 | 2022-06-08 | Sharp Kabushiki Kaisha | PICTURE DECODING DEVICE AND PICTURE CODING DEVICE |
CN114514751B (zh) * | 2019-10-07 | 2024-03-15 | 华为技术有限公司 | 避免多层视频码流中的冗余指示 |
WO2021134015A1 (en) | 2019-12-26 | 2021-07-01 | Bytedance Inc. | Profile, tier and layer indication in video coding |
EP4062634A4 (en) | 2019-12-26 | 2022-12-28 | ByteDance Inc. | LIMITATIONS ON SIGNALING HYPOTHETICAL REFERENCE DECODER PARAMETERS IN VIDEO BITSTREAMS |
EP4066386A4 (en) | 2019-12-27 | 2023-01-25 | ByteDance Inc. | SIGNALING SUBPICTURES IN VIDEO ENCODING |
CN117395441A (zh) | 2020-01-09 | 2024-01-12 | 字节跳动有限公司 | 视频流中的填充数据单元的处理 |
US11330305B2 (en) * | 2020-02-24 | 2022-05-10 | Qualcomm Incorporated | Signaling constraints and sequence parameter set sharing in video coding |
US11778215B2 (en) * | 2020-02-28 | 2023-10-03 | Qualcomm Incorporated | Coding output layer set data and conformance window data of high level syntax for video coding |
US11943429B2 (en) * | 2020-03-04 | 2024-03-26 | Qualcomm Incorporated | Subpicture signaling in video coding |
US11297350B1 (en) * | 2020-03-27 | 2022-04-05 | Tencent America LLC | Method for output layer set for multilayered video stream |
US20230156230A1 (en) * | 2020-03-30 | 2023-05-18 | Lg Electronics Inc. | Image encoding/decoding method and device for signaling information relating to ptl, dpb, and hrd in sps, and computer-readable recording medium storing bitstream |
CN115668908A (zh) * | 2020-05-22 | 2023-01-31 | 字节跳动有限公司 | 视频子比特流提取过程中访问单元的处理 |
JP7506185B2 (ja) | 2020-05-22 | 2024-06-25 | 北京字節跳動網絡技術有限公司 | サブビットストリーム抽出処理におけるコーディングされた映像の操作 |
WO2021252531A1 (en) | 2020-06-09 | 2021-12-16 | Bytedance Inc. | Handling different nal types in video sub-bitstream extraction |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014167817A1 (en) * | 2013-04-07 | 2014-10-16 | Sharp Kabushiki Kaisha | Signaling change in output layer sets |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10595031B2 (en) * | 2013-07-12 | 2020-03-17 | Qualcomm Incorporated | Selection of target output layers in high efficiency video coding extensions |
-
2015
- 2015-03-12 JP JP2016507815A patent/JP6465863B2/ja not_active Expired - Fee Related
- 2015-03-12 WO PCT/JP2015/057251 patent/WO2015137432A1/ja active Application Filing
- 2015-03-12 US US15/124,407 patent/US20170019673A1/en not_active Abandoned
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014167817A1 (en) * | 2013-04-07 | 2014-10-16 | Sharp Kabushiki Kaisha | Signaling change in output layer sets |
Non-Patent Citations (6)
Title |
---|
HAHYUN LEE ET AL.: "MV-HEVC/SHVC HLS: On output_layer_flag", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JCTVC-P0078, 16TH MEETING, January 2014 (2014-01-01), San Jose , US, pages 1 - 3 * |
JIANLE CHEN ET AL.: "SHVC Draft Text 1", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-L1008, 12TH MEETING, March 2013 (2013-03-01), Geneva, CH, pages i - iii, 1-30 * |
KEMAL UGUR ET AL.: "MV-HEVC/ SHVC HLS: On default output layer sets", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT- VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JCTVC-P0110V2, 16TH MEETING, January 2014 (2014-01-01), pages 1 - 4 * |
SACHIN DESHPANDE: "On Output Layer Sets Change Signaling", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-M0212, 13TH MEETING: INCHEON, April 2013 (2013-04-01), pages 1 - 4 * |
TAKESHI TSUKUBA ET AL.: "MV-HEVC/SHVC HLS: On decoding non-output/non-reference layers", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JCTVC-Q0163R2, 17TH MEETING, March 2014 (2014-03-01), Valencia, ES, pages 1 - 3 * |
YONGJIN CHO ET AL.: "MV-HEVC/SHVC HLS: On picture output marking process", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JCTVC-00110, 15TH MEETING, October 2013 (2013-10-01), Geneva, CH, pages 1 - 2 * |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7163461B2 (ja) | 2014-08-07 | 2022-10-31 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | デコード方法およびデコーダシステム |
JP2021193795A (ja) * | 2014-08-07 | 2021-12-23 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | デコード方法およびデコーダシステム |
JP2022515557A (ja) * | 2019-03-15 | 2022-02-18 | テンセント・アメリカ・エルエルシー | ビデオ符号化のための復号されたピクチャバッファ管理 |
JP7250934B2 (ja) | 2019-03-15 | 2023-04-03 | テンセント・アメリカ・エルエルシー | ビデオ符号化のための復号されたピクチャバッファ管理 |
US11516486B2 (en) | 2019-03-15 | 2022-11-29 | Tencent America LLC | Decoded picture buffer management for video coding |
JP2022549647A (ja) * | 2019-09-24 | 2022-11-28 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | ビデオコーディングにおけるマルチビューのためのサイマルキャストレイヤ |
CN114424572B (zh) * | 2019-09-24 | 2023-08-22 | 华为技术有限公司 | 用于多层视频码流的dpb参数的指示 |
JP2022549649A (ja) * | 2019-09-24 | 2022-11-28 | 華為技術有限公司 | マルチレイヤービデオビットストリームにおける不要なレイヤーの禁止 |
JP7419507B2 (ja) | 2019-09-24 | 2024-01-22 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | Olsのためのスケーラブルネスティングseiメッセージ |
US20220210458A1 (en) * | 2019-09-24 | 2022-06-30 | Huawei Technologies Co., Ltd. | OLS For Multiview Scalability |
JP2022549836A (ja) * | 2019-09-24 | 2022-11-29 | 華為技術有限公司 | マルチレイヤービデオビットストリームにおける未使用レイヤーの禁止 |
CN114424572A (zh) * | 2019-09-24 | 2022-04-29 | 华为技术有限公司 | 用于多层视频码流的dpb参数的指示 |
JP2022549004A (ja) * | 2019-09-24 | 2022-11-22 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | マルチレイヤビデオビットストリームのためのdpbパラメータのシグナリング |
JP7335433B2 (ja) | 2019-09-24 | 2023-08-29 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | ビデオコーディングにおけるマルチビューのためのサイマルキャストレイヤ |
JP7381731B2 (ja) | 2019-09-24 | 2023-11-15 | 華為技術有限公司 | マルチレイヤービデオビットストリームにおける不要なレイヤーの禁止 |
JP7381732B2 (ja) | 2019-09-24 | 2023-11-15 | 華為技術有限公司 | マルチレイヤービデオビットストリームにおける未使用レイヤーの禁止 |
JP7388612B2 (ja) | 2019-09-24 | 2023-11-29 | ホアウェイ・テクノロジーズ・カンパニー・リミテッド | マルチレイヤビデオビットストリームのためのdpbパラメータのシグナリング |
US11849134B2 (en) | 2019-09-24 | 2023-12-19 | Huawei Technologies Co., Ltd. | Signaling of DPB parameters for multi-layer video bitstreams |
WO2021117644A1 (en) * | 2019-12-11 | 2021-06-17 | Sharp Kabushiki Kaisha | Systems and methods for signaling output layer set information in video coding |
Also Published As
Publication number | Publication date |
---|---|
JP6465863B2 (ja) | 2019-02-06 |
JPWO2015137432A1 (ja) | 2017-04-06 |
US20170019673A1 (en) | 2017-01-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6585223B2 (ja) | 画像復号装置 | |
JP6465863B2 (ja) | 画像復号装置、画像復号方法及び記録媒体 | |
JP6800837B2 (ja) | 画像復号装置、及び画像復号方法 | |
CN105519119B (zh) | 图像解码装置 | |
US20190014351A1 (en) | Moving image coding device, a moving image coding method, and a moving image decoding device | |
WO2015053120A1 (ja) | 画像復号装置、画像復号方法、画像符号化装置、及び画像符号化方法 | |
WO2014050597A1 (ja) | 画像復号装置 | |
JP2015195543A (ja) | 画像復号装置、画像符号化装置 | |
WO2014007131A1 (ja) | 画像復号装置、および画像符号化装置 | |
JP2015119402A (ja) | 画像復号装置、画像符号化装置、及び符号化データ | |
WO2015098713A1 (ja) | 画像復号装置および画像符号化装置 | |
JP2015126507A (ja) | 画像復号装置、画像符号化装置、及び符号化データ | |
JP2015076807A (ja) | 画像復号装置、画像符号化装置、および符号化データのデータ構造 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15762068 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016507815 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15124407 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15762068 Country of ref document: EP Kind code of ref document: A1 |