WO2015163456A1 - 画像復号装置、画像符号化装置、および、符号化データ変換装置 - Google Patents
画像復号装置、画像符号化装置、および、符号化データ変換装置 Download PDFInfo
- Publication number
- WO2015163456A1 WO2015163456A1 PCT/JP2015/062555 JP2015062555W WO2015163456A1 WO 2015163456 A1 WO2015163456 A1 WO 2015163456A1 JP 2015062555 W JP2015062555 W JP 2015062555W WO 2015163456 A1 WO2015163456 A1 WO 2015163456A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- layer
- information
- offset
- target
- area
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
Definitions
- the present invention relates to an image decoding apparatus that decodes hierarchically encoded data in which an image is hierarchically encoded, and an image encoding apparatus that generates hierarchically encoded data by hierarchically encoding an image.
- One of information transmitted in a communication system or information recorded in a storage device is an image or a moving image. 2. Description of the Related Art Conventionally, a technique for encoding an image for transmitting and storing these images (hereinafter including moving images) is known.
- Non-patent Document 1 As video encoding methods, AVC (H.264 / MPEG-4 Advanced Video Coding) and HEVC (High-Efficiency Video Coding), which is a successor codec, are known (Non-patent Document 1).
- a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and obtained by subtracting the predicted image from the input image (original image).
- the resulting prediction residual is encoded.
- Examples of the method for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).
- predicted images in a picture are sequentially generated based on a locally decoded image in the same picture.
- inter prediction a predicted image is generated by motion compensation between pictures.
- a decoded picture used for predictive image generation in inter prediction is called a reference picture.
- a technique for generating encoded data from a plurality of moving images by encoding a plurality of mutually related moving images into layers (hierarchies) is also known, which is called a hierarchical encoding technique.
- the encoded data generated by the hierarchical encoding technique is also referred to as hierarchical encoded data.
- SHVC Scalable HEVC
- Non-patent Document 2 As a representative hierarchical encoding technique, SHVC (Scalable HEVC) based on HEVC is known (Non-patent Document 2).
- SHVC supports spatial scalability, temporal scalability, and SNR scalability.
- spatial scalability hierarchical encoded data is generated by dividing a plurality of moving images having different resolutions into layers. For example, an image downsampled from the original image to a desired resolution is encoded as a lower layer. Next, the original image is encoded as an upper layer after applying inter-layer prediction in order to remove redundancy between layers.
- MV-HEVC Multi View HEVC
- HEVC High Efficiency Video Coding
- MV-HEVC Multi View HEVC
- a moving image corresponding to a plurality of different viewpoints (views) is divided into layers and encoded to generate hierarchical encoded data. For example, a moving image corresponding to a basic viewpoint (base view) is encoded as a lower layer. Next, a moving image corresponding to a different viewpoint is encoded as an upper layer after applying inter-layer prediction.
- Inter-layer prediction in SHVC and MV-HEVC includes inter-layer image prediction and inter-layer motion prediction.
- inter-layer image prediction a predicted image is generated using a decoded image of a lower layer.
- inter-layer motion prediction motion information prediction values are derived using motion information of lower layers.
- a picture used for prediction in inter-layer prediction is called an inter-layer reference picture.
- a layer including an inter-layer reference picture is called a reference layer.
- reference pictures used for inter prediction and reference pictures used for inter-layer prediction are generically referred to simply as reference pictures.
- any of inter prediction, intra prediction, and inter-layer image prediction can be used to generate a predicted image.
- One of the applications that use SHVC and MV-HEVC is a video application that takes into account the attention area.
- a video playback terminal normally plays back video in the entire area with a relatively low resolution.
- the attention area When a part of the video displayed by the viewer of the video reproduction terminal is designated as the attention area, the attention area is displayed on the reproduction terminal with high resolution.
- the video application considering the attention area as described above is a hierarchical code in which a relatively low resolution video of the entire area is encoded as lower layer encoded data, and a high resolution video of the attention area is encoded as upper layer encoded data.
- This can be realized using the data. That is, when reproducing the entire region, only the encoded data of the lower layer is decoded and reproduced, and when reproducing the high-resolution video of the region of interest, the encoded data of the upper layer is converted into the encoded data of the lower layer.
- the application can be realized with a smaller transmission band than when both encoded data for low-resolution video and encoded data for high-resolution video are sent. At that time, the transmission band can be further suppressed by extracting and transmitting the encoded data corresponding to the region including the region of interest from the upper layer and the lower layer, respectively.
- Non-Patent Document 3 additional information indicating the position of an alternative picture on a lower layer is sent, and the reference pixel position and scale are calculated using the additional information, so that partial data corresponding to a region of interest is obtained from hierarchically encoded data.
- a method for realizing matching before and after extraction of reference pixel positions (corresponding reference positions) and scales even when extracted is disclosed.
- Non-Patent Document 3 an additional syntax element is used to encode or decode additional data for realizing reference pixel position and scale maintenance when extracting partial data corresponding to a region of interest.
- the image decoding device and the image encoding device require additional processing for handling the additional syntax element, which causes an increase in the amount of decoding / encoding processing and the circuit scale.
- the present invention has been made in view of the above problems, and its object is to maintain reference pixel positions and scales when extracting partial data corresponding to a region of interest using fewer additional syntax elements compared to the prior art.
- an image encoding device and an image decoding device capable of maintaining a reference pixel position and a scale when extracting partial data corresponding to a region of interest by encoding or decoding processing with a smaller processing amount and a smaller circuit scale than before are realized. There is to do.
- an image encoding device is an image decoding device that decodes hierarchically encoded data and restores a decoded picture of an upper layer that is a target layer, and includes a parameter A parameter set decoding unit that decodes a set, and a prediction image generation unit that generates a prediction image by inter-layer prediction with reference to a decoded pixel of a reference layer picture, and the parameter set decoding unit decodes reference region information;
- the predicted image generation unit derives a target layer reference region position and a reference layer reference region position from reference region position information included in the reference region information, and uses the reference region position and the reference layer reference region position to It is characterized in that at least one of a scale used for prediction and a corresponding reference position is derived.
- the reference region position information is reference region offset information
- the target layer reference region position is a target layer reference region offset
- the reference layer reference region position is a reference layer target region.
- An offset is preferred.
- the reference region position information is a two-dimensional array variable having the first layer identification information and the second layer identification information as indexes
- the first layer identification information is:
- the second layer identification information is information for specifying an inter-layer processing target layer
- the predicted image generation unit is configured to identify a layer in which a reference area exists
- the target layer reference is performed using the reference region position information using the first layer identification information indicating the target layer and the second layer identification information indicating the reference layer as indexes.
- the reference region position which sets the region position and uses the first layer identification information indicating the reference layer and the second layer identification information indicating the target layer as an index To set the reference layer reference region position by using the distribution, it is preferable.
- the first layer identification information is information that directly or indirectly specifies an index in the VPS of a layer in which a reference area exists
- the second layer identification information is a layer This is information for directly or indirectly designating the intra-VPS index of the inter-process target layer
- the predicted image generation unit sets the intra-VPS index C during inter-layer prediction for the target layer whose intra-VPS index value is C.
- the target layer reference region position is set using the reference region position information using the first layer identification information to indicate and the second layer identification information indicating the index R in the VPS smaller than C as an index
- the first layer identification information indicating the intra-VPS index R and the second layer identification information indicating the intra-VPS index C are used as indexes.
- To set the reference layer reference area location with the serial reference area location information it is preferable.
- the reference region position information is a two-dimensional array variable having the first layer identification information and the second layer identification information as indexes
- the first layer identification information is:
- the second layer identification information is information for specifying an inter-layer processing target layer
- the predicted image generation unit is configured to identify a layer in which a reference area exists
- the target layer reference is performed using the reference region position information using the first layer identification information indicating the target layer and the second layer identification information indicating the reference layer as indexes.
- the region position is set, and the first layer identification information indicating the target layer and the second layer identification information indicating the target reference layer are used as indexes.
- the parameter set decoding unit decodes reference region target selection information, derives first layer identification information and second layer identification information from the reference region target selection information, and The combination of the first layer identification information and the second layer identification information and the reference region position information are recorded in association with each other, and the prediction image generation unit performs the inter-layer prediction between the target layer and the reference layer,
- the target layer reference area position is determined using the reference area position information recorded in association with the combination of the first layer identification information indicating the target layer and the second layer identification information indicating the reference layer.
- the parameter set decoding unit decodes display area information of a target layer and a reference layer, and the prediction image generation unit performs the inter-layer prediction between the target layer and the reference layer,
- the display indicated by the display area information of the target layer When the region position is set as the target layer reference region position and the first layer identification information indicating the reference layer and the reference region position information using the second layer identification information indicating the target layer as an index do not exist, the display indicated by the display area information of the target layer
- the region position is set as the target layer reference region position and the first layer identification information indicating the reference layer and the reference region position information using the second layer identification information indicating the target layer as an index do not exist
- the display area indicated by the display area information of the reference layer is set as the reference layer reference area position.
- the predicted image generation unit may be a ratio of a reference area size indicated by the target layer reference area position and a reference area size indicated by the reference layer reference area position, or a value approximating the ratio. It is preferable to derive the scale.
- an image encoding device is an image encoding device that generates encoded data of an upper layer that is a target layer from an input image, and is a parameter set that encodes a parameter set.
- An encoding unit, and a prediction image generation unit that generates a prediction image by inter-layer prediction with reference to a decoded pixel of a reference layer picture, wherein the parameter set encoding unit encodes reference region information, and generates the prediction image
- the unit derives the target layer reference region position and the reference layer reference region position from the reference region position information included in the reference region information, and uses the reference region position and the reference layer reference region position for a scale used for inter-layer prediction. And at least one of the corresponding reference positions is derived.
- an encoded data conversion apparatus converts input hierarchical encoded data based on input attention area information, generates attention area hierarchical encoded data, and outputs the generated attention area hierarchical encoded data.
- the parameter-encoded data conversion apparatus includes a parameter set correcting unit, wherein the parameter set correcting unit matches the target layer reference region position and the reference layer reference region position derived from the hierarchically encoded data before and after conversion, respectively.
- the reference area position information is corrected as described above.
- an image encoding device is an image decoding device that decodes hierarchically encoded data and restores a decoded picture of an upper layer that is a target layer, and includes a parameter A parameter set decoding unit that decodes the set, and a prediction image generation unit that generates a prediction image by inter-layer prediction with reference to a decoded pixel of the reference layer picture, and the parameter set decoding unit stores the inter-layer position correspondence information
- the inter-layer position correspondence information includes an expanded reference layer offset syntax, a reference layer offset syntax, and an inter-layer phase offset syntax, which are associated with reference layers, respectively, and the parameter set decoding unit
- the first reference layer identifier indicating the reference layer is decoded and is linked in the encoded data.
- the extended reference layer offset syntax associated with the first reference layer identifier includes the reference layer offset syntax associated with the first reference layer identifier, the reference layer offset syntax associated with the first reference layer identifier, and the first reference layer identifier.
- the predicted image generation unit generates a predicted image when a predetermined reference layer is used
- the expansion associated with the reference layer At least one of a scale used for inter-layer prediction and a corresponding reference position is derived using a reference layer offset syntax, the reference layer offset syntax, and the inter-layer phase offset syntax.
- the inter-layer position correspondence information includes a reference layer offset information presence / absence flag associated with a specific reference layer, and the predicted image generation unit sets a value of the reference layer offset information presence / absence flag. Accordingly, the reference layer offset syntax related to the reference layer associated with the reference layer offset is decoded.
- the inter-layer position correspondence information includes an inter-layer phase offset information presence / absence flag associated with a specific reference layer
- the predicted image generation unit includes an inter-layer phase offset information presence / absence flag. According to the value, the inter-layer phase offset syntax associated with the reference layer offset is decoded.
- the inter-layer position correspondence information includes a layer offset information presence / absence flag associated with a specific reference layer
- the predicted image generation unit is configured to respond to a value of the layer offset information presence / absence flag.
- the reference layer offset syntax associated with the reference layer offset and the reference layer offset syntax are decoded.
- An image decoding device (image encoding device) according to the present invention includes a parameter set decoding unit that decodes a parameter set including reference region position information, a target layer reference region position and a reference layer reference region derived from the reference region position information
- a prediction image generation unit that derives a scale and a corresponding reference position in inter-layer prediction with reference to the position is provided.
- the image decoding apparatus (image encoding apparatus) according to the present invention can cope with extraction of partial data corresponding to a region of interest by decoding processing (encoding processing) with a smaller processing amount and a smaller scale circuit than before.
- the reference position and scale can be maintained.
- (C) shows the slice layer that defines the slice S
- (d) shows the CTU layer that defines the coding tree unit CTU
- (e) shows the code layer 3 shows a CU layer that defines a coding unit (Coding Unit; CU) included in a coding tree unit CTU.
- CU Coding Unit
- (A) shows a transmission device equipped with a hierarchical video encoding device, and (b) shows a reception device equipped with a hierarchical video decoding device. It is the figure which showed the structure of the recording device carrying the said hierarchy moving image encoder, and the reproducing
- (A) shows a recording device equipped with a hierarchical video encoding device, and (b) shows a playback device equipped with a hierarchical video decoding device. It is a figure which illustrates the relationship between a target layer picture, a target layer corresponding area, a reference layer picture, a reference layer target area, and an expanded reference layer offset.
- the hierarchical moving picture decoding apparatus 1, the hierarchical moving picture encoding apparatus 2, and the encoded data conversion apparatus 3 according to an embodiment of the present invention are described as follows based on FIGS.
- a hierarchical video decoding device (image decoding device) 1 decodes encoded data that has been hierarchically encoded by a hierarchical video encoding device (image encoding device) 2.
- Hierarchical coding is a coding scheme that hierarchically encodes moving images from low quality to high quality.
- Hierarchical coding is standardized in SVC and SHVC, for example.
- the quality of a moving image here widely means an element that affects the appearance of a subjective and objective moving image.
- the quality of the moving image includes, for example, “resolution”, “frame rate”, “image quality”, and “pixel representation accuracy”.
- the quality of the moving image is different, it means that, for example, “resolution” is different, but it is not limited thereto.
- the quality of moving images is different from each other.
- Hierarchical coding technology is classified into (1) spatial scalability, (2) temporal scalability, (3) SNR (Signal to Noise Ratio) scalability, and (4) view scalability from the viewpoint of the type of information layered.
- Spatial scalability is a technique for hierarchizing resolution and image size.
- Time scalability is a technique for layering at a frame rate (number of frames per unit time).
- SNR scalability is a technique for layering in coding noise.
- view scalability is a technique for hierarchizing at the viewpoint position associated with each image.
- the encoded data conversion device 3 converts the encoded data that has been hierarchically encoded by the hierarchical moving image encoding device 2, and converts the encoded data related to a predetermined attention region (the attention region encoded data). ) Is generated.
- the attention area encoded data can be decoded by the hierarchical moving picture decoding apparatus 1 according to the present embodiment.
- the hierarchical video decoding device 1 Prior to detailed description of the hierarchical video encoding device 2, the hierarchical video decoding device 1, and the hierarchical encoded data conversion device 3 according to the present embodiment, first, (1) the hierarchical video encoding device 2 or the hierarchical code. A layer structure of hierarchically encoded data generated by the encoded data conversion device 3 and decoded by the hierarchical video decoding device 1 will be described, and then (2) a specific example of a data structure that can be adopted in each layer will be described.
- FIG. 2 is a diagram schematically illustrating a case where a moving image is hierarchically encoded / decoded by three layers of a lower layer L3, a middle layer L2, and an upper layer L1. That is, in the example shown in FIGS. 2A and 2B, of the three layers, the upper layer L1 is the highest layer and the lower layer L3 is the lowest layer.
- a decoded image corresponding to a specific quality that can be decoded from hierarchically encoded data is referred to as a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, in the upper hierarchy L1).
- Decoded image POUT # A a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, in the upper hierarchy L1).
- FIG. 2A shows a hierarchical moving image encoding apparatus 2 # A to 2 # C that generates encoded data DATA # A to DATA # C by hierarchically encoding input images PIN # A to PIN # C, respectively. Is shown.
- FIG. 2B shows a hierarchical moving picture decoding apparatus 1 # A ⁇ that generates decoded images POUT # A ⁇ POUT # C by decoding the encoded data DATA # A ⁇ DATA # C, which are encoded hierarchically. 1 # C is shown.
- the input images PIN # A, PIN # B, and PIN # C that are input on the encoding device side have the same original image but different image quality (resolution, frame rate, image quality, and the like).
- the image quality decreases in the order of the input images PIN # A, PIN # B, and PIN # C.
- the hierarchical video encoding device 2 # C of the lower hierarchy L3 encodes the input image PIN # C of the lower hierarchy L3 to generate encoded data DATA # C of the lower hierarchy L3.
- Basic information necessary for decoding the decoded image POUT # C of the lower layer L3 is included (indicated by “C” in FIG. 2). Since the lower layer L3 is the lowest layer, the encoded data DATA # C of the lower layer L3 is also referred to as basic encoded data.
- the hierarchical video encoding apparatus 2 # B of the middle hierarchy L2 encodes the input image PIN # B of the middle hierarchy L2 with reference to the encoded data DATA # C of the lower hierarchy, and performs the middle hierarchy L2 Encoded data DATA # B is generated.
- additional data necessary for decoding the decoded image POUT # B of the intermediate hierarchy is added to the encoded data DATA # B of the intermediate hierarchy L2.
- Information (indicated by “B” in FIG. 2) is included.
- the hierarchical video encoding apparatus 2 # A of the upper hierarchy L1 encodes the input image PIN # A of the upper hierarchy L1 with reference to the encoded data DATA # B of the intermediate hierarchy L2 to Encoded data DATA # A is generated.
- the encoded data DATA # A of the upper layer L1 is used to decode the basic information “C” necessary for decoding the decoded image POUT # C of the lower layer L3 and the decoded image POUT # B of the middle layer L2.
- additional information indicated by “A” in FIG. 2 necessary for decoding the decoded image POUT # A of the upper layer is included.
- the encoded data DATA # A of the upper layer L1 includes information related to decoded images of different qualities.
- the decoding device side will be described with reference to FIG.
- the decoding devices 1 # A, 1 # B, and 1 # C corresponding to the layers of the upper layer L1, the middle layer L2, and the lower layer L3 are encoded data DATA # A and DATA # B, respectively.
- And DATA # C are decoded to output decoded images POUT # A, POUT # B, and POUT # C.
- the hierarchy decoding apparatus 1 # B of the middle hierarchy L2 receives information necessary for decoding the decoded image POUT # B from the hierarchy encoded data DATA # A of the upper hierarchy L1 (that is, the hierarchy encoded data DATA # A decoded image POUT # B may be decoded by extracting “B” and “C”) included in A.
- the decoded images POUT # A, POUT # B, and POUT # C can be decoded based on information included in the hierarchically encoded data DATA # A of the upper hierarchy L1.
- the hierarchical encoded data is not limited to the above three-layer hierarchical encoded data, and the hierarchical encoded data may be hierarchically encoded with two layers or may be hierarchically encoded with a number of layers larger than three. Good.
- Hierarchically encoded data may be configured as described above. For example, in the example described above with reference to FIGS. 2A and 2B, it has been described that “C” and “B” are referred to for decoding the decoded image POUT # B, but the present invention is not limited thereto. It is also possible to configure the hierarchically encoded data so that the decoded image POUT # B can be decoded using only “B”. For example, it is possible to configure a hierarchical video decoding apparatus that receives the hierarchically encoded data composed only of “B” and the decoded image POUT # C for decoding the decoded image POUT # B.
- Hierarchically encoded data can also be generated so that In that case, the lower layer hierarchical video encoding device generates hierarchical encoded data by quantizing the prediction residual using a larger quantization width than the upper layer hierarchical video encoding device. To do.
- Upper layer A layer located above a certain layer is referred to as an upper layer.
- the upper layers of the lower layer L3 are the middle layer L2 and the upper layer L1.
- the decoded image of the upper layer means a decoded image with higher quality (for example, high resolution, high frame rate, high image quality, etc.).
- Lower layer A layer located below a certain layer is referred to as a lower layer.
- the lower layers of the upper layer L1 are the middle layer L2 and the lower layer L3.
- the decoded image of the lower layer refers to a decoded image with lower quality.
- Target layer A layer that is the target of decoding or encoding.
- a decoded image corresponding to the target layer is referred to as a target layer picture.
- pixels constituting the target layer picture are referred to as target layer pixels.
- Reference layer A specific lower layer referred to for decoding a decoded image corresponding to the target layer is referred to as a reference layer.
- a decoded image corresponding to the reference layer is referred to as a reference layer picture.
- pixels constituting the reference layer are referred to as reference layer pixels.
- the reference layers of the upper hierarchy L1 are the middle hierarchy L2 and the lower hierarchy L3.
- the hierarchically encoded data can be configured so that it is not necessary to refer to all of the lower layers in decoding of the specific layer.
- the hierarchical encoded data can be configured such that the reference layer of the upper hierarchy L1 is either the middle hierarchy L2 or the lower hierarchy L3.
- Base layer A layer located at the lowest layer is called a base layer.
- the decoded image of the base layer is the lowest quality decoded image that can be decoded from the encoded data, and is referred to as a basic decoded image.
- the basic decoded image is a decoded image corresponding to the lowest layer.
- the partially encoded data of the hierarchically encoded data necessary for decoding the basic decoded image is referred to as basic encoded data.
- the basic information “C” included in the hierarchically encoded data DATA # A of the upper hierarchy L1 is the basic encoded data.
- Extension layer The upper layer of the base layer is called the extension layer.
- the layer identifier is for identifying the hierarchy, and corresponds to the hierarchy one-to-one.
- the hierarchically encoded data includes a hierarchical identifier used for selecting partial encoded data necessary for decoding a decoded image of a specific hierarchy.
- a subset of hierarchically encoded data associated with a layer identifier corresponding to a specific layer is also referred to as a layer representation.
- a layer representation of the layer and / or a layer representation corresponding to a lower layer of the layer is used. That is, in decoding the decoded image of the target layer, layer representation of the target layer and / or layer representation of one or more layers included in a lower layer of the target layer are used.
- Inter-layer prediction is based on the syntax element value, the value derived from the syntax element value included in the layer expression of the layer (reference layer) different from the layer expression of the target layer, and the decoded image. It is to predict the syntax element value of the target layer, the encoding parameter used for decoding of the target layer, and the like. Inter-layer prediction that predicts information related to motion prediction from reference layer information is sometimes referred to as motion information prediction. In addition, inter-layer prediction predicted from a lower layer decoded image may be referred to as inter-layer image prediction (or inter-layer texture prediction). Note that the hierarchy used for inter-layer prediction is, for example, a lower layer of the target layer. In addition, performing prediction within a target layer without using a reference layer may be referred to as intra-layer prediction.
- the lower layer and the upper layer may be encoded by different encoding methods.
- the encoded data of each layer may be supplied to the hierarchical video decoding device 1 via different transmission paths, or may be supplied to the hierarchical video decoding device 1 via the same transmission path. .
- the base layer when transmitting ultra-high-definition video (moving image, 4K video data) with a base layer and one extended layer in a scalable encoding, the base layer downscales 4K video data, and interlaced video data. It may be encoded by MPEG-2 or H.264 / AVC and transmitted over a television broadcast network, and the enhancement layer may encode 4K video (progressive) with HEVC and transmit over the Internet.
- FIG. 3 is a diagram illustrating a data structure of encoded data (hierarchically encoded data DATA # C in the example of FIG. 2) that can be employed in the base layer.
- Hierarchically encoded data DATA # C illustratively includes a sequence and a plurality of pictures constituting the sequence.
- FIG. 3 shows a hierarchical structure of data in the hierarchical encoded data DATA # C.
- 3A to 3E respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, and a coding tree unit (CTU).
- CTU coding tree unit
- sequence layer a set of data referred to by the hierarchical video decoding device 1 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined.
- the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), and pictures PICT1 to PICTNP ( The NP includes the total number of pictures included in the sequence SEQ) and supplemental enhancement information (SEI).
- VPS Video Parameter Set
- sequence parameter set SPS Sequence Parameter Set
- PPS Picture Parameter Set
- SEI Supplemental Enhancement Information
- the number of layers included in the encoded data and the dependency relationship between the layers are defined.
- the sequence parameter set SPS defines a set of encoding parameters that the hierarchical video decoding device 1 refers to in order to decode the target sequence.
- a plurality of SPSs may exist in the encoded data.
- an SPS used for decoding is selected from a plurality of candidates for each target sequence.
- An SPS used for decoding a specific sequence is also called an active SPS. In the following, unless otherwise specified, it means an active SPS for the target sequence.
- a set of encoding parameters referred to by the hierarchical video decoding device 1 for decoding each picture in the target sequence is defined.
- a plurality of PPS may exist in the encoded data. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.
- a PPS used for decoding a specific picture is also called an active PPS.
- PPS means active PPS for the current picture.
- the active SPS and the active PPS may be set to different SPSs and PPSs for each layer.
- Picture layer In the picture layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 3B, the picture PICT includes slice headers SH1 to SHNS and slices S1 to SNS (NS is the total number of slices included in the picture PICT).
- the slice header SHk includes a coding parameter group that the hierarchical video decoding device 1 refers to in order to determine a decoding method for the corresponding slice Sk.
- a coding parameter group that the hierarchical video decoding device 1 refers to in order to determine a decoding method for the corresponding slice Sk.
- an SPS identifier (seq_parameter_set_id) that specifies SPS
- a PPS identifier (pic_parameter_set_id) that specifies PPS are included.
- the slice type designation information (slice_type) for designating the slice type is an example of an encoding parameter included in the slice header SH.
- I slice that uses only intra prediction at the time of encoding (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
- slice layer In the slice layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a slice S (also referred to as a target slice) to be processed is defined. As shown in FIG. 3C, the slice S includes coding tree units CTU1 to CTUNC (NC is the total number of CTUs included in the slice S).
- CTU layer In the CTU layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a coding tree unit CTU to be processed (hereinafter also referred to as a target CTU) is defined.
- the coding tree unit may be referred to as a coding tree block (CTB) or a maximum coding unit (LCU).
- CTB coding tree block
- LCU maximum coding unit
- the coding tree unit CTU includes a CTU header CTUH and coding unit information CU1 to CUNL (NL is the total number of coding unit information included in the CTU).
- NL is the total number of coding unit information included in the CTU.
- the coding tree unit CTU is divided into units for specifying a block size for each process of intra prediction or inter prediction and conversion.
- the above unit of the coding tree unit CTU is divided by recursive quadtree division.
- the tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.
- a unit corresponding to a leaf that is a node at the end of a coding tree is referred to as a coding node.
- the encoding node is a basic unit of the encoding process, hereinafter, the encoding node is also referred to as an encoding unit (CU).
- CU encoding unit
- coding unit information (hereinafter referred to as CU information)
- CU1 to CUNL is information corresponding to each coding node (coding unit) obtained by recursively dividing the coding tree unit CTU into quadtrees. is there.
- the root of the coding tree is associated with the coding tree unit CTU.
- the coding tree unit CTU is associated with the highest node of the tree structure of the quadtree partition that recursively includes a plurality of coding nodes.
- each coding node is half of the size of the coding node that is the parent node of the coding node (that is, the node that is one layer higher than the coding node).
- the size of the coding tree unit CTU and the size that each coding unit can take are the size designation information of the minimum coding node and the maximum coding node and the minimum coding node included in the sequence parameter set SPS.
- the size of the coding tree unit CTU is 64 ⁇ 64 pixels.
- the size of the encoding node can take any of four sizes, that is, 64 ⁇ 64 pixels, 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, and 8 ⁇ 8 pixels.
- the CTU header CTUH includes an encoding parameter referred to by the hierarchical video decoding device 1 in order to determine a decoding method of the target CTU. Specifically, as shown in FIG. 3 (d), CTU division information SP_CTU for designating a division pattern of the target CTU into each CU, and a quantization parameter difference ⁇ qp (for designating the quantization step size) qp_delta).
- the CTU division information SP_CTU is information representing a coding tree for dividing the CTU, and specifically, information specifying the shape and size of each CU included in the target CTU and the position in the target CTU. It is.
- the CTU partition information SP_CTU does not need to explicitly include the shape or size of the CU.
- the CTU division information SP_CTU may be a set of flags indicating whether or not the entire target CTU or a partial region of the CTU is to be divided into four. In that case, the shape and size of each CU can be specified by using the shape and size of the CTU together.
- CU layer In the CU layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a CU to be processed (hereinafter also referred to as a target CU) is defined.
- the encoding node is a node at the root of a prediction tree (PT) and a transformation tree (TT).
- PT prediction tree
- TT transformation tree
- the encoding node is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined.
- the prediction block is one or a plurality of non-overlapping areas constituting the encoding node.
- the prediction tree includes one or a plurality of prediction blocks obtained by the above division.
- Prediction processing is performed for each prediction block.
- a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).
- PU partitioning There are roughly two types of partitioning in the prediction tree (hereinafter abbreviated as PU partitioning): intra prediction and inter prediction.
- the encoding node is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined.
- the transform block is one or a plurality of non-overlapping areas constituting the encoding node.
- the conversion tree includes one or a plurality of conversion blocks obtained by the above division.
- the division in the transformation tree includes the one in which an area having the same size as the encoding node is assigned as the transformation block, and the one in the recursive quadtree division as in the above-described division of the tree block.
- transform processing is performed for each conversion block.
- the transform block which is a unit of transform is also referred to as a transform unit (TU).
- the CU information CU specifically includes a skip flag SKIP, prediction tree information (hereinafter abbreviated as PT information) PTI, and conversion tree information (hereinafter abbreviated as TT information). Include TTI).
- PT information prediction tree information
- TT information conversion tree information
- the skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU.
- the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, A part of the PT information PTI and the TT information TTI in the CU information CU are omitted. Note that the skip flag SKIP is omitted for the I slice.
- the PT information PTI is information related to a prediction tree (hereinafter abbreviated as PT) included in the CU.
- PT prediction tree
- the PT information PTI is a set of information related to each of one or a plurality of PUs included in the PT, and is referred to when a predicted image is generated by the hierarchical video decoding device 1.
- the PT information PTI includes prediction type information PType and prediction information PInfo.
- Prediction type information PType is information that specifies a predicted image generation method for the target PU. In the base layer, it is information that specifies whether intra prediction or inter prediction is used.
- the prediction information PInfo is prediction information used in the prediction method specified by the prediction type information PType.
- intra prediction information PP_Intra is included in the case of intra prediction.
- inter prediction information PP_Inter is included in the case of inter prediction.
- Inter prediction information PP_Inter includes prediction information that is referred to when the hierarchical video decoding device 1 generates an inter prediction image by inter prediction. More specifically, the inter prediction information PP_Inter includes inter PU division information that specifies a division pattern of the target CU into each inter PU, and inter prediction parameters (motion compensation parameters) for each inter PU. Examples of inter prediction parameters include a merge flag (merge_flag), a merge index (merge_idx), an estimated motion vector index (mvp_idx), a reference picture index (ref_idx), an inter prediction flag (inter_pred_flag), and a motion vector residual (mvd) including.
- merge_flag merge flag
- merge_idx merge index
- mvp_idx estimated motion vector index
- ref_idx reference picture index
- inter_pred_flag inter prediction flag
- mvd motion vector residual
- the intra prediction information PP_Intra includes an encoding parameter that is referred to when the hierarchical video decoding device 1 generates an intra predicted image by intra prediction. More specifically, the intra prediction information PP_Intra includes intra PU division information that specifies a division pattern of the target CU into each intra PU, and intra prediction parameters for each intra PU.
- the intra prediction parameter is a parameter for designating an intra prediction method (prediction mode) for each intra PU.
- the TT information TTI is information regarding a conversion tree (hereinafter abbreviated as TT) included in the CU.
- TT conversion tree
- the TT information TTI is a set of information regarding each of one or a plurality of transform blocks included in the TT, and is referred to when the hierarchical video decoding device 1 decodes residual data.
- the TT information TTI includes TT division information SP_TT for designating a division pattern for each transform block of the target CU, and quantized prediction residuals QD1 to QDNT (NT is assigned to the target CU. The total number of blocks included).
- TT division information SP_TT is information for determining the shape of each transformation block included in the target CU and the position in the target CU.
- the TT division information SP_TT can be realized from information (split_transform_unit_flag) indicating whether or not the target node is divided and information (trafoDepth) indicating the division depth.
- each conversion block obtained by the division can take a size from 32 ⁇ 32 pixels to 4 ⁇ 4 pixels.
- Each quantization prediction residual QD is encoded data generated by the hierarchical video encoding device 2 performing the following processes 1 to 3 on a target block that is a conversion block to be processed.
- Process 1 The prediction residual obtained by subtracting the prediction image from the encoding target image is subjected to frequency conversion (for example, DCT conversion (Discrete Cosine Transform) and DST conversion (Discrete Sine Transform));
- Process 2 Quantize the transform coefficient obtained in Process 1;
- Process 3 Variable length coding is performed on the transform coefficient quantized in Process 2;
- the PU partition type specified by the PU partition information includes the following eight patterns in total, assuming that the size of the target CU is 2N ⁇ 2N pixels. That is, 4 symmetric splittings of 2N ⁇ 2N pixels, 2N ⁇ N pixels, N ⁇ 2N pixels, and N ⁇ N pixels, and 2N ⁇ nU pixels, 2N ⁇ nD pixels, nL ⁇ 2N pixels, And four asymmetric splittings of nR ⁇ 2N pixels.
- N 2m (m is an arbitrary integer of 1 or more).
- a prediction unit obtained by dividing the target CU is referred to as a prediction block or a partition.
- enhancement layer encoded data For encoded data included in the layer representation of the enhancement layer (hereinafter, enhancement layer encoded data), for example, a data structure substantially similar to the data structure shown in FIG. 3 can be adopted. However, in the enhancement layer encoded data, additional information can be added or parameters can be omitted as follows.
- spatial scalability, temporal scalability, SNR scalability, and view scalability hierarchy identification information may be encoded.
- the prediction type information PType included in the CU information CU is information that specifies whether the prediction image generation method for the target CU is intra prediction, inter prediction, or inter-layer image prediction.
- the prediction type information PType includes a flag (inter-layer image prediction flag) that specifies whether or not to apply the inter-layer image prediction mode.
- the inter-layer image prediction flag may be referred to as texture_rl_flag, inter_layer_pred_flag, or base_mode_flag.
- the CU type of the target CU is an intra CU, an inter-layer CU, an inter CU, or a skip CU.
- the intra CU can be defined in the same manner as the intra CU in the base layer.
- the inter-layer image prediction flag is set to “0”, and the prediction mode flag is set to “0”.
- An inter-layer CU can be defined as a CU that uses a decoded image of a picture in a reference layer for generating a predicted image.
- the inter-layer image prediction flag is set to “1” and the prediction mode flag is set to “0”.
- the skip CU can be defined in the same manner as in the HEVC method described above. For example, in the skip CU, “1” is set in the skip flag.
- the inter CU may be defined as a CU that applies non-skip and motion compensation (MC).
- MC non-skip and motion compensation
- the encoded data of the enhancement layer may be generated by an encoding method different from the encoding method of the lower layer. That is, the encoding / decoding process of the enhancement layer does not depend on the type of the lower layer codec.
- the lower layer may be encoded by, for example, MPEG-2 or H.264 / AVC format.
- the VPS may be extended to include a parameter representing a reference structure between layers.
- SPS, PPS, and slice header are extended, and information related to a decoded image of a reference layer used for inter-layer image prediction (for example, an inter-layer reference picture set, an inter-layer reference picture list described later) , Syntax for deriving base control information or the like directly or indirectly).
- a reference layer used for inter-layer image prediction for example, an inter-layer reference picture set, an inter-layer reference picture list described later
- the parameters described above may be encoded independently, or a plurality of parameters may be encoded in combination.
- an index is assigned to the combination of parameter values, and the assigned index is encoded.
- the encoding of the parameter can be omitted.
- FIG. 4 is a diagram for explaining the relationship between pictures and tile slices in hierarchically encoded data.
- a tile is associated with a rectangular partial area in a picture and encoded data relating to the partial area.
- a slice is associated with a partial area in a picture and encoded data related to the partial area, that is, a slice header and slice data related to the partial area.
- FIG. 4A illustrates a divided area when a picture is divided by tile slices.
- the picture is divided into six rectangular tiles (T00, T01, T02, T10, T11, T12).
- Each of the tile T00, the tile T02, the tile T10, and the tile T12 includes one slice (in order, a slice S00, a slice S02, a slice S10, and a slice S12).
- the tile T01 includes two slices (slice S01a and slice S01b)
- the tile T11 includes two slices (slice S11a and slice S11b).
- FIG. 4B illustrates the relationship between tiles and slices in the configuration of encoded data.
- encoded data includes a plurality of VCL (Video Coding Layer) NAL units and non-VCL (non-VCL) NAL units.
- the encoded data of the video encoding layer corresponding to one picture is composed of a plurality of VCL NALs.
- the encoded data corresponding to the picture includes encoded data corresponding to the tiles in the tile raster order. That is, as shown in FIG. 4A, when a picture is divided into tiles, encoded data corresponding to tiles is included in the order of tiles T00, T01, T02, T10, T11, and T12.
- the encoded data corresponding to the slice is changed to the encoded data corresponding to the tile in the order of the CTU at the head of the slice starting from the slice positioned first in the CTU raster scan order within the tile. included.
- the encoded data corresponding to the slices are included in the encoded data corresponding to the tile T01 in order of the slices S01a and S01b. .
- encoded data corresponding to a specific tile in a picture is associated with encoded data corresponding to one or more slices. Therefore, if a decoded image of a slice associated with a tile can be generated, a decoded image of a partial region in a picture corresponding to the tile can be generated.
- FIG. 5 is a functional block diagram showing a schematic configuration of the hierarchical video decoding device 1.
- the hierarchical video decoding apparatus 1 receives hierarchical encoded data DATA (hierarchical encoded data DATAF provided from the hierarchical video encoding apparatus 2 or hierarchical encoded data DATAR provided from the encoded data conversion apparatus 3). Decoding is performed to generate a decoded image POUT # T of the target layer.
- the target layer is an extension layer having the base layer as a reference layer. Therefore, the target layer is also an upper layer with respect to the reference layer. Conversely, the reference layer is also a lower layer with respect to the target layer.
- the hierarchical video decoding device 1 includes a NAL demultiplexing unit 11, a parameter set decoding unit 12, a tile setting unit 13, a slice decoding unit 14, a base decoding unit 15, and a decoded picture management unit 16.
- the NAL demultiplexing unit 11 demultiplexes hierarchically encoded data DATA transmitted in units of NAL units in NAL (Network Abstraction Layer).
- NAL is a layer provided to abstract communication between VCL (Video Coding Layer) and lower systems that transmit and store encoded data.
- VCL Video Coding Layer
- VCL is a layer that performs video encoding processing, and encoding is performed in VCL.
- the lower system here corresponds to the H.264 / AVC and HEVC file formats and the MPEG-2 system.
- NAL the bit stream generated by VCL is divided into units called NAL units and transmitted to the lower system as the destination.
- the NAL unit includes encoded data encoded by the VCL and a header for appropriately delivering the encoded data to a destination lower system.
- the encoded data in each layer is stored in NAL units, is NAL multiplexed, and is transmitted to the hierarchical video decoding device 1.
- Hierarchical encoded data DATA includes NAL including parameter sets (VPS, SPS, PPS), SEI, etc. in addition to NAL generated by VCL. Those NALs are called non-VCL NALs versus VCL NALs.
- the NAL demultiplexing unit 11 demultiplexes the hierarchical encoded data DATA, and extracts the target layer encoded data DATA # T and the reference layer encoded data DATA # $. Further, the NAL demultiplexing unit 11 supplies non-VCL NAL to the parameter set decoding unit 12 and VCL NAL to the slice decoding unit 14 among the NALs included in the target layer encoded data DATA # T.
- the parameter set decoding unit 12 decodes the parameter set, that is, VPS, SPS, and PPS, from the input non-VCL NAL and supplies them to the tile setting unit 13 and the slice decoding unit 14. Details of processing highly relevant to the present invention in the parameter set decoding unit 12 will be described later.
- the tile setting unit 13 derives the tile information of the picture based on the input parameter set and supplies it to the slice decoding unit 14.
- the tile information includes at least tile division information of a picture.
- the slice decoding unit 14 generates a decoded picture or a partial area of the decoded picture based on the input VCL NAL, parameter set, tile information, and reference picture, and records them in a buffer in the decoded picture management unit 16. .
- a detailed description of the slice decoding unit will be described later.
- the decoded picture management unit 16 records the input decoded picture and the base decoded picture in an internal decoded picture buffer (DPB: “Decoded” Picture ”Buffer), and performs reference picture list generation and output picture determination. Further, the decoded picture management unit 16 outputs the decoded picture recorded in the DPB to the outside as an output picture POUT # T at a predetermined timing.
- DPB internal decoded picture buffer
- the base decoding unit 15 decodes the base decoded picture from the reference layer encoded data DATA # R.
- the base decoded picture is a decoded picture of the reference layer used when decoding the decoded picture of the target layer.
- the base decoding unit 15 records the decoded base decoded picture in the DPB in the decoded picture management unit 16.
- FIG. 6 is a functional block diagram illustrating the configuration of the base decoding unit 15.
- the base decoding unit 15 includes a base NAL demultiplexing unit 151, a base parameter set decoding unit 152, a base tile setting unit 153, a base slice decoding unit 154, and a base decoded picture management unit 156.
- the base NAL demultiplexing unit 151 demultiplexes the reference layer encoded data DATA # R to extract VCL NAL and non-VCL NAL, non-VCL NAL to the base parameter set decoding unit 152, and VCL NAL to base slice Each is supplied to the decryption unit 154.
- the base parameter set decoding unit 152 decodes the parameter set, that is, VPS, SPS, and PPS, from the input non-VCL NAL and supplies them to the base tile setting unit 153 and the base slice decoding unit 154.
- the base style setting unit 153 derives picture tile information based on the input parameter set and supplies it to the base slice decoding unit 154.
- the base slice decoding unit 154 generates a decoded picture or a partial area of the decoded picture based on the input VCL NAL, parameter set, tile information, and reference picture, and stores the decoded picture in the buffer in the base decoded picture management unit 156. Record.
- the base decoded picture management unit 156 records the input decoded picture in the internal DPB, and performs reference picture list generation and output picture determination. Further, the base decoded picture management unit 156 outputs the decoded picture recorded in the DPB as a base decoded picture at a predetermined timing.
- the parameter set decoding unit 12 decodes and outputs a parameter set (VPS, SPS, PPS) used for decoding the target layer from the input encoded data of the target layer.
- a parameter set (VPS, SPS, PPS) used for decoding the target layer from the input encoded data of the target layer.
- the decoding of the parameter set is performed based on a predetermined syntax table. That is, a bit string is read from the encoded data according to a procedure defined by the syntax table, and syntax values of syntax elements included in the syntax table are decoded.
- the parameter set decoding unit 12 decodes picture information from input target layer encoded data.
- the picture information is information that determines the size of the decoded picture of the target layer.
- the picture information includes information indicating the width and height of the decoded picture of the target layer.
- Picture information is included in SPS, for example.
- the picture information decoded from the SPS includes the width of the decoded picture (pic_width_in_luma_samples) and the height of the decoded picture (pic_height_in_luma_samples).
- the value of the syntax pic_width_in_luma_samples corresponds to the width of the decoded picture in luminance pixel units.
- the value of the syntax pic_height_in_luma_samples corresponds to the height of the decoded picture in luminance pixel units.
- picture information is shared between layers. That is, picture information of a layer different from the target layer can be referred to when the target layer is decoded and encoded.
- the parameter set decoding unit 12 decodes the display area information from the input target layer encoded data.
- the display area information is included in the SPS, for example.
- the display area information decoded from the SPS includes a display area flag (conformance_flag).
- the display area flag indicates whether information indicating the position of the display area (display area position information) is additionally included in the SPS. That is, when the display area flag is 1, it indicates that the display area position information is additionally included, and when the display area flag is 0, it indicates that the display area position information is not additionally included.
- the display area information decoded from the SPS further includes display area left offset (conf_win_left_offset), display area right offset (conf_win_right_offset), display area upper offset (conf_win_top_offset), and display area. Contains the lower offset (conf_win_bottom_offset).
- the entire picture is set as the display area.
- the display area flag is 1
- a partial area in the picture indicated by the display area position information is set.
- the display area is also referred to as a conformance window.
- FIG. 7 is a diagram illustrating a relationship between a display area which is a partial area in a picture and display area position information.
- the display area is included in the picture
- the display area offset is the distance between the picture upper edge and the display area upper edge
- the display area left offset is the distance between the picture left edge and the display area left edge
- the display area right offset Represents the distance between the right side of the picture and the right side of the display area
- the lower offset of the display area represents the distance between the lower side of the picture and the lower side of the display area. Therefore, the position and size of the display area in the picture can be uniquely specified by the display area position information.
- the display area information may be other information that can uniquely identify the position and size of the display area in the picture.
- the display area information is shared between layers. That is, display area information of a layer different from the target layer can be referred to when the target layer is decoded and encoded.
- the parameter set decoding unit 12 decodes the inter-layer position correspondence information from the input target layer encoded data.
- the inter-layer position correspondence information schematically indicates the positional relationship between corresponding areas of the target layer and the reference layer. For example, when an object (object A) in a picture of the target layer and a picture of the reference layer is included, an area corresponding to the object A on the picture of the target layer and an area corresponding to the object A on the picture of the reference layer , Corresponding to the regions corresponding to the target layer and the reference layer.
- the inter-layer position correspondence information may not necessarily be information that accurately indicates the positional relationship between the corresponding regions of the target layer and the reference layer, but generally, in order to improve the accuracy of inter-layer prediction. The correct positional relationship between the corresponding layers of the target layer and the reference layer is shown.
- the inter-layer position correspondence information includes reference area information.
- the reference area information is information representing the spatial positional relationship between the following two areas in a specific layer (hereinafter referred to as layer A).
- a reference area offset is used as reference area information.
- the reference area offset is information representing the displacement between each vertex of the reference area and the corresponding vertex of the picture area. From the following four offsets corresponding to left, top, right, and bottom, respectively. Composed.
- Reference area left offset horizontal offset between reference area upper left pixel and picture area upper left pixel 2. Offset on reference area: vertical offset between upper left pixel in reference area and upper left pixel in picture area 3.
- Reference area right offset horizontal offset between the reference area lower right pixel and the picture area lower right pixel.
- Reference area lower offset vertical offset between the reference area lower right pixel and picture area lower right pixel
- FIG. 8 is a diagram illustrating the relationship among the target layer picture, the target layer reference region, the reference layer picture, the reference layer reference region, and the reference region offset.
- each reference area offset target layer reference area left offset, (Target layer reference area upper offset, target layer reference area right offset, target layer reference area lower offset).
- Offset reference layer reference area upper offset, reference layer reference area right offset, reference layer reference area lower offset.
- Corresponding pixel position derivation and scale derivation in inter-layer processing typified by inter-layer prediction are executed using the target layer reference region and the reference layer reference region.
- FIG. 8 illustrates the case where the reference area is included in the picture area
- the present invention is not limited thereto, and the picture area may be included in the reference area, or a part of the reference area and the picture area overlap each other. It doesn't matter.
- the reference area offset is positive or negative when the reference area is included in the picture area, and negative when the reference area is included in the picture area. That is, when the reference area is included in the picture area shown in FIG. 8, all the reference area offset values are positive. Note that the positive / negative definition of the reference area offset may be reversed, but in that case, it is necessary to appropriately read and apply a corresponding reference position and a scale derivation formula described later.
- the purpose of the reference area offset is to define the shape of the reference area and the spatial positional relationship between the picture area and the reference area. Therefore, if the purpose is achieved, parameters other than the reference offset other than those described above can be used. For example, a combination of the horizontal and vertical offsets of the upper left pixel of the reference area with respect to the upper left pixel of the picture area and the width and height of the reference area rectangle may be used.
- Information that defines the shape of the reference area and the spatial positional relationship between the picture area and the reference area is referred to as reference area position information.
- the reference area position information corresponds to a superordinate concept of the reference area offset. In the description of the present embodiment, it is assumed that a reference area offset is used as reference area position information.
- FIG. 9 shows a part of the syntax table that is referred to when the parameter set decoding unit 12 decodes the VPS extension (vps_extension ()) included in the VPS, and relates to the reference region offset.
- the VPS includes multiple syntax elements related to the reference area offset. Hereinafter, they are collectively referred to as a reference area offset-related syntax.
- the syntax table shown in FIG. 9 indicates that the following syntax elements are included in the VPS extension as syntax elements constituting the reference area offset-related syntax.
- A0 Reference region offset presence / absence flag (ref_region_offsets_in_vps_flag)
- A1 Number of reference region offsets (num_ref_region_offsets [i])
- A2 Reference region layer index information (ref_region_layer_idx_delta_minus1 [i] [j])
- A3L Reference region left offset information (ref_region_left_offset [i] [j])
- A3T Reference area offset information (ref_region_top_offset [i] [j])
- A3R Reference region right offset information (ref_region_right_offset [i] [j])
- A3B Reference region lower offset information (ref_region_bottom_offset [i] [j])
- the syntax element A0 is a sign of a 1-bit flag (u (1)).
- the syntax elements A1 and A2 are encoded by a non-negative integer 0th-order exponent Golomb code (ue (v)) defined by HEVC.
- A3L, A3T, A3R, and A3B are encoded by a signed integer 0th-order exponent Golomb code (se (v)) defined by HEVC.
- the reference area offset presence / absence flag (A0) is a flag indicating whether or not the reference area offset related syntax excluding the flag is included in the VPS extension. When the value of the flag is 1, it indicates that the reference area offset related syntax is included. Otherwise (the value of the flag is 0), it indicates that the reference area offset related syntax is not included.
- the reference area offset number (A1) represents the number of reference area offsets for a layer (hereinafter referred to as layer i) specified by an index i included in the VPS extension.
- the index i is an index indicating a layer included in the VPS extension.
- the number of reference area offsets corresponds to the number of combinations of four offsets (A3L, A3T, A3R, A3B) corresponding to left, top, right, and bottom, respectively.
- the reference region offset number is encoded / decoded for each layer specified by each index value between 0 and MaxLayersMinus1.
- the number of reference region offsets included in the VPS extension is preferably the above number in order to realize functionality that allows transmission of reference region offsets to any layer.
- the number of reference area offsets corresponding to the layer is omitted, and the value of the reference area offset for the layer is set to 0. May be estimated.
- Reference area layer index information (A2) is information for specifying the j-th layer associated with layer i. Specifically, the reference area layer index information is used to derive a layer index (LIdx [i] [j]) in the VPS of the j-th layer associated with layer i. LIdx [i] [j] is derived by the following equation when the syntax value ref_region_layer_idx_delta_minus1 [i] [j] of the reference region layer index information is RRL [i] [j].
- the maximum value of RRL [i] [j] is less than the maximum number of layers (MaxLayersMinus1 + 1) in the bitstream. That is, the maximum value of the index of the j-th layer associated with layer i can take a value larger than the index of layer i (index i).
- the reference region layer index information is defined as described above, there is an effect of suppressing the code amount of the reference region layer index information.
- another syntax for associating the index j with a specific layer and a combination of derivation means may be used.
- an intra-VPS layer index may be used as the reference area layer index information.
- LIdx [i] [j] is derived by the following equation.
- Reference area offset information (reference area left offset information (A3L), reference area upper offset information (A3T), reference area right offset information (A3R), reference area lower offset information (A3B)) is a reference area on layer i. It is information for deriving an offset (reference area offset) representing a reference area referred to in inter-layer processing (inter-layer prediction) between layer i and the j-th layer associated with layer i. is there.
- the reference area offset information is also information representing the position and size (shape) of the reference area, and is also referred to as reference area position information.
- the above reference area offset information can be expressed as follows. That is, the reference area position information is an array variable that uses the first layer identification information (i) and the second layer identification information (j) as indexes. I which is the first layer identification information is an intra-VPS layer index of the layer in which the reference area exists.
- the second layer identification information j is information for identifying the target layer of the inter-layer processing to which the reference region is applied, and the target layer of the inter-layer processing through the above Lidx [i] [j]. It is associated with the layer index within VPS.
- the slice decoding unit 14 generates and outputs a decoded picture based on the input VCL NAL, parameter set, and tile information.
- FIG. 13 is a functional block diagram illustrating a schematic configuration of the slice decoding unit 14.
- the slice decoding unit 14 includes a slice header decoding unit 141, a slice position setting unit 142, and a CTU decoding unit 144.
- the CTU decoding unit 144 further includes a prediction residual restoration unit 1441, a predicted image generation unit 1442, and a CTU decoded image generation unit 1443.
- the slice header decoding unit 141 decodes the slice header based on the input VCL NAL and the parameter set, and outputs the decoded slice header to the slice position setting unit 142, the skip slice determination unit 143, and the CTU decoding unit 144.
- the slice header includes information related to the slice position in the picture (SH slice position information) and information related to the skip slice (SH skip slice information).
- the slice header includes an in-picture first slice flag (first_slice_segment_in_pic_flag) as slice position information.
- first_slice_segment_in_pic_flag an in-picture first slice flag
- the slice header includes a slice PPS identifier (slice_pic_parameter_set_id) as slice position information.
- the slice PPS identifier is an identifier of a PPS associated with the target slice, and tile information to be associated with the target slice is specified via the PPS identifier.
- the slice position setting unit 142 specifies the slice position in the picture based on the input slice header and tile information, and outputs the slice position to the CTU decoding unit 144.
- the slice position in the picture derived by the slice position setting unit 142 includes the position in the picture of each CTU included in the slice.
- the CTU decoding unit 144 decodes a slice by decoding a decoded image of a region corresponding to each CTU included in the slice based on the input slice header, slice data, and parameter set. Generate an image. The decoded image of the slice is output as a part of the decoded picture at the position indicated by the input slice position.
- the decoded image of the CTU is generated by the prediction residual restoration unit 1441, the prediction image generation unit 1442, and the CTU decoded image generation unit 1443 inside the CTU decoding unit 144.
- the prediction residual restoration unit 1441 decodes prediction residual information (TT information) included in the input slice data to generate and output a prediction residual of the target CTU.
- the predicted image generation unit 1442 generates and outputs a predicted image based on the prediction method and the prediction parameter indicated by the prediction information (PT information) included in the input slice data. At that time, a decoded picture of a reference picture or a coding meter is used as necessary.
- the CTU decoded image generation unit 1443 adds the input predicted image and the prediction residual to generate and output a decoded image of the target CTU.
- the generation process of the predicted pixel value of the target pixel included in the target CTU to which the inter-layer image prediction is applied is executed according to the following procedure.
- a reference picture position derivation process is executed to derive a corresponding reference position.
- the corresponding reference position is a position on the reference layer corresponding to the target pixel on the target layer picture. Since the pixels of the target layer and the reference layer do not necessarily correspond one-to-one, the corresponding reference position is expressed with an accuracy of less than the pixel unit in the reference layer.
- the prediction pixel value of the target pixel is generated by executing the interpolation filter process using the derived corresponding reference position as an input.
- the corresponding reference position is derived based on the picture information and the reference area information included in the parameter set.
- the detailed procedure of the corresponding reference position deriving process will be described with reference to FIG.
- the target layer of the corresponding reference position derivation process is layer C (layer C index in VPS is c), and the reference layer of the target layer is layer R (layer R index in VPS is r) .
- the reference layer of the target layer is a lower layer than the target layer, and the layer corresponding to the small intra-VPS index is a lower layer, there is a relationship that r is smaller than c. It holds.
- FIG. 1 is a flowchart of the corresponding reference position derivation process.
- the corresponding reference position derivation process in the inter-layer image prediction process between layer C and layer R when layer C is the target layer is sequentially executed in the following steps S101 to S107.
- a reference region offset is derived from the reference region offset information decoded by the parameter set decoding unit 12 as reference region information using i and j as indexes.
- the derived reference region offset is referred to as RRO [k] [l].
- reference region offsets and symbols corresponding to left, top, right, and bottom that constitute RRO [k] [l] are defined as follows.
- Reference area left offset RRLO [k] [l]
- Reference area offset RRTO [k] [l]
- Reference area right offset RRRO [k] [l]
- Reference area lower offset RRBO [k] [l]
- RRO [i] [j] [k] [l] offsets RRLO [k] [l], RRTO [k] [l], RRRO [k] [l], RRBO [k] [l] Is derived by the following calculation based on the corresponding reference region offset information.
- RRLO [k] [l] (ref_region_left_offset [i] [j] ⁇ 1)
- RRTO [k] [l] (ref_region_top_offset [i] [j] ⁇ 1)
- RRRO [k] [l] (ref_region_right_offset [i] [j] ⁇ 1)
- RRBO [k] [l] (ref_region_bottom_offset [i] [j] ⁇ 1)
- LIdx [i] [j] is an array derived based on the reference region layer index information decoded by the parameter set decoding unit 12, and is the VPS of the layer with which the j-th reference region offset information for layer i is associated Represents the inner index.
- RRO [k] [l] a reference area offset corresponding to a corresponding combination of i and j for a reference area offset (RRO [k] [l]) corresponding to a specific combination of k and l
- RRO [k The value of each offset constituting [] [l] is set to 0.
- a reference region on layer k and a region corresponding to the entire picture on layer k is set as a default value of the reference region that is referred to in inter-layer prediction between layers k and l Is done.
- a value obtained by doubling the syntax value of the corresponding reference area offset information is set as the reference area offset, but the relationship between the reference area offset and the reference area offset information is not limited thereto.
- the relationship between the reference area offset information and the reference area offset is merely an example, and the two may be associated by another appropriate method.
- a reference layer reference region offset (reference layer reference region position) indicating the position and shape of the reference region on the reference layer (layer r) is derived.
- the reference layer reference region offset is composed of four offsets (RL_RRLO, RL_RRTO, RL_RRRO, and RL_RRBO, respectively) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation.
- RL_RRLO RRLO [r] [c]
- RL_RRTO RRTO [r] [c]
- RL_RRRO RRRO [r] [c]
- RL_RRBO RRBO [r] [c] That is, a reference region offset that is a reference region on the reference layer R and is referred to in inter-layer prediction between the reference layer R and the target layer C is set as the value of the reference layer reference region offset.
- the reference region offset (RRO [r] [c]) used for the reference layer reference region offset value is a reference region offset that satisfies the condition “r ⁇ c”.
- the reference layer reference region offset (reference layer reference region position) at the time of performing inter-layer prediction between the target layer and the reference layer is the index within the VPS of the reference layer as the first array index, and the index within the VPS of the target layer. It is set using the reference area offset element as the second array index.
- each element of the reference area offset corresponding to the above is a number that indirectly represents the index r in the VPS of the reference layer (first layer identification information) as the first array index and the index c within the VPS of the target layer.
- a value of reference area position information (RRO [r] [c]) having i (second layer identification information) as the second array index is set.
- the size of the reference area on the reference layer is derived based on the reference layer reference area offset derived in S102 and the reference layer picture size.
- the width (RL_RRW) and height (RL_RRH) of the reference area on the reference layer are derived from the following equations based on the width (RL_PICW) and height (RL_PICH) of the reference layer picture, respectively.
- n is a parameter for aligning the pixel unit of the reference layer picture with the unit of the reference area offset.
- the width of the reference area on the reference layer is obtained by adding the sum of the reference layer reference area left offset and the reference layer reference area right offset to the width of the reference layer picture. Derived. The same applies to the height of the reference area on the reference layer.
- the target layer reference region offset (target layer reference region position) representing the position and shape of the reference region on the target layer (layer c) is derived.
- the target layer reference area offset is composed of four offsets (CL_RRLO, CL_RRTO, CL_RRRO, CL_RRBO in order) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation.
- a reference region offset that is a reference region on the target layer C and is referred to in inter-layer prediction between the target layer C and the reference layer R is set as the value of the target layer reference region offset.
- the reference region offset (RRO [c] [r]) used for the target layer reference region offset value is a reference region offset that satisfies the condition “c> r”.
- the target layer reference region offset (target layer reference region position) when performing inter-layer prediction between the target layer and the reference layer is the index within the VPS of the target layer c as the first array index, and the index within the VPS of the reference layer It is set with reference to a reference area offset element with r as the second array index.
- each element of the reference area offset corresponding to the above is the number j that indirectly represents the index i within the VPS (first layer identification information) of the target layer as the first array index and the index within the VPS of the reference layer in S101.
- a value of reference area position information (RRO [i] [j]) having (second layer identification information) as the second array index is set.
- the size of the target area on the target layer is derived based on the target layer reference area offset derived in S104 and the target layer picture size.
- the width (CL_RRW) and height (CL_RRH) of the reference area on the target layer are derived from the following equations based on the width (CL_PICW) and height (CL_PICH) of the target layer picture, respectively.
- n is the same as n described in S103, and is a parameter for aligning the pixel unit of the target layer picture and the unit of the target region offset.
- the width of the reference layer on the target layer is obtained by adding the sum of the target layer reference region left offset and the target layer reference region right offset to the width of the target layer picture in units. Derived. The same applies to the height of the reference area on the target layer.
- a scale used for inter-layer prediction is derived based on the size of the reference layer reference region derived in S103 and the size of the target layer reference region derived in S105.
- the horizontal scale sx and the vertical scale sy are derived by the following equations, respectively.
- a reference pixel position with 1/16 pixel accuracy is derived.
- the values of the horizontal component (x component) xRef16 and the vertical component (y component) yRef16 of the reference pixel position of 1/16 pixel accuracy on the reference layer corresponding to the pixel position (xP, yP) on the target layer are Each is derived by the following equation.
- addX, addY, deltaX, and deltaY are parameters representing a shift between a pixel on the target layer and a pixel on the reference layer due to upsampling or interlace.
- the reference pixel position is calculated based on the reference layer reference region offset and scale.
- the derived reference pixel position with 1/16 pixel accuracy is set as the corresponding reference position, and the corresponding reference position deriving process is terminated.
- the scale is derived not as a picture size but as a ratio of sizes between areas specified by the reference area. Therefore, even when the picture size of the target layer picture or reference layer picture is changed for the purpose of extracting the attention area, the same scale is set before and after extraction by setting the related syntax value so that the reference area is the same. Can be derived.
- the pixel value at the position corresponding to the corresponding reference position derived in the corresponding reference position deriving process is applied to the decoded pixels of the pixels near the corresponding reference position on the reference layer picture. Generate.
- the hierarchical moving picture decoding apparatus 1 (hierarchical picture decoding apparatus) according to the present embodiment described above includes a parameter set decoding unit 12 that decodes a parameter set, and a prediction image by inter-layer prediction with reference to a decoded pixel of a reference layer picture. Is provided with a predicted image generation unit 1442.
- the parameter set decoding unit 12 decodes the reference region offset, and the predicted image generation unit 1442 calculates a scale based on the reference layer reference region offset and the target layer reference region offset derived from the reference region offset, and calculates the scale.
- the corresponding reference position is derived with reference to perform inter-layer prediction.
- the hierarchical video decoding device 1 decodes the syntax value necessary for deriving the reference region offset from the encoded data, and calculates the corresponding reference position based on the parameter derived based on the syntax value.
- the hierarchical video decoding apparatus 1 can provide a function of maintaining the accuracy of the positional relationship between the upper layer pixel and the lower layer pixel before and after extracting the attention area by using fewer syntax elements than in the conventional art.
- the reference region layer index information included in the reference region information is decoded by the parameter set decoding unit 12 and is associated with the layer i derived based on the reference region layer index information. It was described that the index LIdx [i] [j] of the jth layer can take a value larger than i. This is used to indicate the reference region in the inter-layer processing between the j-th reference region offset information associated with layer i and LIdx [i] [j], and is indicated by LIdx [i] [j] This means that a layer can be specified independently for each layer above layer i.
- the j-th reference region offset information associated with layer i is a reference region on layer i, and is used in inter-layer processing between layer i and any layer above layer i.
- the information indicates the offset of the reference area.
- the reference layer reference region offset derivation process described in S102 of the corresponding reference position derivation process is modified as follows. That is, a target layer reference region offset indicating a reference region on the reference layer (layer R, VPS index r) is derived.
- the reference layer reference region offset is composed of four offsets (RL_RRLO, RL_RRTO, RL_RRRO, and RL_RRBO, respectively) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation.
- RL_RRLO RRLO [r] [r]
- RL_RRTO RRTO [r] [r]
- RL_RRRO RRRO [r] [r]
- RL_RRBO RRBO [r] [r] That is, the reference region offset that is a reference region on the reference layer R and is referred to in inter-layer prediction between the layer R and an arbitrary layer higher than the layer R is set as the target layer reference region offset value. ing.
- the amount of code required for the reference area information can be reduced. This is because, when referring to a common layer (referenced layer), the same reference area on the referenced layer can be specified by a set of reference area offsets. In general, since the reference regions referred to by inter-layer prediction are often the same, it is functionally sufficient for many applications if the same reference region can be set on the referenced layer as described above.
- FIG. 10 is a part of the syntax table that the parameter set decoding unit 12 refers to when decoding the SPS extension (sps_multilayer_extension ()) included in the SPS, and is a part related to the reference region offset.
- the SPS extension includes multiple syntax elements related to the reference area offset. Hereinafter, these are collectively referred to as SPS reference area offset related syntax.
- the syntax table shown in FIG. 10 indicates that the following syntax elements are included in the SPS extension as syntax elements constituting the SPS reference area offset related syntax.
- B1 Number of reference region offsets (num_ref_region_offsets)
- B2 Reference region layer identifier information (ref_region_layer_id [i])
- B3L Reference region left offset information (ref_region_left_offset [i])
- B3T Reference region offset information (ref_region_top_offset [i])
- B3R Reference region right offset information (ref_region_right_offset [i])
- B3B Reference region lower offset information (ref_region_bottom_offset [i])
- B1 is a 6-bit binary representation code
- B2 is a non-negative integer 0th order exponential Golomb code (ue (v)) defined by HEVC
- B3L, B3T, A3R, and A3B are defined by HEVC. It is decoded or encoded using a signed integer 0th-order exponent Golomb code (se (v)).
- the reference area offset number (B1) represents the number of reference area offsets for a layer (hereinafter referred to as a target layer) to which a slice (picture) referring to the SPS extension belongs.
- the number of reference area offsets corresponds to the number of combinations of four offsets (B3L, B3T, B3R, B3B) corresponding to left, top, right, and bottom, respectively.
- the reference area layer identifier information (B2) is information for specifying a layer associated with the i-th reference area offset in the target layer.
- the reference area layer identifier information is a layer identifier (nuh_layer_id) of the i-th layer associated with the target layer.
- the value of ref_region_layer_id [i] is less than or equal to the layer identifier of the target layer. If the value of ref_region_layer_id [i] is less than the layer identifier of the target layer, the reference region offset information associated with index i is the reference region on the target layer and is between the target layer and the layer indicated by ref_region_layer_id [i] The offset of the reference area referred by the prediction between layers is shown.
- the reference region offset information associated with the index i is a reference region on the target layer, and the layer having the target layer as the reference layer, the target layer, The reference area offset referred in the inter-layer prediction during is shown.
- the reference region layer identifier information is preferably included in the SPS extension in the order in which the corresponding layer identifiers are in ascending order. If the layer identifiers are ordered, the reference region layer identifier information with the largest layer identifier can be easily derived by, for example, ref_region_layer_id [num_ref_region_offsets-1].
- the reference area offset information (reference area left offset information (B3L), reference area upper offset information (B3T), reference area right offset information (B3R), reference area lower offset information (B3B)) is the reference area on the target layer. It is information for deriving an offset (reference region offset) representing a reference region that is referred to in inter-layer processing (inter-layer prediction) between the target layer and the i-th layer associated with the target layer. is there.
- the corresponding reference position derivation process in the inter-layer image prediction process between the target layer and the reference layer is sequentially executed in the following steps S201 to S207.
- the reference area on the target layer is the target layer (layer C, layer identifier cLId).
- a reference layer offset that is referred to in inter-layer prediction between the reference layer (layer R, layer identifier rLId).
- the derived reference region offset is referred to as RRO [cLId] [rLId].
- reference region offsets and symbols respectively corresponding to left, top, right, and bottom constituting RRO [cLId] [rLId] are defined as follows.
- RRLO [cLId] [rLId] (ref_region_left_offset [rLId] ⁇ 1)
- RRTO [cLId] [rLId] (ref_region_top_offset [rLId] ⁇ 1)
- RRRO [cLId] [rLId] (ref_region_right_offset [rLId] ⁇ 1)
- RRBO [cLId] [rLId] (ref_region_bottom_offset [rLId] ⁇ 1)
- a reference layer reference area offset indicating a reference area on the reference layer is derived.
- the reference layer reference area offset is composed of four offsets (RL_RRLO, RL_RRTO, RL_RRRO, and RL_RRBO, respectively) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation.
- RL_RRLO RRLO [cLId] [cLId]
- RL_RRTO RRTO [cLId] [cLId]
- RL_RRRO RRRO [cLId] [cLId]
- RL_RRBO RRBO [cLId] [cLId] That is, as a reference layer reference region offset value, a reference region offset which is a reference region on the reference layer and is referred to in inter-layer prediction between the reference layer and a layer higher than the reference layer is set.
- the reference area offset having the first array index as rLIdx can be referred to, and the following may be used.
- RL_RRLO RRLO [rLId] [rLId]
- RL_RRTO RRTO [rLId] [rLId]
- RL_RRRO RRRO [rLId] [rLId]
- RL_RRBO RRBO [rLId] [rLId]
- the size of the reference region on the reference layer is derived based on the reference layer reference region offset derived in S202 and the reference layer picture size.
- the specific process is the same as the process of S103 except that the value of the reference layer reference region offset is the value derived in S202, and the description is omitted.
- a target layer reference area offset indicating a reference area on the target layer is derived.
- the target layer reference area offset is composed of four offsets (CL_RRLO, CL_RRTO, CL_RRRO, CL_RRBO in order) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation.
- CL_RRLO RRLO [cLId] [rLId]
- CL_RRTO RRTO [cLId] [rLId]
- CL_RRRO RRRO [cLId] [rLId]
- CL_RRBO RRBO [cLId] [rLId] That is, as a value of the target layer reference region offset, a reference region on the target layer, which is referred to in inter-layer prediction between the target layer (layer identifier cLId) and the reference layer (layer identifier rLId) Is set.
- the size of the reference area on the reference layer is derived based on the target layer reference area offset derived in S204 and the target layer picture size.
- the specific process is the same as the process of S105 except that the value of the target layer reference region offset is the value derived in S204, and the description is omitted.
- a scale used for inter-layer prediction is derived based on the size of the reference layer reference region derived in S203 and the size of the target layer reference region derived in S205.
- the specific process is the same as the process of S106 except that the size of the reference layer reference area is the value derived in S203, and the size of the target layer reference area is the value derived in S205. Omitted.
- the hierarchical video decoding device performs the attention region extraction.
- a function for deriving an appropriate corresponding reference position can be provided.
- the reference area offset that follows the reference area on the target layer and the reference area on the reference layer depend on whether the value of the reference area layer identifier is less than or equal to the layer identifier of the target layer. It was determined which offset was to be shown. A method of determining a layer to which a reference region offset is associated by another syntax and a derivation method may be used.
- reference region target selection information is sent, and on which layer the reference region is a reference region based on the reference region target selection information in a predetermined method, the reference region Is a reference region to be referenced in the layer processing between which two layers.
- FIG. 11 is a syntax table that is referred to when decoding the SPS reference region offset-related syntax in the parameter set decoding unit 12, and can be used instead of the syntax table described in FIG. .
- the syntax table of FIG. 11 shows that the following syntax elements are included instead of the syntax elements of B2, B3L, B3T, B3R, and B3B included in the syntax table of FIG. ing.
- C2 Reference region target selection information (ref_region_target_info [i])
- C3L Reference region left offset information (ref_region_left_offset [lid1] [lid2])
- C3T Reference region upper offset information (ref_region_top_offset [lid1] [lid2])
- C3R Reference region right offset information (ref_region_right_offset [lid1] [lid2])
- C3B Reference region lower offset information (ref_region_bottom_offset [lid1] [lid2])
- the reference area target selection information is decoded, and then two layer identifiers lid1 and lid2 are derived from the reference area target selection information.
- the reference area offset information decoded thereafter is set to the reference area offset on the layer lid1 and referred to in the inter-layer prediction between the layers lid1 and lid2.
- the reference area target selection information (C2) is information for specifying a layer to which a subsequent reference area offset is associated. Two layer identifiers lid1 and lid2 are derived from the reference area target selection information.
- the layer identifier lid1 (first layer identification information) represents the layer identifier of the layer in which the reference area exists. lid1 is derived by the following procedure. Note that the value of the reference region target selection information (ref_region_target_info [i]) is abbreviated as RRTI [i].
- the layer identifier lid2 (second layer identification information) represents whether the reference region is a reference region that is referred to in inter-layer prediction between the layer lid1 and which layer. lid2 is derived by the following procedure.
- the two layer identifiers lid1 and lid2 are derived from the value of the reference area target selection information by the above procedure.
- the reference area target selection information is smaller than the value of the layer identifier of the target layer (the layer that refers to the SPS)
- the subsequent reference area offset is used as the reference area offset on the target layer.
- the reference area target selection information is greater than or equal to the value of the layer identifier of the target layer
- the subsequent reference area offset is changed from the layer identifier of the target layer to the layer identifier whose value is the current value of the reference area target selection information. Used as a reference area offset.
- mapping from the reference area target selection information to the two layer identifiers may be executed by other methods. However, in that case, it is necessary to perform a derivation process in which any layer identifier becomes the layer identifier of the target layer.
- the layer identifier nuh_layer_id of the target layer is used in the determination formula or subtraction, but the maximum value of the layer identifier may be used instead.
- the code amount of the reference area target selection information increases, but even if rewriting is required so that the SPS is referred to from another layer, it is possible to omit rewriting the value of the reference area target selection information. There is.
- the reference area offset information (reference area left offset information (C3L), reference area upper offset information (C3T), reference area right offset information (C3R), reference area lower offset information (C3B)) is the reference area on the target layer.
- the reference area offset information is recorded in association with the combination of lid1 and lid2.
- Mode 4 Display area use control information
- a method using display area information is known as a method of implicitly determining a reference area when performing inter-layer prediction between two layers.
- the display area information is used for the purpose of designating the reference area, there is a problem that the display area information is restricted from being changed for designating the display area.
- the display area of a specific layer is changed, the inter-layer prediction image of the upper layer that uses the layer as a reference layer changes, so that it is necessary to regenerate the encoded data of the upper layer There is. Therefore, it is preferable that whether or not the display area is used as a reference area can be controlled by another information.
- RRLO [c] [r] (ref_region_left_offset [i] [j] ⁇ 1)
- RRTO [c] [r] (ref_region_top_offset [i] [j] ⁇ 1)
- RRRO [c] [r] (ref_region_right_offset [i] [j] ⁇ 1)
- RRBO [c] [r] (ref_region_bottom_offset [i] [j] ⁇ 1)
- LIdx [i] [j] is an array derived based on the reference region layer index information decoded by the parameter set decoding unit 12, and is the VPS of the layer with which the j-th reference region offset information for layer i is associated Represents the inner index.
- RRO [c] [r] is set to a value corresponding to the display area information of the reference layer R (intra-VPS index r).
- the display area on the reference layer R is set as the default value of the reference area on the target layer C that is referred to in inter-layer prediction between the target layer C and the reference layer R. Is done.
- the offsets that make up RRO [c] [r], RRLO [c] [r], RRTO [c] [r], RRRO [c] [r], RRBO [c] [r] are Based on the display area information of the reference layer R, the following formula is set.
- RRLO [c] [r] conf_win_left_offset [r]
- RRTO [c] [r] conf_win_top_offset [r]
- RRRO [c] [r] conf_win_right_offset [r]
- RRBO [c] [r] conf_win_bottom_offset [r]
- conf_win_left_offset [r] conf_win_top_offset [r]
- a flag (display area use control flag) indicating whether the reference area is set as a display area can be included in the parameter set, decoded by the parameter set decoding unit 12, and used for the corresponding reference position derivation process in the predicted image generation unit 1442. Any of the following can be used as the display area use control flag.
- Display area use control flag A When the value of the display area use control flag A is 1, it indicates that the display area is used as a reference area in inter-layer prediction. When the value of the flag is 0, it indicates that the entire picture is used as a reference area in inter-layer prediction.
- Display area use control flag B When the value of the display area use control flag B is 1, it indicates that the display area is used as a reference area in inter-layer prediction. When the value of the flag is 0, it indicates that the reference area specified by the reference area offset information is set as the reference area.
- the display area use control flag A (use_conf_win_il_flag) is included in the SPS extension and is decoded or encoded as a 1-bit flag.
- the reference layer reference area is set by the following procedure.
- each offset value included in the display area information for specifying the display area of the reference layer is set as each offset of the reference layer reference area offset.
- Using the display area use control flag as described above can control whether or not to use the display area as a reference area. Therefore, even when the display area is used for a different purpose from the reference area in the inter-layer prediction, the reference area in the inter-layer prediction can be avoided.
- reference area position information (reference area offset information) is directly or indirectly associated with the two layer identification information. That is, a certain set of reference area position information is information indicating the shape and position of the reference area on the layer A and referred to in the inter-layer processing between the layer A and the layer B.
- the reference area position information does not necessarily need to be included in the encoded data for all combinations of layer A and layer B, and the above-described layer A so that only the reference area position information relating to the required combination can be decoded from the encoded data.
- layer identification information for specifying layer B may be used.
- the possible range of layer B may be limited to the direct reference layer (direct reference layer) of layer A.
- an index for the list of direct reference layers for layer A can be used as the layer identification information, thereby reducing the code amount of the layer identification information.
- the range of layer B may be limited to a direct reference layer (active reference layer) that is effective in the picture including the slice header.
- the layer identification information of layer B can use an index for a list of direct reference layers that are valid in the picture including the slice header.
- FIG. 14 is a functional block diagram showing a schematic configuration of the hierarchical video encoding device 2.
- the hierarchical video encoding device 2 encodes the input image PIN # T of the target layer with reference to the reference layer encoded data DATA # R to generate hierarchical encoded data DATA of the target layer. It is assumed that the reference layer encoded data DATA # R has been encoded in the hierarchical video encoding apparatus corresponding to the reference layer.
- the hierarchical video encoding device 2 includes a NAL multiplexing unit 21, a parameter set encoding unit 22, a tile setting unit 23, a slice encoding unit 24, a decoded picture management unit 16, and a base decoding unit. 15.
- the NAL multiplexer 21 generates NAL-multiplexed hierarchical moving image encoded data DATA by storing the input target layer encoded data DATA # T and reference layer encoded data DATA # R in the NAL unit. And output to the outside.
- the parameter set encoding unit 22 sets parameter sets (VPS, SPS, and PPS) used for encoding the input image based on the input tile information and the input image, and sets the target layer encoded data DATA #. Packetized as a part of T in the format of VCL NAL and supplied to the NAL multiplexer 21.
- the parameter set encoded by the parameter set encoding unit 22 includes at least the picture information, the display area information, and the reference area information described in relation to the hierarchical video decoding device 1.
- the tile setting unit 23 sets the tile information of a picture based on the input image, and supplies it to the parameter set encoding unit 22 and the slice encoding unit 24. For example, tile information indicating that the picture size is divided into M ⁇ N tiles is set.
- M and N are arbitrary positive integers.
- the slice encoding unit 24 is a part of the input image corresponding to the slice constituting the picture. Is encoded to generate encoded data of the part, and the encoded data is supplied to the NAL multiplexer 21 as a part of the target layer encoded data DATA # T. Detailed description of the slice encoding unit 24 will be described later.
- the decoded picture management unit 16 is the same component as the decoded picture management unit 16 included in the hierarchical video decoding device 1 already described. However, since the decoded picture management unit 16 included in the hierarchical video encoding device 2 does not need to output the picture recorded in the internal DPB as an output picture, the output can be omitted. Note that the description described as “decoding” in the description of the decoded picture management unit 16 of the hierarchical video decoding device 1 is also applied to the decoded picture management unit 16 of the hierarchical video encoding device 2 by replacing “coding”. it can.
- the base decoding unit 15 is the same constituent element as the base decoding unit 15 included in the hierarchical video decoding device 1 already described, and detailed description thereof is omitted.
- FIG. 15 is a functional block diagram showing a schematic configuration of the slice encoding unit 24.
- the slice encoding unit 24 includes a slice header setting unit 241, a slice position setting unit 242, and a CTU encoding unit 244.
- the CTU encoding unit 244 includes a prediction residual encoding unit 2441, a prediction image encoding unit 2442, and a CTU decoded image generation unit 1443 therein.
- the slice header setting unit 241 generates a slice header used for encoding an input image input in units of slices based on the input parameter set and slice position information.
- the generated slice header is output as a part of the slice encoded data, and is supplied to the CTU encoding unit 244 together with the input image.
- the slice header generated by the slice header setting unit 241 includes at least SH slice position information.
- the slice position setting unit 242 determines the slice position in the picture based on the input tile information and supplies the slice position to the slice header setting unit 241.
- the CTU encoding unit 244 encodes an input image (target slice portion) in units of CTUs based on the input parameter set and slice header, and generates slice data and a decoded image (decoded picture) related to the target slice. Output.
- the CTU encoding is performed by a prediction image encoding unit 2442, a prediction residual encoding unit 2441, and a CTU decoded image generation unit.
- the prediction image encoding unit 2442 determines a prediction method and prediction parameters for the target CTU included in the target slice, generates a prediction image based on the determined prediction method, and generates a prediction residual encoding unit 2441 and a CTU decoded image. Output to the unit 1443.
- Information on the prediction method and prediction parameters is variable-length encoded as prediction information (PT information) and output as a part of slice data included in the slice encoded data.
- the prediction methods that can be selected by the prediction image encoding unit 2442 include at least inter-layer image prediction.
- the predicted image encoding unit 2442 When inter-layer image prediction is selected as the prediction method, the predicted image encoding unit 2442 performs a corresponding reference position derivation process, determines a reference layer pixel position corresponding to the prediction target pixel, and performs interpolation based on the position A predicted pixel value is determined by processing.
- the corresponding reference position derivation process each process described for the predicted image generation unit 1442 of the hierarchical video decoding device 1 can be applied. For example, the process of deriving the corresponding reference position based on the reference area information included in the parameter set described with reference to FIG. 1 is applied.
- the prediction residual coding unit 2441 converts a quantized transform coefficient (TT information) obtained by transforming and quantizing a difference image between an input image and a predicted image into one piece of slice data included in the slice coded data. Output as part.
- the prediction residual is restored by applying inverse transformation / inverse quantization to the quantized transform coefficient, and the restored prediction residual is output to the CTU decoded image generation unit 1443.
- the CTU decoded image generation unit 1443 has the same function as the component of the same name of the hierarchical moving image decoding apparatus 1, the same reference numerals are given and description thereof is omitted.
- the hierarchical video encoding device 2 (hierarchical image encoding device) according to the present embodiment described above includes a parameter set encoding unit 22 that encodes a parameter set and an inter-layer referring to a decoded pixel of a reference layer picture.
- a prediction image encoding unit 2442 that generates a prediction image by prediction is provided.
- the parameter set encoding unit 22 encodes the reference region information, and the predictive image encoding unit 2442 uses the target layer reference region offset derived from the reference region information and the value of the reference layer reference region information to generate pixels on the target layer.
- the corresponding reference position for is derived.
- the hierarchical moving image encoding device 2 is the same before and after the conversion even when converting the hierarchical encoded data for the purpose represented by the attention region extraction by appropriately setting the reference region information.
- the same corresponding reference position can be derived for the pixels of the target layer corresponding to the position.
- both the target area and the reference area of the reference layer used for the scale calculation are derived from the reference area information, a function for deriving the same corresponding reference position before and after conversion using fewer syntax elements than in the past. realizable.
- FIG. 16 is a functional block diagram showing a schematic configuration of the hierarchically encoded data conversion device 3.
- the hierarchical encoded data conversion device 3 converts the input hierarchical encoded data DATA to generate hierarchical encoded data DATA-ROI related to the input attention area information.
- the hierarchically encoded data DATA is hierarchically encoded data generated by the hierarchical moving image encoding device 2. Also, by inputting the hierarchically encoded data DATA-ROI to the hierarchical video decoding device 1, it is possible to reproduce the upper layer video related to the attention area information.
- the hierarchical encoded data conversion device 3 includes a NAL demultiplexing unit 11, a NAL multiplexing unit 21, a parameter set decoding unit 12, a tile setting unit 13, a parameter set modification unit 32, and a NAL selection unit 34. including.
- Each of the NAL demultiplexing unit 11, the parameter set decoding unit 12, and the tile setting unit 13 has the same function as the component of the same name included in the hierarchical video decoding device 1, and therefore, the same reference numerals are given and description thereof is omitted. To do.
- the NAL multiplexing unit 21 has the same function as the component of the same name included in the hierarchical video encoding device 2, the same reference numeral is given and the description is omitted.
- the parameter set correction unit 32 corrects and outputs the input parameter set information based on the input attention area information and tile information.
- the parameter set correction unit 32 generally corrects picture information, display area information, inter-layer pixel correspondence information, reference area information, and PPS tile information included in the parameter set.
- the attention area information is a partial area of a picture specified by a user (for example, a viewer of a reproduction moving image) in a picture constituting the moving image.
- the attention area information is specified by a rectangular area, for example.
- an offset of a position from the corresponding side (upper side, lower side, left side, or right side) of the entire picture of the upper side, the lower side, the left side, and the right side of the rectangle representing the target region can be designated as the attention region information.
- an area having a shape other than a rectangle for example, a circle, a polygon, or an area indicating an object extracted by object extraction
- a rectangular attention area is assumed below. To do.
- a rectangle with the smallest area including the region of interest can be regarded as the region of interest in the following description.
- FIG. 17 is a diagram illustrating the relationship among pictures, attention areas, and tiles in hierarchically encoded data before and after conversion.
- hierarchical encoded data pre-conversion hierarchical encoded data
- hierarchical encoded data post-conversion hierarchical encoded data
- the enhancement layer of the pre-conversion hierarchical encoded data is data corresponding to the pre-conversion EL picture
- the base layer is data corresponding to the pre-conversion BL picture
- the enhancement layer of the post-conversion hierarchically encoded data is data corresponding to the post-conversion EL picture
- the base layer is data corresponding to the post-conversion BL picture.
- the hierarchical encoded data conversion apparatus 3 removes tiles that do not have a region of interest and an overlapping region on the enhancement layer from the input pre-conversion hierarchically encoded data, and corrects the related parameter set. Then, the encoded hierarchical encoded data is generated.
- the hierarchical moving image decoding apparatus can generate a decoded image related to a region of interest with the converted hierarchical encoded data as an input.
- the parameter set correction unit 32 refers to the input attention area information and tile information, and sets the PPS tile information of the enhancement layer so that only a tile (extraction target tile) in which a part of the corresponding area overlaps the attention area is included. Update. Update the PPS tile information of the enhancement layer based on the information of the extraction target tile. First, when there is one extraction target tile, tiles_enabled_flag is corrected to 0. If there are two or more extraction target tiles, the correction process can be omitted. Next, the number of tile rows (num_tile_columns_minus1) and the number of tile columns (num_tile_rows_minus1) are corrected based on the number of extraction target tiles included in the horizontal and vertical directions of the picture.
- the bit string corresponding to the syntax related to the width of the tile column not including the extraction target tile and the height of the tile row not including the extraction target tile is set as a parameter. Remove from set.
- the parameter set correction unit 32 removes tiles unnecessary for decoding the tile extracted in the enhancement layer from the tiles included in the pre-conversion BL picture.
- the base layer PPS tile information is updated so that the region on the enhancement layer corresponding to the converted BL picture (the converted reference layer corresponding region) includes the converted EL picture.
- the parameter set correction unit 32 corrects the picture information with the region corresponding to the set of extraction target tiles of the enhancement layer as the converted EL picture size.
- the width and height of the converted EL picture are set as values of pic_width_in_luma_samples and pic_height_in_luma_samples of the enhancement layer SPS, respectively.
- the parameter set correction unit 32 corrects the picture information with the region corresponding to the set of base layer extraction target tiles as the converted BL picture size.
- the width and height of the BL picture after conversion are set as pic_width_in_luma_samples and pic_height_in_luma_samples values of the base layer SPS, respectively.
- the parameter set correction unit 32 corrects the inter-layer pixel correspondence information included in the parameter set based on the change in the picture size. Specifically, the reference area offset included in the inter-layer pixel correspondence information is corrected.
- the layer identifier of the enhancement layer is e and the layer identifier of the base layer is b.
- Reference region left offset ref_region_right_offset [e] [k] for the reference region on layer e included in the reference region offset which is referred to in inter-layer prediction between layers e and b
- the reference area offset values corresponding to the other upper, right, and lower are similarly set.
- the parameter set correction unit 32 rewrites the display area information of the SPS included in the input parameter set so as to match the attention area indicated by the input attention area information.
- the display area information is rewritten by, for example, the following steps S301 to S303.
- Each offset of the display area offset is set to the offset value of the position of each side of the rectangle representing the attention area with the corresponding side of the picture. For example, the position offset of the upper side of the attention area with respect to the upper side of the picture is set to the value of the display area upper offset (conf_win_top_offset). If the value of the display area flag before rewriting is 1, the original attention area offset value is overwritten using the attention area offset value set above. When the value of the display area flag before rewriting is 1, the set attention area offset is inserted immediately after the SPS display area flag.
- the NAL selection unit 34 selects an input video encoding layer NAL (VCL NAL) based on the input attention area information and tile information.
- VCL NAL input video encoding layer NAL
- the selected VCL NAL is sequentially output to the NAL multiplexing unit 21, and the unselected VCLALNAL is discarded.
- the VCL NAL selected by the NAL selection unit 34 is a VCL NAL including a slice header and slice data related to a slice included in the extraction target tile.
- the NAL selection unit 34 determines whether the slice is included in the extraction target tile from the slice address and tile information included in the slice header. If included, the NAL selection unit 34 selects the VCL NAL including the slice. If not, discard the VCL NAL.
- Hierarchical coded data conversion process flow The hierarchical encoded data conversion process by the hierarchical encoded data conversion device 3 is realized by sequentially executing the procedures shown in S501 to S506.
- the NAL demultiplexing unit 11 demultiplexes the input hierarchical encoded data DATA.
- the part related to the parameter set (non-VCL NAL) is output to the parameter decoding unit 12, and the video coding layer NAL (VCL NAL) that is the part related to the slice layer (slice header, slice data) is output to the NAL selection unit 34. .
- the parameter set decoding unit 12 decodes the parameter set (VPS, SPS, PPS) from the input non-VCL NAL and outputs it to the parameter set correction unit 32 and the tile setting unit 13.
- the tile setting unit 13 derives tile information from the input parameter set, and outputs the tile information to the parameter set correction unit 32 and the NAL selection unit 34.
- the parameter set correction unit 32 corrects and outputs the parameter set input based on the input attention area information and tile information.
- the NAL selection unit 34 selects a part of the input VCL NAL based on the input tile information and attention area information, and outputs the selected VCL NAL to the NAL multiplexing unit 21.
- the NAL multiplexing unit 21 receives the input reference layer encoded data DATA # R as the encoded data of the target layer after correction of the corrected parameter set, the corrected slice header, and the slice data. And output to the outside as hierarchically encoded data DATA-ROI.
- the hierarchically encoded data conversion device 3 based on attention area information, a part of the encoded data (VCL NAL) of the video layer included in the encoded data of the target layer (upper layer).
- a NAL selection unit 34 to be corrected and a parameter set correction unit 32 are provided.
- the NAL selection unit 34 selects, as an extraction target tile, a tile having an area overlapping with the attention area based on the attention area indicated by the attention area information, and the code of the video layer related to the slice included in the selected extraction target tile
- the encoded data is included in the converted hierarchical encoded data.
- the parameter set correction unit 32 corrects the picture information, the PPS tile information, the display information, and the reference area information based on the attention area information and the tile information.
- the input hierarchical encoded data is converted, and the VCL NAL related to the extraction target tile (the tile having the area overlapping the attention area) is extracted in the upper layer.
- the hierarchical encoded data after conversion can be configured. Since the VCL NAL related to the tile having no overlap area with the attention area is discarded, the code amount of the hierarchically encoded data after conversion is smaller than that of the hierarchically encoded data before conversion.
- the hierarchically encoded data after conversion since the parameter set is modified to match picture information, PPS tile information, and display information in accordance with the extraction target tile, the hierarchically encoded data after conversion is a hierarchical moving image.
- the decoding device can be decoded by the decoding device, and a decoded picture related to the region of interest can be displayed.
- the reference area information is corrected, the correspondence relationship between the scale, the upper layer pixel, and the reference layer pixel is maintained in the hierarchically encoded data before and after conversion. Therefore, the prediction image of inter-layer prediction generated from the encoded data before conversion and the prediction image of inter-layer prediction generated from the encoded data after conversion can be maintained at the same level.
- a system that displays attention area information can be configured by combining the above-described hierarchical moving picture decoding apparatus 1, hierarchical moving picture encoding apparatus 2, and hierarchical encoded data conversion apparatus 3.
- FIG. 18 is a block diagram illustrating a configuration of a region of interest display system that is a combination of the hierarchical video decoding device 1, the hierarchical video encoding device 2, and the hierarchical encoded data conversion device 3.
- the attention area display system SYS is generally provided by hierarchically encoding and storing input images having different qualities, and converting and providing the hierarchically encoded data accumulated according to attention area information from the user, By decoding the converted hierarchically encoded data, a high-quality reproduced image related to the region of interest (ROI) is displayed.
- ROI region of interest
- the attention area display system SYS includes a hierarchical video encoding unit SYS1A, a hierarchical video encoding unit SYS1B, a hierarchical encoded data storage unit SYS2, a hierarchical encoded data conversion unit SYS3, and a hierarchical video decoding.
- the unit SYS4, the display unit SYS6, and the ROI notification unit SYS8 are included as components.
- the hierarchical video encoding device 2 described above can be used for the hierarchical video encoding units SYS1A and SYS1B.
- the hierarchically encoded data storage unit SYS2 stores hierarchically encoded data and supplies the hierarchically encoded data as required.
- a computer having a recording medium (memory, hard disk, optical disk) can be used as the hierarchically encoded data storage unit SYS2.
- the hierarchical encoded data conversion unit 3 can be used as the hierarchical encoded data conversion unit SYS3. Therefore, the hierarchical encoded data conversion unit SYS3 can set the reference area information included in the input hierarchical encoded data to an appropriate value according to the input attention area.
- the hierarchical video decoding device 1 described above can be used for the hierarchical video decoding unit SYS4. Therefore, the hierarchical video decoding unit SYS4 can decode the reference area information from the parameter set and perform inter-layer prediction with reference to the reference area information.
- the display unit SYS6 displays the decoded image at a predetermined display position in a predetermined display area.
- the display area is a television screen, and the display position is the whole.
- the display unit SYS6 preferably displays the input decoded image in an enlarged or reduced size so as to match the size of the display area.
- the ROI notification unit SYS8 notifies attention area information designated by the user by a predetermined method. For example, the user can inform the ROI notification unit of the attention area by designating an area corresponding to the attention area on the display area where the entire display image is displayed. Note that the ROI notification unit SYS8 notifies information indicating that there is no attention area as attention area information when there is no user designation.
- Processing by the attention area display system can be divided into hierarchical encoded data generation and accumulation processing and attention area data generation and reproduction processing.
- hierarchical encoded data generation and accumulation process hierarchical encoded data is generated and stored from different quality input images.
- the hierarchically encoded data generation / accumulation process is executed in the sequence from T101 to T103.
- the hierarchical moving image encoding unit SYS1B encodes the input low-quality input image and supplies the generated hierarchical encoded data to the hierarchical moving image encoding unit SYS1A. That is, the hierarchical moving image encoding unit SYS1B generates and outputs hierarchical encoded data used as a reference layer (lower layer) in the hierarchical moving image encoding unit SYS1A from the input image.
- the hierarchical moving image encoding unit SYS1A encodes the input high-quality input image using the input hierarchical encoded data as encoded data of the reference layer, generates hierarchical encoded data, and generates a hierarchical code Output to the digitized data storage unit SYS2.
- the hierarchically encoded data storage unit SYS2 attaches an appropriate index to the input hierarchically encoded data and records it on an internal recording medium.
- the hierarchically encoded data is read from the hierarchically encoded data storage unit SYS2, converted into hierarchically encoded data corresponding to the attention area, and the converted hierarchically encoded data is decoded and reproduced and displayed. .
- the attention area data generation / reproduction processing is executed in the following steps T201 to T205.
- T201 The hierarchically encoded data related to the moving image selected by the user is supplied from the hierarchically encoded data storage unit SYS2 to the hierarchically encoded data conversion unit SYS3.
- the ROI notification unit SYS8 notifies the hierarchical encoded data conversion unit SYS3 of the attention area information designated by the user.
- the hierarchical encoded data conversion unit SYS3 converts the input hierarchical encoded data based on the input attention area information, and outputs the converted hierarchical encoded data to the hierarchical video decoding unit SYS4.
- the hierarchical video decoding unit SYS4 decodes the input hierarchical video encoded data (after conversion), and outputs the obtained decoded picture of the upper layer to the display unit SYS6.
- the display unit SYS6 displays the input decoded image.
- the attention area display system SYS includes an attention area notification section (ROI notification section SYS8) that supplies attention area information, and converts the hierarchically encoded data based on the attention area information and after conversion.
- ROI notification section SYS8 that supplies attention area information, and converts the hierarchically encoded data based on the attention area information and after conversion.
- a hierarchical encoded data conversion unit SYS3 that generates hierarchical encoded data
- a hierarchical moving image decoding unit SYS4 that decodes the converted hierarchical encoded data and outputs decoded pictures of an upper layer and a lower layer
- a display unit SYS6 I have.
- the decoded picture of the area specified by the attention area information can be displayed.
- the decoded picture of the region specified by the attention region information is subjected to inter-layer image prediction using the scale derived from the encoded data of the upper layer of the hierarchical encoded data and the corresponding reference position based on the reference region information. Since it is used and decoded, the image quality is high.
- the hierarchically encoded data converted based on the attention area has a smaller code amount than the hierarchically encoded data before conversion. Therefore, by using the attention area display system SYS described above, it is possible to reproduce a decoded picture with high image quality related to the attention area while reducing the bandwidth required for transferring the hierarchically encoded data.
- Hierarchical video decoding device 3 A schematic configuration of the hierarchical video decoding device 3 will be described.
- Hierarchical video decoding device 3 is configured by replacing parameter set decoding unit 12 with parameter set decoding unit 12A and slice decoding unit 14 with slice decoding unit 14A in hierarchical video decoding device 1 described with reference to FIG. It is. That is, the hierarchical video decoding device 3 includes a NAL demultiplexing unit 11, a parameter set decoding unit 12A, a tile setting unit 13, a slice decoding unit 14A, a base decoding unit 15, and a decoded picture management unit 16.
- parameter decoding unit 12A and slice decoding unit 14A which are new components, will be described.
- the parameter set decoding unit 12A decodes the inter-layer position correspondence information from the input target layer encoded data.
- the inter-layer position correspondence information schematically indicates the positional relationship between corresponding areas of the target layer and the reference layer. For example, when an object (object A) in a picture of the target layer and a picture of the reference layer is included, an area corresponding to the object A on the picture of the target layer and an area corresponding to the object A on the picture of the reference layer , Corresponding to the regions corresponding to the target layer and the reference layer.
- the inter-layer position correspondence information may not necessarily be information that accurately indicates the positional relationship between the corresponding regions of the target layer and the reference layer, but generally, in order to improve the accuracy of inter-layer prediction.
- the correct positional relationship between the corresponding layers of the target layer and the reference layer is shown.
- the inter-layer position correspondence information includes expanded reference layer offset information, reference layer offset information, and inter-layer phase information.
- Inter-layer position correspondence information Extended reference layer offset information
- the information for specifying the enlarged reference layer offset is included in the inter-layer position correspondence information.
- a plurality of extended reference layer offsets can be included in the encoded data, and each extended reference layer offset is composed of four offsets corresponding to left, upper, right, and lower, respectively, the target picture, and the reference It is associated with a combination of two pictures.
- information defining the corresponding expanded reference layer offset for each combination of the target picture and a specific reference picture is included in the inter-layer position correspondence information. Note that it is not always necessary to include information defining the reference layer offset for all combinations of the target picture and the reference picture, and a predetermined value can be used by omitting some combinations under specific conditions. .
- FIG. 21 is a diagram illustrating the relationship between the target layer picture, the reference layer picture, and the inter-layer pixel corresponding offset.
- FIG. 21 (a) shows an example in which the entire picture of the reference layer corresponds to a part of the picture of the target layer.
- an area on the target layer corresponding to the entire reference layer picture (target layer corresponding area) is included in the target layer picture.
- FIG. 21B illustrates an example in which a part of the reference layer picture corresponds to the entire picture of the target layer.
- the target layer picture is included inside the reference layer corresponding area. Note that the entire target layer picture includes an offset.
- the enlarged reference layer left offset represents the offset of the left side of the reference layer corresponding area with respect to the left side of the target layer picture.
- SRL left offset When the SRL left offset is larger than 0, it indicates that the left side of the reference layer corresponding area is located on the right side of the left side of the target layer picture.
- the enlarged reference layer offset (in the figure, SRL offset) represents the offset of the upper side of the reference layer corresponding area with respect to the upper side of the target layer picture.
- SRL offset represents the offset of the upper side of the reference layer corresponding area with respect to the upper side of the target layer picture.
- the enlarged reference layer right offset represents the offset of the right side of the reference layer corresponding area to the right side of the target layer picture.
- SRL right offset When the SRL right offset is larger than 0, it indicates that the right side of the reference layer corresponding region is located on the left side of the right side of the target layer picture.
- the enlarged reference layer lower offset represents an offset of the lower side of the reference layer corresponding area to the lower side of the target layer picture.
- SRL lower offset When the SRL lower offset is larger than 0, it indicates that the lower side of the reference layer corresponding region is located above the lower side of the target layer picture.
- the inter-layer position correspondence information includes information that defines a reference layer offset (reference layer offset information).
- a plurality of reference layer offsets can be included in the encoded data, and each reference layer offset is composed of four offsets corresponding to left, top, right, and bottom, respectively, and the target picture and reference picture Associated with a combination of two pictures.
- a corresponding reference layer offset can be included in the inter-layer pixel correspondence information.
- the reference layer left offset represents the offset of the left side of the reference layer standard area with respect to the left side of the reference layer picture.
- the offset value is greater than 0, it means that the left side of the reference layer base area is located on the right side of the left side of the reference layer picture.
- the reference layer upper offset represents an offset of the upper side of the reference layer standard area with respect to the upper side of the reference layer picture. When the offset value is greater than 0, it indicates that the upper side of the reference layer base area is located below the upper side of the reference layer picture.
- the reference layer right offset represents an offset of the right side of the reference layer base area with respect to the right side of the reference layer picture.
- the offset value is larger than 0, it means that the right side of the reference layer base area is located on the left side of the right side of the reference layer picture.
- the reference layer lower offset represents an offset of the lower side of the reference layer standard area with respect to the lower side of the reference layer picture. When the offset value is greater than 0, this indicates that the lower side of the reference layer base area is located above the lower side of the reference layer picture.
- Inter-layer position correspondence information phase information between layers
- the information on the phase difference between layers is included in the phase information between layers.
- a plurality of inter-phase phases can be included in the encoded data, and each inter-phase phase offset is composed of four offsets in the luminance horizontal direction, luminance vertical direction, color difference horizontal direction, and color difference vertical direction.
- the picture and the reference picture are associated with a combination of two pictures. In other words, for each combination of the target picture and a specific reference picture, the corresponding inter-layer phase offset may be included in the inter-layer pixel correspondence information.
- the inter-layer phase luminance horizontal offset represents a horizontal shift of less than one pixel between the upper left pixel of the reference layer corresponding region and the pixel on the reference layer corresponding to the pixel on the luminance image of the target layer.
- the inter-layer phase luminance vertical offset represents a vertical shift of less than one pixel between the upper left pixel of the reference layer corresponding region and the pixel on the reference layer corresponding to the pixel on the luminance image of the target layer.
- the inter-layer phase color difference horizontal offset represents a horizontal shift of less than one pixel between the upper left pixel of the reference layer corresponding region and the pixel on the reference layer corresponding to the pixel on the color difference image of the target layer.
- the inter-layer phase color difference vertical offset represents a vertical shift of less than one pixel between the upper left pixel of the reference layer corresponding region and the pixel on the reference layer corresponding to the pixel on the color difference image of the target layer.
- the expanded reference layer offset information, the reference layer offset information, and the inter-layer phase offset information are included in, for example, a PPS extension (pps_estension) that is a part of the PPS of the higher layer, and follow the syntax table shown in FIG. Decrypted.
- FIG. 22 is a part of a syntax table that the parameter set decoding unit 12A refers to when performing PPS decoding, and is a part related to inter-layer pixel correspondence information.
- the extended reference layer offset information decoded from the PPS includes the number of extended reference layer offsets (num_scaled_ref_layer_offsets) included in the PPS extension.
- the number of extended reference layer offsets indicated by the number of extended reference layer offsets is included together with the reference layer identifier.
- the reference layer identifier (ref_layer_id) is an identifier indicating a reference layer to which a subsequent extended reference layer offset is associated.
- the inter-layer pixel corresponding offset includes an enlarged reference layer left offset (scaled_ref_layer_left_offset), an enlarged reference layer upper offset (scaled_ref_layer_top_offset), an enlarged reference layer right offset (scaled_ref_layer_right_offset), and an enlarged reference layer lower offset (scaled_ref_layer_bottom_offset).
- scaled_ref_layer_left_offset, scaled_ref_layer_top_offset, scaled_ref_layer_right_offset, and scaled_ref_layer_bottom_offset are collectively referred to as an enlarged reference layer offset syntax.
- the array reference “[ref_layer_id [i]]” is added after the expanded reference layer offset syntax because the expanded reference layer offset syntax is indicated by ref_layer_id [i]. Indicates that it is associated with a reference layer.
- the parameter set decoding unit 12A decodes the value of the extended reference layer offset syntax by decoding the code of the signed exponent Golomb code (se (v)) from the encoded data.
- the signed exponent Golomb code specifically, a code with the same name defined in HEVC can be used.
- the value of the decoded expanded reference layer offset syntax is in units of chrominance pixels of the target picture.
- the value of each extended reference layer offset corresponding to the reference layer is set to 0.
- the reference layer offset information decoded from the PPS includes a reference layer offset information presence / absence flag (ref_layer_offset_present_flag) that is a flag indicating whether or not the reference layer offset information associated with the reference layer indicated by the layer identifier “ref_layer_id [i]” is present in the PPS. )including.
- a reference layer offset information presence / absence flag (ref_layer_offset_present_flag [i]) having an array index of “i” is 1, each reference associated with the layer (target layer) to which the slice that refers to the PPS belongs to the PPS and the reference layer Layer offset is included.
- a reference layer left offset (ref_layer_left_offset), a reference layer upper offset (ref_layer_top_offset), a reference layer right offset (ref_layer_right_offset), and a reference layer lower offset (ref_layer_bottom_offset) are included.
- ref_layer_left_offset, ref_layer_top_offset, ref_layer_right_offset, and ref_layer_bottom_offset are collectively referred to as a reference layer offset syntax.
- the parameter set decoding unit 12A decodes the value of the reference layer offset syntax by decoding the code of the signed exponential Golomb code (se (v)) from the encoded data.
- the value of the decoded reference layer offset syntax is based on the chrominance pixel of the reference layer picture associated with the reference layer offset syntax through the reference layer identifier.
- the reference layer offset syntax corresponding to a specific reference layer does not exist in the encoded data, the value of each reference layer offset corresponding to the reference layer is set to 0.
- the inter-layer phase offset information is a flag indicating whether or not the inter-layer phase offset information associated with the reference layer indicated by the layer identifier “ref_layer_id [i]” is present in the PPS.
- a flag (resample_phase_present_flag) is included.
- phase_hor_luma an inter-layer phase luminance horizontal offset
- phase_ver__luma an inter-layer phase luminance vertical offset
- phase_hor_chroma an inter-layer phase color difference horizontal offset
- phase_ver_chroma an inter-layer phase color difference vertical offset
- phase_ver_chroma an inter-layer phase color difference vertical offset
- the parameter set decoding unit 12A decodes the code of the signless exponential Golomb code (ue (v)) from the encoded data, thereby decoding the value of the inter-layer phase offset syntax. Note that when the inter-layer phase offset syntax corresponding to a specific reference layer does not exist in the encoded data, the value of the inter-layer phase offset corresponding to the reference layer is set to 0.
- the inter-layer position correspondence information decoded from the PPS in the parameter set decoding unit 12A described with reference to the syntax table of FIG. 22 has the following characteristics.
- a reference layer identifier for selecting one of the reference layers is included in the PPS, and an extended reference layer offset syntax, a reference layer offset syntax, and an inter-layer phase offset syntax associated with the same reference layer identifier are It is continuously included in the encoded data.
- the PPS includes ref_layer_id [i] and has the same array index “[ref_layer_id [i]]”, the expanded reference layer offset syntax, the reference layer offset syntax, and the inter-layer phase offset syntax.
- the tax is continuously included in the encoded data.
- presence / absence flags of each offset information may be included, and some offsets may be included depending on the value of the presence / absence flag. It may be omitted. Also, the order in the encoded data of the expanded reference layer offset syntax, the reference layer offset syntax, and the inter-layer phase offset syntax associated with the same reference layer identifier does not necessarily have to be the order described above. It may be replaced.
- the extended reference layer offset syntax, the reference layer offset syntax, and the inter-layer phase offset syntax associated with the same reference layer identifier are continuously included in the encoded data, so that each offset type In each case, it is possible to decode the inter-layer position correspondence information from the encoded data with a small code amount, as compared with the case of decoding the number of offsets and the identifier of the associated reference layer.
- the parameter set decoding unit 12A decodes a first reference layer identifier (ref_layer_id [i]) indicating a specific reference layer, and the extended reference layer offset syntax associated with the first reference layer identifier. Less decoding by successively decoding the reference layer offset syntax associated with the first reference layer identifier and the inter-layer phase offset syntax associated with the first reference layer identifier from encoded data Inter-layer position information can be decoded from encoded data of a code amount.
- the inter-layer position correspondence information includes a reference layer offset information presence / absence flag, it is possible to reduce the amount of codes when it is not necessary to explicitly transmit the reference layer offset. Since the reference layer offset has a characteristic that the ratio of applying the default value is high, the average code amount can be reduced by using the inter-reference layer offset information presence / absence flag.
- the inter-layer position correspondence information includes an inter-layer phase offset information presence / absence flag, it is possible to reduce the amount of codes when it is not necessary to explicitly transmit the inter-layer phase offset. Since an inter-layer phase offset has a characteristic that a ratio of applying a default value is high, an average code amount can be reduced by using an inter-layer phase offset information presence / absence flag.
- the slice decoding unit 14A has the same configuration and function as the slice decoding unit 14 described with reference to FIG. The difference between the two is that the slice decoding unit 14A includes a predicted image generation unit 1443A instead of the predicted image generation unit 1442 included in the slice decoding unit 14.
- the predicted image generation unit 1443A executes the corresponding reference position derivation process according to the following steps S401 to S409 based on the inter-layer position correspondence information when executing the predicted image generation process when the inter-layer prediction is selected.
- (S401) It is determined whether the target of predicted image generation processing is a luminance pixel or a color difference pixel. If the target is a luminance pixel, S402 is executed. If the target is a color difference pixel, S404 is executed.
- the extended reference layer offset is derived from the value of the extended reference layer offset syntax corresponding to the reference layer decoded by the parameter set decoding unit 12A.
- the derived extended reference layer offset related to the reference layer (layer R) is referred to as SRLO [r].
- SRLO [r] the expanded reference layer offsets and symbols corresponding to left, top, right, and bottom that constitute SRLO [r] are defined as follows.
- Extended reference layer left offset SRLLO [r]
- Extended reference layer offset SRLTO [r]
- Extended reference layer right offset SRLRO [r]
- Extended reference layer lower offset SRLBO [r]
- SRLLO (scaled_ref_layer_left_offset [r] * SubWidthC)
- SRLTO (scaled_ref_layer_top_offset [r] * SubHeightC)
- SRLRO (scaled_ref_layer_right_offset [r] * SubWidthC)
- SRLBO (scaled_ref_layer_bottom_offset [r] * SubHeightC)
- SubWidthC and SubHeightC are a luminance color difference width ratio and a luminance color difference height ratio derived from the color format associated with the target layer.
- the above derivation process of the extended reference layer offset can be expressed as follows.
- the target of the predicted image generation process is a luminance pixel
- enlargement is performed by multiplying the value of the enlarged reference layer offset syntax by the luminance color difference size ratio (luminance color difference width ratio or luminance color difference height ratio) in the target layer.
- luminance color difference size ratio luminance color difference width ratio or luminance color difference height ratio
- a reference layer offset on the reference layer (layer r) is derived.
- the reference layer offset is composed of four offsets (RLLO, RLTO, RLRO, and RLBO in order) corresponding to left, top, right, and bottom, respectively, and is derived from the following equation based on the value of the reference layer offset syntax.
- RefSubWidthC and RefSubHeightC are luminance color difference size ratios derived from the color format associated with the reference layer.
- the value of SubWidthC derived based on the color format of the reference layer is set to RefSubWidthC
- the value of SubHeightC derived based on the color format of the reference layer is set to RefSubHeightC.
- the above reference layer offset derivation process can be expressed as follows. That is, when the target of the predicted image generation process is a luminance pixel, the reference layer offset is derived from the product of the reference layer offset syntax value and the luminance / color difference size ratio in the reference layer. Next, S406 is executed.
- Each offset, SRLLO, SRLTO, SRLRO, SRLBO constituting SRLO [j] described in S402 is derived by the following calculation based on the value of the corresponding expanded reference layer offset syntax.
- SRLLO scaled_ref_layer_left_offset [r]
- SRLTO scaled_ref_layer_top_offset [r]
- SRLRO scaled_ref_layer_right_offset [r]
- SRLBO scaled_ref_layer_bottom_offset [r]
- the reference layer offset derivation process can be expressed as follows. That is, when the target of the predicted image generation process is a chrominance pixel, the reference layer offset syntax value is derived as the reference layer offset value. Next, S406 is executed.
- the size of the expanded reference layer is derived based on the expanded reference layer offset derived in S402 or S404 and the target layer picture size.
- the width (SRLW) and height (SRLH) of the expanded reference layer are derived from the following equations based on the width (CL_PICW) and height (CL_PICH) of the target layer picture, respectively.
- SRLW CL_PICW-(SRLLO + SRLRO)
- SRLH CL_PICH-(SRLTO + SRLBO) That is, according to the above equation, the width of the expanded reference layer on the target layer is derived by subtracting the sum of the expanded reference layer left offset and the expanded reference layer right offset from the width of the target layer picture. The same applies to the height of the enlarged reference layer on the target layer.
- S407 is executed. It should be noted that the sum of offsets in the above equation is subtracted when the enlarged reference layer (reference layer corresponding area in the figure) is inside the target layer picture, as shown in FIG. 21 (a). This is because the sign of the offset is defined so that the value becomes positive.
- the size of a region (reference layer reference region) serving as a reference for scale calculation on the reference layer is derived.
- the width (RLW) and height (RLH) of the reference layer reference region are derived from the following equations based on the width (RL_PICW) and height (RL_PICH) of the reference layer picture, respectively.
- the width of the reference layer standard region is derived by subtracting the sum of the reference layer left offset and the reference layer right offset from the reference layer picture width. The same applies to the height of the reference layer standard region.
- a scale used for inter-layer prediction is derived based on the size of the enlarged reference layer derived in S406 and the size of the reference layer standard region derived in S407.
- the horizontal scale sx and the vertical scale sy are derived by the following equations, respectively.
- phase_hor_luma [r] is set as the value of phaseX
- phase_ver_luma [r] is set as the value of phaseY.
- the target pixel is a color difference pixel
- the value of phase_hor_chroma [r] is set in phaseX
- the value of phase_ver_chroma [r] is set in phaseY.
- the reference pixel position is calculated based on the expanded reference layer offset, the reference layer offset, the inter-layer phase offset, and the scale.
- the derived reference pixel position with 1/16 pixel accuracy is set as the corresponding reference position, and the corresponding reference position deriving process is terminated.
- the pixel value at the position corresponding to the corresponding reference position derived in the corresponding reference position deriving process is applied to the decoded pixels of the pixels near the corresponding reference position on the reference layer picture. Generate.
- Modification 5 Extended reference layer offset information presence / absence flag
- the expanded reference layer offset information presence / absence flag (scaled_ref_layer_offset_prsent_flag) is a flag indicating whether or not the expanded reference layer offset information exists in the PPS.
- the expanded reference layer offset information presence / absence flag (scaled_ref_layer_offset_present_flag [i]) corresponding to the array index i is the syntax (scaled_ref_layer_left_offset [ref_layer_id [i], scaled_ref_layer_top_offset) of the expanded reference layer offset corresponding to the reference layer whose layer identifier is ref_layer_id [i].
- a syntax table including a layer offset information presence / absence flag and an inter-layer phase offset presence / absence flag may be used in the inter-layer position correspondence information.
- the layer offset information presence / absence flag (layer_offset_present_flag) is a flag indicating the presence of the expanded reference layer offset information and the reference layer offset information in the PPS.
- the layer offset information presence / absence flag (layer_offset_present_flag [i]) corresponding to the array index i is present in the encoded data of the extended reference layer offset and the reference layer offset syntax corresponding to the reference layer whose layer identifier is ref_layer_id [i] When the value is 1, it indicates that the syntax exists, and when the value is 0, the syntax does not exist.
- the above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images.
- the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.
- FIG. 19A is a block diagram illustrating a configuration of a transmission device PROD_A in which the hierarchical video encoding device 2 is mounted.
- the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and the encoded data obtained by the encoding unit PROD_A1.
- a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided.
- the hierarchical moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.
- the transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1.
- An image processing unit A7 that generates or processes an image may be further provided.
- FIG. 19A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but a part of the configuration may be omitted.
- the recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.
- FIG. 19 is a block diagram illustrating a configuration of the receiving device PROD_B in which the hierarchical video decoding device 1 is mounted.
- the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator.
- a decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2.
- the above-described hierarchical video decoding device 1 is used as the decoding unit PROD_B3.
- the receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3.
- PROD_B6 may be further provided.
- FIG. 19B illustrates a configuration in which the reception apparatus PROD_B includes all of these, but a part of the configuration may be omitted.
- the recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
- the transmission medium for transmitting the modulation signal may be wireless or wired.
- the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.
- a terrestrial digital broadcast broadcasting station (broadcasting equipment or the like) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting.
- a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.
- a server workstation etc.
- Client television receiver, personal computer, smart phone etc.
- VOD Video On Demand
- video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication.
- PROD_A / reception device PROD_B usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN.
- the personal computer includes a desktop PC, a laptop PC, and a tablet PC.
- the smartphone also includes a multi-function mobile phone terminal.
- the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.
- FIG. 20A is a block diagram illustrating a configuration of a recording apparatus PROD_C in which the above-described hierarchical video encoding apparatus 2 is mounted.
- the recording device PROD_C has an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M.
- the hierarchical moving image encoding device 2 described above is used as the encoding unit PROD_C1.
- the recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) For example, it may be loaded into a drive device (not shown) built in the recording device PROD_C.
- the recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images.
- the unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided.
- FIG. 20A a configuration in which the recording apparatus PROD_C includes all of these is illustrated, but a part may be omitted.
- the receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.
- Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images).
- a camcorder in this case, the camera PROD_C3 is a main source of moving images
- a personal computer in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images
- a smartphone in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
- the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
- FIG. 20 is a block showing a configuration of a playback device PROD_D in which the above-described hierarchical video decoding device 1 is mounted.
- the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written on the recording medium PROD_M and a coded data read by the read unit PROD_D1.
- a decoding unit PROD_D2 to be obtained.
- the hierarchical moving image decoding apparatus 1 described above is used as the decoding unit PROD_D2.
- the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.
- the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2.
- PROD_D5 may be further provided.
- FIG. 20B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but a part of the configuration may be omitted.
- the transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.
- Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images).
- a television receiver in this case, the display PROD_D3 is a main supply destination of moving images
- a digital signage also referred to as an electronic signboard or an electronic bulletin board
- the display PROD_D3 or the transmission unit PROD_D5 is a main supply of moving images.
- Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image)
- a smartphone which is a main image supply destination
- a smartphone in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination
- the like are also examples of such a playback device PROD_D.
- each block of the hierarchical video decoding device 1 and the hierarchical video encoding device 2 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central It may be realized by software using a Processing Unit).
- IC chip integrated circuit
- CPU Central It may be realized by software using a Processing Unit
- each of the devices includes a CPU that executes instructions of a control program that realizes each function, a ROM (Read Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and A storage device (recording medium) such as a memory for storing various data is provided.
- An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each of the above devices, which is software that realizes the above-described functions, is recorded in a computer-readable manner This can also be achieved by supplying each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU (Micro Processing Unit)).
- a program code execution format program, intermediate code program, source program
- Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, CD-ROMs (Compact Disc-Read-Only Memory) / MO (Magneto-Optical) / Discs including optical discs such as MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable), cards such as IC cards (including memory cards) / optical cards, mask ROM / EPROM (Erasable) Programmable Read-only Memory / EEPROM (registered trademark) (ElectricallyErasable Programmable Read-only Memory) / Semiconductor memories such as flash ROM, or logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) Etc. can be used.
- tapes such as magnetic tapes and cassette tapes
- magnetic disks such as floppy (registered trademark) disks / hard disks
- each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
- the communication network is not particularly limited as long as it can transmit the program code.
- the Internet intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Area Antenna Television) communication network, Virtual Private Network (Virtual Private Network), A telephone line network, a mobile communication network, a satellite communication network, etc. can be used.
- the transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type.
- IEEE Institute of Electrical and Electronic Engineers 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. wired such as IrDA (Infrared Data Association) and remote control
- IrDA Infrared Data Association
- wireless such as Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital network, etc.
- the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.
- the present invention is suitable for a hierarchical image decoding device that decodes encoded data in which image data is hierarchically encoded, and a hierarchical image encoding device that generates encoded data in which image data is hierarchically encoded. Applicable to. Further, the present invention can be suitably applied to the data structure of hierarchically encoded data generated by the hierarchical image encoding device and referenced by the hierarchical image decoding device.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
本実施の形態に係る階層動画像復号装置(画像復号装置)1は、階層動画像符号化装置(画像符号化装置)2によって階層符号化された符号化データを復号する。階層符号化とは、動画像を低品質のものから高品質のものにかけて階層的に符号化する符号化方式のことである。階層符号化は、例えば、SVCやSHVCにおいて標準化されている。なお、ここでいう動画像の品質とは、主観的および客観的な動画像の見栄えに影響する要素のことを広く意味する。動画像の品質には、例えば、“解像度”、“フレームレート”、“画質”、および、“画素の表現精度”が含まれる。よって、以下、動画像の品質が異なるといえば、例示的には、“解像度”等が異なることを指すが、これに限られない。例えば、異なる量子化ステップで量子化された動画像の場合(すなわち、異なる符号化雑音により符号化された動画像の場合)も互いに動画像の品質が異なるといえる。
ここで、図2を用いて、階層符号化データの符号化および復号について説明すると次のとおりである。図2は、動画像を、下位階層L3、中位階層L2、および上位階層L1の3階層により階層的に符号化/復号する場合について模式的に表す図である。つまり、図2(a)および(b)に示す例では、3階層のうち、上位階層L1が最上位層となり、下位階層L3が最下位層となる。
以下、各階層の符号化データを生成する符号化方式として、HEVCおよびその拡張方式を用いる場合について例示する。しかしながら、これに限られず、各階層の符号化データを、MPEG-2や、H.264/AVCなどの符号化方式により生成してもよい。
図3は、基本レイヤにおいて採用できる符号化データ(図2の例でいえば、階層符号化データDATA#C)のデータ構造を例示する図である。階層符号化データDATA#Cは、例示的に、シーケンス、およびシーケンスを構成する複数のピクチャを含む。
シーケンスレイヤでは、処理対象のシーケンスSEQ(以下、対象シーケンスとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。シーケンスSEQは、図3の(a)に示すように、ビデオパラメータセットVPS(Video Parameter Set)、シーケンスパラメータセットSPS(Sequence Parameter Set)、ピクチャパラメータセットPPS(Picture Parameter Set)、ピクチャPICT1~PICTNP(NPはシーケンスSEQに含まれるピクチャの総数)、及び、付加拡張情報SEI(Supplemental Enhancement Information)を含んでいる。
ピクチャレイヤでは、処理対象のピクチャPICT(以下、対象ピクチャとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。ピクチャPICTは、図3の(b)に示すように、スライスヘッダSH1~SHNS、及び、スライスS1~SNSを含んでいる(NSはピクチャPICTに含まれるスライスの総数)。
スライスレイヤでは、処理対象のスライスS(対象スライスとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。スライスSは、図3の(c)に示すように、符号化ツリーユニットCTU1~CTUNC(NCはスライスSに含まれるCTUの総数)を含んでいる。
CTUレイヤでは、処理対象の符号化ツリーユニットCTU(以下、対象CTUとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。なお、符号化ツリーユニットのことを符号化ツリーブロック(CTB: Coding Tree block)、または、最大符号化単位(LCU:Largest Cording Unit)と呼ぶこともある。
CTUヘッダCTUHには、対象CTUの復号方法を決定するために階層動画像復号装置1が参照する符号化パラメータが含まれる。具体的には、図3の(d)に示すように、対象CTUの各CUへの分割パターンを指定するCTU分割情報SP_CTU、および、量子化ステップの大きさを指定する量子化パラメータ差分Δqp(qp_delta)が含まれる。
CUレイヤでは、処理対象のCU(以下、対象CUとも称する)を復号するために階層動画像復号装置1が参照するデータの集合が規定されている。
続いて、図3(e)を参照しながらCU情報CUに含まれるデータの具体的な内容を説明する。図3(e)に示すように、CU情報CUは、具体的には、スキップフラグSKIP、予測ツリー情報(以下、PT情報と略称する)PTI、および、変換ツリー情報(以下、TT情報と略称する)TTIを含む。
PT情報PTIは、CUに含まれる予測ツリー(以下、PTと略称する)に関する情報である。言い換えれば、PT情報PTIは、PTに含まれる1または複数のPUそれぞれに関する情報の集合であり、階層動画像復号装置1により予測画像を生成する際に参照される。PT情報PTIは、図3(e)に示すように、予測タイプ情報PType、および、予測情報PInfoを含んでいる。
TT情報TTIは、CUに含まれる変換ツリー(以下、TTと略称する)に関する情報である。言い換えれば、TT情報TTIは、TTに含まれる1または複数の変換ブロックそれぞれに関する情報の集合であり、階層動画像復号装置1により残差データを復号する際に参照される。
処理2:処理1にて得られた変換係数を量子化する;
処理3:処理2にて量子化された変換係数を可変長符号化する;
なお、上述した量子化パラメータqpは、階層動画像符号化装置2が変換係数を量子化する際に用いた量子化ステップQPの大きさを表す(QP=2qp/6)。
PU分割情報によって指定されるPU分割タイプには、対象CUのサイズを2N×2N画素とすると、次の合計8種類のパターンがある。すなわち、2N×2N画素、2N×N画素、N×2N画素、およびN×N画素の4つの対称的分割(symmetric splittings)、並びに、2N×nU画素、2N×nD画素、nL×2N画素、およびnR×2N画素の4つの非対称的分割(asymmetric splittings)である。なお、N=2m(mは1以上の任意の整数)を意味している。以下、対象CUを分割して得られる予測単位のことを予測ブロック、または、パーティションと称する。
拡張レイヤのレイヤ表現に含まれる符号化データ(以下、拡張レイヤ符号化データ)についても、例えば、図3に示すデータ構造とほぼ同様のデータ構造を採用できる。ただし、拡張レイヤ符号化データでは、以下のとおり、付加的な情報を追加したり、パラメータを省略できる。
次に、ピクチャ、タイル、スライスについて、相互の関係および符号化データとの関係を図4を参照して説明する。図4は、階層符号化データにおけるピクチャとタイル・スライスの関係を説明する図である。タイルは、ピクチャ内の矩形の部分領域、および、該部分領域に係る符号化データに対応付けられる。スライスはピクチャ内の部分領域、および、該部分領域に係る符号化データ、すなわち、該部分領域に係るスライスヘッダおよびスライスデータに対応付けられる。
以下では、本実施形態に係る階層動画像復号装置1の構成について、図1~図13を参照して説明する。
図5を参照して、階層動画像復号装置1の概略的構成を説明する。図5は、階層動画像復号装置1の概略的構成を示した機能ブロック図である。階層動画像復号装置1は、階層符号化データDATA(階層動画像符号化装置2から提供される階層符号化データDATAF、または、符号化データ変換装置3から提供される階層符号化データDATAR)を復号して、対象レイヤの復号画像POUT#Tを生成する。なお、以下では、対象レイヤは基本レイヤを参照レイヤとする拡張レイヤであるとして説明する。そのため、対象レイヤは、参照レイヤに対する上位レイヤでもある。逆に、参照レイヤは、対象レイヤに対する下位レイヤでもある。
パラメータセット復号部12は、入力される対象レイヤの符号化データから、対象レイヤの復号に用いられるパラメータセット(VPS、SPS、PPS)を復号して出力する。一般に、パラメータセットの復号は既定のシンタックス表に基づいて実行される。すなわち、シンタックス表の定める手順に従って符号化データからビット列を読み出して、シンタックス表に含まれるシンタックス要素のシンタックス値を復号する。また、必要に応じて、復号したシンタックス値に基づいて導出した変数を、出力するパラメータセットに含めてもよい。したがって、パラメータセット復号部12から出力されるパラメータセットは、符号化データに含まれるパラメータセット(VPS、SPS、PPS)に係るシンタックス要素のシンタックス値、および、該シンタックス値より導出される変数の集合と表現できる。
パラメータセット復号部12は、入力される対象レイヤ符号化データからピクチャ情報を復号する。ピクチャ情報は、概略的には、対象レイヤの復号ピクチャのサイズを定める情報である。例えば、ピクチャ情報は、対象レイヤの復号ピクチャの幅や高さを表わす情報を含んでいる。
パラメータセット復号部12は、入力される対象レイヤ符号化データから表示領域情報を復号する。表示領域情報は、例えば、SPSに含まれている。SPSから復号される表示領域情報は、表示領域フラグ(conformance_flag)を含む。表示領域フラグは表示領域の位置を表わす情報(表示領域位置情報)が追加でSPSに含まれるか否かを示す。すなわち、表示領域フラグが1の場合、表示領域位置情報が追加で含まれることを示し、表示領域フラグが0の場合、表示領域位置情報が追加で含まれないことを示す。
パラメータセット復号部12は、入力される対象レイヤ符号化データからレイヤ間位置対応情報を復号する。レイヤ間位置対応情報は、概略的には、対象レイヤと参照レイヤの対応する領域の位置関係を示す。例えば、対象レイヤのピクチャと参照レイヤのピクチャにある物体(物体A)が含まれる場合、対象レイヤのピクチャ上の物体Aに対応する領域と、参照レイヤのピクチャ上の物体Aに対応する領域が、前記対象レイヤと参照レイヤの対応する領域に相当する。なお、レイヤ間位置対応情報は、必ずしも上記の対象レイヤと参照レイヤの対応する領域の位置関係を正確に示す情報でなくてもよいが、一般的には、レイヤ間予測の正確性を高めるために正確な対象レイヤと参照レイヤの対応する領域の位置関係を示している。
レイヤ間位置対応情報は、参照領域情報を含む。参照領域情報は、特定のレイヤ(以下、レイヤA)における、以下の2つの領域の空間的な位置関係を表す情報である。
(参照領域):レイヤAと別のレイヤ(レイヤB)の間のレイヤ間処理において、レイヤ間の画素位置の対応関係を示す情報の導出に用いられる、レイヤA上の領域
2.参照領域上オフセット:参照領域左上画素とピクチャ領域左上画素との間の垂直方向のオフセット
3.参照領域右オフセット:参照領域右下画素とピクチャ領域右下画素との間の水平方向のオフセット
4.参照領域下オフセット:参照領域右下画素とピクチャ領域右下画素との間の垂直方向のオフセット
パラメータセット復号部12において、入力される符号化データから復号される参照領域オフセットに係るシンタックスについて、図9を参照して説明する。図9は、パラメータセット復号部12がVPSに含まれるVPS拡張(vps_extension())の復号時に参照されるシンタックス表の一部であって、参照領域オフセットに係る部分である。
・A1:参照領域オフセット数(num_ref_region_offsets[i])
・A2:参照領域レイヤインデックス情報(ref_region_layer_idx_delta_minus1[i][j])
・A3L:参照領域左オフセット情報(ref_region_left_offset[i][j])
・A3T:参照領域上オフセット情報(ref_region_top _offset[i][j])
・A3R:参照領域右オフセット情報(ref_region_right_offset[i][j])
・A3B:参照領域下オフセット情報(ref_region_bottom_offset[i][j])
LIdx[i][j] = LIdx[i][j-1] + (RRL[i][j] + 1) ... (j >= 1)
すなわち、jが0に等しい場合、インデックスiから、参照領域レイヤインデックス情報に1を加えた数だけ小さいレイヤインデックスを設定する。また、jが1以上の場合、レイヤiに関連付けられた(j-1)番目のレイヤのインデックスから、参照領域レイヤインデックス情報に1を加えた数だけ大きいレイヤインデックスを設定する。ここで、RRL[i][j]の最大値は、ビットストリーム中のレイヤ数最大値(MaxLayersMinus1+1)未満である。すなわち、レイヤiに関連付けられたj番目のレイヤのインデックスの最大値は、レイヤiのインデックス(インデックスi)より大きい値を取り得る。
スライス復号部14は、入力されるVCL NAL、パラメータセット、および、タイル情報に基づいて復号ピクチャを生成して出力する。
スライスヘッダ復号部141は、入力されるVCL NALとパラメータセットに基づいてスライスヘッダを復号し、スライス位置設定部142、スキップスライス判定部143、および、CTU復号部144に出力する。
スライス位置設定部142は、入力されるスライスヘッダとタイル情報に基づいてピクチャ内のスライス位置を特定してCTU復号部144に出力する。スライス位置設定部142で導出されるピクチャ内のスライス位置は、スライスに含まれる各CTUのピクチャ内での位置を含む。
CTU復号部144は、概略的には、入力されるスライスヘッダ、スライスデータ、および、パラメータセットに基づいて、スライスに含まれる各CTUに対応する領域の復号画像を復号することで、スライスの復号画像を生成する。スライスの復号画像は、入力されるスライス位置の示す位置に、復号ピクチャの一部として出力される。CTUの復号画像は、CTU復号部144内部の予測残差復元部1441、予測画像生成部1442、および、CTU復号画像生成部1443により生成される。予測残差復元部1441は、入力のスライスデータに含まれる予測残差情報(TT情報)を復号して対象CTUの予測残差を生成して出力する。予測画像生成部1442は、入力のスライスデータに含まれる予測情報(PT情報)の示す予測方法と予測パラメータに基づいて予測画像を生成して出力する。その際、必要に応じて、参照ピクチャの復号画像や符号化らメータが利用される。CTU復号画像生成部1443は、入力される予測画像と予測残差を加算して対象CTUの復号画像を生成して出力する。
前述の予測画像生成部1442による予測画像生成処理のうち、レイヤ間画像予測が選択された場合の予測画像生成処理の詳細を説明する。
参照領域上オフセット:RRTO[k][l]
参照領域右オフセット:RRRO[k][l]
参照領域下オフセット:RRBO[k][l]
RRTO[k][l] = (ref_region_top_offset[i][j] << 1)
RRRO[k][l] = (ref_region_right_offset[i][j] << 1)
RRBO[k][l] = (ref_region_bottom_offset[i][j] << 1)
ここで、k=i、l=LIdx[i][j]の関係がある。LIdx[i][j]は、パラメータセット復号部12において復号された参照領域レイヤインデックス情報に基づいて導出された配列であって、レイヤiに対するj番目の参照領域オフセット情報が関連付けられるレイヤのVPS内インデックスを表す。
RL_RRTO = RRTO[r][c]
RL_RRRO = RRRO[r][c]
RL_RRBO = RRBO[r][c]
すなわち、参照レイヤ参照領域オフセットの値として、参照レイヤR上の参照領域であって、参照レイヤRと対象レイヤCとの間のレイヤ間予測で参照される参照領域オフセットを設定している。
RL_RRH = RL_PICH - n * (RL_RRTO + RL_RRBO)
ここで、nの値は、参照レイヤピクチャの画素単位と参照領域オフセットの単位を揃えるためのパラメータである。例えば、入力画像の色信号が4:2:0形式の場合であって、参照領域オフセットの単位が輝度画素単位である場合、輝度に対してはn=1の値を、色差に対してはn=0.5の値を用いることで単位を揃える。
CL_RRTO = RRTO[c][r]
CL_RRRO = RRRO[c][r]
CL_RRBO = RRBO[c][r]
すなわち、対象レイヤ参照領域オフセットの値として、対象レイヤC上の参照領域であって、対象レイヤCと参照レイヤRとの間のレイヤ間予測で参照される参照領域オフセットを設定している。
CL_RRH = CL_PICH - n * (CL_RRTO + CL_RRBO)
ここで、nの値は、S103で説明したnと同様であり、対象レイヤピクチャの画素単位と対象領域オフセットの単位を揃えるためのパラメータである。
sy = ((RL_RRH << 16) + (CL_RRH >> 1)) / CL_RRH
なお、演算子“/”は整数の除算を表す演算子である。
すなわち、上式によれば、参照レイヤ参照領域サイズに所定の定数(上式では“<<16”)を乗じ、対象レイヤ参照領域サイズで除算した値をスケールとして設定している。なお、“(CL_RRW>>1)”の項は、除算による丸めを調整する項である。したがって、単位や丸め調整の効果を除けば、スケールは参照レイヤ参照領域のサイズ(幅または高さ)の、対象レイヤ参照領域のサイズ(幅または高さ)に対する比率と説明できる。
yRef16 = (((yP - CL_OY) * sy + addY + (1<<11)) >> 12) + deltaY + RL_OY
ここで、
CL_OX = CL_RRLO / crAdjust
CL_OY = CL_RRTO / crAdjust
RL_OX = (RL_RRLO << 4) / crAdjust
RL_OY = (RL_RRTO << 4) / crAdjust
なお、crAdjustは輝度と色差の単位の違いを補正するパラメータであり、例えば、4:2:0の復号画像を扱う場合、輝度であれば1、色差であれば2の値が設定される。また、addX、addY、deltaX、deltaYは、アップサンプリングやインタレースに伴う対象レイヤ上の画素と参照レイヤ上の画素のずれを表すパラメータである。
以上説明した本実施形態に係る階層動画像復号装置1(階層画像復号装置)は、パラメータセットを復号するパラメータセット復号部12と、参照レイヤピクチャの復号画素を参照してレイヤ間予測により予測画像を生成する予測画像生成部1442を備えている。パラメータセット復号部12は参照領域オフセットを復号し、予測画像生成部1442は、該参照領域オフセットから導出される参照レイヤ参照領域オフセットと対象レイヤ参照領域オフセットに基づいてスケールを計算し、該スケールを参照して対応参照位置を導出してレイヤ間予測を実行する。換言すると、階層動画像復号装置1は、参照領域オフセットの導出に必要なシンタックス値を符号化データから復号し、該シンタックス値に基づいて導出されたパラメータに基づいて対応参照位置を計算することで、注目領域抽出を実行する場合に適切な対応参照位置を導出する機能を提供できる。したがって、階層動画像復号装置1は、従来に比べて、より少ないシンタックス要素を用いて、注目領域抽出前後で上位レイヤ画素と下位レイヤ画素の位置関係の正確さを維持する機能を提供できる。
上記の階層動画像復号装置1の説明では、パラメータセット復号部12で参照領域情報に含まれる参照領域レイヤインデックス情報が復号され、該参照領域レイヤインデックス情報に基づいて導出されるレイヤiに関連付けられるj番目のレイヤのインデックスLIdx[i][j]は、iより大きい値を取り得ると記載した。これは、レイヤiに関連付けられるj番目の参照領域オフセット情報がLIdx[i][j]との間のレイヤ間処理における参照領域を示すために利用され、LIdx[i][j]で示されるレイヤはレイヤiより上位の各レイヤに対して独立に指定できることを意味する。
RL_RRTO = RRTO[r][r]
RL_RRRO = RRRO[r][r]
RL_RRBO = RRBO[r][r]
すなわち、対象レイヤ参照領域オフセットの値として、参照レイヤR上の参照領域であって、レイヤRと、レイヤRより上位の任意レイヤとの間のレイヤ間予測で参照される参照領域オフセットを設定している。
上記の階層動画像復号装置1の説明では、参照領域情報をVPS拡張に含む例を説明したが、他のパラメータセット(SPSやPPS)やヘッダ(スライスヘッダ)で送ることも可能である。以下では、SPSに参照領域情報を含む場合の変形例を説明する。
パラメータセット復号部12において、入力される符号化データから復号される参照領域オフセットに係るシンタックスについて、図10を参照して説明する。図10は、パラメータセット復号部12がSPSに含まれるSPS拡張(sps_multilayer_extension())の復号時に参照するシンタックス表の一部であって、参照領域オフセットに係る部分である。
・B2:参照領域レイヤ識別子情報(ref_region_layer_id[i])
・B3L:参照領域左オフセット情報(ref_region_left_offset[i])
・B3T:参照領域上オフセット情報(ref_region_top _offset[i])
・B3R:参照領域右オフセット情報(ref_region_right_offset[i])
・B3B:参照領域下オフセット情報(ref_region_bottom_offset[i])
シンタックス要素B1は6ビットの2進数表現表現の符号、B2はHEVCで規定されている非負整数の0次指数ゴロム符号(ue(v))、B3L、B3T、A3R、A3BはHEVCで規定されている符号付き整数の0次指数ゴロム符号(se(v))を用いて復号または符号化される。
次に、上記の図10のシンタックス表を参照して復号されたSPS参照領域オフセット関連シンタックスを用いて、予測画像生成部1442による予測画像生成処理のうち、レイヤ間画像予測が選択された場合に実行される、対応参照位置導出処理を説明する。
参照領域上オフセット:RRTO[cLId][rLId]
参照領域右オフセット:RRRO[cLId][rLId]
参照領域下オフセット:RRBO[cLId][rLId]
RRO[i][j]を構成する各オフセット、RRLO[i][j]、RRTO[i][j]、RRRO[i][j]、RRBO[i][j]は、対応する参照領域オフセット情報に基づいて、以下の計算により導出される。
RRTO[cLId][rLId] = (ref_region_top_offset[rLId] << 1)
RRRO[cLId][rLId] = (ref_region_right_offset[rLId] << 1)
RRBO[cLId][rLId] = (ref_region_bottom_offset[rLId] << 1)
RL_RRTO = RRTO[cLId][cLId]
RL_RRRO = RRRO[cLId][cLId]
RL_RRBO = RRBO[cLId][cLId]
すなわち、参照レイヤ参照領域オフセットの値として、参照レイヤ上の参照領域であって、参照レイヤと参照レイヤより上位のレイヤとの間のレイヤ間予測で参照される参照領域オフセットを設定している。
RL_RRTO = RRTO[rLId][rLId]
RL_RRRO = RRRO[rLId][rLId]
RL_RRBO = RRBO[rLId][rLId]
CL_RRTO = RRTO[cLId][rLId]
CL_RRRO = RRRO[cLId][rLId]
CL_RRBO = RRBO[cLId][rLId]
すなわち、対象レイヤ参照領域オフセットの値として、対象レイヤ上の参照領域であって、対象レイヤ(レイヤ識別子cLId)と参照レイヤ(レイヤ識別子rLId)との間のレイヤ間予測で参照される参照領域オフセットを設定している。
上記変形例2において、参照領域レイヤ識別子の値が対象レイヤのレイヤ識別子未満か、レイヤ識別子と等しいかに応じて、後続する参照領域オフセットが対象レイヤ上の参照領域と、参照レイヤ上の参照領域の何れのオフセットを示すかを判定していた。別のシンタックスと導出方法により参照領域オフセットが関連付けられるレイヤを判定する方法を用いることもできる。
・C3L:参照領域左オフセット情報(ref_region_left_offset[lid1][lid2])
・C3T:参照領域上オフセット情報(ref_region_top _offset[lid1][lid2])
・C3R:参照領域右オフセット情報(ref_region_right_offset[lid1][lid2])
・C3B:参照領域下オフセット情報(ref_region_bottom_offset[lid1][lid2])
概略的には、まず、参照領域対象選択情報が復号され、次に該参照領域対象選択情報から2つのレイヤ識別子lid1とlid2を導出する。その後に復号される参照領域オフセット情報は、レイヤlid1上の参照領域であって、レイヤlid1とレイヤlid2の間のレイヤ間予測において参照される参照領域のオフセットに設定される。
2つのレイヤ間のレイヤ間予測を実行する場合に参照領域を暗黙的に決定する方法として、表示領域情報を用いる方法が知られている。しかしながら、表示領域情報を参照領域を指定する用途で用いる場合には、表示領域情報を表示領域の指定のために変更することが制限されるという課題がある。また、特定のレイヤの表示領域が変更された場合には、該レイヤを参照レイヤとする上位レイヤのレイヤ間予測画像が変化するため、上位レイヤの符号化データを再生成する必要が生じるという課題がある。そのため、表示領域を参照領域として使用するか否かを別の情報により制御できることが好ましい。
図1を参照して説明した対応参照位置導出処理のS101では、参照領域オフセット情報から参照領域オフセットを導出する際に、対応する参照領域オフセット情報が存在しない場合の参照領域オフセット既定値として、参照領域がピクチャ全体となるよう参照領域オフセットの値を設定すると説明した。参照領域オフセット既定値を表示領域とすることで、参照領域オフセットを送らない場合に表示領域を参照領域の既定値として利用しつつ、表示領域を変更する必要がある場合には、参照領域オフセットを明示的に変更前の表示領域に相当する値を指定することで、上位レイヤの書き換えを避けつつ表示領域の変更が可能となる。
参照領域上オフセット:RRTO[c][r]
参照領域右オフセット:RRRO[c][r]
参照領域下オフセット:RRBO[c][r]
RRO[c][r]を構成する各オフセット、RRLO[c][r]、RRTO[c][r]、RRRO[c][r]、RRBO[c][r]は、対応する参照領域オフセット情報に基づいて、以下の計算により導出される。
RRTO[c][r] = (ref_region_top_offset[i][j] << 1)
RRRO[c][r] = (ref_region_right_offset[i][j] << 1)
RRBO[c][r] = (ref_region_bottom_offset[i][j] << 1)
ここで、c=i、r=LIdx[i][j]の関係がある。LIdx[i][j]は、パラメータセット復号部12において復号された参照領域レイヤインデックス情報に基づいて導出された配列であって、レイヤiに対するj番目の参照領域オフセット情報が関連付けられるレイヤのVPS内インデックスを表す。
RRTO[c][r] = conf_win_top_offset[r]
RRRO[c][r] = conf_win_right_offset[r]
RRBO[c][r] = conf_win_bottom_offset[r]
ここで、conf_win_left_offset[r]、conf_win_top_offset[r]、conf_win_right_offset[r]、conf_win_bottom_offset[r]は、それぞれ、参照レイヤRが参照するSPSから復号される各表示領域位置情報(conf_win_left_offset、conf_win_top_offset、conf_win_right_offset、conf_win_bottom_offset)の対応するシンタックスの値である。
参照領域を表示領域とするかを示すフラグ(表示領域利用制御フラグ)をパラメータセットに含め、パラメータセット復号部12で復号して、予測画像生成部1442における対応参照位置導出処理に利用できる。表示領域利用制御フラグとしては、次の何れかが利用できる。
上記の実施形態や各変形例において、参照領域位置情報(参照領域オフセット情報)は2つのレイヤ識別情報に直接または間接的に関連付けられている。つまり、ある一組の参照領域位置情報は、レイヤA上の参照領域であって、レイヤAとレイヤBの間のレイヤ間処理で参照される参照領域の形状および位置を示す情報である。
図14を用いて、階層動画像符号化装置2の概略構成を説明する。図14は、階層動画像符号化装置2の概略的構成を示した機能ブロック図である。階層動画像符号化装置2は、対象レイヤの入力画像PIN#Tを、参照レイヤ符号化データDATA#Rを参照しながら符号化して、対象レイヤの階層符号化データDATAを生成する。なお、参照レイヤ符号化データDATA#Rは、参照レイヤに対応する階層動画像符号化装置において符号化済みであるとする。
次に図15を参照して、スライス符号化部24の構成の詳細を説明する。図15は、スライス符号化部24の概略的構成を示した機能ブロック図である。
以上説明した本実施形態に係る階層動画像符号化装置2(階層画像符号化装置)は、パラメータセットを符号化するパラメータセット符号化部22と、参照レイヤピクチャの復号画素を参照してレイヤ間予測により予測画像を生成する予測画像符号化部2442を備えている。パラメータセット符号化部22は参照領域情報を符号化し、予測画像符号化部2442は、参照領域情報から導出される対象レイヤ参照領域オフセットと参照レイヤ参照領域情報の値を用いて対象レイヤ上の画素に対する対応参照位置を導出する。
図16を用いて、階層符号化データ変換装置3の概略構成を説明する。図16は、階層符号化データ変換装置3の概略的構成を示した機能ブロック図である。階層符号化データ変換装置3は、入力される階層符号化データDATAを変換して、入力される注目領域情報に係る階層符号化データDATA-ROIを生成する。なお、階層符号化データDATAは階層動画像符号化装置2により生成された階層符号化データである。また、階層符号化データDATA-ROIを階層動画像復号装置1に入力することで注目領域情報に係る上位レイヤの動画像を再生できる。
始めに、階層符号化データ変換装置3による変換処理によるパラメータセット修正の概略について、図17を参照して説明する。図17は、変換前後の階層符号化データにおけるピクチャ、注目領域、および、タイルの関係を例示した図である。図17では、拡張レイヤとベースレイヤの2レイヤから構成される階層符号化データ(変換前階層符号化データ)を変換して注目領域を含む階層符号化データ(変換後階層符号化データ)を生成する場合の、変換前後のピクチャの関係を示している。変換前階層符号化データの拡張レイヤは変換前ELピクチャに相当するデータであり、ベースレイヤは変換前BLピクチャに相当するデータである。同様に、変換後階層符号化データの拡張レイヤは変換後ELピクチャに相当するデータであり、ベースレイヤは変換後BLピクチャに相当するデータである。
パラメータセット修正部32は、入力される注目領域情報とタイル情報を参照して、対応する領域の一部が注目領域と重複するタイル(抽出対象タイル)のみを含むよう拡張レイヤのPPSタイル情報を更新する。抽出対象タイルの情報に基づいて、拡張レイヤのPPSタイル情報を更新する。まず、抽出対象タイルが1個の場合、tiles_enabled_flagを0に修正する。なお、抽出対象タイルが2個以上の場合は修正処理は省略できる。次に、ピクチャの水平方向と垂直方向に含まれる抽出対象タイルの個数に基づいて、タイル行数を表わす(num_tile_columns_minus1)とタイル列数を表わす(num_tile_rows_minus1)を修正する。次に、タイルサイズが不均等(uniform_spacing_flagが0)の場合には、抽出対象タイルを含まないタイル列の幅、抽出対象タイルを含まないタイル行の高さに係るシンタックスに対応するビット列をパラメータセットから削除する。
パラメータセット修正部32は、拡張レイヤの抽出対象タイルの集合に対応する領域を変換後ELピクチャサイズとしてピクチャ情報を修正する。変換後ELピクチャの幅と高さを拡張レイヤSPSのpic_width_in_luma_samplesとpic_height_in_luma_samplesの値としてそれぞれ設定する。
パラメータセット修正部32は、ピクチャサイズの変更を踏まえて、パラメータセットに含まれるレイヤ間画素対応情報を修正する。具体的には、レイヤ間画素対応情報に含まれる参照領域オフセットを修正する。ここで、説明のため、拡張レイヤのレイヤ識別子をe、ベースレイヤのレイヤ識別子をbとする。参照領域オフセットに含まれる、レイヤe上の参照領域であって、レイヤeとレイヤbとの間のレイヤ間予測で参照される参照領域に対する参照領域左オフセットref_region_right_offset[e][k](ただし、b=rLIdx[e][k])の値は、変換前ELピクチャの左上画素と変換後ELピクチャの左上画素の水平方向の距離を2画素単位で表した値を絶対値とする負値が設定される。加えて、レイヤb上の参照領域であって、レイヤbとレイヤeとの間のレイヤ間予測で参照される参照領域に対する参照領域左オフセットref_region_right_offset[b][l](ただしe=rLIdx[b][l])の値は、変換前BLピクチャの左上画素と変換後BLピクチャの左上画素の水平方向の距離を2画素単位で表した値を絶対値とする負値が設定される。他の上、右、下に対応する参照領域オフセットの値も同様に設定される。
パラメータセット修正部32は、入力される注目領域情報の示す注目領域と一致するように、入力されるパラメータセットに含まれるSPSの表示領域情報を書き換える。表示領域情報は、例えば、次のS301からS303の手順で書き換えられる。
階層符号化データ変換装置3による階層符号化データ変換処理は、S501~S506に示す手順を順次実行することで実現される。
以上説明した本実施形態に係る階層符号化データ変換装置3は、対象レイヤ(上位レイヤ)の符号化データに含まれるビデオレイヤの符号化データ(VCL NAL)の一部を注目領域情報に基づいて修正するNAL選択部34と、パラメータセット修正部32を備えている。NAL選択部34は、注目領域情報の示す注目領域に基づいて、注目領域と重複する領域をもつタイルを抽出対象タイルとして選択し、前記選択した抽出対象タイルに含まれるスライスに係るビデオレイヤの符号化データが変換後の階層符号化データに含まれる。パラメータセット修正部32は、注目領域情報とタイル情報に基づいて、ピクチャ情報、PPSタイル情報、表示情報、および、参照領域情報を修正する。
上述した階層動画像復号装置1、階層動画像符号化装置2、及び、階層符号化データ変換装置3を組み合わせて、注目領域情報を表示するシステム(注目領域表示システムSYS)を構成できる。
注目領域表示システムによる処理は、階層符号化データ生成蓄積処理と注目領域データ生成再生処理に分けることができる。
以上説明した本実施形態に係る注目領域表示システムSYSは、注目領域情報を供給する注目領域通知部(ROI通知部SYS8)と、前記注目領域情報に基づいて階層符号化データを変換して変換後階層符号化データを生成する階層符号化データ変換部SYS3と、上記変換後階層符号化データを復号して上位レイヤ及び下位レイヤの復号ピクチャを出力する階層動画像復号部SYS4と、表示部SYS6を備えている。
図21~図24に基づいて、本発明の一実施形態に係る階層動画像復号装置3を説明すれば以下のとおりである。
階層動画像復号装置3の概略的構成を説明する。階層動画像復号装置3は、図5を参照して説明した階層動画像復号装置1において、パラメータセット復号部12をパラメータセット復号部12Aに、スライス復号部14をスライス復号部14Aに置き換えた構成である。すなわち、階層動画像復号装置3は、NAL逆多重化部11、パラメータセット復号部12A、タイル設定部13、スライス復号部14A、ベース復号部15、復号ピクチャ管理部16を含む。以下では、新たな構成要素である、パラメータ復号部12Aおよびスライス復号部14Aについて説明する。
(レイヤ間位置対応情報)
パラメータセット復号部12Aは、入力される対象レイヤ符号化データからレイヤ間位置対応情報を復号する。レイヤ間位置対応情報は、概略的には、対象レイヤと参照レイヤの対応する領域の位置関係を示す。例えば、対象レイヤのピクチャと参照レイヤのピクチャにある物体(物体A)が含まれる場合、対象レイヤのピクチャ上の物体Aに対応する領域と、参照レイヤのピクチャ上の物体Aに対応する領域が、前記対象レイヤと参照レイヤの対応する領域に相当する。なお、レイヤ間位置対応情報は、必ずしも上記の対象レイヤと参照レイヤの対応する領域の位置関係を正確に示す情報でなくてもよいが、一般的には、レイヤ間予測の正確性を高めるために正確な対象レイヤと参照レイヤの対応する領域の位置関係を示している。
レイヤ間位置対応情報には拡大参照レイヤオフセットを規定する情報が含まれる。拡大参照レイヤオフセットは符号化データ内に複数含むことが可能であり、各々の拡大参照レイヤオフセットは、左、上、右、下にそれぞれ対応する4つのオフセットから構成され、対象ピクチャ、および、参照ピクチャの2枚のピクチャの組み合わせに関連付けられている。言い換えると、対象ピクチャと、特定の参照ピクチャの組み合わせそれぞれに対して、対応する拡大参照レイヤオフセットを規定する情報がレイヤ間位置対応情報に含まれる。なお、必ずしも全ての対象ピクチャと参照ピクチャの組み合わせに対して参照レイヤオフセットを規定する情報が含まれている必要はなく、特定の条件で一部の組み合わせについて省略して既定値を用いることもできる。
レイヤ間位置対応情報には参照レイヤオフセットを規定する情報(参照レイヤオフセット情報)が含まれる。参照レイヤオフセットは符号化データ内に複数含むことが可能であり、各々の参照レイヤオフセットは、左、上、右、下にそれぞれ対応する4つのオフセットから構成され、対象ピクチャ、および、参照ピクチャの2枚のピクチャの組み合わせに関連付けられている。言い換えると、対象ピクチャと、特定の参照ピクチャの組み合わせそれぞれに対して、対応する参照レイヤオフセットがレイヤ間画素対応情報に含まれ得る。
レイヤ間位相情報にはレイヤ間の位相差を表す情報(レイヤ間位相オフセット)が含まれる。レイヤ間位相は符号化データ内に複数含むことが可能であり、各々のレイヤ間位相オフセットは、輝度水平方向、輝度垂直方向、色差水平方向、色差垂直方向の4個のオフセットから構成され、対象ピクチャ、および、参照ピクチャの2枚のピクチャの組み合わせに関連付けられている。言い換えると、対象ピクチャと、特定の参照ピクチャの組み合わせそれぞれに対して、対応するレイヤ間位相オフセットがレイヤ間画素対応情報に含まれ得る。
次に符号化データに含まれ、パラメータセット復号部12Aにより復号される、拡大参照レイヤオフセット情報、参照レイヤオフセット情報、および、レイヤ間位相オフセット情報について説明する。
スライス復号部14Aは図13を参照して説明したスライス復号部14と同様の構成と機能を有する。両者の違いは、スライス復号部14が含む予測画像生成部1442に代わりスライス復号部14Aは予測画像生成部1443Aを含んでいる点にある。予測画像生成部1443Aでは、レイヤ間予測が選択された場合の予測画像生成処理の実行時に、レイヤ間位置対応情報に基づいて、対応参照位置導出処理を以下のS401~S409の手順で実行する。
拡大参照レイヤ上オフセット:SRLTO[r]
拡大参照レイヤ右オフセット:SRLRO[r]
拡大参照レイヤ下オフセット:SRLBO[r]
SRLLO = (scaled_ref_layer_left_offset[r] * SubWidthC)
SRLTO = (scaled_ref_layer_top_offset[r] * SubHeightC)
SRLRO = (scaled_ref_layer_right_offset[r] * SubWidthC)
SRLBO = (scaled_ref_layer_bottom_offset[r] * SubHeightC)
ここで、SubWidthCおよびSubHeightCは、対象レイヤに対応付けられる色フォーマットから導出される輝度色差幅比および輝度色差高さ比である。
RLTO = (ref_layer_top_offset[r] * RefSubHeightC)
RLRO = (ref_layer_right_offset[r] * RefSubWidthC)
RLBO = (ref_layer_bottom_offset[r] * RefSubHeightC)
ここで、RefSubWidthCおよびRefSubHeightCは、参照レイヤに対応付けられる色フォーマットから導出される輝度色差サイズ比である。すなわち、参照レイヤの色フォーマットに基づき導出されたSubWidthCの値がRefSubWidthCに、参照レイヤの色フォーマットに基づき導出されたSubHeightCの値がRefSubHeightCにそれぞれ設定される。
SRLTO = scaled_ref_layer_top_offset[r]
SRLRO = scaled_ref_layer_right_offset[r]
SRLBO = scaled_ref_layer_bottom_offset[r]
上記の拡大参照レイヤオフセットの導出処理は、次のように表現することができる。すなわち、予測画像生成処理の対象が色差画素である場合、拡大参照レイヤオフセットシンタックスの値を拡大参照レイヤオフセットとして導出する。
RLTO = ref_layer_top_offset[r]
RLRO = ref_layer_right_offset[r]
RLBO = ref_layer_bottom_offset[r]
上記の参照レイヤオフセットの導出処理は、次のように表現できる。すなわち、予測画像生成処理の対象が色差画素である場合、参照レイヤオフセットシンタックスの値を参照レイヤオフセットの値として導出する。次にS406を実行する。
SRLH = CL_PICH - (SRLTO + SRLBO)
すなわち、上記の式によれば、対象レイヤピクチャの幅に、拡大参照レイヤ左オフセットと拡大参照レイヤ右オフセットの和を減算することで、対象レイヤ上の拡大参照レイヤの幅を導出している。対象レイヤ上の拡大参照レイヤの高さについても同様である。次にS407を実行する。なお、上記式においてオフセットの和を減算するのは、図21(a)に示したように、拡大参照レイヤ(図中の参照レイヤ対応領域)が、対象レイヤピクチャの内部にある場合にオフセットの値が正になるようオフセットの符号が定義されているためである。
RLH = RL_PICH - (RLTO + RLBO)
すなわち、上記の式によれば、参照レイヤピクチャ幅に、参照レイヤ左オフセットと参照レイヤ右オフセットの和を減算することで、参照レイヤ基準領域の幅を導出している。参照レイヤ基準領域の高さについても同様である。次にS408を実行する。
sy = ((RLH << 16) + (SRLH >> 1)) / SRLH
なお、演算子“/”は整数の除算を表す演算子である。
すなわち、上式によれば、参照レイヤ参照領域サイズに所定の定数(上式では“<<16”)を乗じ、対象レイヤ参照領域サイズで除算した値をスケールとして設定している。なお、“(SRLW>>1)”の項は、除算による丸めを調整する項である。したがって、単位や丸め調整の効果を除けば、スケールは参照レイヤ基準領域のサイズ(幅または高さ)の、拡大参照レイヤのサイズ(幅または高さ)に対する比率と説明できる。次にS409を実行する。
yRef16 = (((yP - CL_OY) * sy + addY + (1<<11)) >> 12) + RL_OY
ここで、
CL_OX = SRLLO
CL_OY = SRLTO
RL_OX = (RLLO << 4)
RL_OY = (RLTO << 4)
なお、addX、addYは、アップサンプリングやインタレースに伴う対象レイヤ上の画素と参照レイヤ上の画素のずれを表すパラメータであり、次式により導出される。
addY = (sy * phaseY + 8) >> 4
ここで、参照レイヤ識別子の値をrとすると、対象画素が輝度画素の場合には、phaseXの値にはphase_hor_luma[r]、phaseYの値にはphase_ver_luma[r]が設定される。また、対象画素が色差画素の場合には、phaseXにはphase_hor_chroma[r]の値が、phaseYにはphase_ver_chroma[r]の値が設定される。
上記のパラメータセット復号部12Aにおいて復号されるレイヤ間位置対応情報には、参照レイヤオフセット情報存否フラグとレイヤ間位相オフセット情報存否フラグが含まれている例を説明したが、図23に示すように、拡大参照レイヤオフセット情報存否フラグを追加で含むシンタックス表を用いる構成であってもよい。拡大参照レイヤオフセット情報存否フラグ(scaled_ref_layer_offset_prsent_flag)は、拡大参照レイヤオフセット情報のPPS内での存否を示すフラグである。配列インデックスiに対応する拡大参照レイヤオフセット情報存否フラグ(scaled_ref_layer_offset_present_flag[i])は、レイヤ識別子がref_layer_id[i]の参照レイヤに対応する拡大参照レイヤオフセットのシンタックス(scaled_ref_layer_left_offset[ref_layer_id[i]、scaled_ref_layer_top_offset[ref_layer_id[i]、scaled_ref_layer_right_offset[ref_layer_id[i]、scaled_ref_layer_bottom_offset[ref_layer_id[i])の符号化データ内の存在有無を表し、値が1の場合に該シンタックスが存在することを、値が0の場合に該シンタックスが存在しないことを示す。拡大参照レイヤオフセット情報存否フラグを用いることで、拡大参照レイヤオフセットが不要な場合の符号量を削減できる。
上述した階層動画像符号化装置2及び階層動画像復号装置1は、動画像の送信、受信、記録、再生を行う各種装置に搭載して利用できる。なお、動画像は、カメラ等により撮像された自然動画像であってもよいし、コンピュータ等により生成された人工動画像(CGおよびGUIを含む)であってもよい。
最後に、階層動画像復号装置1、階層動画像符号化装置2の各ブロックは、集積回路(ICチップ)上に形成された論理回路によってハードウェア的に実現してもよいし、CPU(Central Processing Unit)を用いてソフトウェア的に実現してもよい。
11 NAL逆多重化部
12、12A パラメータセット復号部
13 タイル設定部
14、14A スライス復号部
141 スライスヘッダ復号部
142 スライス位置設定部
144 CTU復号部
1441 予測残差復元部
1442、1442A、 予測画像生成部
1443 CTU復号画像生成部
15 ベース復号部
151 ベースNAL逆多重化部
152 ベースパラメータセット復号部
153 ベースタイル設定部
154 ベーススライス復号部
156 ベース復号ピクチャ管理部
16 復号ピクチャ管理部
2 階層動画像符号化装置(画像符号化装置)
21 NAL多重化部
22 パラメータセット符号化部
23 タイル設定部
24 スライス符号化部
241 スライスヘッダ設定部
242 スライス位置設定部
244 CTU符号化部
2441 予測残差符号化部
2442 予測画像符号化部
3 階層符号化データ変換装置(符号化データ変換装置)
32 パラメータセット修正部
34 NAL選択部
Claims (14)
- 階層符号化された符号化データを復号し、対象レイヤである上位レイヤの復号ピクチャを復元する画像復号装置であって、
パラメータセットを復号するパラメータセット復号部と、
参照レイヤピクチャの復号画素を参照して、レイヤ間予測により予測画像を生成する予測画像生成部を備え、
前記パラメータセット復号部は、レイヤ間位置対応情報を復号し、
前記レイヤ間位置対応情報は、拡大参照レイヤオフセットシンタックス、参照レイヤオフセットシンタックス、および、レイヤ間位相オフセットシンタックスを含み、
前記パラメータセット復号部は、特定の参照レイヤを示す第一の参照レイヤ識別子を復号し、符号化データ内で連続して含まれる、前記第一の参照レイヤ識別子に関連付けられる前記拡大参照レイヤオフセットシンタックス、前記第一の参照レイヤ識別子に関連付けられる前記参照レイヤオフセットシンタックス、および、前記第一の参照レイヤ識別子に関連付けられるレイヤ間位相オフセットシンタックスを復号し、
前記予測画像生成部は、所定の参照レイヤを用いる場合の予測画像を生成する場合に、該参照レイヤに関連付けられる前記拡大参照レイヤオフセットシンタックス、前記参照レイヤオフセットシンタックス、および、前記レイヤ間位相オフセットシンタックスを用いて、レイヤ間予測に用いるスケール、および、対応参照位置の少なくとも一方を導出することを特徴とする画像復号装置。 - 前記レイヤ間位置対応情報は、特定の参照レイヤと関連付けられた参照レイヤオフセット情報存否フラグを含み、
前記予測画像生成部は、前記参照レイヤオフセット情報存否フラグの値に応じて、該参照レイヤオフセットが関連付けられた参照レイヤに関する参照レイヤオフセットシンタックスを復号することを特徴とする、請求項1に記載の画像復号装置。 - 前記レイヤ間位置対応情報は、特定の参照レイヤと関連付けられたレイヤ間位相オフセット情報存否フラグを含み、
前記予測画像生成部は、前記レイヤ間位相オフセット情報存否フラグの値に応じて、該参照レイヤオフセットが関連付けられたレイヤ間位相オフセットシンタックスを復号することを特徴とする、請求項1に記載の画像復号装置。 - 前記レイヤ間位置対応情報は、特定の参照レイヤと関連付けられたレイヤオフセット情報存否フラグを含み、
前記予測画像生成部は、前記レイヤオフセット情報存否フラグの値に応じて、該参照レイヤオフセットが関連付けられた拡大参照レイヤオフセットシンタックス、および、参照レイヤオフセットシンタックスを復号することを特徴とする、請求項1に記載の画像復号装置。 - 階層符号化された符号化データを復号し、対象レイヤである上位レイヤの復号ピクチャを復元する画像復号装置であって、
パラメータセットを復号するパラメータセット復号部と、
参照レイヤピクチャの復号画素を参照して、レイヤ間予測により予測画像を生成する予測画像生成部を備え、
前記パラメータセット復号部は参照領域情報を復号し、
前記予測画像生成部は、前記参照領域情報に含まれる参照領域位置情報から対象レイヤ参照領域位置および参照レイヤ参照領域位置を導出し、前記参照領域位置および前記参照レイヤ参照領域位置を用いてレイヤ間予測に用いるスケール、および、対応参照位置の少なくとも一方を導出することを特徴とする画像復号装置。 - 前記参照領域位置情報は、参照領域オフセット情報であり、
前記対象レイヤ参照領域位置は、対象レイヤ参照領域オフセットであり、
前記参照レイヤ参照領域位置は、参照レイヤ対象領域オフセットであることを特徴とする請求項5に記載の画像復号装置。 - 前記参照領域位置情報は、第一のレイヤ識別情報と第二のレイヤ識別情報をインデックスとする二次元の配列変数であって、
前記第一のレイヤ識別情報は、参照領域が存在するレイヤを特定する情報であって、
前記第二のレイヤ識別情報は、レイヤ間処理対象レイヤを特定する情報であって、
前記予測画像生成部は、対象レイヤと参照レイヤの間のレイヤ間予測実行時に、前記対象レイヤを示す前記第一のレイヤ識別情報と、前記参照レイヤを示す前記第二のレイヤ識別情報とをインデックスとする前記参照領域位置情報を用いて前記対象レイヤ参照領域位置を設定し、前記参照レイヤを示す前記第一のレイヤ識別情報と、前記対象レイヤを示す前記第二のレイヤ識別情報をインデックスとする前記参照領域位置情報を用いて前記参照レイヤ参照領域位置を設定することを特徴とする請求項5または請求項6に記載の画像復号装置。 - 前記第一のレイヤ識別情報は、参照領域が存在するレイヤのVPS内インデックスを直接または間接的に指定する情報であって、
前記第二のレイヤ識別情報は、レイヤ間処理対象レイヤのVPS内インデックスを直接または間接的に指定する情報であって、
前記予測画像生成部は、VPS内インデックスの値がCの前記対象レイヤに対するレイヤ間予測実行時に、VPS内インデックスCを示す前記第一のレイヤ識別情報と、Cより小さいVPS内インデックスRを示す前記第二のレイヤ識別情報とをインデックスとする前記参照領域位置情報を用いて前記対象レイヤ参照領域位置を設定し、前記VPS内インデックスRを示す前記第一のレイヤ識別情報と、VPS内インデックスCを示す前記第二のレイヤ識別情報をインデックスとする前記参照領域位置情報を用いて前記参照レイヤ参照領域位置を設定することを特徴とする請求項7に記載の画像復号装置。 - 前記参照領域位置情報は、第一のレイヤ識別情報と第二のレイヤ識別情報をインデックスとする二次元の配列変数であって、
前記第一のレイヤ識別情報は、参照領域が存在するレイヤを特定する情報であって、
前記第二のレイヤ識別情報は、レイヤ間処理対象レイヤを特定する情報であって、
前記予測画像生成部は、対象レイヤと参照レイヤの間のレイヤ間予測実行時に、前記対象レイヤを示す前記第一のレイヤ識別情報と、前記参照レイヤを示す前記第二のレイヤ識別情報とをインデックスとする前記参照領域位置情報を用いて前記対象レイヤ参照領域位置を設定し、前記対象レイヤを示す前記第一のレイヤ識別情報と、前記対象レイヤを示す前記第二のレイヤ識別情報をインデックスとする前記参照領域位置情報を用いて前記参照レイヤ参照領域位置を設定することを特徴とする請求項5または請求項6に記載の画像復号装置。 - 前記パラメータセット復号部は、参照領域対象選択情報を復号し、該参照領域対象選択情報から第一のレイヤ識別情報と第二のレイヤ識別情報を導出するとともに、上記第一のレイヤ識別情報と上記第二のレイヤ識別情報の組み合わせと上記参照領域位置情報を関連付けて記録し、
前記予測画像生成部は、対象レイヤと参照レイヤの間のレイヤ間予測実行時に、前記対象レイヤを示す前記第一のレイヤ識別情報と、前記参照レイヤを示す前記第二のレイヤ識別情報の組み合わせに関連付けられて記録されている前記参照領域位置情報を用いて前記対象レイヤ参照領域位置を設定し、参照レイヤを示す前記第一のレイヤ識別情報と、対象レイヤを示す前記第二のレイヤ識別情報に関連付けられて記録されている前記参照領域位置情報を用いて前記参照レイヤ参照領域位置を設定することを特徴とする請求項5または請求項6に記載の画像復号装置。 - 前記パラメータセット復号部は、対象レイヤと参照レイヤの表示領域情報を復号し、
前記予測画像生成部は、対象レイヤと参照レイヤの間のレイヤ間予測実行時に、前記対象レイヤを示す前記第一のレイヤ識別情報と、前記参照レイヤを示す前記第二のレイヤ識別情報とをインデックスとする前記参照領域位置情報が存在しない場合、対象レイヤの前記表示領域情報の示す表示領域位置を前記対象レイヤ参照領域位置として設定し、参照レイヤを示す前記第一のレイヤ識別情報と、対象レイヤを示す前記第二のレイヤ識別情報をインデックスとする前記参照領域位置情報が存在しない場合、参照レイヤの前記表示領域情報の示す表示領域を前記参照レイヤ参照領域位置として設定することを特徴とする請求項7から請求項10に記載の画像復号装置。 - 前記予測画像生成部は、前記対象レイヤ参照領域位置の示す参照領域サイズと、前記参照レイヤ参照領域位置の示す参照領域サイズの比、または、比を近似する値として、前記スケールを導出することを特徴とする請求項5から請求項11に記載の画像復号装置。
- 入力画像から対象レイヤである上位レイヤの符号化データを生成する画像符号化装置であって、
パラメータセットを符号化するパラメータセット符号化部と、
参照レイヤピクチャの復号画素を参照して、レイヤ間予測により予測画像を生成する予測画像生成部を備え、
前記パラメータセット符号化部は参照領域情報を符号化し、
前記予測画像生成部は、前記参照領域情報に含まれる参照領域位置情報から対象レイヤ参照領域位置および参照レイヤ参照領域位置を導出し、前記参照領域位置および前記参照レイヤ参照領域位置を用いてレイヤ間予測に用いるスケール、および、対応参照位置の少なくとも一方を導出することを特徴とする画像符号化装置。 - 入力される階層符号化データを入力される注目領域情報に基づいて変換し、注目領域階層符号化データを生成して出力する、パラメータセット修正部を備える階層符号化データ変換装置であって、
前記パラメータセット修正部は、階層符号化データから導出される対象レイヤ参照領域位置および参照レイヤ参照領域位置がそれぞれ変換前後で一致するように参照領域位置情報を修正することを特徴とする階層符号化データ変換装置。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201580019567.6A CN106170980A (zh) | 2014-04-25 | 2015-04-24 | 图像解码装置、图像编码装置以及编码数据变换装置 |
JP2016515226A JP6542201B2 (ja) | 2014-04-25 | 2015-04-24 | 画像復号装置および画像復号方法 |
US15/303,585 US10187659B2 (en) | 2014-04-25 | 2015-04-24 | Image decoding device, image coding device, and coding data transform device |
US16/181,509 US20190075323A1 (en) | 2014-04-25 | 2018-11-06 | Image decoding device, image encoding device, and image decoding method |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014091275 | 2014-04-25 | ||
JP2014-091275 | 2014-04-25 | ||
JP2014-124100 | 2014-06-17 | ||
JP2014124100 | 2014-06-17 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/303,585 A-371-Of-International US10187659B2 (en) | 2014-04-25 | 2015-04-24 | Image decoding device, image coding device, and coding data transform device |
US16/181,509 Continuation US20190075323A1 (en) | 2014-04-25 | 2018-11-06 | Image decoding device, image encoding device, and image decoding method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015163456A1 true WO2015163456A1 (ja) | 2015-10-29 |
Family
ID=54332616
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/062555 WO2015163456A1 (ja) | 2014-04-25 | 2015-04-24 | 画像復号装置、画像符号化装置、および、符号化データ変換装置 |
Country Status (5)
Country | Link |
---|---|
US (2) | US10187659B2 (ja) |
JP (1) | JP6542201B2 (ja) |
CN (1) | CN106170980A (ja) |
TW (1) | TWI589152B (ja) |
WO (1) | WO2015163456A1 (ja) |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11336909B2 (en) * | 2016-12-27 | 2022-05-17 | Sony Corporation | Image processing apparatus and method |
KR102424829B1 (ko) * | 2017-02-22 | 2022-07-25 | 에스케이텔레콤 주식회사 | 비디오 데이터가 부호화된 비트스트림을 처리하는 방법 |
WO2018155939A1 (ko) | 2017-02-22 | 2018-08-30 | 에스케이텔레콤 주식회사 | 영상 복호화 방법 및 장치 |
US10234864B2 (en) * | 2017-03-07 | 2019-03-19 | nuTonomy Inc. | Planning for unknown objects by an autonomous vehicle |
WO2019174884A1 (en) * | 2018-03-16 | 2019-09-19 | Inveox Gmbh | Automated identification, orientation and sample detection of a sample container |
AU2019268844B2 (en) * | 2018-05-15 | 2022-09-29 | FG Innovation Company Limited | Image encoding device, encoded stream extraction device, and image decoding device |
CN112204978B (zh) * | 2018-06-01 | 2024-03-15 | 夏普株式会社 | 图像解码装置以及图像编码装置 |
US10798376B2 (en) * | 2018-07-17 | 2020-10-06 | Tencent America LLC | Method and apparatus for video coding |
DK3850847T3 (da) | 2019-03-11 | 2022-07-11 | Dolby Laboratories Licensing Corp | Signalering af information relateret til blændevinkel |
PL3977723T3 (pl) * | 2019-07-08 | 2024-04-22 | Huawei Technologies Co., Ltd. | Obsługa wielu rozmiarów obrazu i okien zgodności w celu ponownego próbkowania obrazu odniesienia w kodowaniu wideo |
US11877000B2 (en) * | 2019-08-06 | 2024-01-16 | Dolby Laboratories Licensing Corporation | Canvas size scalable video coding |
US11140402B2 (en) * | 2019-09-20 | 2021-10-05 | Tencent America LLC | Signaling of reference picture resampling with constant window size indication in video bitstream |
US11336894B2 (en) * | 2019-09-20 | 2022-05-17 | Tencent America LLC | Signaling of reference picture resampling with resampling picture size indication in video bitstream |
US11317093B2 (en) | 2019-09-24 | 2022-04-26 | Tencent America LLC | Method for reference picture resampling with offset in video bitstream |
GB2587365B (en) * | 2019-09-24 | 2023-02-22 | Canon Kk | Method, device, and computer program for coding and decoding a picture |
WO2021121419A1 (en) | 2019-12-19 | 2021-06-24 | Beijing Bytedance Network Technology Co., Ltd. | Interaction between adaptive color transform and quantization parameters |
JP7436680B2 (ja) * | 2020-01-05 | 2024-02-22 | 北京字節跳動網絡技術有限公司 | 映像コーディングのための一般制約情報 |
WO2021143896A1 (en) | 2020-01-18 | 2021-07-22 | Beijing Bytedance Network Technology Co., Ltd. | Adaptive colour transform in image/video coding |
CN115299064A (zh) | 2020-03-11 | 2022-11-04 | 抖音视界有限公司 | 基于颜色格式的自适应参数集信令通知 |
WO2021235759A1 (ko) * | 2020-05-22 | 2021-11-25 | 엘지전자 주식회사 | 비디오 또는 영상 코딩 시스템에서의 다중 레이어 기반 영상 코딩 방법 |
WO2021244420A1 (en) | 2020-05-31 | 2021-12-09 | Beijing Bytedance Network Technology Co., Ltd. | Palette mode with local dual tree modetype definition |
CN116156203A (zh) * | 2021-11-22 | 2023-05-23 | 中兴通讯股份有限公司 | 语法元素的编码、解码方法、装置、电子设备和存储介质 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20060105409A (ko) * | 2005-04-01 | 2006-10-11 | 엘지전자 주식회사 | 영상 신호의 스케일러블 인코딩 및 디코딩 방법 |
WO2008060125A1 (en) * | 2006-11-17 | 2008-05-22 | Lg Electronics Inc. | Method and apparatus for decoding/encoding a video signal |
KR20140138538A (ko) * | 2013-05-24 | 2014-12-04 | 주식회사 케이티 | 복수의 레이어를 지원하는 비디오 코딩 방법 및 장치 |
CA2943121C (en) * | 2014-03-18 | 2020-09-08 | Arris Enterprises Llc | Scalable video coding using reference and scaled reference layer offsets |
-
2015
- 2015-04-24 CN CN201580019567.6A patent/CN106170980A/zh active Pending
- 2015-04-24 US US15/303,585 patent/US10187659B2/en not_active Expired - Fee Related
- 2015-04-24 JP JP2016515226A patent/JP6542201B2/ja not_active Expired - Fee Related
- 2015-04-24 WO PCT/JP2015/062555 patent/WO2015163456A1/ja active Application Filing
- 2015-04-24 TW TW104113139A patent/TWI589152B/zh not_active IP Right Cessation
-
2018
- 2018-11-06 US US16/181,509 patent/US20190075323A1/en not_active Abandoned
Non-Patent Citations (6)
Title |
---|
DO-KYOUNG KWON ET AL.: "Reference- layer cropping offsets signaling in SHVC", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 13TH MEETING, April 2013 (2013-04-01), Incheon, KR, pages 1 - 5 * |
JIANLE CHEN ET AL.: "On phase offset for resampling process in SHVC", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 17TH MEETING, 18 March 2014 (2014-03-18), Valencia, ES, pages 1 - 3 * |
KOOHYAR MINOO ET AL.: "Increased resolution for scaled reference layer offset", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 17TH MEETING, 27 March 2014 (2014-03-27), Valencia, ES, pages 1 - 6 * |
TOMOYUKI YAMAMOTO ET AL.: "AHG 13: Sub-region extraction - position calculation and comparison of different approaches", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 17TH MEETING, 29 March 2014 (2014-03-29), Valencia, ES, pages 1 - 7 * |
TOMOYUKI YAMAMOTO ET AL.: "AHG13: On reference position calculation", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 18TH MEETING, 21 June 2014 (2014-06-21), Sapporo, JP, pages 1 - 4 * |
TOMOYUKI YAMAMOTO ET AL.: "MV-HEVC/SHVC HLS: On conversion to ROI-oriented multi-layer bitstream", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11 15TH MEETING, 24 October 2013 (2013-10-24), Geneva, CH, pages 1 - 5 * |
Also Published As
Publication number | Publication date |
---|---|
US20170034532A1 (en) | 2017-02-02 |
CN106170980A (zh) | 2016-11-30 |
TW201545544A (zh) | 2015-12-01 |
JPWO2015163456A1 (ja) | 2017-04-20 |
US10187659B2 (en) | 2019-01-22 |
TWI589152B (zh) | 2017-06-21 |
US20190075323A1 (en) | 2019-03-07 |
JP6542201B2 (ja) | 2019-07-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6542201B2 (ja) | 画像復号装置および画像復号方法 | |
JP6229904B2 (ja) | 画像復号装置、画像符号化装置、および、符号化データ変換装置 | |
JP6363088B2 (ja) | 画像復号装置、画像復号方法、画像符号化装置、および画像符号化方法 | |
US10841600B2 (en) | Image decoding device, an image encoding device and a decoding method | |
US10136151B2 (en) | Image decoding device and image decoding method | |
US20190014351A1 (en) | Moving image coding device, a moving image coding method, and a moving image decoding device | |
US10136161B2 (en) | DMM prediction section, image decoding device, and image coding device | |
WO2014115283A1 (ja) | 画像復号装置、および画像符号化装置 | |
WO2014162954A1 (ja) | 画像復号装置、および画像符号化装置 | |
JP5995448B2 (ja) | 画像復号装置、および画像符号化装置 | |
WO2014007131A1 (ja) | 画像復号装置、および画像符号化装置 | |
JP2015073213A (ja) | 画像復号装置、画像符号化装置、符号化データ変換装置、および、注目領域表示システム | |
JP2014176039A (ja) | 画像復号装置、および画像符号化装置 | |
JP6162289B2 (ja) | 画像復号装置および画像復号方法 | |
WO2015098713A1 (ja) | 画像復号装置および画像符号化装置 | |
JP2015177318A (ja) | 画像復号装置、画像符号化装置 | |
JP2015126508A (ja) | 画像復号装置、画像符号化装置、符号化データ変換装置、領域再生装置 | |
JP2014082729A (ja) | 画像復号装置、および画像符号化装置 | |
JP2015076807A (ja) | 画像復号装置、画像符号化装置、および符号化データのデータ構造 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15782812 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016515226 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15303585 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15782812 Country of ref document: EP Kind code of ref document: A1 |