WO2015056620A1 - Image decoding device and image coding device - Google Patents

Image decoding device and image coding device Download PDF

Info

Publication number
WO2015056620A1
WO2015056620A1 PCT/JP2014/076977 JP2014076977W WO2015056620A1 WO 2015056620 A1 WO2015056620 A1 WO 2015056620A1 JP 2014076977 W JP2014076977 W JP 2014076977W WO 2015056620 A1 WO2015056620 A1 WO 2015056620A1
Authority
WO
WIPO (PCT)
Prior art keywords
prediction
unit
image
flag
merge
Prior art date
Application number
PCT/JP2014/076977
Other languages
French (fr)
Japanese (ja)
Inventor
貴也 山本
知宏 猪飼
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2015056620A1 publication Critical patent/WO2015056620A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present invention relates to an image decoding device and an image encoding device.
  • the multi-view image encoding technique includes a parallax predictive encoding that reduces the amount of information by predicting a parallax between images when encoding images of a plurality of viewpoints, and a decoding method corresponding to the encoding method.
  • a vector representing the parallax between viewpoint images is called a displacement vector.
  • the displacement vector is a two-dimensional vector having a horizontal element (x component) and a vertical element (y component), and is calculated for each block which is an area obtained by dividing one image.
  • x component horizontal element
  • y component vertical element
  • each viewpoint image is encoded as a different layer in each of a plurality of layers.
  • a method for encoding a moving image composed of a plurality of layers is generally referred to as scalable encoding or hierarchical encoding.
  • scalable coding high coding efficiency is realized by performing prediction between layers.
  • a reference layer without performing prediction between layers is called a base layer, and other layers are called enhancement layers.
  • Scalable encoding in the case where a layer is composed of viewpoint images is referred to as view scalable encoding.
  • the base layer is also called a base view
  • the enhancement layer is also called a non-base view.
  • scalable coding when a layer is composed of a texture layer (image layer) and a depth layer (distance image layer) is called three-dimensional scalable coding.
  • scalable coding in addition to view scalable coding, spatial scalable coding (pictures with low resolution as the base layer and pictures with high resolution in the enhancement layer), SNR scalable coding (image quality as the base layer) Low picture, high resolution picture as an enhancement layer).
  • a base layer picture may be used as a reference picture in coding an enhancement layer picture.
  • Non-Patent Document 1 a technique called residual prediction is known in which a predicted image with higher accuracy is obtained by adding a predicted residual to a prediction target block obtained by normal motion compensation prediction. .
  • a technique called illuminance-compensated prediction that obtains a predicted image with higher accuracy by compensating for a difference in luminance between layers is known.
  • a technique called viewpoint synthesis prediction is known in which a prediction target block is divided into smaller subblocks and prediction is performed using a displacement vector for each subblock, thereby obtaining a prediction image with higher accuracy.
  • Non-Patent Document 1 The parameter required for the view synthesis prediction of Non-Patent Document 1 is derived as one of merge candidates.
  • viewpoint synthesis prediction is not used simultaneously with residual prediction or illumination compensation prediction. Therefore, when residual prediction or illumination compensation prediction is selected as a predicted image generation method, there is a problem that merge candidates for viewpoint synthesis that are not used are derived.
  • the present invention has been made in view of the above points, and by preventing an increase in the number of candidates in merging, an image decoding apparatus, an image decoding method, an image decoding program, and an image that improve the encoding efficiency and reduce the amount of calculation
  • An encoding apparatus, an image encoding method, an image encoding program, an image display system, and an image transmission system are provided.
  • the present invention has been made to solve the above-described problem, and one aspect of the present invention is a residual prediction unit that generates a predicted image using residual prediction, and whether or not to use the residual prediction.
  • a residual prediction weight index decoding unit for decoding the residual prediction weight index shown, and a merge mode parameter deriving unit for deriving a merge candidate list, wherein the merge mode parameter deriving unit selects a VSP merge candidate used for view synthesis prediction.
  • the residual prediction weight index is 0, the VSP merge candidate is added to the merge candidate list.
  • an illumination compensation unit that applies illumination compensation to a predicted image, an illumination compensation flag decoding unit that decodes an illumination compensation flag indicating whether to apply illumination compensation, and a merge candidate list
  • Another aspect of the present invention provides a residual prediction unit that generates a prediction image using residual prediction, and a residual prediction weight index that decodes a residual prediction weight index indicating whether to use residual prediction.
  • an illumination compensation unit that applies illumination compensation to a predicted image, an illumination compensation flag decoding unit that decodes an illumination compensation flag indicating whether to apply illumination compensation, and a prediction parameter are derived.
  • a merge mode parameter deriving unit that performs a view synthesis prediction when the VSP mode flag is 1, and the merge mode parameter deriving unit performs a VSP mode when the illumination compensation flag is other than 0. It is characterized in that 0 is set as a flag.
  • the encoding efficiency in view synthesis prediction is improved and the amount of calculation is reduced.
  • FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system according to an embodiment of the present invention. It is a figure which shows the hierarchical structure of the data of the encoding stream which concerns on this embodiment. It is a conceptual diagram which shows an example of a reference picture list. It is a conceptual diagram which shows the example of a reference picture. It is the schematic which shows the structure of the image decoding apparatus which concerns on this embodiment. It is the schematic which shows the structure of the inter prediction parameter decoding part which concerns on this embodiment. It is the schematic which shows the structure of the merge mode parameter derivation
  • FIG. 1 is a schematic diagram showing a configuration of an image transmission system 1 according to the present embodiment.
  • the image transmission system 1 is a system that transmits a code obtained by encoding a plurality of layer images and displays an image obtained by decoding the transmitted code.
  • the image transmission system 1 includes an image encoding device 11, a network 21, an image decoding device 31, and an image display device 41.
  • the signal T indicating a plurality of layer images (also referred to as texture images) is input to the image encoding device 11.
  • a layer image is an image that is viewed or photographed at a certain resolution and a certain viewpoint.
  • each of the plurality of layer images is referred to as a viewpoint image.
  • the viewpoint corresponds to the position or observation point of the photographing apparatus.
  • the plurality of viewpoint images are images taken by the left and right photographing devices toward the subject.
  • the image encoding device 11 encodes each of the signals to generate an encoded stream Te (encoded data). Details of the encoded stream Te will be described later.
  • a viewpoint image is a two-dimensional image (planar image) observed at a certain viewpoint.
  • the viewpoint image is indicated by, for example, a luminance value or a color signal value for each pixel arranged in a two-dimensional plane.
  • one viewpoint image or a signal indicating the viewpoint image is referred to as a picture.
  • the plurality of layer images include a base layer image having a low resolution and an enhancement layer image having a high resolution.
  • SNR scalable encoding is performed using a plurality of layer images
  • the plurality of layer images are composed of a base layer image with low image quality and an extended layer image with high image quality.
  • view scalable coding, spatial scalable coding, and SNR scalable coding may be arbitrarily combined.
  • encoding and decoding of an image including at least a base layer image and an image other than the base layer image is handled as the plurality of layer images.
  • the image on the reference side is referred to as a first layer image
  • the image on the reference side is referred to as a second layer image.
  • the base layer image is treated as a first layer image and the enhancement layer image is treated as a second layer image.
  • the enhancement layer image include an image of a viewpoint other than the base view and a depth image.
  • a depth image (also referred to as depth map, “depth image”, or “distance image”) is a signal value (“depth value”) corresponding to the distance from the viewpoint (shooting device, etc.) of the subject or background included in the subject space. ”,“ Depth value ”,“ depth ”, etc.), and is an image signal composed of signal values (pixel values) for each pixel arranged in a two-dimensional plane.
  • the pixels constituting the depth image correspond to the pixels constituting the viewpoint image. Therefore, the depth map is a clue for representing the three-dimensional object space by using the viewpoint image which is a reference image signal obtained by projecting the object space onto the two-dimensional plane.
  • the network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31.
  • the network 21 is the Internet, a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof.
  • the network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional or bidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting.
  • the network 21 may be replaced by a storage medium that records an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray Disc).
  • the image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates a plurality of decoded layer images Td (decoded viewpoint images Td).
  • the image display device 41 displays all or part of the plurality of decoded layer images Td generated by the image decoding device 31. For example, in view scalable coding, a 3D image (stereoscopic image) and a free viewpoint image are displayed in all cases, and a 2D image is displayed in some cases.
  • the image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display.
  • a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display.
  • the spatial scalable coding and SNR scalable coding when the image decoding device 31 and the image display device 41 have a high processing capability, a high-quality enhancement layer image is displayed and only a lower processing capability is provided. Displays a base layer image that does not require higher processing capability and display capability as an extension layer.
  • FIG. 2 is a diagram showing a hierarchical structure of data in the encoded stream Te.
  • the encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence.
  • (A) to (f) of FIG. 2 respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, a slice data layer that defines slice data, and a slice data.
  • Coding Unit CU
  • sequence layer a set of data referred to by the image decoding device 31 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined.
  • the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an additional extension.
  • Information SEI Supplemental Enhancement Information
  • # indicates the layer ID.
  • FIG. 2 shows an example in which encoded data of # 0 and # 1, that is, layer 0 and layer 1, exists, but the type of layer and the number of layers are not dependent on this.
  • the video parameter set VPS is a set of encoding parameters common to a plurality of moving images, a plurality of layers included in the moving image, and encoding parameters related to individual layers in a moving image composed of a plurality of layers.
  • a set is defined.
  • the sequence parameter set SPS defines a set of encoding parameters that the image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are defined.
  • a set of encoding parameters referred to by the image decoding device 31 in order to decode each picture in the target sequence is defined.
  • a quantization width reference value (pic_init_qp_minus26) used for picture decoding and a flag (weighted_pred_flag) indicating application of weighted prediction are included.
  • a plurality of PPS may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.
  • Picture layer In the picture layer, a set of data referred to by the image decoding device 31 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 2 (b), the picture PICT includes slices S0 to SNS-1 (NS is the total number of slices included in the picture PICT).
  • slice layer In the slice layer, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 2C, the slice S includes a slice header SH and slice data SDATA.
  • the slice header SH includes a coding parameter group that the image decoding device 31 refers to in order to determine a decoding method of the target slice.
  • the slice type designation information (slice_type) that designates the slice type is an example of an encoding parameter included in the slice header SH.
  • I slice using only intra prediction at the time of encoding (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
  • the slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the sequence layer.
  • the slice data layer a set of data referred to by the image decoding device 31 in order to decode the slice data SDATA to be processed is defined.
  • the slice data SDATA includes a coded tree block (CTB) as shown in FIG.
  • CTB is a fixed-size block (for example, 64 ⁇ 64) constituting a slice, and may be referred to as a maximum coding unit (LCU).
  • the coding tree layer defines a set of data that the image decoding device 31 refers to in order to decode the coding tree block to be processed.
  • the coding tree unit is divided by recursive quadtree division.
  • a tree-structured node obtained by recursive quadtree partitioning is called a coding tree.
  • An intermediate node of the quadtree is a coded tree unit (CTU), and the coded tree block itself is also defined as the highest CTU.
  • the CTU includes a split flag (splif_flag). When the split_flag is 1, the CTU is split into four coding tree units CTU.
  • the coding tree unit CTU is divided into four coding units (CU: Coded Unit).
  • the coding unit CU is a terminal node of the coding tree layer and is not further divided in this layer.
  • the encoding unit CU is a basic unit of the encoding process.
  • the size of the coding unit is any of 64 ⁇ 64 pixels, 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, and 8 ⁇ 8 pixels. It can take.
  • the encoding unit layer defines a set of data referred to by the image decoding device 31 in order to decode the processing target encoding unit.
  • the encoding unit includes a CU header CUH, a prediction tree, a conversion tree, and a CU header CUF.
  • the CU header CUH it is defined whether the coding unit is a unit using intra prediction or a unit using inter prediction.
  • the CU header CUH includes a residual prediction weight index iv_res_pred_weight_idx indicating whether the coding unit is a unit using residual prediction, and an illuminance compensation flag ic_flag indicating whether the coding unit is a unit using illuminance compensation prediction.
  • the encoding unit is the root of a prediction tree (PT) and a transform tree (TT).
  • the CU header CUF is included between the prediction tree and the conversion tree or after the conversion tree.
  • the coding unit is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined.
  • the prediction block is one or a plurality of non-overlapping areas constituting the coding unit.
  • the prediction tree includes one or a plurality of prediction blocks obtained by the above division.
  • Prediction processing is performed for each prediction block.
  • a prediction block which is a unit of prediction is also referred to as a prediction unit (PU, prediction unit).
  • Intra prediction is prediction within the same picture
  • inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).
  • the division method is encoded by part_mode of encoded data, and 2N ⁇ 2N (the same size as the encoding unit), 2N ⁇ N, 2N ⁇ nU, 2N ⁇ nD, N ⁇ 2N, nL X2N, nRx2N, and NxN.
  • 2N ⁇ nU indicates that a 2N ⁇ 2N encoding unit is divided into two regions of 2N ⁇ 0.5N and 2N ⁇ 1.5N in order from the top.
  • 2N ⁇ nD indicates that a 2N ⁇ 2N encoding unit is divided into two regions of 2N ⁇ 1.5N and 2N ⁇ 0.5N in order from the top.
  • nL ⁇ 2N indicates that a 2N ⁇ 2N encoding unit is divided into two regions of 0.5N ⁇ 2N and 1.5N ⁇ 2N in order from the left.
  • nR ⁇ 2N indicates that a 2N ⁇ 2N encoding unit is divided into two regions of 1.5N ⁇ 2N and 0.5N ⁇ 1.5N in order from the left. Since the number of divisions is one of 1, 2, and 4, PUs included in the CU are 1 to 4. These PUs are expressed as PU0, PU1, PU2, and PU3 in order.
  • the encoding unit is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined.
  • the transform block is one or a plurality of non-overlapping areas constituting the encoding unit.
  • the conversion tree includes one or a plurality of conversion blocks obtained by the above division.
  • the division in the transformation tree includes the one in which an area having the same size as that of the encoding unit is assigned as the transformation block, and the one in the recursive quadtree division like the above-described division in the tree block.
  • a transform block that is a unit of transformation is also referred to as a transform unit (TU).
  • the prediction image of the prediction unit is derived by a prediction parameter associated with the prediction unit.
  • the prediction parameters include a prediction parameter for intra prediction or a prediction parameter for inter prediction.
  • prediction parameters for inter prediction inter prediction (inter prediction parameters) will be described.
  • the inter prediction parameter includes prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and vectors mvL0 and mvL1.
  • the prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list and L1 list are used, respectively, and a reference picture list corresponding to a value of 1 is used.
  • prediction list use flag information can also be expressed by an inter prediction flag inter_pred_idc described later.
  • a prediction list use flag is used in a prediction image generation unit and a prediction parameter memory described later, and an inter prediction flag inter_pred_idc is used when decoding information on which reference picture list is used from encoded data. It is done.
  • Syntax elements for deriving inter prediction parameters included in the encoded data include, for example, a partition mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference There is a vector mvdLX.
  • FIG. 3 is a conceptual diagram illustrating an example of a reference picture list.
  • the reference picture list 601 five rectangles arranged in a line on the left and right indicate reference pictures, respectively.
  • the codes P1, P2, Q0, P3, and P4 shown in order from the left end to the right are codes indicating respective reference pictures.
  • P such as P1 indicates the viewpoint P
  • Q of Q0 indicates a viewpoint Q different from the viewpoint P.
  • the subscripts P and Q indicate the picture order number POC.
  • a downward arrow directly below refIdxLX indicates that the reference picture index refIdxLX is an index that refers to the reference picture Q0 in the reference picture memory 306.
  • FIG. 4 is a conceptual diagram illustrating an example of a reference picture.
  • the horizontal axis indicates the display time
  • the vertical axis indicates the viewpoint.
  • the rectangles shown in FIG. 4 with 2 rows and 3 columns (6 in total) indicate pictures.
  • the rectangle in the second column from the left in the lower row indicates a picture to be decoded (target picture), and the remaining five rectangles indicate reference pictures.
  • a reference picture Q0 indicated by an upward arrow from the target picture is a picture that has the same display time as the target picture and a different viewpoint. In the displacement prediction based on the target picture, the reference picture Q0 is used.
  • a reference picture P1 indicated by a left-pointing arrow from the target picture is a past picture at the same viewpoint as the target picture.
  • a reference picture P2 indicated by a right-pointing arrow from the target picture is a future picture at the same viewpoint as the target picture. In motion prediction based on the target picture, the reference picture P1 or P2 is used.
  • >> is a right shift
  • ⁇ is a left shift. Therefore, as the inter prediction parameter, the prediction list use flags predFlagL0 and predFlagL1 may be used, or the inter prediction flag inter_pred_idc may be used.
  • the determination using the prediction list use flags predFlagL0 and predFlagL1 may be replaced with the inter prediction flag inter_pred_idc.
  • the determination using the inter prediction flag inter_pred_idc can be replaced with the prediction list use flags predFlagL0 and predFlagL1.
  • the prediction parameter decoding (encoding) method includes a merge mode and an AMVP (Adaptive Motion Vector Prediction) mode.
  • the merge flag merge_flag is a flag for identifying these.
  • the prediction parameter of the target PU is derived using the prediction parameter of the already processed block.
  • the merge mode is a mode that uses the prediction parameters already derived without including the prediction list use flag predFlagLX (inter prediction flag inter_pred_idc), the reference picture index refIdxLX, and the vector mvLX in the encoded data
  • the AMVP mode is an inter prediction.
  • the flag inter_pred_idc, the reference picture index refIdxLX, and the vector mvLX are included in the encoded data.
  • the vector mvLX is encoded as a prediction vector index mvp_LX_idx indicating a prediction vector and a difference vector (mvdLX).
  • the inter prediction flag inter_pred_idc is data indicating the type and number of reference pictures, and takes one of the values Pred_L0, Pred_L1, and Pred_Bi.
  • Pred_L0 and Pred_L1 indicate that reference pictures stored in reference picture lists called an L0 list and an L1 list are used, respectively, and that both use one reference picture (single prediction). Prediction using the L0 list and the L1 list are referred to as L0 prediction and L1 prediction, respectively.
  • Pred_Bi indicates that two reference pictures are used (bi-prediction), and indicates that two reference pictures stored in the L0 list and the L1 list are used.
  • the prediction vector index mvp_LX_idx is an index indicating a prediction vector
  • the reference picture index refIdxLX is an index indicating a reference picture stored in the reference picture list.
  • LX is a description method used when L0 prediction and L1 prediction are not distinguished.
  • refIdxL0 is a reference picture index used for L0 prediction
  • refIdxL1 is a reference picture index used for L1 prediction
  • refIdx (refIdxLX) is a notation used when refIdxL0 and refIdxL1 are not distinguished.
  • the merge index merge_idx is an index indicating which one of the prediction parameter candidates (merge candidates) derived from the processed block is used as the prediction parameter of the decoding target block.
  • the vector mvLX includes a motion vector and a displacement vector (disparity vector).
  • a motion vector is a positional shift between the position of a block in a picture at a certain display time of a layer and the position of the corresponding block in a picture of the same layer at a different display time (for example, an adjacent discrete time). It is a vector which shows.
  • the displacement vector is a vector indicating a positional shift between the position of a block in a picture at a certain display time of a certain layer and the position of a corresponding block in a picture of a different layer at the same display time.
  • the pictures in different layers may be pictures from different viewpoints or pictures with different resolutions.
  • a displacement vector corresponding to pictures of different viewpoints is called a disparity vector.
  • a vector mvLX A prediction vector and a difference vector related to the vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively.
  • Whether the vector mvLX and the difference vector mvdLX are motion vectors or displacement vectors is determined using a reference picture index refIdxLX associated with the vectors.
  • FIG. 5 is a schematic diagram illustrating a configuration of the image decoding device 31 according to the present embodiment.
  • the image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit 302, a reference picture memory (reference image storage unit, frame memory) 306, a prediction parameter memory (prediction parameter storage unit, frame memory) 307, and a prediction image generation unit 308.
  • An inverse quantization / inverse DCT unit 311, an addition unit 312, a residual storage unit 313 (residual recording unit), and a depth DV derivation unit 351 (not shown).
  • the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304.
  • the predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.
  • the entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside, and separates and decodes individual codes (syntax elements).
  • the separated codes include prediction information for generating a prediction image and residual information for generating a difference image.
  • the entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302.
  • Some of the separated codes are, for example, prediction mode PredMode, split mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, residual prediction weight The index iv_res_pred_weight_idx and the illumination compensation flag ic_flag. Control of which code to decode is performed based on an instruction from the prediction parameter decoding unit 302.
  • the entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311.
  • This quantization coefficient is a coefficient obtained by performing quantization and performing DCT (Discrete Cosine Transform) on the residual signal in the encoding process.
  • the entropy decoding unit 301 outputs the depth DV conversion table DepthToDisparityB to the depth DV deriving unit 351.
  • BitDepthY indicates the bit depth of the pixel value corresponding to the luminance signal, and takes, for example, 8 as the value.
  • the prediction parameter decoding unit 302 receives a part of the code from the entropy decoding unit 301 as an input.
  • the prediction parameter decoding unit 302 decodes the prediction parameter corresponding to the prediction mode indicated by the prediction mode PredMode that is a part of the code.
  • the prediction parameter decoding unit 302 outputs the prediction mode PredMode and the decoded prediction parameter to the prediction parameter memory 307 and the prediction image generation unit 308.
  • the inter prediction parameter decoding unit 303 decodes the inter prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301.
  • the inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.
  • the intra prediction parameter decoding unit 304 refers to the prediction parameter stored in the prediction parameter memory 307 on the basis of the code input from the entropy decoding unit 301 and decodes the intra prediction parameter.
  • the intra prediction parameter is a parameter used in a process of predicting a picture block within one picture, for example, an intra prediction mode IntraPredMode.
  • the intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.
  • the intra prediction parameter decoding unit 304 may derive different intra prediction modes depending on luminance and color difference.
  • the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the color difference prediction mode IntraPredModeC as the color difference prediction parameter.
  • the luminance prediction mode IntraPredModeY is a 35 mode and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2 to 34).
  • the color difference prediction mode IntraPredModeC uses one of planar prediction (0), DC prediction (1), direction prediction (2, 3, 4), and LM mode (5).
  • the reference picture memory 306 stores the reference picture block (reference picture block) generated by the adding unit 312 at a predetermined position for each picture and block to be decoded.
  • the prediction parameter memory 307 stores the prediction parameter in a predetermined position for each decoding target picture and block. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. .
  • the stored inter prediction parameters include, for example, a prediction list use flag predFlagLX (inter prediction flag inter_pred_idc), a reference picture index refIdxLX, and a vector mvLX.
  • the prediction mode predMode and the prediction parameter are input from the prediction parameter decoding unit 302 to the prediction image generation unit 308. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates prediction picture blocks predSmaples (prediction images) using the input prediction parameter and the read reference picture in the prediction mode indicated by the prediction mode predMode.
  • the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture to perform prediction picture block predSmaples by inter prediction. Is generated.
  • the prediction picture block predSmaples corresponds to the prediction unit PU.
  • the PU corresponds to a part of a picture composed of a plurality of pixels as a unit for performing the prediction process as described above, that is, a decoding target block on which the prediction process is performed at a time.
  • the inter predicted image generation unit 309 For the reference picture list (L0 list or L1 list) for which the prediction list use flag predFlagLX is 1, the inter predicted image generation unit 309 generates a vector mvLX based on the decoding target block from the reference picture indicated by the reference picture index refIdxLX. The reference picture block at the position indicated by is read from the reference picture memory 306. The inter prediction image generation unit 309 performs prediction on the read reference picture block to generate prediction picture blocks predSmaples. The inter prediction image generation unit 309 outputs the generated prediction picture block predSmaples to the addition unit 312.
  • the intra predicted image generation unit 310 When the prediction mode predMode indicates the intra prediction mode, the intra predicted image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference picture. Specifically, the intra predicted image generation unit 310 reads, from the reference picture memory 306, a reference picture block that is a decoding target picture and is in a predetermined range from the decoding target block among blocks that have already been decoded.
  • the predetermined range is, for example, any of the left, upper left, upper, and upper right adjacent blocks when the decoding target block sequentially moves in a so-called raster scan order, and varies depending on the intra prediction mode.
  • the raster scan order is an order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.
  • the intra predicted image generation unit 310 performs prediction in the prediction mode indicated by the intra prediction mode IntraPredMode for the read reference picture block, and generates a predicted picture block.
  • the intra predicted image generation unit 310 outputs the generated predicted picture block predSmaples to the addition unit 312.
  • the intra prediction image generation unit 310 performs planar prediction (0), DC prediction (1), direction according to the luminance prediction mode IntraPredModeY.
  • a prediction picture block of luminance is generated according to any of prediction (2 to 34), and planar prediction (0), DC prediction (1), direction prediction (2, 3, 4), LM according to the color difference prediction mode IntraPredModeC
  • a color difference prediction picture block is generated in any one of modes (5).
  • the inverse quantization / inverse DCT unit 311 inversely quantizes the quantization coefficient input from the entropy decoding unit 301 to obtain a DCT coefficient.
  • the inverse quantization / inverse DCT unit 311 performs inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient to calculate a decoded residual signal.
  • the inverse quantization / inverse DCT unit 311 outputs the calculated decoded residual signal to the addition unit 312 and the residual storage unit 313.
  • the adder 312 outputs the prediction picture block predSmaples input from the inter prediction image generation unit 309 and the intra prediction image generation unit 310 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. Addition to generate a reference picture block.
  • the adder 312 stores the generated reference picture block in the reference picture memory 306, and outputs a decoded layer image Td in which the generated reference picture block is integrated for each picture to the outside.
  • FIG. 6 is a schematic diagram illustrating a configuration of the inter prediction parameter decoding unit 303 according to the present embodiment.
  • the inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter derivation unit 3032, an addition unit 3035, and a merge mode parameter derivation unit 3036.
  • the inter prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode a code related to the inter prediction (the syntax element) includes, for example, a division mode part_mode, a merge included in the encoded data.
  • a flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, residual prediction weight index iv_res_pred_weight_idx, and illumination compensation flag ic_flag are extracted.
  • the inter prediction parameter decoding control unit 3031 first extracts a residual prediction weight index iv_res_pred_weight_idx and an illumination compensation flag ic_flag from the encoded data.
  • a residual prediction weight index iv_res_pred_weight_idx and an illumination compensation flag ic_flag from the encoded data.
  • the inter prediction parameter decoding control unit 3031 expresses that a certain syntax element is to be extracted, it means that the entropy decoding unit 301 is instructed to decode a certain syntax element, and the corresponding syntax element is read from the encoded data. To do.
  • the inter prediction parameter decoding control unit 3031 extracts a merge flag from the encoded data.
  • the inter prediction parameter decoding control unit 3031 extracts the merge index merge_idx as a prediction parameter related to the merge mode.
  • the inter prediction parameter decoding control unit 3031 outputs the extracted residual prediction weight index iv_res_pred_weight_idx, the illumination compensation flag ic_flag, and the merge index merge_idx to the merge mode parameter deriving unit 3036.
  • the inter prediction parameter decoding control unit 3031 uses the entropy decoding unit 301 to extract AMVP prediction parameters from the encoded data.
  • AMVP prediction parameters include an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a vector index mvp_LX_idx, and a difference vector mvdLX.
  • the inter prediction parameter decoding control unit 3031 outputs the prediction list use flag predFlagLX derived from the extracted inter prediction flag inter_pred_idc and the reference picture index refIdxLX to the AMVP prediction parameter derivation unit 3032 and the prediction image generation unit 308 (FIG.
  • the inter prediction parameter decoding control unit 3031 outputs the extracted vector index mvp_LX_idx to the AMVP prediction parameter derivation unit 3032.
  • the inter prediction parameter decoding control unit 3031 outputs the extracted difference vector mvdLX to the addition unit 3035.
  • the inter prediction parameter decoding control unit 3031 displays a displacement vector (NBDV) derived at the time of deriving the inter prediction parameter and a VSP mode flag VSPModeFlag that is a flag indicating whether or not to perform viewpoint synthesis prediction.
  • NBDV displacement vector
  • VSPModeFlag VSPModeFlag
  • FIG. 7 is a schematic diagram illustrating a configuration of the merge mode parameter deriving unit 3036 according to the present embodiment.
  • the merge mode parameter deriving unit 3036 includes a merge candidate deriving unit 30361 and a merge candidate selecting unit 30362.
  • the merge candidate derivation unit 30361 includes a merge candidate storage unit 303611, an extended merge candidate derivation unit 303612, and a basic merge candidate derivation unit 303613.
  • the merge candidate storage unit 303611 stores the merge candidates input from the extended merge candidate derivation unit 303612 and the basic merge candidate derivation unit 303613 in the merge candidate list mergeCandList.
  • the merge candidate includes a prediction list use flag predFlagLX, a vector mvLX, a reference picture index refIdxLX, a VSP mode flag VspModeFlag, a displacement vector MvDisp, and a layer ID RefViewIdx.
  • an index is assigned to the merge candidates stored in the merge candidate list mergeCandList according to a predetermined rule. For example, “0” is assigned as an index to the merge candidate input from the extended merge candidate derivation unit 303612.
  • VSP mode flag VspModeFlag is 0, 0 is set in the X and Y components of the displacement vector MvDisp, and ⁇ 1 is set in the layer ID RefViewIdx.
  • FIG. 18 shows an example of the merge candidate list mergeCandList derived by the merge candidate storage unit 303611. If two merge candidates have the same prediction parameter, excluding the processing of reducing the order, the merge index order, layer merge candidate (lower left), spatial merge candidate (upper right), spatial merge candidate (upper right), Displacement merge candidates, VSP merge candidates, spatial merge candidates (lower left), spatial merge candidates (upper left), and temporal merge candidates. Further, there are a merge merge candidate and a zero merge candidate after that, but they are omitted in FIG.
  • the extended merge candidate derivation unit 303612 includes a displacement vector acquisition unit 3036122, an inter-layer merge candidate derivation unit 3036121, a displacement merge candidate derivation unit 3036123, and a VSP merge candidate derivation unit 3036124.
  • the displacement vector acquisition unit 3036122 first acquires displacement vectors in order from a plurality of candidate blocks adjacent to the decoding target block (for example, blocks adjacent to the left, upper, and upper right). Specifically, one of the candidate blocks is selected, and whether the selected candidate block vector is a displacement vector or a motion vector is determined by using a reference picture index refIdxLX of the candidate block as a reference layer determination unit 303111 (described later). If there is a displacement vector, it is set as the displacement vector. If there is no displacement vector in the candidate block, the next candidate block is scanned in order.
  • the displacement vector acquisition unit 3036122 When there is no displacement vector in the adjacent block, the displacement vector acquisition unit 3036122 attempts to acquire the displacement vector of the block at the position corresponding to the target block of the block included in the reference picture in the temporally different display order. When the displacement vector cannot be acquired, the displacement vector acquisition unit 3036122 sets a zero vector as the displacement vector.
  • the obtained displacement vector is called NBDV (Neighbour Base Disparity Vector).
  • the displacement vector acquisition unit 3036122 outputs the obtained NBDV to the depth DV deriving unit 351, and receives the horizontal component of the depth base DV derived by the depth DV deriving unit 351 as an input.
  • the displacement vector acquisition unit 3036122 obtains an updated displacement vector by replacing the horizontal component of the NBDV with the horizontal component of the depth base DV input from the depth DV deriving unit 351 (the vertical component of the NBDV is unchanged).
  • the updated displacement vector is called DoNBDV (Depth Orientated Neighbour Base Disparity Vector).
  • DoNBDV Depth Orientated Neighbour Base Disparity Vector
  • the displacement vector acquisition unit 3036122 outputs the displacement vector (DoNBDV) to the inter-layer merge candidate derivation unit 3036121, the displacement merge candidate derivation unit 3036123, and the VSP merge candidate derivation unit 3036124. Further, the obtained displacement vector (NBDV) is output to the inter predicted image generation unit 309.
  • the inter-layer merge candidate derivation unit 3036121 receives the displacement vector from the displacement vector acquisition unit 3036122.
  • the inter-layer merge candidate derivation unit 3036121 selects a block indicated only by the displacement vector input from the displacement vector acquisition unit 3036122 from a picture having the same POC as the decoding target picture of another layer (eg, base layer, base view).
  • the prediction parameter which is a motion vector included in the block, is read from the prediction parameter memory 307. More specifically, the prediction parameter read by the inter-layer merge candidate derivation unit 3036121 is a prediction parameter of a block including coordinates obtained by adding a displacement vector to the coordinates of the starting point when the center point of the target block is the starting point. .
  • the coordinates (xRef, yRef) of the reference block are the coordinates of the target block (xP, yP), the displacement vector (mvDisp [0], mvDisp [1]), and the width and height of the target block are nPSW, nPSH.
  • xRef Clip3 (0, PicWidthInSamples L -1, xP + ((nPSW-1) >> 1) + ((mvDisp [0] + 2) >> 2)
  • yRef Clip3 (0, PicHeightInSamples L -1, yP + ((nPSH-1) >> 1) + ((mvDisp [1] + 2) >> 2)) It is derived by the following formula.
  • PicWidthInSamples L and PicHeightInSamples L represent the width and height of the image, respectively, and the function Clip3 (x, y, z) restricts (clips) z to not less than x and not more than y, and returns the restricted result. It is a function.
  • the inter-layer merge candidate derivation unit 3036121 determines whether or not the prediction parameter is a motion vector in the determination method of a reference layer determination unit 303111 (described later) included in the inter prediction parameter decoding control unit 3031 (not a displacement vector). The determination is made according to the determined method.
  • the inter-layer merge candidate derivation unit 3036121 outputs the read prediction parameter as a merge candidate to the merge candidate storage unit 303611. Further, when the prediction parameter cannot be derived, the inter-layer merge candidate deriving unit 3036121 outputs that fact to the displacement merge candidate deriving unit.
  • This merge candidate is a motion prediction inter-layer candidate (inter-view candidate) and is also described as an inter-layer merge candidate (motion prediction).
  • the displacement merge candidate derivation unit 3036123 receives the displacement vector from the displacement vector acquisition unit 3036122.
  • the displacement merge candidate derivation unit 3036123 generates a vector whose horizontal component is the horizontal component of the displacement vector to which the horizontal component is input and whose vertical component is zero.
  • the displacement merge candidate derivation unit 3036123 stores the generated vector and the reference picture index refIdxLX of the previous layer image pointed to by the displacement vector (for example, the index of the base layer image having the same POC as the decoding target picture) as a merge candidate.
  • This merge candidate is a displacement prediction inter-layer candidate (inter-view candidate) and is also described as an inter-layer merge candidate (displacement prediction).
  • the VSP merge candidate derivation unit 3036124 derives a VSP (View Synthesis Prediction) merge candidate.
  • the VSP merge candidate is a merge candidate used in a predicted image generation process by viewpoint synthesis prediction performed by the inter predicted image generation unit 309.
  • the VSP merge candidate derivation unit 3036124 receives the displacement vector from the displacement vector acquisition unit 3036122.
  • the VSP merge candidate derivation unit 3036124 inputs the input displacement vector mvDisp to the vector mvLX and the displacement vector MvDisp, the reference picture index of the reference picture indicating the previous layer image pointed to by the displacement vector to the reference picture index refIdxLX, and the displacement vector to The layer ID refViewIdx of the layer is set to the layer ID RefViewIdx, and a VSP merge candidate is derived by setting the VSP mode flag VspModeFlag to 1.
  • the VSP merge candidate derivation unit 3036124 outputs the derived VSP merge candidate to the merge candidate storage unit 303611.
  • the VSP merge candidate derivation unit 3036124 of the present embodiment receives the residual prediction weight index iv_res_pred_weight_idx and the illumination compensation flag ic_flag from the inter prediction parameter decoding control unit.
  • the VSP merge candidate derivation unit 3036124 performs VSP merge candidate derivation processing only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0. That is, only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0, the VSP merge candidate is added to the elements of the merge candidate list mergeCandList.
  • the VSP merge candidate derivation unit 3036124 does not add the VSP merge candidate to the elements of the merge candidate list mergeCandList when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0.
  • the calculation amount reduction is reduced by skipping the derivation process of VSP merge candidates that are not used. Since the variation of the merge index merge_idx can be suppressed by preventing an increase in merge candidates, the coding efficiency is improved.
  • the VSP merge candidate derivation unit 3036124 performs the VSP merge candidate derivation process only when the residual prediction weight index iv_res_pred_weight_idx is 0. That is, only when the residual prediction weight index iv_res_pred_weight_idx is 0, VSP merge candidates are added to the elements of the merge candidate list mergeCandList. Conversely, when the residual prediction weight index iv_res_pred_weight_idx is other than 0, no VSP merge candidate is added to the elements of the merge candidate list mergeCandList.
  • the VSP merge candidate derivation unit 3036124 performs VSP merge candidate derivation processing only when the illumination compensation flag ic_flag is 0. That is, only when the illumination compensation flag ic_flag is 0, the VSP merge candidate is added to the element of the merge candidate list mergeCandList. Conversely, when the illumination compensation flag ic_flag is other than 0, no VSP merge candidate is added to the elements of the merge candidate list mergeCandList.
  • VSP merge candidate derivation unit (Variation 1 of VSP merge candidate derivation unit)
  • the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0
  • the VSP merge candidate is not added to the element of the merge candidate list mergeCandList.
  • an alternative merge candidate that replaces the VSP merge candidate may be derived and added to the elements of the merge candidate list mergeCandList.
  • a merge candidate that is already stored at the top of the merge candidate list mergeCandList is changed to an alternative merge candidate.
  • 4 is added to the X component of the vector mvL0 of the merge candidate L0 prediction.
  • the method for changing the merge candidate is not limited to the above. For example, 8 may be subtracted from the Y component of the vector mvL0, or the vector mvL1 of the L1 prediction may be changed.
  • FIG. 19 shows an example of the merge candidate list mergeCandList in this modification.
  • the merge candidate whose merge index is 5 is either a VSP merge candidate or an alternative merge candidate.
  • the number of candidates stored in the merge candidate list mergeCandList does not change depending on the residual prediction weight index iv_res_pred_weight_idx and the value of the illumination compensation flag ic_flag. Therefore, when the encoding method is selected by the image encoding device 11, if the value of the residual prediction weight index iv_res_pred_weight_idx or the illumination compensation flag ic_flag is changed, the VSP candidate and the alternative candidate in the merge candidate list mergeCandList are simply replaced. In this case, there is no need to regenerate the entire merge candidate list mergeCandList, so that an increase in processing amount can be suppressed.
  • VSP merge candidate derivation unit As another form of the VSP merge candidate derivation unit, when the size of the target block is equal to or smaller than a predetermined size (for example, the width nPSW is 8 or less or the height nPSH is 8 or less), the element of the merge candidate list mergeCandList is A configuration in which merge candidates are not added may be adopted. That is, when the size of the target block is larger than a predetermined size (for example, the width nPSW is larger than 8 and the height nPSH is larger than 8), the VSP merge candidate is added to the element of the merge candidate list mergeCandList. .
  • a predetermined size for example, the width nPSW is 8 or less or the height nPSH is 8 or less
  • the viewpoint synthesis prediction unit 3094 described later divides the target block into sub-blocks, reads out a block at a position shifted by the disparity array disparitySampleArray from the reference picture for each sub-block, and generates a predicted image.
  • encoding efficiency is improved by setting a reading position for each sub-block and performing motion displacement prediction using different displacement vectors.
  • the effect is small because it is not divided into subblocks (or the number of divisions is small even when divided into subblocks).
  • VSP merge candidates are not derived.
  • the size of the target block is equal to or smaller than the predetermined size, that is, if it is not divided into sub-blocks (or the number of divisions is small even when divided into sub-blocks)
  • VSP merge candidates are not derived.
  • the basic merge candidate derivation unit 303613 includes a spatial merge candidate derivation unit 3036131, a temporal merge candidate derivation unit 3036132, a merge merge candidate derivation unit 3036133, and a zero merge candidate derivation unit 3036134.
  • the spatial merge candidate derivation unit 3036131 reads the prediction parameters (prediction list use flag predFlagLX, vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule, and uses the read prediction parameters as spatial merge candidates. Derived as The prediction parameter to be read out is for each adjacent block that is a block within a predetermined range from the decoding target block (for example, all or a part of the blocks in contact with the lower left end, upper left upper end, and upper right end of the decoding target block). This is a prediction parameter.
  • the derived spatial merge candidate is stored in the merge candidate storage unit 303611.
  • the spatial merge candidate derivation unit 3036131 inherits the VSP mode flag VspModeFlag of the adjacent block as the VSP mode flag VspModeFlag of the spatial merge candidate. That is, when the VSP mode flag VspModeFlag of the adjacent block is 1, the VSP mode flag VspModeFlag of the corresponding spatial merge candidate is 1, and in other cases, the VSP mode flag VspModeFlag is 0.
  • the spatial merge candidate derivation unit 3036131 inherits the displacement vector of the adjacent block and the layer ID of the layer indicated by the displacement vector. That is, the spatial merge candidate derivation unit 3036131 sets the displacement vector MvDisp of the adjacent block and the layer ID refViewIdx of the layer indicated by the displacement vector of the adjacent block as the displacement vector MvDisp and the layer ID RefViewIdx of the spatial merge candidate, respectively.
  • VSP mode flag VspModeFlag is set to 0.
  • the temporal merge candidate derivation unit 3036132 reads the prediction parameter of the block in the reference image including the lower right coordinate of the decoding target block from the prediction parameter memory 307 and sets it as a merge candidate.
  • the reference picture designation method may be, for example, the reference picture index refIdxLX designated in the slice header, or may be designated using the smallest reference picture index refIdxLX of the block adjacent to the decoding target block. .
  • the derived merge candidates are stored in the merge candidate storage unit 303611.
  • the merge merge candidate derivation unit 3036133 derives merge merge candidates by combining two different derived merge candidate vectors and reference picture indexes already derived and stored in the merge candidate storage unit 303611 as L0 and L1 vectors, respectively. To do.
  • the derived merge candidates are stored in the merge candidate storage unit 303611.
  • the zero merge candidate derivation unit 3036134 derives a merge candidate in which the reference picture index refIdxLX is 0 and both the X component and the Y component of the vector mvLX are 0.
  • the derived merge candidates are stored in the merge candidate storage unit 303611.
  • the merge candidate selection unit 30362 selects, from the merge candidates stored in the merge candidate storage unit 303611, a merge candidate to which an index corresponding to the merge index merge_idx input from the inter prediction parameter decoding control unit 3031 is assigned. As an inter prediction parameter. That is, when the merge candidate list is mergeCandList, the prediction parameter indicated by mergeCandList [merge_idx] is selected.
  • the merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 (FIG. 5) and outputs it to the prediction image generation unit 308 (FIG. 5).
  • FIG. 8 is a schematic diagram showing the configuration of the AMVP prediction parameter derivation unit 3032 according to this embodiment.
  • the AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033 and a prediction vector selection unit 3034.
  • the vector candidate derivation unit 3033 reads a vector (motion vector or displacement vector) stored in the prediction parameter memory 307 (FIG. 5) as a vector candidate mvpLX based on the reference picture index refIdx.
  • the vector to be read is a vector related to each of the blocks within a predetermined range from the decoding target block (for example, all or a part of the blocks in contact with the lower left end, the upper left upper end, and the upper right end of the decoding target block, respectively).
  • the prediction vector selection unit 3034 selects a vector candidate indicated by the vector index mvp_LX_idx input from the inter prediction parameter decoding control unit 3031 among the vector candidates read by the vector candidate derivation unit 3033 as the prediction vector mvpLX.
  • the prediction vector selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3035.
  • FIG. 9 is a conceptual diagram showing an example of vector candidates.
  • a predicted vector list 602 illustrated in FIG. 9 is a list including a plurality of vector candidates derived by the vector candidate deriving unit 3033.
  • five rectangles arranged in a line on the left and right indicate areas indicating prediction vectors, respectively.
  • the downward arrow directly below the second mvp_LX_idx from the left end and mvpLX below the mvp_LX_idx indicate that the vector index mvp_LX_idx is an index referring to the vector mvpLX in the prediction parameter memory 307.
  • the candidate vector is a block for which the decoding process has been completed, and is generated based on a vector related to the referenced block with reference to a block (for example, an adjacent block) in a predetermined range from the decoding target block.
  • the adjacent block has a block that is spatially adjacent to the target block, for example, the left block and the upper block, and a block that is temporally adjacent to the target block, for example, the same position as the target block, and has a different display time. Contains blocks derived from blocks.
  • the addition unit 3035 adds the prediction vector mvpLX input from the prediction vector selection unit 3034 and the difference vector mvdLX input from the inter prediction parameter decoding control unit to calculate a vector mvLX.
  • the adding unit 3035 outputs the calculated vector mvLX to the predicted image generation unit 308 (FIG. 5).
  • FIG. 10 is a block diagram illustrating a configuration of the inter prediction parameter decoding control unit 3031 according to the first embodiment.
  • the inter prediction parameter decoding control unit 3031 includes a residual prediction weight index decoding unit 30311, an illuminance compensation flag decoding unit 30312, and a split mode decoding unit, a merge flag decoding unit, a merge index decoding unit, not shown,
  • An inter prediction flag decoding unit, a reference picture index decoding unit, a vector candidate index decoding unit, and a vector difference decoding unit are configured.
  • the partition mode decoding unit, the merge flag decoding unit, the merge index decoding unit, the inter prediction flag decoding unit, the reference picture index decoding unit, the vector candidate index decoding unit, and the vector difference decoding unit are respectively divided mode part_mode, merge flag merge_flag, and merge index.
  • the merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX are decoded.
  • the residual prediction weight index decoding unit 30311 uses the entropy decoding unit 301 to decode the residual prediction weight index iv_res_pred_weight_idx.
  • the residual prediction weight index decoding unit 30311 outputs the decoded residual prediction weight index iv_res_pred_weight_idx to the merge mode parameter derivation unit 3036 and the inter prediction image generation unit 309.
  • the illuminance compensation flag decoding unit 30312 uses the entropy decoding unit 301 to decode the illuminance compensation flag ic_flag.
  • the illuminance compensation flag decoding unit 30312 outputs the decoded illuminance compensation flag ic_flag to the merge mode parameter derivation unit 3036 and the inter predicted image generation unit 309.
  • the displacement vector acquisition unit extracts the displacement vector from the prediction parameter memory 307, refers to the prediction parameter memory 307, and predicts the prediction flag of the block adjacent to the target PU.
  • the displacement vector acquisition unit includes a reference layer determination unit 303111 therein.
  • the displacement vector acquisition unit sequentially reads prediction parameters of blocks adjacent to the target PU, and determines whether the adjacent block has a displacement vector from the reference picture index of the adjacent block using the reference layer determination unit 303111. If the adjacent block has a displacement vector, the displacement vector is output. If there is no displacement vector in the prediction parameter of the adjacent block, the zero vector is output as the displacement vector.
  • Reference layer determination unit 303111 Based on the input reference picture index refIdxLX, the reference layer determination unit 303111 determines reference layer information reference_layer_info indicating a relationship between the reference picture indicated by the reference picture index refIdxLX and the target picture.
  • Reference layer information reference_layer_info is information indicating whether the vector mvLX to the reference picture is a displacement vector or a motion vector.
  • Prediction when the target picture layer and the reference picture layer are the same layer is called the same layer prediction, and the vector obtained in this case is a motion vector.
  • Prediction when the target picture layer and the reference picture layer are different layers is called inter-layer prediction, and the vector obtained in this case is a displacement vector.
  • the depth DV deriving unit 351 includes the depth DV conversion table DepthToDisparityB decoded from the encoded data by the entropy decoding unit 301, the width nSubBlkW and the height nSubBlkH of the subblock obtained from the inter prediction parameter decoding unit 303, and the division flag.
  • splitFlag depth image refDepPels, coordinates of corresponding blocks on depth image refDepPels (xTL, yTL), and layer ID refViewIdx, the following processing is used to derive disparity array disparitySamples that is the horizontal component of the depth-derived displacement vector To do.
  • the depth DV deriving unit 351 derives a depth representative value maxDep by using a plurality of sub-sub-block corners and points in the vicinity thereof for each sub-sub-block obtained by further dividing the sub-block.
  • the depth DV deriving unit 351 determines the width nSubSubBlkW and the height nSubSubBlkH of the sub-subblock.
  • the split flag splitFlag is 1, the sub-block width nSubBlkW and the height nSubBlkH are halved, or the sub-block width nSubBlkW is halved and the height nSubBlkH is the sub-sub-block width nSubSubBlkW and high.
  • the split flag splitFlag is 1
  • the sub-block width nSubBlkW and the height nSubBlkH are halved
  • the sub-block width nSubBlkW is halved
  • the height nSubBlkH is the sub-sub-block width nSubSubBlkW and high.
  • the pixel value of the depth image at the upper left corner of the sub block is refDepPelsP0
  • the upper right pixel value is refDepPelsP1
  • the lower left pixel value is refDepPelsP2
  • the width nSubSubBlkW and the height nSubSubBlkH of the sub-sub block are set to the same width nSubBlkW and height nSubBlkH as the sub-block.
  • the method for determining the width nSubSubBlkW and the height nSubSubBlkH of the sub-subblock is not limited to the above method.For example, a value obtained by halving the width and height of the sub-block may be set. The same width and height may be used.
  • the depth DV deriving unit 351 sets the left upper X coordinate xP0, the right end X coordinate xP1, and the upper end Y coordinate yP0 when the upper left relative coordinates of the sub sub block are (xSubB, ySubB).
  • xP0 Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB)
  • yP0 Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB)
  • xP1 Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB + nSubSubBlkW-1)
  • yP1 Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB + nSubSubBlkH-1) It is derived by the following formula. Note that pic_width_in_luma_samples and pic_height_in_luma_samples represent the width and height of the image, respectively.
  • the depth DV deriving unit 351 derives a representative value of the depth of the sub-subblock. Specifically, the pixel values refDepPels [xP0] [yP0], refDepPels [xP0], refDepPels [xP1] [yP0], refDepPels [xP1] [yP1] of the depth image at the corner of the sub-subblock and the four points in the vicinity thereof ]
  • maxDep Max (maxDep, refDepPels [xP0] [yP0])
  • maxDep Max (maxDep, refDepPels [xP0] [yP1])
  • maxDep Max (maxDep, refDepPels [xP1] [yP0])
  • maxDep Max (maxDep, refDepPels [xP1] [yP1])
  • maxDep Max (maxDep, refDepP
  • the depth DV deriving unit 351 performs the above processing on all sub-subblocks in the sub-block.
  • the depth DV derivation unit 351 outputs the derived parallax array disparitySamples to the displacement vector acquisition unit 3036122 and the viewpoint synthesis prediction unit 3094.
  • FIG. 11 is a schematic diagram illustrating a configuration of the inter predicted image generation unit 309 according to the present embodiment.
  • the inter prediction image generation unit 309 includes a motion displacement compensation unit 3091, a residual prediction unit 3092, an illuminance compensation unit 3093, a viewpoint synthesis prediction unit 3094, and an inter prediction image generation control unit 3096.
  • the inter prediction image generation control unit 3096 receives the VSP mode flag VspModeFlag and the prediction parameter from the inter prediction parameter decoding unit 303.
  • the inter prediction image generation control unit 3096 outputs the prediction parameter to the view synthesis prediction unit 3094.
  • the inter predicted image generation control unit 3096 outputs the prediction parameters to the motion displacement compensation unit 3091, the residual prediction unit 3092, and the illuminance compensation unit 3093.
  • the inter prediction image generation control unit 3096 performs residual prediction on the motion displacement compensation unit 3091 and the residual prediction execution flag resPredFlag when the residual prediction flag iv_res_pred_weight_idx is not 0 and the target block is motion compensation. 1 indicating execution is set and output to the residual prediction unit 3092.
  • the residual prediction flag iv_res_pred_weight_idx is 0, or when the target block is not motion compensation (in the case of disparity compensation)
  • the residual prediction execution flag resPredFlag is set to 0, and the motion displacement compensation unit 3091 and the residual The result is output to the prediction unit 3092.
  • the motion displacement compensation unit 3091 is based on the prediction list input flag predFlagLX, the reference picture index refIdxLX, and the vector mvLX (motion vector or displacement vector) that are the prediction parameters input from the inter prediction image generation control unit 3096. Is generated.
  • the motion displacement compensation unit 3091 reads out a block at a position shifted by the vector mvLX from the reference picture memory 306 from the position of the target block of the reference picture specified by the reference picture index refIdxLX, and interpolates the predicted image. Generate.
  • a prediction image is generated by applying a filter called a motion compensation filter (or displacement compensation filter) for generating a pixel at a decimal position.
  • a motion compensation filter or displacement compensation filter
  • the above processing is called motion compensation
  • the vector mvLX is a displacement vector
  • it is collectively referred to as motion displacement compensation.
  • the prediction image of L0 prediction is referred to as predSamplesL0
  • the prediction image of L1 prediction is referred to as predSamplesL1. If the two are not distinguished, they are called predSamplesLX.
  • the refResSamples residual prediction is a prediction image predSamplesLX that is an image obtained by predicting a residual of a reference layer (first layer image) different from a target layer (second layer image) that is a target of prediction image generation. This is done by adding to That is, assuming that the same residual as that of the reference layer also occurs in the target layer, the already derived residual of the reference layer is used as an estimated value of the residual of the target layer.
  • the reference layer In the base layer (base view), only the image of the same layer becomes the reference image. Therefore, when the reference layer (first layer image) is a base layer (base view), the predicted image of the reference layer is a predicted image by motion compensation, and thus depends on the target layer (second layer image). Also in prediction, residual prediction is effective in the case of a predicted image by motion compensation. That is, the residual prediction has a characteristic that it is effective when the target block is motion compensation.
  • FIG. 14 is a block diagram showing a configuration of the residual prediction unit 3092.
  • the residual prediction unit 3092 includes a reference image acquisition unit 30922 and a residual synthesis unit 30923.
  • the reference image acquisition unit 30922 stores the motion vector mvLX and the residual prediction displacement vector mvDisp input from the inter prediction parameter decoding unit 303 and the reference picture memory 306. Read the corresponding block currIvSamplesLX and the reference block refIvSamplesLX of the corresponding block.
  • FIG. 15 is a diagram for explaining the corresponding block currIvSamplesLX.
  • the corresponding block corresponding to the target block on the target layer is a displacement vector mvDisp that is a vector indicating the positional relationship between the reference layer and the target layer, starting from the position of the target block of the image on the reference layer. It is located in a block that is shifted by a distance.
  • the reference image acquisition unit 30922 derives a pixel at a position where the coordinates (x, y) of the pixel of the target block are shifted by the displacement vector mvDisp of the target block.
  • the reference image acquisition unit 30922 uses the X coordinate of the pixel R0 with integer precision corresponding to the pixel coordinate of the target block being (xP, yP).
  • xInt xPb + (mvLX [0] >> 2)
  • yInt yPb + (mvLX [1] >> 2)
  • xFrac mvLX [0] & 3
  • yFrac mvLX [1] & 3 It is derived by the following formula.
  • X & 3 is a mathematical expression for extracting only the lower 2 bits of X.
  • the reference image acquisition unit 30922 generates an interpolation pixel predPartLX [x] [y] in consideration of the fact that the displacement vector mvDisp has a pel resolution of 1/4 pel.
  • xA Clip3 (0, picWidthInSamples 1, xInt)
  • xB Clip3 (0, picWidthInSamples 1, xInt + 1)
  • xC Clip3 (0, picWidthInSamples 1, xInt)
  • xD Clip3 (0, picWidthInSamples 1, xInt + 1)
  • yA Clip3 (0, picHeightInSamples 1, yInt)
  • yB Clip3 (0, picHeightInSamples 1, yInt)
  • yC Clip3 (0, picHeightInSamples 1, yInt)
  • the integer pixel A is a pixel corresponding to the pixel R0
  • the integer pixels B, C, and D are integer precision pixels adjacent to the right, bottom, and bottom right of the integer pixel A, respectively.
  • the reference image acquisition unit 30922 includes reference pixels refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], and refPicLX [corresponding to the integer pixels A, B, C, and D, respectively.
  • xD] [yD] is read from the reference picture memory 306.
  • the reference image acquisition unit 30922 includes the reference pixel refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], refPicLX [xD] [yD] and the X component of the displacement vector mvDisp.
  • An interpolated pixel predPartLX [x] [y] which is a pixel shifted by the decimal part of the displacement vector mvDisp from the pixel R0, is derived using the fractional part xFrac and the fractional part yFrac of the Y component.
  • predPartLX [x] [y] (refPicLX [xA] [yA] * (8 xFrac) * (8 yFrac) + refPicLX [xB] [yB] * (8 yFrac) * xFrac + refPicLX [xC] [yC] * (8 xFrac) * yFrac + refPicLX [xD] [yD] * xFrac * yFrac)
  • the reference image acquisition unit 30922 performs the above interpolation pixel derivation process on each pixel in the target block, and sets a set of interpolation pixels as an interpolation block predPartLX.
  • the reference image acquisition unit 30922 outputs the derived interpolation block predPartLX to the residual synthesis unit 30923 as the corresponding block currIvSamplesLX.
  • FIG. 16 is a diagram for explaining the reference block refIvSamplesLX. As shown in FIG. 16, the reference block corresponding to the corresponding block on the reference layer is located at the block that is shifted by the motion vector mvLX of the target block, starting from the position of the corresponding block of the reference image on the reference layer. To do.
  • the reference image acquisition unit 30922 except for the process of deriving the corresponding block currIvSamplesLX and replacing the displacement vector mvDisp with a vector (mvDisp [0] + mvLX [0], mvDisp [1] + mvLX [1])
  • the corresponding block refIvSamplesLX is derived by performing the same processing.
  • the reference image acquisition unit 30922 outputs the corresponding block refIvSamplesLX to the residual synthesis unit 30923.
  • the residual synthesis unit 30923 derives a corrected predicted image predSamplesLX ′ from the predicted image predSamplesLX, the corresponding block currIvSamplesLX, the reference block refIvSamplesLX, and the residual prediction flag iv_res_pred_weight_idx.
  • the corrected predicted image predSamplesLX ⁇ predSamplesLX ⁇ predSamplesLX + ((currIvSamplesLX-refIvSamplesLX) >> (iv_res_pred_weight_idx-1)) It is calculated using the following formula.
  • the residual prediction implementation flag resPredFlag is 0, the residual synthesis unit 30923 outputs the predicted image predSamplesLX as it is.
  • the illumination compensation unit 3093 performs illumination compensation on the input predicted image predSamplesLX.
  • the input predicted image predSamplesLX is output as it is.
  • the prediction image predSamplesLX input to the illuminance compensation unit 3093 is an output image of the motion displacement compensation unit 3091 when the residual prediction execution flag resPredFlag is 0, and when the residual prediction execution flag resPredFlag is 1, It is an output image of the residual prediction unit 3092.
  • VSP mode flag VspModeFlag 1
  • the view synthesis prediction unit 3094 performs view synthesis prediction using the prediction parameter input from the inter prediction image generation control unit 3096.
  • the viewpoint synthesis prediction unit 3094 does not perform processing when the VSP mode flag VspModeFlag is 0.
  • View synthesis prediction is a process of dividing a target block into sub-blocks, and generating predicted images predSamples by reading out and interpolating blocks at positions shifted by the disparity array disparitySampleArray from the reference picture memory 306 in sub-block units. It is.
  • FIG. 17 is a block diagram showing a configuration of the viewpoint synthesis prediction unit 3094.
  • the viewpoint synthesis prediction unit 3094 includes a parallax array derivation unit 30941 and a reference image acquisition unit 30942.
  • the disparity array deriving unit 30941 derives a disparity array disparitySampleArray in units of sub blocks.
  • the disparity array deriving unit 30941 has the same POC as the decoding target picture from the reference picture memory 306, and the depth image refDepPels having the same layer ID as the layer ID RefViewIdx of the layer image indicated by the displacement vector. Is read.
  • the layer of the depth image refDepPels to be read may be the same layer as the reference picture indicated by the reference picture index refIdxLX, or may be the same layer as the image to be decoded.
  • the derived coordinates (xTL, yTL) indicate the coordinates of the block corresponding to the target block on the depth image refDepPels.
  • split flag splitFlag is a flag that is set to 1 when both the width nPSW and the height nPSH of the target block are larger than 4, and is set to 0 otherwise. That is, if the width nPSW and the height nPSH of the target block are both greater than 4, the width nSubBlkW and the height nSubBlkH of the subblock are both set to 8, and the width nPSW and the height nPSH of the target block are both set.
  • the width nPSW and the height nPSH of the target block are set.
  • the width nSubBlkW and the height nSubBlkH of the sub-block are not limited to the derivation result by the above formula, and values may be set within a range that does not exceed the size of the target block, such as 16 or 32, for example.
  • the disparity array deriving unit 30941 for every subblock in the target block, the width nSubBlkW and height nSubBlkH of the sub-block when the upper left pixel of the block is the origin, the split flag splitFlag, and the depth image refDepPels And the coordinates (xTL, yTL) of the corresponding block and the layer ID refViewIdx of the layer to which the reference picture indicated by the reference picture index refIdxLX belongs are output from the depth DV deriving unit 351 to the disparity array disparitySampleArray. Get.
  • the parallax array derivation unit 30941 outputs the derived parallax array disparitySampleArray to the reference image acquisition unit 30942.
  • the reference image acquisition unit 30942 calculates a prediction block predSamples from the disparity array disparitySampleArray input from the disparity array deriving unit 30941 and the reference picture index refIdxLX input from the inter prediction parameter decoding unit 303. To derive.
  • the reference image acquisition unit 30942 For each pixel in the target block, the reference image acquisition unit 30942 extracts, from the reference picture refPic specified by the reference picture index refIdxLX, a pixel at a position where the X coordinate is shifted from the coordinates of the corresponding pixel by the value of the corresponding disparity array disparitySampleArray. Extract.
  • the reference image acquisition unit 30942 has the coordinates of the upper left pixel of the target block as (xP, yP), and each pixel in the target block
  • the coordinates are (xL, yL) (xL takes a value from 0 to nPbWyL takes a value from 0 to nPbH)
  • the decimal part xFrac and yFrac of the disparity array disparitySampleArray [xL] [yL] corresponding to xIntL xP + xL + disparitySamples [xL] [yL]
  • the reference image acquisition unit 30942 performs an interpolation pixel derivation process similar to that of the reference image acquisition unit 30922 on each pixel in the target block, and sets a set of interpolation pixels as an interpolation block predPartLX.
  • the reference image acquisition unit 30942 outputs the derived interpolation block predPartLX to the addition unit 312 as the prediction block predSamples.
  • the inter prediction image generation unit can take a configuration that does not perform illuminance compensation prediction.
  • the motion displacement compensation unit 3091 ′ and the residual prediction unit 3092 ′ output the predicted image predSamplesLX to the addition unit 312.
  • the inter-prediction image generation unit can be configured not to perform residual prediction.
  • the motion displacement compensation unit 3091 ′′ always derives the predicted image predSamplesLX.
  • FIG. 20 is a block diagram illustrating a configuration of the image encoding device 11 according to the present embodiment.
  • the image encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a prediction parameter memory (prediction parameter storage). Unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, prediction parameter coding unit 111, and residual storage unit 313 (residual recording unit). Is done.
  • the prediction parameter encoding unit 111 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113.
  • the predicted image generation unit 101 generates predicted picture blocks predSmaples for each block that is an area obtained by dividing the picture for each viewpoint of the layer image T input from the outside.
  • the predicted image generation unit 101 reads the reference picture block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter encoding unit 111.
  • the prediction parameter input from the prediction parameter encoding unit 111 is, for example, a motion vector or a displacement vector.
  • the predicted image generation unit 101 reads the reference picture block of the block at the position indicated by the motion vector or the displacement vector predicted from the encoding target block.
  • the predicted image generation unit 101 generates predicted picture blocks predSmaples using one of the plurality of prediction schemes for the read reference picture block.
  • the predicted image generation unit 101 outputs the generated predicted picture block predSmaples to the subtraction unit 102 and the addition unit 106. Note that since the predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 already described, details of generation of the predicted picture block predSmaples are omitted.
  • the predicted image generation unit 101 calculates an error value based on the difference between the signal value for each pixel of the block included in the layer image and the signal value for each corresponding pixel of the predicted picture block predSmaples. Select the prediction method to minimize. Note that the method of selecting the prediction method is not limited to this.
  • the plurality of prediction methods are intra prediction, motion prediction, and merge mode.
  • Motion prediction is prediction between display times among the above-mentioned inter predictions.
  • the merge mode is a prediction that uses the same reference picture block and prediction parameter as a block that has already been encoded and is within a predetermined range from the encoding target block.
  • the plurality of prediction methods are intra prediction, motion prediction, merge mode (including viewpoint synthesis prediction), and displacement prediction.
  • the displacement prediction (disparity prediction) is prediction between different layer images (different viewpoint images) in the above-described inter prediction. For displacement prediction (disparity prediction), there are predictions with and without additional prediction (residual prediction and illuminance compensation).
  • the prediction image generation unit 101 When the intra prediction is selected, the prediction image generation unit 101 outputs a prediction mode predMode indicating the intra prediction mode used when generating the prediction picture block predSmaples to the prediction parameter encoding unit 111.
  • the prediction image generation unit 101 stores the motion vector mvLX used when generating the prediction picture block predSmaples in the prediction parameter memory 108 and outputs the motion vector mvLX to the inter prediction parameter encoding unit 112.
  • the motion vector mvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block predSmaples is generated.
  • the information indicating the motion vector mvLX may include information indicating a reference picture (for example, a reference picture index refIdxLX, a picture order number POC), and may represent a prediction parameter.
  • the predicted image generation unit 101 outputs a prediction mode predMode indicating the inter prediction mode to the prediction parameter encoding unit 111.
  • the prediction image generation unit 101 When the prediction image generation unit 101 selects the displacement prediction, the prediction image generation unit 101 stores the displacement vector used when generating the prediction picture block predSmaples in the prediction parameter memory 108 and outputs it to the inter prediction parameter encoding unit 112.
  • the displacement vector dvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block predSmaples is generated.
  • the information indicating the displacement vector dvLX may include information indicating a reference picture (for example, reference picture index refIdxLX, view IDview_id) and may represent a prediction parameter.
  • the predicted image generation unit 101 outputs a prediction mode predMode indicating the inter prediction mode to the prediction parameter encoding unit 111.
  • the predicted image generation unit 101 When the merge mode is selected, the predicted image generation unit 101 outputs a merge index merge_idx indicating the selected reference picture block to the inter prediction parameter encoding unit 112. Further, the predicted image generation unit 101 outputs a prediction mode predMode indicating the merge mode to the prediction parameter encoding unit 111.
  • the prediction image generation unit 101 performs the viewpoint synthesis prediction unit 3094 included in the prediction image generation unit 101 as described above. Perform viewpoint synthesis prediction. Further, in the motion prediction, displacement prediction, and merge mode, the prediction image generation unit 101 includes the prediction image generation unit 101 as described above when the residual prediction execution flag resPredFlag indicates that the residual prediction is performed. The residual prediction unit 3092 performs residual prediction.
  • the subtraction unit 102 subtracts the signal value of the predicted picture block predSmaples input from the predicted image generation unit 101 for each pixel from the signal value of the corresponding block of the layer image T input from the outside, and generates a residual signal. Generate.
  • the subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103 and the encoding parameter determination unit 110.
  • the DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102 and calculates a DCT coefficient.
  • the DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain a quantization coefficient.
  • the DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy encoding unit 104 and the inverse quantization / inverse DCT unit 105.
  • the entropy coding unit 104 receives the quantization coefficient from the DCT / quantization unit 103 and the coding parameter from the coding parameter determination unit 110.
  • the input encoding parameters include, for example, codes such as a reference picture index refIdxLX, a vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, a merge index merge_idx, a residual prediction weight index iv_res_pred_weight_idx, and an illumination compensation flag ic_flag.
  • the entropy encoding unit 104 generates an encoded stream Te by entropy encoding the input quantization coefficient and encoding parameter, and outputs the generated encoded stream Te to the outside.
  • the inverse quantization / inverse DCT unit 105 inversely quantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain a DCT coefficient.
  • the inverse quantization / inverse DCT unit 105 performs inverse DCT on the obtained DCT coefficient to calculate a decoded residual signal.
  • the inverse quantization / inverse DCT unit 105 outputs the calculated decoded residual signal to the addition unit 106, the residual storage unit 313, and the coding parameter determination unit 110.
  • the addition unit 106 adds the signal value of the prediction picture block predSmaples input from the prediction image generation unit 101 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and refers to them. Generate a picture block.
  • the adding unit 106 stores the generated reference picture block in the reference picture memory 109.
  • the prediction parameter memory 108 stores the prediction parameter generated by the prediction parameter encoding unit 111 at a predetermined position for each picture and block to be encoded.
  • the reference picture memory 109 stores the reference picture block generated by the adding unit 106 at a predetermined position for each picture and block to be encoded.
  • the encoding parameter determination unit 110 selects one set from among a plurality of sets of encoding parameters.
  • the encoding parameter is a parameter to be encoded that is generated in association with the above-described prediction parameter or the prediction parameter.
  • the predicted image generation unit 101 generates predicted picture blocks predSmaples using each of these sets of encoding parameters.
  • the encoding parameter determination unit 110 calculates a cost value indicating the amount of information and the encoding error for each of a plurality of sets.
  • the cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient ⁇ .
  • the code amount is the information amount of the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameter.
  • the square error is the sum between pixels regarding the square value of the residual value of the residual signal calculated by the subtracting unit 102.
  • the coefficient ⁇ is a real number larger than a preset zero.
  • the encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the set of unselected encoding parameters.
  • the prediction parameter encoding unit 111 derives a prediction parameter used when generating a prediction picture based on the parameter input from the prediction image generation unit 101, and encodes the derived prediction parameter to generate a set of encoding parameters. To do.
  • the prediction parameter encoding unit 111 outputs the generated set of encoding parameters to the entropy encoding unit 104.
  • the prediction parameter encoding unit 111 stores, in the prediction parameter memory 108, a prediction parameter corresponding to the set of the generated encoding parameters selected by the encoding parameter determination unit 110.
  • the prediction parameter encoding unit 111 operates the inter prediction parameter encoding unit 112 when the prediction mode predMode input from the prediction image generation unit 101 indicates the inter prediction mode.
  • the prediction parameter encoding unit 111 operates the intra prediction parameter encoding unit 113 when the prediction mode predMode indicates the intra prediction mode.
  • the inter prediction parameter encoding unit 112 derives an inter prediction parameter based on the prediction parameter input from the encoding parameter determination unit 110.
  • the inter prediction parameter encoding unit 112 includes the same configuration as the configuration in which the inter prediction parameter decoding unit 303 (see FIG. 5 and the like) derives the inter prediction parameter as a configuration for deriving the inter prediction parameter.
  • the configuration of the inter prediction parameter encoding unit 112 will be described later.
  • the intra prediction parameter encoding unit 113 determines the intra prediction mode IntraPredMode indicated by the prediction mode predMode input from the encoding parameter determination unit 110 as a set of inter prediction parameters.
  • the inter prediction parameter encoding unit 112 is means corresponding to the inter prediction parameter decoding unit 303.
  • FIG. 21 is a schematic diagram illustrating a configuration of the inter prediction parameter encoding unit 112 according to the present embodiment.
  • the inter prediction parameter encoding unit 112 includes a merge mode parameter deriving unit 1121, an AMVP prediction parameter deriving unit 1122, a subtracting unit 1123, and an inter prediction parameter encoding control unit 1126.
  • the merge mode parameter deriving unit 1121 has the same configuration as the merge mode parameter deriving unit 3036 (see FIG. 7).
  • the AMVP prediction parameter derivation unit 1122 has the same configuration as the AMVP prediction parameter derivation unit 3032 (see FIG. 7).
  • the subtraction unit 1123 subtracts the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the vector mvLX input from the coding parameter determination unit 110 to generate a difference vector mvdLX.
  • the difference vector mvdLX is output to the inter prediction parameter encoding control unit 1126.
  • the inter prediction parameter coding control unit 1126 instructs the entropy coding unit 104 to decode a code related to inter prediction (the syntax element) includes, for example, a code (syntax element) included in the coded data. , Merge flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX are encoded.
  • the inter prediction parameter encoding control unit 1126 includes an additional prediction flag encoding unit 10311, a merge index encoding unit 10312, a vector candidate index encoding unit 10313, a split mode encoding unit, a merge flag encoding unit, and an inter prediction flag.
  • An encoding unit, a reference picture index encoding unit, and a vector difference encoding unit are configured.
  • the division mode encoding unit, the merge flag encoding unit, the merge index encoding unit, the inter prediction flag encoding unit, the reference picture index encoding unit, the vector candidate index encoding unit 10313, and the vector difference encoding unit are respectively divided modes.
  • merge flag merge_flag merge index merge_idx
  • inter prediction flag inter_pred_idc reference picture index refIdxLX
  • prediction vector index mvp_LX_idx reference picture index refIdxLX
  • difference vector mvdLX difference vector
  • the additional prediction flag encoding unit 10311 encodes the illumination compensation flag ic_flag and the residual prediction weight index iv_res_pred_weight_idx to indicate whether or not additional prediction is performed.
  • the inter prediction parameter encoding control unit 1126 uses the merge index merge_idx input from the encoding parameter determination unit 110 as the entropy encoding unit 104. To be encoded.
  • the inter prediction parameter encoding control unit 1126 performs the following process.
  • the inter prediction parameter encoding control unit 1126 integrates the reference picture index refIdxLX and the vector index mvp_LX_idx input from the encoding parameter determination unit 110, and the difference vector mvdLX input from the subtraction unit 1123.
  • the inter prediction parameter encoding control unit 1126 outputs the integrated code to the entropy encoding unit 104 to be encoded.
  • the VSP mode flag VspModeFlag and the layer ID of the layer indicated by the displacement vector and the displacement vector are inherited from adjacent blocks, but when the viewpoint synthesis prediction is not performed. These inherited parameters are never referenced.
  • the inheritance of the layer ID of the layer indicated by the VSP mode flag VspModeFlag, the displacement vector, and the displacement vector from the adjacent block is zero in the residual prediction weight index iv_res_pred_weight_idx, and This is performed only when the illumination compensation flag ic_flag is 0.
  • the spatial merge candidate derivation unit 3036131A does not perform inheritance from adjacent blocks when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0.
  • the image transmission system 1A in the present embodiment is different from the image transmission system 1 in FIG. 1 in that an image encoding device 11A is provided instead of the image encoding device 11, and an image decoding device 31A is provided instead of the image decoding device 31. .
  • the image decoding device 31A differs from the image decoding device 31 of FIG. 5 only in that it has an inter prediction parameter decoding unit 303A instead of the inter prediction parameter decoding unit 303.
  • the inter prediction parameter decoding unit 303A is different from the inter prediction parameter decoding unit 303 in FIG. 6 only in that it includes a merge mode parameter deriving unit 3036A instead of the merge mode parameter deriving unit 3036.
  • FIG. 22 is a block diagram illustrating a configuration of the merge mode parameter deriving unit 3036A according to the second embodiment.
  • the merge mode parameter deriving unit 3036A differs from the merge mode parameter deriving unit 3036 of FIG. 7 only in that it has a spatial merge candidate deriving unit 3036131A instead of the spatial merge candidate deriving unit 3036131.
  • the spatial merge candidate derivation unit 3036131A is different from the spatial merge candidate derivation unit 3036131 in that it receives the residual prediction weight index iv_res_pred_weight_idx and the illumination compensation flag ic_flag as inputs.
  • the spatial merge candidate derivation unit 3036131A derives a prediction parameter read from the prediction parameter memory 307 as a spatial merge candidate by the same method as the spatial merge candidate derivation unit 3036131.
  • the spatial merge candidate derivation unit 3036131A sets the adjacent block as the VSP mode flag VspModeFlag, the displacement vector MvDisp, and the layer IDRefViewIdx of the spatial merge candidate only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0.
  • the VSP mode flag VspModeFlag, the displacement vector, and the layer ID of the layer indicated by the displacement vector are inherited.
  • the spatial merge candidate derivation unit 3036131A performs the spatial merge candidate VSP mode flag VspModeFlag, the displacement vector MvDisp, and the layer ID RefViewIdx. Is not carried out, and 0 is set as the VSP mode flag VspModeFlag of the spatial merge candidate.
  • the image encoding device 11A differs from the image encoding device 11 of FIG. 20 only in that it has a merge mode parameter deriving unit 1121A instead of the merge mode parameter deriving unit 1121.
  • the merge mode parameter deriving unit 1121A differs from the merge mode parameter deriving unit 3036 only in that it has a spatial merge candidate deriving unit 3036131A instead of the spatial merge candidate deriving unit 3036131.
  • the image transmission system 1B in this embodiment is different from the image transmission system 1 in FIG. 1 in that an image encoding device 11B is provided instead of the image encoding device 11, and an image decoding device 31B is provided instead of the image decoding device 31. .
  • the image decoding device 31B is different from the image decoding device 31 in FIG. 5 only in that an inter prediction parameter decoding unit 303B is provided instead of the inter prediction parameter decoding unit 303.
  • the inter prediction parameter decoding unit 303B is different from the inter prediction parameter decoding unit 303 in FIG. 6 only in that it has an inter prediction parameter decoding control unit 3031B instead of the inter prediction parameter decoding control unit 3031.
  • FIG. 23 is a block diagram illustrating a configuration of the inter prediction parameter decoding control unit 3031B according to the third embodiment.
  • the inter prediction parameter decoding control unit 3031B replaces the residual prediction weight index decoding unit 30311 with a residual prediction weight index decoding unit 30311B, and an illuminance compensation flag decoding unit 30312.
  • an illumination compensation flag decoding unit 30312B is provided instead of.
  • the residual prediction weight index decoding unit 30311B uses the entropy decoding unit 301 to decode the residual prediction weight index iv_res_pred_weight_idx, but the output destination of the residual prediction weight index iv_res_pred_weight_idx Is different in that an illuminance compensation flag decoding unit 30312B is included.
  • the illumination compensation flag decoding unit 30312B receives the residual prediction weight index iv_res_pred_weight_idx from the residual prediction weight index decoding unit 30311B as an input.
  • the illuminance compensation flag decoding unit 30312B decodes the illuminance compensation flag ic_flag only when the residual prediction weight index iv_res_pred_weight_idx is 0.
  • the illuminance compensation flag decoding unit 30312B does not decode the illuminance compensation flag ic_flag but derives the illuminance compensation flag ic_flag set to 0.
  • the illuminance compensation flag decoding unit 30312B outputs the illuminance compensation flag ic_flag to the merge mode parameter derivation unit 3036 and the inter predicted image generation unit 309.
  • FIG. 24 is a part of a syntax table showing a configuration of encoded data to be decoded by the illumination compensation flag decoding unit 30312B.
  • the portions indicated by S1001 and S1002 are portions related to the illuminance compensation flag ic_flag, and at least the residual prediction weight index iv_res_pred_weight_idx needs to be 0 for the illuminance compensation flag ic_flag to exist in the encoded data. Show.
  • the image encoding device 11B differs from the image encoding device 11 of FIG. 20 in that an encoding parameter determination unit 110B replaces the encoding parameter determination unit 110, and an inter prediction parameter encoding unit replaces the inter prediction parameter encoding unit 112. The only difference is that it has 112B.
  • the encoding parameter determination unit 110B is different from the encoding parameter determination unit 110 in that the illumination compensation flag ic_flag is always set to 0 when the residual prediction weight index iv_res_pred_weight_idx is set to other than 0.
  • the inter prediction parameter encoding unit 112B is different from the inter prediction parameter encoding unit 112 of FIG. 21 only in that it has an inter prediction parameter encoding control 1126B instead of the inter prediction parameter encoding control unit 1126.
  • the inter prediction parameter coding control unit 1126B is different from the inter prediction parameter coding control unit 1126 in that the illuminance compensation flag ic_flag is not output to the entropy coding unit when the residual prediction weight index iv_res_pred_weight_idx is other than 0. .
  • the luminance compensation prediction that is effective when performing disparity compensation is disabled, thereby affecting the coding efficiency.
  • the residual prediction is performed while stopping the above, it is possible to omit the encoding of the luminance compensation flag, and the encoding efficiency is improved.
  • the entropy decoding unit 301 the prediction parameter decoding unit 302
  • the predicted image generation unit 101 the DCT / Quantization unit 103
  • entropy encoding unit 104 the inverse quantization / inverse DCT unit 105
  • encoding parameter determination unit 110 prediction parameter encoding unit 111
  • entropy decoding unit 301 prediction parameter decoding unit 302
  • prediction image generation unit 308 The inverse quantization / inverse DCT unit 311 may be realized by a computer.
  • the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed.
  • the “computer system” here is a computer system built in either the image encoding device 11-11B or the image decoding device 31-31B, and includes an OS and hardware such as peripheral devices.
  • the “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk incorporated in a computer system.
  • the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line,
  • a volatile memory inside a computer system serving as a server or a client may be included and a program that holds a program for a certain period of time.
  • the program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
  • part or all of the image encoding devices 11, 11A, 11B and the image decoding devices 31, 31A, 31B in the above-described embodiments may be realized as an integrated circuit such as an LSI (Large Scale Integration).
  • LSI Large Scale Integration
  • Each functional block of the image encoding devices 11, 11A, 11B and the image decoding devices 31, 31A, 31B may be individually made into a processor, or a part or all of them may be integrated into a processor.
  • the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology may be used.
  • the present invention has been made to solve the above-described problems, and one aspect of the present invention is an image decoding apparatus that generates and decodes a predicted image of a target block, and performs prediction using residual prediction.
  • a residual prediction unit that generates an image
  • a viewpoint synthesis prediction unit that generates a predicted image using viewpoint synthesis prediction
  • a residual prediction weight index that decodes a residual prediction weight index indicating whether or not to use residual prediction
  • the merge mode parameter deriving unit comprising a view synthesis prediction merge candidate deriving unit for deriving a VSP merge candidate that is a prediction parameter used for view synthesis prediction
  • the view synthesis prediction merge candidate derivation unit derives the VSP merge candidate only when the residual prediction weight index is 0.
  • the other aspect of this invention is an image decoding apparatus which produces
  • the merge mode parameter derivation unit includes a view synthesis prediction merge candidate derivation unit that derives a VSP merge candidate that is a prediction parameter used for view synthesis prediction, and the view synthesis prediction merge candidate derivation unit includes the illuminance compensation flag. Only when V is 0, the VSP merge candidate is derived.
  • the image decoding apparatus wherein the merge mode parameter deriving unit further derives a space candidate and further derives a space candidate.
  • the viewpoint synthesis prediction is used for decoding the adjacent block referred to for this purpose, a spatial candidate derivation unit that inherits prediction parameters for the viewpoint synthesis prediction of the neighboring block is provided, and the spatial candidate derivation unit includes When the residual prediction weight index is 0, the prediction parameter for the viewpoint synthesis prediction of the adjacent block is inherited.
  • another aspect of the present invention is the image decoding device according to (2), wherein the merge mode parameter deriving unit further derives a space candidate and further derives a space candidate.
  • the viewpoint synthesis prediction is used for decoding the adjacent block referred to for this purpose, a spatial candidate derivation unit that inherits prediction parameters for the viewpoint synthesis prediction of the neighboring block is provided, and the spatial candidate derivation unit includes When the illumination compensation flag is 0, the prediction parameter for the viewpoint synthesis prediction of the adjacent block is inherited.
  • the prediction parameter for the viewpoint synthesis prediction of the adjacent block is the viewpoint synthesis of the adjacent block. This is a viewpoint synthesis prediction flag indicating whether or not prediction has been performed.
  • a prediction parameter for viewpoint synthesis prediction of the adjacent block includes a vector of the adjacent block.
  • the image decoding device wherein the viewpoint synthesis prediction merge candidate derivation unit is configured such that the residual prediction weight index is other than 0.
  • the alternative merge candidate is derived instead of the VSP merge candidate.
  • the viewpoint synthesis prediction merge candidate derivation unit includes the illuminance compensation flag other than 0,
  • An alternative merge candidate is derived instead of the VSP merge candidate.
  • Another aspect of the present invention is the image decoding device according to (7) or (8), in which the viewpoint synthesis prediction merge candidate derivation unit has already been derived as the alternative merge candidate A merge candidate is derived by changing the values of the X and Y components of the merge candidate vector.
  • Another aspect of the present invention is an image decoding apparatus that generates and decodes a predicted image of a target block, and includes a viewpoint synthesis prediction unit that generates a predicted image using viewpoint synthesis prediction, and a prediction parameter.
  • the deriving unit does not derive a VSP merge candidate when the size of the target block is equal to or smaller than a predetermined size.
  • an image decoding apparatus that generates and decodes a predicted image of a target block, a residual prediction unit that generates a predicted image using residual prediction, and a prediction image
  • An illuminance compensation unit that applies illuminance compensation, a residual prediction weight index decoding unit that decodes a residual prediction weight index that indicates whether to use residual prediction, and an illuminance compensation flag that indicates whether to apply illuminance compensation
  • the illuminance compensation flag decoding unit decodes the illuminance compensation flag only when the residual prediction weight index is 0.
  • the present invention can be suitably applied to an image decoding apparatus that decodes encoded data obtained by encoding image data and an image encoding apparatus that generates encoded data obtained by encoding image data. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.
  • Vector candidate index decoding unit 3032 ... AMVP prediction parameter derivation unit 3033 ...
  • Vector candidate derivation unit 3034 ... Prediction vector selection unit 3035 ...
  • Addition unit 3036, 3036A ... Merge mode parameter derivation unit 30361 ... Merge candidate derivation unit 303611 ... Merge candidate storage Unit 303612 ... extended merge candidate derivation unit 3036121 ... inter-layer merge candidate derivation unit 3036122 ... displacement vector acquisition unit 3036123 ... displacement merge candidate derivation unit 3036124 ... VSP merge candidate derivation unit 303613 ... basic merge candidate derivation units 3036131 and 3036131A ... spatial merge candidates Deriving unit 3036132 ... Time merge candidate deriving unit 3036133 ...
  • Join merge candidate deriving unit 3036134 Zero merge candidate deriving unit 30362 ... Merge candidate selecting unit 304 ...
  • Intra prediction parameter decoding unit 306 Reference picture memory (frame memory) 307 ... Prediction parameter memory (frame memory) 308 ... Prediction image generation unit 309 ...
  • Inter prediction image generation unit 3091 ... Displacement compensation unit 3092 . Residual prediction unit 30922 ... Reference image acquisition unit 30923 ... Residual synthesis unit 3093 ... Illuminance compensation unit 3094 ... Viewpoint synthesis prediction unit 30941 ... Parallax Array derivation unit 30942 ... reference image acquisition unit 3096 ... inter prediction image generation control unit 310 ... intra prediction image generation unit 311 ... inverse quantization / inverse DCT unit 312 ... addition unit 313 ... residual storage unit 351 ... depth DV derivation unit 41 ... Image display device

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Even if residual prediction or illumination compensation is performed, that is, view synthesis prediction is not performed when a prediction image is generated, processing for deriving a VSP candidate has been performed. When a residual prediction weight index that is an index indicating whether the residual prediction is used is 1 or more, or an illumination compensation flag indicating whether the illumination compensation is used is 1, the processing for deriving the VSP candidate, which is never used, is not performed, thereby making it possible to reduce the calculation amount.

Description

画像復号装置、画像符号化装置Image decoding apparatus and image encoding apparatus
　本発明は、画像復号装置、画像符号化装置に関する。 The present invention relates to an image decoding device and an image encoding device.
　複数視点の画像符号化技術には、複数の視点の画像を符号化する際に画像間の視差を予測することによって情報量を低減する視差予測符号化や、その符号化方法に対応した復号方法が提案されている。視点画像間の視差を表すベクトルを変位ベクトルと呼ぶ。変位ベクトルは、水平方向の要素（ｘ成分）と垂直方向の要素（ｙ成分）を有する２次元のベクトルであり、１つの画像を分割した領域であるブロック毎に算出される。また、複数視点の画像を取得するには、それぞれの視点に配置されたカメラを用いることが一般的である。複数視点の符号化では、各視点画像は、複数のレイヤにおいてそれぞれ異なるレイヤとして符号化される。複数のレイヤから構成される動画像の符号化方法は、一般に、スケーラブル符号化又は階層符号化と呼ばれる。スケーラブル符号化では、レイヤ間で予測を行うことで、高い符号化効率を実現する。レイヤ間で予測を行わずに基準となるレイヤは、ベースレイヤ、それ以外のレイヤは拡張レイヤと呼ばれる。レイヤが視点画像から構成される場合のスケーラブル符号化を、ビュースケーラブル符号化と呼ぶ。このとき、ベースレイヤはベースビュー、拡張レイヤは非ベースビューとも呼ばれる。さらに、ビュースケーラブルに加え、レイヤがテクスチャレイヤ（画像レイヤ）とデプスレイヤ（距離画像レイヤ）から構成される場合のスケーラブル符号化は、３次元スケーラブル符号化と呼ばれる。 The multi-view image encoding technique includes a parallax predictive encoding that reduces the amount of information by predicting a parallax between images when encoding images of a plurality of viewpoints, and a decoding method corresponding to the encoding method. Has been proposed. A vector representing the parallax between viewpoint images is called a displacement vector. The displacement vector is a two-dimensional vector having a horizontal element (x component) and a vertical element (y component), and is calculated for each block which is an area obtained by dividing one image. In order to acquire images from a plurality of viewpoints, it is common to use cameras arranged at the respective viewpoints. In multi-viewpoint encoding, each viewpoint image is encoded as a different layer in each of a plurality of layers. A method for encoding a moving image composed of a plurality of layers is generally referred to as scalable encoding or hierarchical encoding. In scalable coding, high coding efficiency is realized by performing prediction between layers. A reference layer without performing prediction between layers is called a base layer, and other layers are called enhancement layers. Scalable encoding in the case where a layer is composed of viewpoint images is referred to as view scalable encoding. At this time, the base layer is also called a base view, and the enhancement layer is also called a non-base view. Furthermore, in addition to view scalable, scalable coding when a layer is composed of a texture layer (image layer) and a depth layer (distance image layer) is called three-dimensional scalable coding.
　また、スケーラブル符号化には、ビュースケーラブル符号化の他、空間的スケーラブル符号化（ベースレイヤとして解像度の低いピクチャ、拡張レイヤが解像度の高いピクチャを処理）、ＳＮＲスケーラブル符号化（ベースレイヤとして画質の低いピクチャ、拡張レイヤとして解像度の高いピクチャを処理）等がある。スケーラブル符号化では、例えばベースレイヤのピクチャを、拡張レイヤのピクチャの符号化において、参照ピクチャとして用いることがある。 For scalable coding, in addition to view scalable coding, spatial scalable coding (pictures with low resolution as the base layer and pictures with high resolution in the enhancement layer), SNR scalable coding (image quality as the base layer) Low picture, high resolution picture as an enhancement layer). In scalable coding, for example, a base layer picture may be used as a reference picture in coding an enhancement layer picture.
　また、非特許文献１では、通常の動き補償予測でえられる予測対象ブロックに予測された残差を足し合わせることにより、より精度の高い予測画像を得る残差予測と呼ばれる技術が知られている。また、レイヤ間の輝度の差を補償することにより、より精度の高い予測画像を得る照度補償予測と呼ばれる技術が知られている。また、予測対象ブロックをさらに小さいサブブロックに分割し、サブブロックごとに変位ベクトルを用いて予測をすることにより、より精度の高い予測画像を得る視点合成予測と呼ばれる技術が知られている。 Further, in Non-Patent Document 1, a technique called residual prediction is known in which a predicted image with higher accuracy is obtained by adding a predicted residual to a prediction target block obtained by normal motion compensation prediction. . In addition, a technique called illuminance-compensated prediction that obtains a predicted image with higher accuracy by compensating for a difference in luminance between layers is known. In addition, a technique called viewpoint synthesis prediction is known in which a prediction target block is divided into smaller subblocks and prediction is performed using a displacement vector for each subblock, thereby obtaining a prediction image with higher accuracy.
　非特許文献１の視点合成予測に必要なパラメータはマージ候補の１つとして導出される。しかしながら、非特許文献１において、視点合成予測は残差予測又は照度補償予測と同時に用いられることはない。そのため、予測画像の生成方法として残差予測又は照度補償予測が選択された場合には、利用されない視点合成用のマージ候補が導出されてしまうという課題がある。 The parameter required for the view synthesis prediction of Non-Patent Document 1 is derived as one of merge candidates. However, in Non-Patent Document 1, viewpoint synthesis prediction is not used simultaneously with residual prediction or illumination compensation prediction. Therefore, when residual prediction or illumination compensation prediction is selected as a predicted image generation method, there is a problem that merge candidates for viewpoint synthesis that are not used are derived.
　本発明は上記の点に鑑みてなされたものであり、マージにおける候補数の増加を防ぐことにより、符号化効率の改善と計算量を低減させる画像復号装置、画像復号方法、画像復号プログラム、画像符号化装置、画像符号化方法、画像符号化プログラム、画像表示システム及び画像伝送システムを提供する。 The present invention has been made in view of the above points, and by preventing an increase in the number of candidates in merging, an image decoding apparatus, an image decoding method, an image decoding program, and an image that improve the encoding efficiency and reduce the amount of calculation An encoding apparatus, an image encoding method, an image encoding program, an image display system, and an image transmission system are provided.
　この発明は上述した課題を解決するためになされたもので、本発明の一態様は、残差予測を用いて予測画像を生成する残差予測部と、前記残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、マージ候補リストを導出するマージモードパラメータ導出部とを備え、前記マージモードパラメータ導出部は、視点合成予測に用いるVSPマージ候補を導出し、前記残差予測重みインデックスが０の場合に、前記VSPマージ候補をマージ候補リストに追加することを特徴とする。 The present invention has been made to solve the above-described problem, and one aspect of the present invention is a residual prediction unit that generates a predicted image using residual prediction, and whether or not to use the residual prediction. A residual prediction weight index decoding unit for decoding the residual prediction weight index shown, and a merge mode parameter deriving unit for deriving a merge candidate list, wherein the merge mode parameter deriving unit selects a VSP merge candidate used for view synthesis prediction. When the residual prediction weight index is 0, the VSP merge candidate is added to the merge candidate list.
　また、本発明の他の態様は、予測画像に照度補償を適用する照度補償部と、照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部と、マージ候補リストを導出するマージモードパラメータ導出部とを備え、前記マージモードパラメータ導出部は、視点合成予測に用いる予測パラメータであるVSPマージ候補を導出し、前記照度補償フラグが０である場合に、前記VSPマージ候補をVSPマージ候補をマージ候補リストに追加することを特徴とする。 In another aspect of the present invention, an illumination compensation unit that applies illumination compensation to a predicted image, an illumination compensation flag decoding unit that decodes an illumination compensation flag indicating whether to apply illumination compensation, and a merge candidate list A merge mode parameter deriving unit for deriving, the merge mode parameter deriving unit deriving a VSP merge candidate that is a prediction parameter used for view synthesis prediction, and when the illumination compensation flag is 0, the VSP merge candidate Is added to the merge candidate list.
　また、本発明の他の態様は、残差予測を用いて予測画像を生成する残差予測部と、残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、マージ候補リストを導出するマージモードパラメータ導出部と、VSPモードフラグが１の場合に視点合成予測を行う視点合成予測部を備え、前記マージモードパラメータ導出部は、前記残差予測重みインデックスが０以外の場合に、VSPモードフラグとして０を設定することを特徴とする。 Another aspect of the present invention provides a residual prediction unit that generates a prediction image using residual prediction, and a residual prediction weight index that decodes a residual prediction weight index indicating whether to use residual prediction. A decoding unit; a merge mode parameter deriving unit that derives a merge candidate list; and a view synthesis prediction unit that performs view synthesis prediction when the VSP mode flag is 1, wherein the merge mode parameter deriving unit includes the residual prediction weight When the index is other than 0, 0 is set as the VSP mode flag.
　また、本発明の他の態様は、予測画像に照度補償を適用する照度補償部と、照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部と、予測パラメータを導出するマージモードパラメータ導出部と、VSPモードフラグが１の場合に、視点合成予測を行う視点合成予測部を備え、前記マージモードパラメータ導出部は、前記照度補償フラグが０以外の場合に、VSPモードフラグとして０を設定することを特徴とする。 According to another aspect of the present invention, an illumination compensation unit that applies illumination compensation to a predicted image, an illumination compensation flag decoding unit that decodes an illumination compensation flag indicating whether to apply illumination compensation, and a prediction parameter are derived. A merge mode parameter deriving unit that performs a view synthesis prediction when the VSP mode flag is 1, and the merge mode parameter deriving unit performs a VSP mode when the illumination compensation flag is other than 0. It is characterized in that 0 is set as a flag.
　本発明によれば、視点合成予測における符号化効率が向上し、計算量が低減される。 According to the present invention, the encoding efficiency in view synthesis prediction is improved and the amount of calculation is reduced.
本発明の実施形態に係る画像伝送システムの構成を示す概略図である。1 is a schematic diagram illustrating a configuration of an image transmission system according to an embodiment of the present invention. 本実施形態に係る符号化ストリームのデータの階層構造を示す図である。It is a figure which shows the hierarchical structure of the data of the encoding stream which concerns on this embodiment. 参照ピクチャリストの一例を示す概念図である。It is a conceptual diagram which shows an example of a reference picture list. 参照ピクチャの例を示す概念図である。It is a conceptual diagram which shows the example of a reference picture. 本実施形態に係る画像復号装置の構成を示す概略図である。It is the schematic which shows the structure of the image decoding apparatus which concerns on this embodiment. 本実施形態に係るインター予測パラメータ復号部の構成を示す概略図である。It is the schematic which shows the structure of the inter prediction parameter decoding part which concerns on this embodiment. 本実施形態に係るマージモードパラメータ導出部の構成を示す概略図である。It is the schematic which shows the structure of the merge mode parameter derivation | leading-out part which concerns on this embodiment. 本実施形態に係るAMVP予測パラメータ導出部の構成を示す概略図である。It is the schematic which shows the structure of the AMVP prediction parameter derivation | leading-out part which concerns on this embodiment. ベクトル候補の一例を示す概念図である。It is a conceptual diagram which shows an example of a vector candidate. 本実施形態に係るインター予測パラメータ復号制御復号部の構成を示す概略図である。It is the schematic which shows the structure of the inter prediction parameter decoding control decoding part which concerns on this embodiment. 本実施形態に係るインター予測画像生成部の構成を示す概略図である。It is the schematic which shows the structure of the inter estimated image generation part which concerns on this embodiment. 本実施形態に係るインター予測画像生成部の別構成である、照度補償部を含まない構成を示す概略図である。It is the schematic which shows the structure which does not contain the illumination intensity compensation part which is another structure of the inter estimated image generation part which concerns on this embodiment. 本実施形態に係るインター予測画像生成部の別構成である、残差予測部を含まない構成を示す概略図である。It is the schematic which shows the structure which does not contain the residual prediction part which is another structure of the inter estimated image generation part which concerns on this embodiment. 本実施形態に係る残差予測部の構成を示す概略図である。It is the schematic which shows the structure of the residual prediction part which concerns on this embodiment. 本実施形態に係る残差予測の概念図（その１）である。It is a conceptual diagram (the 1) of the residual prediction which concerns on this embodiment. 本実施形態に係る残差予測の概念図（その２）である。It is a conceptual diagram (the 2) of the residual prediction which concerns on this embodiment. 本実施形態に係る視点合成予測部の構成を示す概略図である。It is the schematic which shows the structure of the viewpoint synthetic | combination prediction part which concerns on this embodiment. マージ候補リストの一例を示す図である。It is a figure which shows an example of a merge candidate list. マージ候補リストの一例を示す図である。It is a figure which shows an example of a merge candidate list. 本実施形態に係る画像符号化装置の構成を示すブロック図である。It is a block diagram which shows the structure of the image coding apparatus which concerns on this embodiment. 本実施形態に係るインター予測パラメータ符号化部の構成を示す概略図である。It is the schematic which shows the structure of the inter prediction parameter encoding part which concerns on this embodiment. 本実施形態に係るマージモードパラメータ導出部の構成を示す概略図である。It is the schematic which shows the structure of the merge mode parameter derivation | leading-out part which concerns on this embodiment. 本実施形態に係るインター予測パラメータ復号制御部の構成を示す概略図である。It is the schematic which shows the structure of the inter prediction parameter decoding control part which concerns on this embodiment. 本実施形態に係る照度補償フラグ復号部が符号化ユニットを復号対象とする際に参照するシンタックス表の一部であって、照度補償フラグに係る部分を示す図である。It is a figure which is a part of syntax tables referred when the illumination intensity compensation flag decoding part which concerns on this embodiment makes an encoding unit decoding object, Comprising: It is a figure which shows the part regarding an illumination intensity compensation flag.
　　（第１の実施形態）
　以下、図面を参照しながら本発明の実施形態について説明する。 (First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
　図１は、本実施形態に係る画像伝送システム１の構成を示す概略図である。 FIG. 1 is a schematic diagram showing a configuration of an image transmission system 1 according to the present embodiment.
　画像伝送システム１は、複数のレイヤ画像を符号化した符号を伝送し、伝送された符号を復号した画像を表示するシステムである。画像伝送システム１は、画像符号化装置１１、ネットワーク２１、画像復号装置３１及び画像表示装置４１を含んで構成される。 The image transmission system 1 is a system that transmits a code obtained by encoding a plurality of layer images and displays an image obtained by decoding the transmitted code. The image transmission system 1 includes an image encoding device 11, a network 21, an image decoding device 31, and an image display device 41.
　画像符号化装置１１には、複数のレイヤ画像（テクスチャ画像ともいう）を示す信号Ｔが入力される。レイヤ画像とは、ある解像度及びある視点で視認もしくは撮影される画像である。複数のレイヤ画像を用いて３次元画像を符号化するビュースケーラブル符号化を行う場合、複数のレイヤ画像のそれぞれは、視点画像と呼ばれる。ここで、視点は撮影装置の位置又は観測点に相当する。例えば、複数の視点画像は、被写体に向かって左右の撮影装置のそれぞれが撮影した画像である。画像符号化装置１１は、この信号のそれぞれを符号化して符号化ストリームＴｅ（符号化データ）を生成する。符号化ストリームＴｅの詳細については、後述する。視点画像とは、ある視点において観測される２次元画像（平面画像）である。視点画像は、例えば２次元平面内に配置された画素毎の輝度値、又は色信号値で示される。以下では、１枚の視点画像又は、その視点画像を示す信号をピクチャ（ｐｉｃｔｕｒｅ）と呼ぶ。また、複数のレイヤ画像を用いて空間スケーラブル符号化を行う場合、その複数のレイヤ画像は、解像度の低いベースレイヤ画像と、解像度の高い拡張レイヤ画像からなる。複数のレイヤ画像を用いてＳＮＲスケーラブル符号化を行う場合、その複数のレイヤ画像は、画質の低いベースレイヤ画像と、画質の高い拡張レイヤ画像からなる。なお、ビュースケーラブル符号化、空間スケーラブル符号化、ＳＮＲスケーラブル符号化を任意に組み合わせて行っても良い。本実施形態では、複数のレイヤ画像として、少なくともベースレイヤ画像と、ベースレイヤ画像以外の画像（拡張レイヤ画像）を含む画像の符号化および復号を扱う。複数のレイヤのうち、画像もしくは符号化パラメータにおいて参照関係（依存関係）にある２つのレイヤについて、参照される側の画像を、第１レイヤ画像、参照する側の画像を第２レイヤ画像と呼ぶ。例えば、ベースレイヤを参照して符号化される（ベースレイヤ以外の）エンハンスレイヤ画像がある場合、ベースレイヤ画像を第１レイヤ画像、エンハンスレイヤ画像を第２レイヤ画像として扱う。なお、エンハンスレイヤ画像の例としては、ベースビュー以外の視点の画像やデプス画像などがある。 The signal T indicating a plurality of layer images (also referred to as texture images) is input to the image encoding device 11. A layer image is an image that is viewed or photographed at a certain resolution and a certain viewpoint. When performing view scalable coding in which a three-dimensional image is coded using a plurality of layer images, each of the plurality of layer images is referred to as a viewpoint image. Here, the viewpoint corresponds to the position or observation point of the photographing apparatus. For example, the plurality of viewpoint images are images taken by the left and right photographing devices toward the subject. The image encoding device 11 encodes each of the signals to generate an encoded stream Te (encoded data). Details of the encoded stream Te will be described later. A viewpoint image is a two-dimensional image (planar image) observed at a certain viewpoint. The viewpoint image is indicated by, for example, a luminance value or a color signal value for each pixel arranged in a two-dimensional plane. Hereinafter, one viewpoint image or a signal indicating the viewpoint image is referred to as a picture. In addition, when performing spatial scalable coding using a plurality of layer images, the plurality of layer images include a base layer image having a low resolution and an enhancement layer image having a high resolution. When SNR scalable encoding is performed using a plurality of layer images, the plurality of layer images are composed of a base layer image with low image quality and an extended layer image with high image quality. Note that view scalable coding, spatial scalable coding, and SNR scalable coding may be arbitrarily combined. In the present embodiment, encoding and decoding of an image including at least a base layer image and an image other than the base layer image (enhancement layer image) is handled as the plurality of layer images. Of the multiple layers, for two layers that have a reference relationship (dependency relationship) in the image or encoding parameter, the image on the reference side is referred to as a first layer image, and the image on the reference side is referred to as a second layer image. . For example, when there is an enhancement layer image (other than the base layer) that is encoded with reference to the base layer, the base layer image is treated as a first layer image and the enhancement layer image is treated as a second layer image. Note that examples of the enhancement layer image include an image of a viewpoint other than the base view and a depth image.
　デプス画像（ｄｅｐｔｈ　ｍａｐ、「深度画像」、「距離画像」とも言う）とは、被写空間に含まれる被写体や背景の、視点（撮影装置等）からの距離に対応する信号値（「デプス値」、「深度値」、「デプス」等と呼ぶ）であって、二次元平面に配置された画素毎の信号値（画素値）からなる画像信号である。デプス画像を構成する画素は、視点画像を構成する画素と対応する。従って、デプスマップは、被写空間を二次元平面に射影した基準となる画像信号である視点画像を用いて、三次元の被写空間を表すための手がかりとなる。 A depth image (also referred to as depth map, “depth image”, or “distance image”) is a signal value (“depth value”) corresponding to the distance from the viewpoint (shooting device, etc.) of the subject or background included in the subject space. ”,“ Depth value ”,“ depth ”, etc.), and is an image signal composed of signal values (pixel values) for each pixel arranged in a two-dimensional plane. The pixels constituting the depth image correspond to the pixels constituting the viewpoint image. Therefore, the depth map is a clue for representing the three-dimensional object space by using the viewpoint image which is a reference image signal obtained by projecting the object space onto the two-dimensional plane.
　ネットワーク２１は、画像符号化装置１１が生成した符号化ストリームＴｅを画像復号装置３１に伝送する。ネットワーク２１は、インターネット（internet）、広域ネットワーク（WAN:Wide Area Network）、小規模ネットワーク（LAN:Local Area Network）又はこれらの組み合わせである。ネットワーク２１は、必ずしも双方向の通信網に限らず、地上波ディジタル放送、衛星放送等の放送波を伝送する一方向又は双方向の通信網であっても良い。また、ネットワーク２１は、ＤＶＤ（Digital Versatile Disc）、ＢＤ（Blue-ray Disc）等の符号化ストリームＴｅを記録した記憶媒体で代替されても良い。 The network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31. The network 21 is the Internet, a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional or bidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may be replaced by a storage medium that records an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray Disc).
　画像復号装置３１は、ネットワーク２１が伝送した符号化ストリームＴｅのそれぞれを復号し、それぞれ復号した複数の復号レイヤ画像Ｔｄ（復号視点画像Ｔｄ）を生成する。 The image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates a plurality of decoded layer images Td (decoded viewpoint images Td).
　画像表示装置４１は、画像復号装置３１が生成した複数の復号レイヤ画像Ｔｄの全部又は一部を表示する。例えば、ビュースケーラブル符号化においては、全部の場合、３次元画像（立体画像）や自由視点画像が表示され、一部の場合、２次元画像が表示される。画像表示装置４１は、例えば、液晶ディスプレイ、有機ＥＬ（Electro-luminescence）ディスプレイ等の表示デバイスを備える。また、空間スケーラブル符号化、ＳＮＲスケーラブル符号化では、画像復号装置３１、画像表示装置４１が高い処理能力を有する場合には、画質の高い拡張レイヤ画像を表示し、より低い処理能力しか有しない場合には、拡張レイヤほど高い処理能力、表示能力を必要としないベースレイヤ画像を表示する。 The image display device 41 displays all or part of the plurality of decoded layer images Td generated by the image decoding device 31. For example, in view scalable coding, a 3D image (stereoscopic image) and a free viewpoint image are displayed in all cases, and a 2D image is displayed in some cases. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. In addition, in the spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have a high processing capability, a high-quality enhancement layer image is displayed and only a lower processing capability is provided. Displays a base layer image that does not require higher processing capability and display capability as an extension layer.
　　＜符号化ストリームＴｅの構造＞
　本実施形態に係る画像符号化装置１１および画像復号装置３１の詳細な説明に先立って、画像符号化装置１１によって生成され、画像復号装置３１によって復号される符号化ストリームＴｅのデータ構造について説明する。 <Structure of Encoded Stream Te>
Prior to detailed description of the image encoding device 11 and the image decoding device 31 according to the present embodiment, the data structure of the encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described. .
　図２は、符号化ストリームＴｅにおけるデータの階層構造を示す図である。符号化ストリームＴｅは、例示的に、シーケンス、およびシーケンスを構成する複数のピクチャを含む。図２の（ａ）～（ｆ）は、それぞれ、シーケンスＳＥＱを既定するシーケンスレイヤ、ピクチャＰＩＣＴを規定するピクチャレイヤ、スライスＳを規定するスライスレイヤ、スライスデータを規定するスライスデータレイヤ、スライスデータに含まれる符号化ツリーユニットを規定する符号化ツリーレイヤ、符号化ツリーに含まれる符号化単位（Ｃｏｄｉｎｇ　Ｕｎｉｔ；ＣＵ）を規定する符号化ユニットレイヤを示す図である。 FIG. 2 is a diagram showing a hierarchical structure of data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. (A) to (f) of FIG. 2 respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, a slice data layer that defines slice data, and a slice data. It is a figure which shows the encoding unit layer which prescribes | regulates the encoding tree layer which prescribes | regulates the encoding tree unit contained, and the coding unit (Coding Unit; CU) contained in a coding tree.
　　（シーケンスレイヤ）
　シーケンスレイヤでは、処理対象のシーケンスＳＥＱ（以下、対象シーケンスとも称する）を復号するために画像復号装置３１が参照するデータの集合が規定されている。シーケンスＳＥＱは、図２の（ａ）に示すように、ビデオパラメータセット（Video Parameter Set）シーケンスパラメータセットＳＰＳ（Sequence Parameter Set）、ピクチャパラメータセットＰＰＳ（Picture Parameter Set）、ピクチャＰＩＣＴ、及び、付加拡張情報ＳＥＩ（Supplemental Enhancement Information）を含んでいる。ここで＃の後に示される値はレイヤＩＤを示す。図２では、＃０と＃１すなわちレイヤ０とレイヤ１の符号化データが存在する例を示すが、レイヤの種類およびレイヤの数はこれによらない。 (Sequence layer)
In the sequence layer, a set of data referred to by the image decoding device 31 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 2A, the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an additional extension. Information SEI (Supplemental Enhancement Information) is included. Here, the value indicated after # indicates the layer ID. FIG. 2 shows an example in which encoded data of # 0 and # 1, that is, layer 0 and layer 1, exists, but the type of layer and the number of layers are not dependent on this.
　ビデオパラメータセットＶＰＳは、複数のレイヤから構成されている動画像において、複数の動画像に共通する符号化パラメータの集合および動画像に含まれる複数のレイヤおよび個々のレイヤに関連する符号化パラメータの集合が規定されている。 The video parameter set VPS is a set of encoding parameters common to a plurality of moving images, a plurality of layers included in the moving image, and encoding parameters related to individual layers in a moving image composed of a plurality of layers. A set is defined.
　シーケンスパラメータセットＳＰＳでは、対象シーケンスを復号するために画像復号装置３１が参照する符号化パラメータの集合が規定されている。例えば、ピクチャの幅や高さが規定される。 The sequence parameter set SPS defines a set of encoding parameters that the image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are defined.
　ピクチャパラメータセットＰＰＳでは、対象シーケンス内の各ピクチャを復号するために画像復号装置３１が参照する符号化パラメータの集合が規定されている。例えば、ピクチャの復号に用いられる量子化幅の基準値（pic_init_qp_minus26）や重み付き予測の適用を示すフラグ（weighted_pred_flag）が含まれる。なお、ＰＰＳは複数存在してもよい。その場合、対象シーケンス内の各ピクチャから複数のＰＰＳの何れかを選択する。 In the picture parameter set PPS, a set of encoding parameters referred to by the image decoding device 31 in order to decode each picture in the target sequence is defined. For example, a quantization width reference value (pic_init_qp_minus26) used for picture decoding and a flag (weighted_pred_flag) indicating application of weighted prediction are included. A plurality of PPS may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.
　　（ピクチャレイヤ）
　ピクチャレイヤでは、処理対象のピクチャＰＩＣＴ（以下、対象ピクチャとも称する）を復号するために画像復号装置３１が参照するデータの集合が規定されている。ピクチャＰＩＣＴは、図２の（ｂ）に示すように、スライスＳ０～ＳＮＳ－１を含んでいる（ＮＳはピクチャＰＩＣＴに含まれるスライスの総数）。 (Picture layer)
In the picture layer, a set of data referred to by the image decoding device 31 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 2 (b), the picture PICT includes slices S0 to SNS-1 (NS is the total number of slices included in the picture PICT).
　なお、以下、スライスＳ０～ＳＮＳ－１のそれぞれを区別する必要が無い場合、符号の添え字を省略して記述することがある。また、以下に説明する符号化ストリームＴｅに含まれるデータであって、添え字を付している他のデータについても同様である。 It should be noted that, hereinafter, when it is not necessary to distinguish each of the slices S0 to SNS-1, the subscripts may be omitted. The same applies to data included in an encoded stream Te described below and to which other subscripts are attached.
　　（スライスレイヤ）
　スライスレイヤでは、処理対象のスライスＳ（対象スライスとも称する）を復号するために画像復号装置３１が参照するデータの集合が規定されている。スライスＳは、図２の（ｃ）に示すように、スライスヘッダＳＨ、および、スライスデータＳＤＡＴＡを含んでいる。 (Slice layer)
In the slice layer, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 2C, the slice S includes a slice header SH and slice data SDATA.
　スライスヘッダＳＨには、対象スライスの復号方法を決定するために画像復号装置３１が参照する符号化パラメータ群が含まれる。スライスタイプを指定するスライスタイプ指定情報（ｓｌｉｃｅ＿ｔｙｐｅ）は、スライスヘッダＳＨに含まれる符号化パラメータの一例である。 The slice header SH includes a coding parameter group that the image decoding device 31 refers to in order to determine a decoding method of the target slice. The slice type designation information (slice_type) that designates the slice type is an example of an encoding parameter included in the slice header SH.
　スライスタイプ指定情報により指定可能なスライスタイプとしては、（１）符号化の際にイントラ予測のみを用いるＩスライス、（２）符号化の際に単方向予測、または、イントラ予測を用いるＰスライス、（３）符号化の際に単方向予測、双方向予測、または、イントラ予測を用いるＢスライスなどが挙げられる。 As slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
　なお、スライスヘッダＳＨには、上記シーケンスレイヤに含まれる、ピクチャパラメータセットＰＰＳへの参照（pic_parameter_set_id）を含んでいても良い。 In addition, the slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the sequence layer.
　　（スライスデータレイヤ）
　スライスデータレイヤでは、処理対象のスライスデータＳＤＡＴＡを復号するために画像復号装置３１が参照するデータの集合が規定されている。スライスデータＳＤＡＴＡは、図２の（ｄ）に示すように、符号化ツリーブロック（CTB:Coded Tree Block）を含んでいる。ＣＴＢは、スライスを構成する固定サイズ（例えば６４×６４）のブロックであり、最大符号化単位（LCU:Largest Cording Unit）と呼ぶこともある。 (Slice data layer)
In the slice data layer, a set of data referred to by the image decoding device 31 in order to decode the slice data SDATA to be processed is defined. The slice data SDATA includes a coded tree block (CTB) as shown in FIG. The CTB is a fixed-size block (for example, 64 × 64) constituting a slice, and may be referred to as a maximum coding unit (LCU).
　　（符号化ツリーレイヤ）
　符号化ツリーレイヤは、図２の（ｅ）に示すように、処理対象の符号化ツリーブロックを復号するために画像復号装置３１が参照するデータの集合が規定されている。符号化ツリーユニットは、再帰的な４分木分割により分割される。再帰的な４分木分割により得られる木構造のノードのことを符号化ツリー（coding tree）と称する。４分木の中間ノードは、符号化ツリーユニット（CTU:Coded Tree Unit）であり、符号化ツリーブロック自身も最上位のＣＴＵとして規定される。ＣＴＵは、分割フラグ（splif_flag）を含み、splif_flagが１の場合には、４つの符号化ツリーユニットＣＴＵに分割される。splif_flagが０の場合には、符号化ツリーユニットＣＴＵは４つの符号化ユニット（CU:Coded Unit）に分割される。符号化ユニットＣＵは符号化ツリーレイヤの末端ノードであり、このレイヤではこれ以上分割されない。符号化ユニットＣＵは、符号化処理の基本的な単位となる。 (Encoding tree layer)
As shown in FIG. 2E, the coding tree layer defines a set of data that the image decoding device 31 refers to in order to decode the coding tree block to be processed. The coding tree unit is divided by recursive quadtree division. A tree-structured node obtained by recursive quadtree partitioning is called a coding tree. An intermediate node of the quadtree is a coded tree unit (CTU), and the coded tree block itself is also defined as the highest CTU. The CTU includes a split flag (splif_flag). When the split_flag is 1, the CTU is split into four coding tree units CTU. When splif_flag is 0, the coding tree unit CTU is divided into four coding units (CU: Coded Unit). The coding unit CU is a terminal node of the coding tree layer and is not further divided in this layer. The encoding unit CU is a basic unit of the encoding process.
　また、符号化ツリーブロックＣＴＢのサイズ６４×６４画素の場合には、符号化ユニットのサイズは、６４×６４画素、３２×３２画素、１６×１６画素、および、８×８画素の何れかをとり得る。 In the case where the size of the coding tree block CTB is 64 × 64 pixels, the size of the coding unit is any of 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels. It can take.
　　（符号化ユニットレイヤ）
　符号化ユニットレイヤは、図２の（ｆ）に示すように、処理対象の符号化ユニットを復号するために画像復号装置３１が参照するデータの集合が規定されている。具体的には、符号化ユニットは、ＣＵヘッダＣＵＨ、予測ツリー、変換ツリー、ＣＵヘッダＣＵＦから構成される。ＣＵヘッダＣＵＨでは、符号化ユニットが、イントラ予測を用いるユニットであるか、インター予測を用いるユニットであるかなどが規定される。また、ＣＵヘッダＣＵＨには、符号化ユニットが、残差予測を用いるユニットであるかを示す残差予測重みインデックスiv_res_pred_weight_idxや、照度補償予測を用いるユニットであるかを示す照度補償フラグic_flagを含む。符号化ユニットは、予測ツリー（prediction tree;PT）および変換ツリー（transform tree;TT）のルートとなる。ＣＵヘッダＣＵＦは、予測ツリーと変換ツリーの間、もしくは、変換ツリーの後に含まれる。 (Encoding unit layer)
As shown in (f) of FIG. 2, the encoding unit layer defines a set of data referred to by the image decoding device 31 in order to decode the processing target encoding unit. Specifically, the encoding unit includes a CU header CUH, a prediction tree, a conversion tree, and a CU header CUF. In the CU header CUH, it is defined whether the coding unit is a unit using intra prediction or a unit using inter prediction. Further, the CU header CUH includes a residual prediction weight index iv_res_pred_weight_idx indicating whether the coding unit is a unit using residual prediction, and an illuminance compensation flag ic_flag indicating whether the coding unit is a unit using illuminance compensation prediction. The encoding unit is the root of a prediction tree (PT) and a transform tree (TT). The CU header CUF is included between the prediction tree and the conversion tree or after the conversion tree.
　予測ツリーは、符号化ユニットが１または複数の予測ブロックに分割され、各予測ブロックの位置とサイズとが規定される。別の表現でいえば、予測ブロックは、符号化ユニットを構成する１または複数の重複しない領域である。また、予測ツリーは、上述の分割により得られた１または複数の予測ブロックを含む。 In the prediction tree, the coding unit is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the coding unit. The prediction tree includes one or a plurality of prediction blocks obtained by the above division.
　予測処理は、この予測ブロックごとに行われる。以下、予測の単位である予測ブロックのことを、予測単位（prediction unit;PU、予測ユニット）とも称する。 Prediction processing is performed for each prediction block. Hereinafter, a prediction block which is a unit of prediction is also referred to as a prediction unit (PU, prediction unit).
　予測ツリーにおける分割の種類は、大まかにいえば、イントラ予測の場合と、インター予測の場合との２つがある。イントラ予測とは、同一ピクチャ内の予測であり、インター予測とは、互いに異なるピクチャ間（例えば、表示時刻間、レイヤ画像間）で行われる予測処理を指す。 There are roughly two types of division in the prediction tree: intra prediction and inter prediction. Intra prediction is prediction within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).
　イントラ予測の場合、分割方法は、２Ｎ×２Ｎ（符号化ユニットと同一サイズ）と、Ｎ×Ｎとがある。 In the case of intra prediction, there are 2N × 2N (the same size as the encoding unit) and N × N division methods.
　また、インター予測の場合、分割方法は、符号化データのpart_modeにより符号化され、２Ｎ×２Ｎ（符号化ユニットと同一サイズ）、２Ｎ×Ｎ、２Ｎ×ｎＵ、２Ｎ×ｎＤ、Ｎ×２Ｎ、ｎＬ×２Ｎ、ｎＲ×２Ｎ、および、Ｎ×Ｎなどがある。なお、２Ｎ×ｎＵは、２Ｎ×２Ｎの符号化ユニットを上から順に２Ｎ×０．５Ｎと２Ｎ×１．５Ｎの２領域に分割することを示す。２Ｎ×ｎＤは、２Ｎ×２Ｎの符号化ユニットを上から順に２Ｎ×１．５Ｎと２Ｎ×０．５Ｎの２領域に分割することを示す。ｎＬ×２Ｎは、２Ｎ×２Ｎの符号化ユニットを左から順に０．５Ｎ×２Ｎと１．５Ｎ×２Ｎの２領域に分割することを示す。ｎＲ×２Ｎは、２Ｎ×２Ｎの符号化ユニットを左から順に１．５Ｎ×２Ｎと０．５Ｎ×１．５Ｎの２領域に分割することを示す。分割数は１、２、４のいずれかであるため、ＣＵに含まれるＰＵは１個から４個である。これらのＰＵを順にＰＵ０、ＰＵ１、ＰＵ２、ＰＵ３と表現する。 Further, in the case of inter prediction, the division method is encoded by part_mode of encoded data, and 2N × 2N (the same size as the encoding unit), 2N × N, 2N × nU, 2N × nD, N × 2N, nL X2N, nRx2N, and NxN. Note that 2N × nU indicates that a 2N × 2N encoding unit is divided into two regions of 2N × 0.5N and 2N × 1.5N in order from the top. 2N × nD indicates that a 2N × 2N encoding unit is divided into two regions of 2N × 1.5N and 2N × 0.5N in order from the top. nL × 2N indicates that a 2N × 2N encoding unit is divided into two regions of 0.5N × 2N and 1.5N × 2N in order from the left. nR × 2N indicates that a 2N × 2N encoding unit is divided into two regions of 1.5N × 2N and 0.5N × 1.5N in order from the left. Since the number of divisions is one of 1, 2, and 4, PUs included in the CU are 1 to 4. These PUs are expressed as PU0, PU1, PU2, and PU3 in order.
　また、変換ツリーにおいては、符号化ユニットが１または複数の変換ブロックに分割され、各変換ブロックの位置とサイズとが規定される。別の表現でいえば、変換ブロックは、符号化ユニットを構成する１または複数の重複しない領域のことである。また、変換ツリーは、上述の分割より得られた１または複数の変換ブロックを含む。 Also, in the transform tree, the encoding unit is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding unit. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.
　変換ツリーにおける分割には、符号化ユニットと同一のサイズの領域を変換ブロックとして割り付けるものと、上述したツリーブロックの分割と同様、再帰的な４分木分割によるものがある。 The division in the transformation tree includes the one in which an area having the same size as that of the encoding unit is assigned as the transformation block, and the one in the recursive quadtree division like the above-described division in the tree block.
　変換処理は、この変換ブロックごとに行われる。以下、変換の単位である変換ブロックのことを、変換単位（transform unit;TU）とも称する。 Conversion processing is performed for each conversion block. Hereinafter, a transform block that is a unit of transformation is also referred to as a transform unit (TU).
　　（予測パラメータ）
　予測ユニットの予測画像は、予測ユニットに付随する予測パラメータによって導出される。予測パラメータには、イントラ予測の予測パラメータもしくはインター予測の予測パラメータがある。以下、インター予測の予測パラメータ（インター予測パラメータ）について説明する。インター予測パラメータは、予測リスト利用フラグpredFlagL0、predFlagL1と、参照ピクチャインデックスrefIdxL0、refIdxL1と、ベクトルｍｖＬ０、ｍｖＬ１から構成される。予測リスト利用フラグpredFlagL0、predFlagL1は、各々Ｌ０リスト、Ｌ１リストと呼ばれる参照ピクチャリストが用いられるか否かを示すフラグであり、値が１の場合に対応する参照ピクチャリストが用いられる。なお、本明細書中「ＸＸであるか否かを示すフラグ」と記す場合、１をＸＸである場合、０をＸＸではない場合とし、論理否定、論理積などでは１を真、０を偽と扱う（以下同様）。但し、実際の装置や方法では真値、偽値として他の値を用いることもできる。２つの参照ピクチャリストが用いられる場合、つまり、(predFlagL0，predFlagL1) ＝ (1, 1)の場合が、双予測に対応し、１つの参照ピクチャリストを用いる場合、すなわち(predFlagL0, predFlagL1) = (1, 0)もしくは(predFlagL0, predFlagL1) = (0, 1)の場合が単予測に対応する。なお、予測リスト利用フラグの情報は、後述のインター予測フラグinter_pred_idcで表現することもできる。通常、後述の予測画像生成部、予測パラメータメモリでは、予測リスト利用フラグが用いれ、符号化データから、どの参照ピクチャリストが用いられるか否かの情報を復号する場合にはインター予測フラグinter_pred_idcが用いられる。 (Prediction parameter)
The prediction image of the prediction unit is derived by a prediction parameter associated with the prediction unit. The prediction parameters include a prediction parameter for intra prediction or a prediction parameter for inter prediction. Hereinafter, prediction parameters for inter prediction (inter prediction parameters) will be described. The inter prediction parameter includes prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and vectors mvL0 and mvL1. The prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list and L1 list are used, respectively, and a reference picture list corresponding to a value of 1 is used. In this specification, when “flag indicating whether or not XX” is described, 1 is XX, 0 is not XX, 1 is true and 0 is false in logical negation and logical product. (The same applies hereinafter). However, other values can be used as true values and false values in an actual apparatus or method. When two reference picture lists are used, that is, (predFlagL0, predFlagL1) = (1, 1) corresponds to bi-prediction, and when one reference picture list is used, that is, (predFlagL0, predFlagL1) = ( 1, 0) or (predFlagL0, predFlagL1) = (0, 1) corresponds to single prediction. Note that the prediction list use flag information can also be expressed by an inter prediction flag inter_pred_idc described later. Normally, a prediction list use flag is used in a prediction image generation unit and a prediction parameter memory described later, and an inter prediction flag inter_pred_idc is used when decoding information on which reference picture list is used from encoded data. It is done.
　符号化データに含まれるインター予測パラメータを導出するためのシンタックス要素には、例えば、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLXがある。 Syntax elements for deriving inter prediction parameters included in the encoded data include, for example, a partition mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference There is a vector mvdLX.
　　（参照ピクチャリストの一例）
　次に、参照ピクチャリストの一例について説明する。参照ピクチャリストとは、参照ピクチャメモリ３０６（図５）に記憶された参照ピクチャからなる列である。図３は、参照ピクチャリストの一例を示す概念図である。参照ピクチャリスト６０１において、左右に一列に配列された５個の長方形は、それぞれ参照ピクチャを示す。左端から右へ順に示されている符号、Ｐ１、Ｐ２、Ｑ０、Ｐ３、Ｐ４は、それぞれの参照ピクチャを示す符号である。Ｐ１等のＰとは、視点Ｐを示し、そしてＱ０のＱとは、視点Ｐとは異なる視点Ｑを示す。Ｐ及びＱの添字は、ピクチャ順序番号ＰＯＣを示す。refIdxLXの真下の下向きの矢印は、参照ピクチャインデックスrefIdxLXが、参照ピクチャメモリ３０６において参照ピクチャＱ０を参照するインデックスであることを示す。 (Example of reference picture list)
Next, an example of the reference picture list will be described. The reference picture list is a sequence of reference pictures stored in the reference picture memory 306 (FIG. 5). FIG. 3 is a conceptual diagram illustrating an example of a reference picture list. In the reference picture list 601, five rectangles arranged in a line on the left and right indicate reference pictures, respectively. The codes P1, P2, Q0, P3, and P4 shown in order from the left end to the right are codes indicating respective reference pictures. P such as P1 indicates the viewpoint P, and Q of Q0 indicates a viewpoint Q different from the viewpoint P. The subscripts P and Q indicate the picture order number POC. A downward arrow directly below refIdxLX indicates that the reference picture index refIdxLX is an index that refers to the reference picture Q0 in the reference picture memory 306.
　　（参照ピクチャの例）
　次に、ベクトルを導出する際に用いる参照ピクチャの例について説明する。図４は、参照ピクチャの例を示す概念図である。図４において、横軸は表示時刻を示し、縦軸は視点を示す。図４に示されている、縦２行、横３列（計６個）の長方形は、それぞれピクチャを示す。６個の長方形のうち、下行の左から２列目の長方形は復号対象のピクチャ（対象ピクチャ）を示し、残りの５個の長方形がそれぞれ参照ピクチャを示す。対象ピクチャから上向きの矢印で示される参照ピクチャＱ０は対象ピクチャと同表示時刻であって視点が異なるピクチャである。対象ピクチャを基準とする変位予測においては、参照ピクチャＱ０が用いられる。対象ピクチャから左向きの矢印で示される参照ピクチャＰ１は、対象ピクチャと同じ視点であって、過去のピクチャである。対象ピクチャから右向きの矢印で示される参照ピクチャＰ２は、対象ピクチャと同じ視点であって、未来のピクチャである。対象ピクチャを基準とする動き予測においては、参照ピクチャＰ１又はＰ２が用いられる。 (Reference picture example)
Next, an example of a reference picture used for deriving a vector will be described. FIG. 4 is a conceptual diagram illustrating an example of a reference picture. In FIG. 4, the horizontal axis indicates the display time, and the vertical axis indicates the viewpoint. The rectangles shown in FIG. 4 with 2 rows and 3 columns (6 in total) indicate pictures. Among the six rectangles, the rectangle in the second column from the left in the lower row indicates a picture to be decoded (target picture), and the remaining five rectangles indicate reference pictures. A reference picture Q0 indicated by an upward arrow from the target picture is a picture that has the same display time as the target picture and a different viewpoint. In the displacement prediction based on the target picture, the reference picture Q0 is used. A reference picture P1 indicated by a left-pointing arrow from the target picture is a past picture at the same viewpoint as the target picture. A reference picture P2 indicated by a right-pointing arrow from the target picture is a future picture at the same viewpoint as the target picture. In motion prediction based on the target picture, the reference picture P1 or P2 is used.
　　（インター予測フラグと予測リスト利用フラグ）
　インター予測フラグinter_pred_idcと、予測リスト利用フラグpredFlagL0、predFlagL1の関係は、
　inter_pred_idc　＝　（predFlagL1＜＜１）　＋　predFlagL0
　predFlagL0　＝inter_pred_idc　＆　１
　predFlagL1　＝inter_pred_idc　＞＞　１
の式を用いて相互に変換可能である。ここで、＞＞は右シフト、＜＜は左シフトである。そのため、インター予測パラメータとしては、予測リスト利用フラグpredFlagL0、predFlagL1を用いても良いし、インター予測フラグinter_pred_idcを用いてもよい。また、以下、予測リスト利用フラグpredFlagL0、predFlagL1を用いた判定は、インター予測フラグinter_pred_idcに置き替えても可能である。逆に、インター予測フラグinter_pred_idcを用いた判定は、予測リスト利用フラグpredFlagL0、predFlagL1に置き替えても可能である。 (Inter prediction flag and prediction list usage flag)
The relationship between the inter prediction flag inter_pred_idc and the prediction list use flags predFlagL0 and predFlagL1 is as follows:
inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
Can be converted to each other. Here, >> is a right shift, and << is a left shift. Therefore, as the inter prediction parameter, the prediction list use flags predFlagL0 and predFlagL1 may be used, or the inter prediction flag inter_pred_idc may be used. In addition, hereinafter, the determination using the prediction list use flags predFlagL0 and predFlagL1 may be replaced with the inter prediction flag inter_pred_idc. Conversely, the determination using the inter prediction flag inter_pred_idc can be replaced with the prediction list use flags predFlagL0 and predFlagL1.
　　（マージモードとAMVP予測）
　予測パラメータの復号（符号化）方法には、マージ（ｍｅｒｇｅ）モードとAMVP（Adaptive Motion Vector Prediction、適応動きベクトル予測）モードがある、マージフラグmerge_flagは、これらを識別するためのフラグである。マージモードでも、AMVPモードでも、既に処理済みのブロックの予測パラメータを用いて、対象ＰＵの予測パラメータが導出される。マージモードは、予測リスト利用フラグpredFlagLX（インター予測フラグinter_pred_idc）、参照ピクチャインデックスrefIdxLX、ベクトルmvLXを符号化データに含めずに、既に導出した予測パラメータをそのまま用いるモードであり、AMVPモードは、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、ベクトルmvLXを符号化データに含めるモードである。なおベクトルmvLXは、予測ベクトルを示す予測ベクトルインデックスmvp_LX_idxと差分ベクトル（mvdLX）として符号化される。 (Merge mode and AMVP prediction)
The prediction parameter decoding (encoding) method includes a merge mode and an AMVP (Adaptive Motion Vector Prediction) mode. The merge flag merge_flag is a flag for identifying these. In both the merge mode and the AMVP mode, the prediction parameter of the target PU is derived using the prediction parameter of the already processed block. The merge mode is a mode that uses the prediction parameters already derived without including the prediction list use flag predFlagLX (inter prediction flag inter_pred_idc), the reference picture index refIdxLX, and the vector mvLX in the encoded data, and the AMVP mode is an inter prediction. In this mode, the flag inter_pred_idc, the reference picture index refIdxLX, and the vector mvLX are included in the encoded data. The vector mvLX is encoded as a prediction vector index mvp_LX_idx indicating a prediction vector and a difference vector (mvdLX).
　インター予測フラグinter_pred_idcは、参照ピクチャの種類および数を示すデータであり、Ｐｒｅｄ＿Ｌ０、Ｐｒｅｄ＿Ｌ１、Ｐｒｅｄ＿Ｂｉの何れかの値をとる。Ｐｒｅｄ＿Ｌ０、Ｐｒｅｄ＿Ｌ１は、各々Ｌ０リスト、Ｌ１リストと呼ばれる参照ピクチャリストに記憶された参照ピクチャが用いられることを示し、共に１枚の参照ピクチャを用いること（単予測）を示す。Ｌ０リスト、Ｌ１リストを用いた予測を各々Ｌ０予測、Ｌ１予測と呼ぶ。Ｐｒｅｄ＿Ｂｉは２枚の参照ピクチャを用いること（双予測）を示し、Ｌ０リストとＬ１リストに記憶された参照ピクチャの２つを用いることを示す。予測ベクトルインデックスmvp_LX_idxは予測ベクトルを示すインデックスであり、参照ピクチャインデックスrefIdxLXは、参照ピクチャリストに記憶された参照ピクチャを示すインデックスである。なお、ＬＸは、Ｌ０予測とＬ１予測を区別しない場合に用いられる記述方法であり、ＬＸをＬ０、Ｌ１に置き換えることでＬ０リストに対するパラメータとＬ１リストに対するパラメータを区別するする。例えば、refIdxL0はＬ０予測に用いる参照ピクチャインデックス、refIdxL1はＬ１予測に用いる参照ピクチャインデックス、refIdx（refIdxLX）は、refIdxL0とrefIdxL1を区別しない場合に用いられる表記である。 The inter prediction flag inter_pred_idc is data indicating the type and number of reference pictures, and takes one of the values Pred_L0, Pred_L1, and Pred_Bi. Pred_L0 and Pred_L1 indicate that reference pictures stored in reference picture lists called an L0 list and an L1 list are used, respectively, and that both use one reference picture (single prediction). Prediction using the L0 list and the L1 list are referred to as L0 prediction and L1 prediction, respectively. Pred_Bi indicates that two reference pictures are used (bi-prediction), and indicates that two reference pictures stored in the L0 list and the L1 list are used. The prediction vector index mvp_LX_idx is an index indicating a prediction vector, and the reference picture index refIdxLX is an index indicating a reference picture stored in the reference picture list. Note that LX is a description method used when L0 prediction and L1 prediction are not distinguished. By replacing LX with L0 and L1, parameters for the L0 list and parameters for the L1 list are distinguished. For example, refIdxL0 is a reference picture index used for L0 prediction, refIdxL1 is a reference picture index used for L1 prediction, and refIdx (refIdxLX) is a notation used when refIdxL0 and refIdxL1 are not distinguished.
　マージインデックスmerge_idxは、処理が完了したブロックから導出される予測パラメータ候補（マージ候補）のうち、いずれかの予測パラメータを復号対象ブロックの予測パラメータとして用いるかを示すインデックスである。 The merge index merge_idx is an index indicating which one of the prediction parameter candidates (merge candidates) derived from the processed block is used as the prediction parameter of the decoding target block.
　　（動きベクトルと変位ベクトル）
　ベクトルmvLXには、動きベクトルと変位ベクトル（disparity vector、視差ベクトル）がある。動きベクトルとは、あるレイヤのある表示時刻でのピクチャにおけるブロックの位置と、異なる表示時刻（例えば、隣接する離散時刻）における同一のレイヤのピクチャにおける対応するブロックの位置との間の位置のずれを示すベクトルである。変位ベクトルとは、あるレイヤのある表示時刻でのピクチャにおけるブロックの位置と、同一の表示時刻における異なるレイヤのピクチャにおける対応するブロックの位置との間の位置のずれを示すベクトルである。異なるレイヤのピクチャとしては、異なる視点のピクチャである場合、もしくは、異なる解像度のピクチャである場合などがある。特に、異なる視点のピクチャに対応する変位ベクトルを視差ベクトルと呼ぶ。以下の説明では、動きベクトルと変位ベクトルを区別しない場合には、単にベクトルmvLXと呼ぶ。ベクトルmvLXに関する予測ベクトル、差分ベクトルを、それぞれ予測ベクトルmvpLX、差分ベクトルmvdLXと呼ぶ。ベクトルmvLXおよび差分ベクトルmvdLXが、動きベクトルであるか、変位ベクトルであるかは、ベクトルに付随する参照ピクチャインデックスrefIdxLXを用いて行われる。 (Motion vector and displacement vector)
The vector mvLX includes a motion vector and a displacement vector (disparity vector). A motion vector is a positional shift between the position of a block in a picture at a certain display time of a layer and the position of the corresponding block in a picture of the same layer at a different display time (for example, an adjacent discrete time). It is a vector which shows. The displacement vector is a vector indicating a positional shift between the position of a block in a picture at a certain display time of a certain layer and the position of a corresponding block in a picture of a different layer at the same display time. The pictures in different layers may be pictures from different viewpoints or pictures with different resolutions. In particular, a displacement vector corresponding to pictures of different viewpoints is called a disparity vector. In the following description, when a motion vector and a displacement vector are not distinguished, they are simply referred to as a vector mvLX. A prediction vector and a difference vector related to the vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively. Whether the vector mvLX and the difference vector mvdLX are motion vectors or displacement vectors is determined using a reference picture index refIdxLX associated with the vectors.
　　（画像復号装置の構成）
　次に、本実施形態に係る画像復号装置３１の構成について説明する。図５は、本実施形態に係る画像復号装置３１の構成を示す概略図である。画像復号装置３１は、エントロピー復号部３０１、予測パラメータ復号部３０２、参照ピクチャメモリ（参照画像記憶部、フレームメモリ）３０６、予測パラメータメモリ（予測パラメータ記憶部、フレームメモリ）３０７、予測画像生成部３０８、逆量子化・逆ＤＣＴ部３１１、加算部３１２、残差格納部３１３（残差記録部）及び図示されないデプスＤＶ導出部３５１を含んで構成される。 (Configuration of image decoding device)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 5 is a schematic diagram illustrating a configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit 302, a reference picture memory (reference image storage unit, frame memory) 306, a prediction parameter memory (prediction parameter storage unit, frame memory) 307, and a prediction image generation unit 308. , An inverse quantization / inverse DCT unit 311, an addition unit 312, a residual storage unit 313 (residual recording unit), and a depth DV derivation unit 351 (not shown).
　また、予測パラメータ復号部３０２は、インター予測パラメータ復号部３０３及びイントラ予測パラメータ復号部３０４を含んで構成される。予測画像生成部３０８は、インター予測画像生成部３０９及びイントラ予測画像生成部３１０を含んで構成される。 The prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.
　エントロピー復号部３０１は、外部から入力された符号化ストリームＴｅに対してエントロピー復号を行って、個々の符号（シンタックス要素）を分離し復号する。分離された符号には、予測画像を生成するための予測情報および、差分画像を生成するための残差情報などがある。 The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside, and separates and decodes individual codes (syntax elements). The separated codes include prediction information for generating a prediction image and residual information for generating a difference image.
　エントロピー復号部３０１は、分離した符号の一部を予測パラメータ復号部３０２に出力する。分離した符号の一部とは、例えば、予測モードPredMode、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックmvp_LX_idx、差分ベクトルmvdLX、残差予測重みインデックスiv_res_pred_weight_idx、照度補償フラグic_flagである。どの符号を復号するか否かの制御は、予測パラメータ復号部３０２の指示に基づいて行われる。エントロピー復号部３０１は、量子化係数を逆量子化・逆ＤＣＴ部３１１に出力する。この量子化係数は、符号化処理において、残差信号に対してＤＣＴ（Discrete Cosine Transform、離散コサイン変換）を行い量子化して得られる係数である。エントロピー復号部３０１は、デプスＤＶ変換テーブルDepthToDisparityBを、デプスＤＶ導出部３５１に出力する。このデプスＤＶ変換テーブルDepthToDisparityBは、デプス画像の画素値を視点画像間の変位を示す視差に変換するためのテーブルであり、デプスＤＶ変換テーブルDepthToDisparityBの要素DepthToDisparityB[d]は、傾きcp_scaleとオフセットcp_off、傾きの精度cp_precisionを用いて、
　　log2Div = BitDepthY 1 + cp_precision
　　offset = ( cp_off << BitDepthY ) + ( ( 1 << log2Div ) >> 1 )
　　scale = cp_scale
　　DepthToDisparityB[ d ] = ( scale * d + offset ) >> log2Div
の式によって求めることができる。パラメータcp_scale、cp_off、cp_precisionは参照する視点毎に符号化データ中のパラメータセットから復号する。なお、BitDepthYは輝度信号に対応する画素値のビット深度を示しており、例えば、値として８をとる。 The entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302. Some of the separated codes are, for example, prediction mode PredMode, split mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, residual prediction weight The index iv_res_pred_weight_idx and the illumination compensation flag ic_flag. Control of which code to decode is performed based on an instruction from the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311. This quantization coefficient is a coefficient obtained by performing quantization and performing DCT (Discrete Cosine Transform) on the residual signal in the encoding process. The entropy decoding unit 301 outputs the depth DV conversion table DepthToDisparityB to the depth DV deriving unit 351. The depth DV conversion table DepthToDisparityB is a table for converting the pixel value of the depth image into a parallax indicating the displacement between the viewpoint images, and an element DepthToDisparityB [d] of the depth DV conversion table DepthToDisparityB has an inclination cp_scale and an offset cp_off, Using the slope precision cp_precision,
log2Div = BitDepth Y 1 + cp_precision
offset = (cp_off << BitDepthY) + ((1 << log2Div) >> 1)
scale = cp_scale
DepthToDisparityB [d] = (scale * d + offset) >> log2Div
It can be calculated by the following formula. The parameters cp_scale, cp_off, and cp_precision are decoded from the parameter set in the encoded data for each viewpoint to be referred to. BitDepthY indicates the bit depth of the pixel value corresponding to the luminance signal, and takes, for example, 8 as the value.
　予測パラメータ復号部３０２は、エントロピー復号部３０１から符号の一部を入力として受け取る。予測パラメータ復号部３０２は、符号の一部である予測モードPredModeが示す予測モードに対応する予測パラメータを復号する。予測パラメータ復号部３０２は、予測モードPredModeと復号した予測パラメータを予測パラメータメモリ３０７と予測画像生成部３０８に出力する。 The prediction parameter decoding unit 302 receives a part of the code from the entropy decoding unit 301 as an input. The prediction parameter decoding unit 302 decodes the prediction parameter corresponding to the prediction mode indicated by the prediction mode PredMode that is a part of the code. The prediction parameter decoding unit 302 outputs the prediction mode PredMode and the decoded prediction parameter to the prediction parameter memory 307 and the prediction image generation unit 308.
　インター予測パラメータ復号部３０３は、エントロピー復号部３０１から入力された符号に基づいて、予測パラメータメモリ３０７に記憶された予測パラメータを参照してインター予測パラメータを復号する。インター予測パラメータ復号部３０３は、復号したインター予測パラメータを予測画像生成部３０８に出力し、また予測パラメータメモリ３０７に記憶する。インター予測パラメータ復号部３０３の詳細については後述する。 The inter prediction parameter decoding unit 303 decodes the inter prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.
　イントラ予測パラメータ復号部３０４は、エントロピー復号部３０１から入力された符号に基づいて、予測パラメータメモリ３０７に記憶された予測パラメータを参照してイントラ予測パラメータを復号する。イントラ予測パラメータとは、ピクチャブロックを１つのピクチャ内で予測する処理で用いるパラメータ、例えば、イントラ予測モードＩｎｔｒａPredModeである。イントラ予測パラメータ復号部３０４は、復号したイントラ予測パラメータを予測画像生成部３０８に出力し、また予測パラメータメモリ３０７に記憶する。 The intra prediction parameter decoding unit 304 refers to the prediction parameter stored in the prediction parameter memory 307 on the basis of the code input from the entropy decoding unit 301 and decodes the intra prediction parameter. The intra prediction parameter is a parameter used in a process of predicting a picture block within one picture, for example, an intra prediction mode IntraPredMode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.
　イントラ予測パラメータ復号部３０４は、輝度と色差で異なるイントラ予測モードを導出しても良い。この場合、イントラ予測パラメータ復号部３０４は、輝度の予測パラメータとして輝度予測モードIntraPredModeY、色差の予測パラメータとして、色差予測モードIntraPredModeCを復号する。輝度予測モードIntraPredModeYは、３５モードであり、プレーナ予測（０）、ＤＣ予測（１）、方向予測（２～３４）が対応する。色差予測モードIntraPredModeCは、プレーナ予測（０）、ＤＣ予測（１）、方向予測（２、３、４）、ＬＭモード（５）の何れかを用いるもの。 The intra prediction parameter decoding unit 304 may derive different intra prediction modes depending on luminance and color difference. In this case, the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the color difference prediction mode IntraPredModeC as the color difference prediction parameter. The luminance prediction mode IntraPredModeY is a 35 mode and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2 to 34). The color difference prediction mode IntraPredModeC uses one of planar prediction (0), DC prediction (1), direction prediction (2, 3, 4), and LM mode (5).
　参照ピクチャメモリ３０６は、加算部３１２が生成した参照ピクチャのブロック（参照ピクチャブロック）を、復号対象のピクチャ及びブロック毎に予め定めた位置に記憶する。 The reference picture memory 306 stores the reference picture block (reference picture block) generated by the adding unit 312 at a predetermined position for each picture and block to be decoded.
　予測パラメータメモリ３０７は、予測パラメータを、復号対象のピクチャ及びブロック毎に予め定めた位置に記憶する。具体的には、予測パラメータメモリ３０７は、インター予測パラメータ復号部３０３が復号したインター予測パラメータ、イントラ予測パラメータ復号部３０４が復号したイントラ予測パラメータ及びエントロピー復号部３０１が分離した予測モードpredModeを記憶する。記憶されるインター予測パラメータには、例えば、予測リスト利用フラグpredFlagLX（インター予測フラグinter_pred_idc）、参照ピクチャインデックスrefIdxLX、ベクトルmvLXがある。 The prediction parameter memory 307 stores the prediction parameter in a predetermined position for each decoding target picture and block. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. . The stored inter prediction parameters include, for example, a prediction list use flag predFlagLX (inter prediction flag inter_pred_idc), a reference picture index refIdxLX, and a vector mvLX.
　予測画像生成部３０８には、予測パラメータ復号部３０２から予測モードpredMode及び予測パラメータが入力される。また、予測画像生成部３０８は、参照ピクチャメモリ３０６から参照ピクチャを読み出す。予測画像生成部３０８は、予測モードpredModeが示す予測モードで、入力された予測パラメータと読み出した参照ピクチャを用いて予測ピクチャブロックpredSmaples（予測画像）を生成する。 The prediction mode predMode and the prediction parameter are input from the prediction parameter decoding unit 302 to the prediction image generation unit 308. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates prediction picture blocks predSmaples (prediction images) using the input prediction parameter and the read reference picture in the prediction mode indicated by the prediction mode predMode.
　ここで、予測モードpredModeがインター予測モードを示す場合、インター予測画像生成部３０９は、インター予測パラメータ復号部３０３から入力されたインター予測パラメータと読み出した参照ピクチャを用いてインター予測により予測ピクチャブロックpredSmaplesを生成する。予測ピクチャブロックpredSmaplesは予測単位ＰＵに対応する。ＰＵは、上述したように予測処理を行う単位となる複数の画素からなるピクチャの一部分、つまり１度に予測処理が行われる復号対象ブロックに相当する。 Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture to perform prediction picture block predSmaples by inter prediction. Is generated. The prediction picture block predSmaples corresponds to the prediction unit PU. The PU corresponds to a part of a picture composed of a plurality of pixels as a unit for performing the prediction process as described above, that is, a decoding target block on which the prediction process is performed at a time.
　インター予測画像生成部３０９は、予測リスト利用フラグpredFlagLXが１である参照ピクチャリスト（Ｌ０リスト、もしくはＬ１リスト）に対し、参照ピクチャインデックスrefIdxLXで示される参照ピクチャから、復号対象ブロックを基準としてベクトルmvLXが示す位置にある参照ピクチャブロックを参照ピクチャメモリ３０６から読み出す。インター予測画像生成部３０９は、読み出した参照ピクチャブロックについて予測を行って予測ピクチャブロックpredSmaplesを生成する。インター予測画像生成部３０９は、生成した予測ピクチャブロックpredSmaplesを加算部３１２に出力する。 For the reference picture list (L0 list or L1 list) for which the prediction list use flag predFlagLX is 1, the inter predicted image generation unit 309 generates a vector mvLX based on the decoding target block from the reference picture indicated by the reference picture index refIdxLX. The reference picture block at the position indicated by is read from the reference picture memory 306. The inter prediction image generation unit 309 performs prediction on the read reference picture block to generate prediction picture blocks predSmaples. The inter prediction image generation unit 309 outputs the generated prediction picture block predSmaples to the addition unit 312.
　予測モードpredModeがイントラ予測モードを示す場合、イントラ予測画像生成部３１０は、イントラ予測パラメータ復号部３０４から入力されたイントラ予測パラメータと読み出した参照ピクチャを用いてイントラ予測を行う。具体的には、イントラ予測画像生成部３１０は、復号対象のピクチャであって、既に復号されたブロックのうち復号対象ブロックから予め定めた範囲にある参照ピクチャブロックを参照ピクチャメモリ３０６から読み出す。予め定めた範囲とは、復号対象ブロックがいわゆるラスタースキャンの順序で順次移動する場合、例えば、左、左上、上、右上の隣接ブロックのうちのいずれかであり、イントラ予測モードによって異なる。ラスタースキャンの順序とは、各ピクチャにおいて、上端から下端まで各行について、順次左端から右端まで移動させる順序である。 When the prediction mode predMode indicates the intra prediction mode, the intra predicted image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference picture. Specifically, the intra predicted image generation unit 310 reads, from the reference picture memory 306, a reference picture block that is a decoding target picture and is in a predetermined range from the decoding target block among blocks that have already been decoded. The predetermined range is, for example, any of the left, upper left, upper, and upper right adjacent blocks when the decoding target block sequentially moves in a so-called raster scan order, and varies depending on the intra prediction mode. The raster scan order is an order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.
　イントラ予測画像生成部３１０は、読み出した参照ピクチャブロックについてイントラ予測モードIntraPredModeが示す予測モードで予測を行って予測ピクチャブロックを生成する。イントラ予測画像生成部３１０は、生成した予測ピクチャブロックpredSmaplesを加算部３１２に出力する。 The intra predicted image generation unit 310 performs prediction in the prediction mode indicated by the intra prediction mode IntraPredMode for the read reference picture block, and generates a predicted picture block. The intra predicted image generation unit 310 outputs the generated predicted picture block predSmaples to the addition unit 312.
　イントラ予測パラメータ復号部３０４において、輝度と色差で異なるイントラ予測モードを導出する場合、イントラ予測画像生成部３１０は、輝度予測モードIntraPredModeYに応じて、プレーナ予測（０）、ＤＣ予測（１）、方向予測（２～３４）の何れかによって輝度の予測ピクチャブロックを生成し、色差予測モードIntraPredModeCに応じて、プレーナ予測（０）、ＤＣ予測（１）、方向予測（２、３、４）、ＬＭモード（５）の何れかによって色差の予測ピクチャブロックを生成する。 When the intra prediction parameter decoding unit 304 derives an intra prediction mode different in luminance and color difference, the intra prediction image generation unit 310 performs planar prediction (0), DC prediction (1), direction according to the luminance prediction mode IntraPredModeY. A prediction picture block of luminance is generated according to any of prediction (2 to 34), and planar prediction (0), DC prediction (1), direction prediction (2, 3, 4), LM according to the color difference prediction mode IntraPredModeC A color difference prediction picture block is generated in any one of modes (5).
　逆量子化・逆ＤＣＴ部３１１は、エントロピー復号部３０１から入力された量子化係数を逆量子化してＤＣＴ係数を求める。逆量子化・逆ＤＣＴ部３１１は、求めたＤＣＴ係数について逆ＤＣＴ（Inverse Discrete Cosine Transform、逆離散コサイン変換）を行い、復号残差信号を算出する。逆量子化・逆ＤＣＴ部３１１は、算出した復号残差信号を加算部３１２および残差格納部３１３に出力する。 The inverse quantization / inverse DCT unit 311 inversely quantizes the quantization coefficient input from the entropy decoding unit 301 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 311 performs inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 311 outputs the calculated decoded residual signal to the addition unit 312 and the residual storage unit 313.
　加算部３１２は、インター予測画像生成部３０９及びイントラ予測画像生成部３１０から入力された予測ピクチャブロックpredSmaplesと逆量子化・逆ＤＣＴ部３１１から入力された復号残差信号の信号値を画素毎に加算して、参照ピクチャブロックを生成する。加算部３１２は、生成した参照ピクチャブロックを参照ピクチャメモリ３０６に記憶し、生成した参照ピクチャブロックをピクチャ毎に統合した復号レイヤ画像Ｔｄを外部に出力する。 The adder 312 outputs the prediction picture block predSmaples input from the inter prediction image generation unit 309 and the intra prediction image generation unit 310 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. Addition to generate a reference picture block. The adder 312 stores the generated reference picture block in the reference picture memory 306, and outputs a decoded layer image Td in which the generated reference picture block is integrated for each picture to the outside.
　（インター予測パラメータ復号部の構成）
　次に、インター予測パラメータ復号部３０３の構成について説明する。 (Configuration of inter prediction parameter decoding unit)
Next, the configuration of the inter prediction parameter decoding unit 303 will be described.
　図６は、本実施形態に係るインター予測パラメータ復号部３０３の構成を示す概略図である。インター予測パラメータ復号部３０３は、インター予測パラメータ復号制御部３０３１、AMVP予測パラメータ導出部３０３２、加算部３０３５及びマージモードパラメータ導出部３０３６を含んで構成される。 FIG. 6 is a schematic diagram illustrating a configuration of the inter prediction parameter decoding unit 303 according to the present embodiment. The inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter derivation unit 3032, an addition unit 3035, and a merge mode parameter derivation unit 3036.
　インター予測パラメータ復号制御部３０３１は、インター予測に関連する符号（シンタックス要素の復号をエントロピー復号部３０１に指示し、符号化データに含まれる符号（シンタックス要素）を例えば、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLX、残差予測重みインデックスiv_res_pred_weight_idx、照度補償フラグic_flagを抽出する。 The inter prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode a code related to the inter prediction (the syntax element) includes, for example, a division mode part_mode, a merge included in the encoded data. A flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, residual prediction weight index iv_res_pred_weight_idx, and illumination compensation flag ic_flag are extracted.
　インター予測パラメータ復号制御部３０３１は、まず、符号化データから残差予測重みインデックスiv_res_pred_weight_idx及び照度補償フラグic_flagを抽出する。インター予測パラメータ復号制御部３０３１が、あるシンタックス要素を抽出すると表現する場合は、あるシンタックス要素の復号をエントロピー復号部３０１に指示し、該当のシンタックス要素を符号化データから読み出すことを意味する。 The inter prediction parameter decoding control unit 3031 first extracts a residual prediction weight index iv_res_pred_weight_idx and an illumination compensation flag ic_flag from the encoded data. When the inter prediction parameter decoding control unit 3031 expresses that a certain syntax element is to be extracted, it means that the entropy decoding unit 301 is instructed to decode a certain syntax element, and the corresponding syntax element is read from the encoded data. To do.
　次に、インター予測パラメータ復号制御部３０３１は、符号化データからマージフラグを抽出する。ここで、マージフラグmerge_flagが示す値が１、すなわち、マージモードを示す場合、インター予測パラメータ復号制御部３０３１は、マージモードに係る予測パラメータとして、マージインデックスmerge_idxを抽出する。インター予測パラメータ復号制御部３０３１は、抽出した残差予測重みインデックスiv_res_pred_weight_idx、照度補償フラグic_flag及びマージインデックスmerge_idxをマージモードパラメータ導出部３０３６に出力する。 Next, the inter prediction parameter decoding control unit 3031 extracts a merge flag from the encoded data. Here, when the value indicated by the merge flag merge_flag is 1, that is, indicates the merge mode, the inter prediction parameter decoding control unit 3031 extracts the merge index merge_idx as a prediction parameter related to the merge mode. The inter prediction parameter decoding control unit 3031 outputs the extracted residual prediction weight index iv_res_pred_weight_idx, the illumination compensation flag ic_flag, and the merge index merge_idx to the merge mode parameter deriving unit 3036.
　マージフラグmerge_flagが示す値が０、すなわち、AMVP予測モードを示す場合、インター予測パラメータ復号制御部３０３１は、エントロピー復号部３０１を用いて符号化データからAMVP予測パラメータを抽出する。AMVP予測パラメータとして、例えば、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLXがある。インター予測パラメータ復号制御部３０３１は、抽出したインター予測フラグinter_pred_idcから導出した予測リスト利用フラグpredFlagLXと、参照ピクチャインデックスrefIdxLXをAMVP予測パラメータ導出部３０３２及び予測画像生成部３０８（図５）に出力し、また予測パラメータメモリ３０７（図５）に記憶する。インター予測パラメータ復号制御部３０３１は、抽出したベクトルインデックスmvp_LX_idxをAMVP予測パラメータ導出部３０３２に出力する。インター予測パラメータ復号制御部３０３１は、抽出した差分ベクトルmvdLXを加算部３０３５に出力する。 When the value indicated by the merge flag merge_flag is 0, that is, indicates the AMVP prediction mode, the inter prediction parameter decoding control unit 3031 uses the entropy decoding unit 301 to extract AMVP prediction parameters from the encoded data. Examples of AMVP prediction parameters include an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a vector index mvp_LX_idx, and a difference vector mvdLX. The inter prediction parameter decoding control unit 3031 outputs the prediction list use flag predFlagLX derived from the extracted inter prediction flag inter_pred_idc and the reference picture index refIdxLX to the AMVP prediction parameter derivation unit 3032 and the prediction image generation unit 308 (FIG. 5). Moreover, it memorize | stores in the prediction parameter memory 307 (FIG. 5). The inter prediction parameter decoding control unit 3031 outputs the extracted vector index mvp_LX_idx to the AMVP prediction parameter derivation unit 3032. The inter prediction parameter decoding control unit 3031 outputs the extracted difference vector mvdLX to the addition unit 3035.
　また、インター予測パラメータ復号制御部３０３１は、インター予測パラメータ導出時に導出された変位ベクトル（NBDV）、及び視点合成予測を行うか否かを示すフラグであるVSPモードフラグVSPModeFlagを、インター予測画像生成部３０９に出力する。 In addition, the inter prediction parameter decoding control unit 3031 displays a displacement vector (NBDV) derived at the time of deriving the inter prediction parameter and a VSP mode flag VSPModeFlag that is a flag indicating whether or not to perform viewpoint synthesis prediction. To 309.
　図７は、本実施形態に係るマージモードパラメータ導出部３０３６の構成を示す概略図である。マージモードパラメータ導出部３０３６は、マージ候補導出部３０３６１とマージ候補選択部３０３６２を備える。マージ候補導出部３０３６１は、マージ候補格納部３０３６１１と、拡張マージ候補導出部３０３６１２と基本マージ候補導出部３０３６１３を含んで構成される。 FIG. 7 is a schematic diagram illustrating a configuration of the merge mode parameter deriving unit 3036 according to the present embodiment. The merge mode parameter deriving unit 3036 includes a merge candidate deriving unit 30361 and a merge candidate selecting unit 30362. The merge candidate derivation unit 30361 includes a merge candidate storage unit 303611, an extended merge candidate derivation unit 303612, and a basic merge candidate derivation unit 303613.
　マージ候補格納部３０３６１１は、拡張マージ候補導出部３０３６１２及び基本マージ候補導出部３０３６１３から入力されたマージ候補をマージ候補リストmergeCandListに格納する。なお、マージ候補は、予測リスト利用フラグpredFlagLX、ベクトルmvLX、参照ピクチャインデックスrefIdxLX、VSPモードフラグVspModeFlag、変位ベクトルMvDisp、レイヤＩＤRefViewIdxを含んで構成されている。マージ候補格納部３０３６１１において、マージ候補リストmergeCandListに格納されたマージ候補には、所定の規則に従ってインデックスが割り当てられる。例えば、拡張マージ候補導出部３０３６１２から入力されたマージ候補には、インデックスとして「０」を割り当てる。なお、マージ候補のVSPモードフラグVspModeFlagが０の場合は、変位ベクトルMvDispのＸ，Ｙ成分には０が、レイヤＩＤRefViewIdxには－１が設定されるものとする。 The merge candidate storage unit 303611 stores the merge candidates input from the extended merge candidate derivation unit 303612 and the basic merge candidate derivation unit 303613 in the merge candidate list mergeCandList. The merge candidate includes a prediction list use flag predFlagLX, a vector mvLX, a reference picture index refIdxLX, a VSP mode flag VspModeFlag, a displacement vector MvDisp, and a layer ID RefViewIdx. In the merge candidate storage unit 303611, an index is assigned to the merge candidates stored in the merge candidate list mergeCandList according to a predetermined rule. For example, “0” is assigned as an index to the merge candidate input from the extended merge candidate derivation unit 303612. When the merge candidate VSP mode flag VspModeFlag is 0, 0 is set in the X and Y components of the displacement vector MvDisp, and −1 is set in the layer ID RefViewIdx.
　図１８は、マージ候補格納部３０３６１１が導出するマージ候補リストmergeCandListの例を示すものである。２つのマージ候補が同じ予測パラメータである場合に順番を詰める処理を除くと、マージインデックス順に、レイヤ間マージ候補、空間マージ候補（左下）、空間マージ候補（右上）、空間マージ候補（右上）、変位マージ候補、VSPマージ候補、空間マージ候補（左下）、空間マージ候補（左上）、時間マージ候補の順になる。また、それ以降に、結合マージ候補、ゼロマージ候補があるが、図１８では省略している。 FIG. 18 shows an example of the merge candidate list mergeCandList derived by the merge candidate storage unit 303611. If two merge candidates have the same prediction parameter, excluding the processing of reducing the order, the merge index order, layer merge candidate (lower left), spatial merge candidate (upper right), spatial merge candidate (upper right), Displacement merge candidates, VSP merge candidates, spatial merge candidates (lower left), spatial merge candidates (upper left), and temporal merge candidates. Further, there are a merge merge candidate and a zero merge candidate after that, but they are omitted in FIG.
　拡張マージ候補導出部３０３６１２は、変位ベクトル取得部３０３６１２２と、レイヤ間マージ候補導出部３０３６１２１と変位マージ候補導出部３０３６１２３、VSPマージ候補導出部３０３６１２４を含んで構成される。 The extended merge candidate derivation unit 303612 includes a displacement vector acquisition unit 3036122, an inter-layer merge candidate derivation unit 3036121, a displacement merge candidate derivation unit 3036123, and a VSP merge candidate derivation unit 3036124.
　変位ベクトル取得部３０３６１２２は、まず、復号対象ブロックに隣接する複数の候補ブロック（例えば、左、上、右上に隣接するブロック）から順に変位ベクトルを取得する。具体的には、候補ブロックの一つを選択し、選択した候補ブロックのベクトルが変位ベクトルであるか動きベクトルであるかを、候補ブロックの参照ピクチャインデックスrefIdxLXを用いてリファレンスレイヤ判定部３０３１１１（後述）により判定し、変位ベクトルが有る場合には、それを変位ベクトルとする。候補ブロックに変位ベクトルがない場合には、次の候補ブロックを順に走査する。隣接するブロックに変位ベクトルがない場合、変位ベクトル取得部３０３６１２２は、時間的に別の表示順の参照ピクチャに含まれるブロックの対象ブロックに対応する位置のブロックの変位ベクトルの取得を試みる。変位ベクトルが取得できなかった場合には、変位ベクトル取得部３０３６１２２は、変位ベクトルとしてゼロベクトルを設定する。得られた変位ベクトルはNBDV(Neighbour Base Disparity Vector)と呼ばれる。変位ベクトル取得部３０３６１２２は、得られたＮＢＤＶをデプスＤＶ導出部３５１に出力し、デプスＤＶ導出部３５１が導出するデプスベースＤＶの水平成分を入力として受け取る。変位ベクトル取得部３０３６１２２は、ＮＢＤＶの水平成分を、デプスＤＶ導出部３５１から入力されたデプスベースＤＶの水平成分に置き換えることで更新された変位ベクトルを得る（ＮＢＤＶの垂直成分はそのまま）。更新された変位ベクトルは、DoNBDV(Depth Orientated Neighbour Base Disparity Vector)と呼ばれる。変位ベクトル取得部３０３６１２２は、変位ベクトル(DoNBDV)をレイヤ間マージ候補導出部３０３６１２１、変位マージ候補導出部３０３６１２３及びＶＳＰマージ候補導出部３０３６１２４に出力する。さらに、得られた変位ベクトル(NBDV)を、インター予測画像生成部３０９に出力する。 The displacement vector acquisition unit 3036122 first acquires displacement vectors in order from a plurality of candidate blocks adjacent to the decoding target block (for example, blocks adjacent to the left, upper, and upper right). Specifically, one of the candidate blocks is selected, and whether the selected candidate block vector is a displacement vector or a motion vector is determined by using a reference picture index refIdxLX of the candidate block as a reference layer determination unit 303111 (described later). If there is a displacement vector, it is set as the displacement vector. If there is no displacement vector in the candidate block, the next candidate block is scanned in order. When there is no displacement vector in the adjacent block, the displacement vector acquisition unit 3036122 attempts to acquire the displacement vector of the block at the position corresponding to the target block of the block included in the reference picture in the temporally different display order. When the displacement vector cannot be acquired, the displacement vector acquisition unit 3036122 sets a zero vector as the displacement vector. The obtained displacement vector is called NBDV (Neighbour Base Disparity Vector). The displacement vector acquisition unit 3036122 outputs the obtained NBDV to the depth DV deriving unit 351, and receives the horizontal component of the depth base DV derived by the depth DV deriving unit 351 as an input. The displacement vector acquisition unit 3036122 obtains an updated displacement vector by replacing the horizontal component of the NBDV with the horizontal component of the depth base DV input from the depth DV deriving unit 351 (the vertical component of the NBDV is unchanged). The updated displacement vector is called DoNBDV (Depth Orientated Neighbour Base Disparity Vector). The displacement vector acquisition unit 3036122 outputs the displacement vector (DoNBDV) to the inter-layer merge candidate derivation unit 3036121, the displacement merge candidate derivation unit 3036123, and the VSP merge candidate derivation unit 3036124. Further, the obtained displacement vector (NBDV) is output to the inter predicted image generation unit 309.
　レイヤ間マージ候補導出部３０３６１２１は、変位ベクトル取得部３０３６１２２から変位ベクトルを入力される。レイヤ間マージ候補導出部３０３６１２１は、別レイヤ（例えばベースレイヤ、ベースビュー）の復号対象ピクチャと同一ＰＯＣを持つピクチャ内から、変位ベクトル取得部３０３６１２２から入力された変位ベクトルだけが示すブロックを選択し、該ブロックが有する動きベクトルである予測パラメータを予測パラメータメモリ３０７から読み出す。より具体的には、レイヤ間マージ候補導出部３０３６１２１が読みだす予測パラメータは、対象ブロックの中心点を起点にしたときに、起点の座標に変位ベクトルを加算した座標を含むブロックの予測パラメータである。 The inter-layer merge candidate derivation unit 3036121 receives the displacement vector from the displacement vector acquisition unit 3036122. The inter-layer merge candidate derivation unit 3036121 selects a block indicated only by the displacement vector input from the displacement vector acquisition unit 3036122 from a picture having the same POC as the decoding target picture of another layer (eg, base layer, base view). The prediction parameter, which is a motion vector included in the block, is read from the prediction parameter memory 307. More specifically, the prediction parameter read by the inter-layer merge candidate derivation unit 3036121 is a prediction parameter of a block including coordinates obtained by adding a displacement vector to the coordinates of the starting point when the center point of the target block is the starting point. .
　参照ブロックの座標（ｘＲｅｆ、ｙＲｅｆ）は、対象ブロックの座標が（ｘＰ、ｙＰ）、変位ベクトルが（mvDisp［０］、mvDisp［１］）、対象ブロックの幅と高さがｎＰＳＷ、ｎＰＳＨの場合に、
　xRef = Clip3( 0, PicWidthInSamplesL - 1, xP + ( ( nPSW - 1 ) >> 1 ) + ( ( mvDisp[0] + 2 ) >> 2 ) )
　yRef = Clip3( 0, PicHeightInSamplesL - 1, yP + ( ( nPSH - 1 ) >> 1 ) + ( ( mvDisp[1] + 2 ) >> 2 ))
の式により導出する。なお、PicWidthInSamplesLとPicHeightInSamplesL は、それぞれ画像の幅と高さを表し、関数Clip3(x,y,z)は、ｚをｘ以上、ｙ以下に制限（クリップ）し、その制限した結果を返す関数である。 The coordinates (xRef, yRef) of the reference block are the coordinates of the target block (xP, yP), the displacement vector (mvDisp [0], mvDisp [1]), and the width and height of the target block are nPSW, nPSH. In addition,
xRef = Clip3 (0, PicWidthInSamples L -1, xP + ((nPSW-1) >> 1) + ((mvDisp [0] + 2) >> 2))
yRef = Clip3 (0, PicHeightInSamples L -1, yP + ((nPSH-1) >> 1) + ((mvDisp [1] + 2) >> 2))
It is derived by the following formula. Note that PicWidthInSamples L and PicHeightInSamples L represent the width and height of the image, respectively, and the function Clip3 (x, y, z) restricts (clips) z to not less than x and not more than y, and returns the restricted result. It is a function.
　なお、レイヤ間マージ候補導出部３０３６１２１は、予測パラメータが動きベクトルか否かを、インター予測パラメータ復号制御部３０３１に含まれる後述するリファレンスレイヤ判定部３０３１１１の判定方法において偽（変位ベクトルではない）と判定した方法により判定する。レイヤ間マージ候補導出部３０３６１２１は、読みだした予測パラメータをマージ候補としてマージ候補格納部３０３６１１に出力する。また、レイヤ間マージ候補導出部３０３６１２１は、予測パラメータを導出出来なかった際には、その旨を変位マージ候補導出部に出力する。本マージ候補は、動き予測のインターレイヤ候補（インタービュー候補）でありレイヤ間マージ候補（動き予測）とも記載する。 Note that the inter-layer merge candidate derivation unit 3036121 determines whether or not the prediction parameter is a motion vector in the determination method of a reference layer determination unit 303111 (described later) included in the inter prediction parameter decoding control unit 3031 (not a displacement vector). The determination is made according to the determined method. The inter-layer merge candidate derivation unit 3036121 outputs the read prediction parameter as a merge candidate to the merge candidate storage unit 303611. Further, when the prediction parameter cannot be derived, the inter-layer merge candidate deriving unit 3036121 outputs that fact to the displacement merge candidate deriving unit. This merge candidate is a motion prediction inter-layer candidate (inter-view candidate) and is also described as an inter-layer merge candidate (motion prediction).
　変位マージ候補導出部３０３６１２３は、変位ベクトル取得部３０３６１２２から変位ベクトルを入力される。変位マージ候補導出部３０３６１２３は、水平成分が入力された変位ベクトルの水平成分であり、垂直成分が０であるベクトルを生成する。変位マージ候補導出部３０３６１２３は、生成したベクトルと、変位ベクトルが指す先のレイヤ画像の参照ピクチャインデックスrefIdxLX（例えば、復号対象ピクチャと同一ＰＯＣを持つベースレイヤ画像のインデックス）をマージ候補としてマージ候補格納部３０３６１１に出力する。本マージ候補は、変位予測のインターレイヤ候補（インタービュー候補）でありレイヤ間マージ候補（変位予測）とも記載する。 The displacement merge candidate derivation unit 3036123 receives the displacement vector from the displacement vector acquisition unit 3036122. The displacement merge candidate derivation unit 3036123 generates a vector whose horizontal component is the horizontal component of the displacement vector to which the horizontal component is input and whose vertical component is zero. The displacement merge candidate derivation unit 3036123 stores the generated vector and the reference picture index refIdxLX of the previous layer image pointed to by the displacement vector (for example, the index of the base layer image having the same POC as the decoding target picture) as a merge candidate. Output to the unit 303611. This merge candidate is a displacement prediction inter-layer candidate (inter-view candidate) and is also described as an inter-layer merge candidate (displacement prediction).
　VSPマージ候補導出部３０３６１２４は、VSP（視点合成予測：View Synthesis Prediction）マージ候補を導出する。VSPマージ候補は、インター予測画像生成部３０９で行われる視点合成予測による予測画像生成処理の際に用いられるマージ候補である。VSPマージ候補導出部３０３６１２４は、変位ベクトル取得部３０３６１２２から変位ベクトルを入力される。VSPマージ候補導出部３０３６１２４は、入力された変位ベクトルmvDispをベクトルmvLXと変位ベクトルMvDispに、変位ベクトルが指す先のレイヤ画像を示す参照ピクチャの参照ピクチャインデックスを参照ピクチャインデックスrefIdxLXに、変位ベクトルが指すレイヤのレイヤＩＤrefViewIdxをレイヤＩＤRefViewIdxに設定し、VSPモードフラグVspModeFlagに１を設定することでVSPマージ候補を導出する。VSPマージ候補導出部３０３６１２４は、導出したVSPマージ候補をマージ候補格納部３０３６１１に出力する。 The VSP merge candidate derivation unit 3036124 derives a VSP (View Synthesis Prediction) merge candidate. The VSP merge candidate is a merge candidate used in a predicted image generation process by viewpoint synthesis prediction performed by the inter predicted image generation unit 309. The VSP merge candidate derivation unit 3036124 receives the displacement vector from the displacement vector acquisition unit 3036122. The VSP merge candidate derivation unit 3036124 inputs the input displacement vector mvDisp to the vector mvLX and the displacement vector MvDisp, the reference picture index of the reference picture indicating the previous layer image pointed to by the displacement vector to the reference picture index refIdxLX, and the displacement vector to The layer ID refViewIdx of the layer is set to the layer ID RefViewIdx, and a VSP merge candidate is derived by setting the VSP mode flag VspModeFlag to 1. The VSP merge candidate derivation unit 3036124 outputs the derived VSP merge candidate to the merge candidate storage unit 303611.
　本実施形態のVSPマージ候補導出部３０３６１２４は、インター予測パラメータ復号制御部から残差予測重みインデックスiv_res_pred_weight_idx及び照度補償フラグic_flagを入力として受け取る。VSPマージ候補導出部３０３６１２４は、残差予測重みインデックスiv_res_pred_weight_idxが０、かつ、照度補償フラグic_flagが０の場合のみ、VSPマージ候補の導出処理を行う。つまり、残差予測重みインデックスiv_res_pred_weight_idxが０、かつ、照度補償フラグic_flagが０の場合のみ、マージ候補リストmergeCandListの要素に、VSPマージ候補が追加される。逆に、VSPマージ候補導出部３０３６１２４は、残差予測重みインデックスiv_res_pred_weight_idxが０以外、又は、照度補償フラグic_flagが０以外の場合には、マージ候補リストmergeCandListの要素に、VSPマージ候補が追加されない。これにより、残差予測又は照度補償予測が実施される場合、即ち、視点合成予測が実施されない場合に、利用されることのないVSPマージ候補の導出処理がスキップされる事による計算量の削減と、マージ候補の増大を防ぐことによりマージインデックスmerge_idxのばらつきを抑えられるため、符号化効率が改善するという効果を奏する。 The VSP merge candidate derivation unit 3036124 of the present embodiment receives the residual prediction weight index iv_res_pred_weight_idx and the illumination compensation flag ic_flag from the inter prediction parameter decoding control unit. The VSP merge candidate derivation unit 3036124 performs VSP merge candidate derivation processing only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0. That is, only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0, the VSP merge candidate is added to the elements of the merge candidate list mergeCandList. Conversely, the VSP merge candidate derivation unit 3036124 does not add the VSP merge candidate to the elements of the merge candidate list mergeCandList when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0. As a result, when residual prediction or illumination compensation prediction is performed, that is, when viewpoint synthesis prediction is not performed, the calculation amount reduction is reduced by skipping the derivation process of VSP merge candidates that are not used. Since the variation of the merge index merge_idx can be suppressed by preventing an increase in merge candidates, the coding efficiency is improved.
　なお、残差予測のみを行い照度補償予測を行わない構成も可能である。この構成では、VSPマージ候補導出部３０３６１２４は、残差予測重みインデックスiv_res_pred_weight_idxが０の場合のみ、VSPマージ候補の導出処理を行う。つまり、残差予測重みインデックスiv_res_pred_weight_idxが０の場合のみ、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加する。逆に、残差予測重みインデックスiv_res_pred_weight_idxが０以外の場合には、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加しない。 Note that a configuration in which only residual prediction is performed and illuminance compensation prediction is not performed is also possible. In this configuration, the VSP merge candidate derivation unit 3036124 performs the VSP merge candidate derivation process only when the residual prediction weight index iv_res_pred_weight_idx is 0. That is, only when the residual prediction weight index iv_res_pred_weight_idx is 0, VSP merge candidates are added to the elements of the merge candidate list mergeCandList. Conversely, when the residual prediction weight index iv_res_pred_weight_idx is other than 0, no VSP merge candidate is added to the elements of the merge candidate list mergeCandList.
　なお、照度補償予測のみを行い残差予測を行わない構成も可能である。この構成では、VSPマージ候補導出部３０３６１２４は、照度補償フラグic_flagが０の場合のみ、VSPマージ候補の導出処理を行う。つまり、照度補償フラグic_flagが０の場合のみ、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加する。逆に、照度補償フラグic_flagが０以外の場合には、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加しない。 Note that a configuration is possible in which only illumination compensation prediction is performed and residual prediction is not performed. In this configuration, the VSP merge candidate derivation unit 3036124 performs VSP merge candidate derivation processing only when the illumination compensation flag ic_flag is 0. That is, only when the illumination compensation flag ic_flag is 0, the VSP merge candidate is added to the element of the merge candidate list mergeCandList. Conversely, when the illumination compensation flag ic_flag is other than 0, no VSP merge candidate is added to the elements of the merge candidate list mergeCandList.
　　（VSPマージ候補導出部の変形例１）
　上記構成では、残差予測重みインデックスiv_res_pred_weight_idxが０以外、又は、照度補償フラグic_flagが０以外の場合には、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加しないことについて説明したが、別の形態として、VSPマージ候補の代わりとなる代替マージ候補を導出し、マージ候補リストmergeCandListの要素に追加してもよい。 (Variation 1 of VSP merge candidate derivation unit)
In the above configuration, when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0, it has been described that the VSP merge candidate is not added to the element of the merge candidate list mergeCandList. As a form, an alternative merge candidate that replaces the VSP merge candidate may be derived and added to the elements of the merge candidate list mergeCandList.
　代替マージ候補の導出方法としては、例えば、既にマージ候補リストmergeCandListの先頭に格納されているマージ候補に変更を加えたものを代替マージ候補とする。具体的な変更方法としては、例えば、マージ候補のL０予測のベクトルmvL0のＸ成分に４を加算する。なお、マージ候補の変更方法は上記に限定されるものではなく、例えば、ベクトルmvL0のＹ成分から８を減算してもよいし、L1予測のベクトルmvL1を変更対象としてもよい。 As an alternative merge candidate derivation method, for example, a merge candidate that is already stored at the top of the merge candidate list mergeCandList is changed to an alternative merge candidate. As a specific change method, for example, 4 is added to the X component of the vector mvL0 of the merge candidate L0 prediction. The method for changing the merge candidate is not limited to the above. For example, 8 may be subtracted from the Y component of the vector mvL0, or the vector mvL1 of the L1 prediction may be changed.
　図１９は、本変形例におけるマージ候補リストmergeCandListの例を示すものである。この例においては、マージインデックスが５であるマージ候補は、VSPマージ候補又は代替マージ候補のいずれかとなる。 FIG. 19 shows an example of the merge candidate list mergeCandList in this modification. In this example, the merge candidate whose merge index is 5 is either a VSP merge candidate or an alternative merge candidate.
　上記変形例では、マージ候補リストmergeCandListに格納される候補数が残差予測重みインデックスiv_res_pred_weight_idxや照度補償フラグic_flagの値によって変化しない。そのため、画像符号化装置１１で符号化方法を選択する際に、残差予測重みインデックスiv_res_pred_weight_idxや照度補償フラグic_flagの値を変更した場合には、マージ候補リストmergeCandListのVSP候補と代替候補を入れ替えるだけでよく、マージ候補リストmergeCandList全体を生成し直す必要がないため、処理量の増加を抑制できるという効果を奏する。 In the above modification, the number of candidates stored in the merge candidate list mergeCandList does not change depending on the residual prediction weight index iv_res_pred_weight_idx and the value of the illumination compensation flag ic_flag. Therefore, when the encoding method is selected by the image encoding device 11, if the value of the residual prediction weight index iv_res_pred_weight_idx or the illumination compensation flag ic_flag is changed, the VSP candidate and the alternative candidate in the merge candidate list mergeCandList are simply replaced. In this case, there is no need to regenerate the entire merge candidate list mergeCandList, so that an increase in processing amount can be suppressed.
　　（VSPマージ候補導出部の変形例２）
　VSPマージ候補導出部の別の形態として、対象ブロックのサイズが所定のサイズ以下（例えば、幅nPSWが８以下または高さnPSHが８以下）の場合には、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加しない構成をとってもよい。つまり、対象ブロックのサイズが所定のサイズより大きい（例えば、幅nPSWが８より大きく、なお且つ、高さnPSHが８より大きい）場合に、マージ候補リストmergeCandListの要素に、VSPマージ候補を追加する。 (Modification 2 of VSP merge candidate derivation unit)
As another form of the VSP merge candidate derivation unit, when the size of the target block is equal to or smaller than a predetermined size (for example, the width nPSW is 8 or less or the height nPSH is 8 or less), the element of the merge candidate list mergeCandList is A configuration in which merge candidates are not added may be adopted. That is, when the size of the target block is larger than a predetermined size (for example, the width nPSW is larger than 8 and the height nPSH is larger than 8), the VSP merge candidate is added to the element of the merge candidate list mergeCandList. .
　後述する視点合成予測部３０９４では、対象ブロックをサブブロックに分割し、サブブロックごとに参照ピクチャから視差配列disparitySampleArrayだけずれた位置のブロックを読み出し、予測画像を生成している。つまり、視点合成予測では、サブブロックごとに読み出す位置を設定し異なる変位ベクトルを用いて動き変位予測を行うことで、符号化効率を向上させている。しかし、対象ブロックのサイズが小さい場合には、サブブロックに分割されない（もしくはサブブロックに分割した場合にも分割数が小さい）ため効果が小さい。効果の小さいＶＳＰマージ候補をマージ候補リストmergeCandListの要素に追加する場合、ＶＳＰマージ候補より効果の大きいマージ候補の位置が後方になり、良いマージ候補の選択により多くの符号量（より大きなmerge_idx）が必要となる。従って、ＶＳＰマージ候補を追加することで、逆に、符号化効率が低下することが生じる。 The viewpoint synthesis prediction unit 3094 described later divides the target block into sub-blocks, reads out a block at a position shifted by the disparity array disparitySampleArray from the reference picture for each sub-block, and generates a predicted image. In other words, in view synthesis prediction, encoding efficiency is improved by setting a reading position for each sub-block and performing motion displacement prediction using different displacement vectors. However, when the size of the target block is small, the effect is small because it is not divided into subblocks (or the number of divisions is small even when divided into subblocks). When a VSP merge candidate having a small effect is added to an element of the merge candidate list mergeCandList, the position of the merge candidate having a larger effect than that of the VSP merge candidate is behind, and a larger code amount (a larger merge_idx) is selected by selecting a good merge candidate. Necessary. Therefore, by adding VSP merge candidates, conversely, encoding efficiency may be reduced.
　上記変形例では、対象ブロックのサイズが所定のサイズ以下の場合、つまり、サブブロックに分割されない（もしくはサブブロックに分割した場合にも分割数が小さい）場合には、VSPマージ候補を導出しないため、上記性能の低下を抑制しつつ、ＶＳＰマージ候補導出（およびサブブロック分割により行われるＶＳＰブロックの視点合成処理の）計算量が削減されるという効果を奏する。 In the above modification, if the size of the target block is equal to or smaller than the predetermined size, that is, if it is not divided into sub-blocks (or the number of divisions is small even when divided into sub-blocks), VSP merge candidates are not derived. Thus, it is possible to reduce the calculation amount of the VSP merge candidate derivation (and the VSP block viewpoint synthesis processing performed by sub-block division) while suppressing the above-described deterioration in performance.
　基本マージ候補導出部３０３６１３は、空間マージ候補導出部３０３６１３１と時間マージ候補導出部３０３６１３２と結合マージ候補導出部３０３６１３３とゼロマージ候補導出部３０３６１３４を含んで構成される。 The basic merge candidate derivation unit 303613 includes a spatial merge candidate derivation unit 3036131, a temporal merge candidate derivation unit 3036132, a merge merge candidate derivation unit 3036133, and a zero merge candidate derivation unit 3036134.
　空間マージ候補導出部３０３６１３１は、所定の規則に従って、予測パラメータメモリ３０７が記憶している予測パラメータ（予測リスト利用フラグpredFlagLX、ベクトルmvLX、参照ピクチャインデックスrefIdxLX）を読み出し、読み出した予測パラメータを空間マージ候補として導出する。読み出される予測パラメータは、復号対象ブロックから予め定めた範囲内にあるブロック（例えば、復号対象ブロックの左下端、左上端、右上端にそれぞれ接するブロックの全部又は一部）である隣接ブロックのそれぞれに係る予測パラメータである。導出された空間マージ候補はマージ候補格納部３０３６１１に格納される。 The spatial merge candidate derivation unit 3036131 reads the prediction parameters (prediction list use flag predFlagLX, vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule, and uses the read prediction parameters as spatial merge candidates. Derived as The prediction parameter to be read out is for each adjacent block that is a block within a predetermined range from the decoding target block (for example, all or a part of the blocks in contact with the lower left end, upper left upper end, and upper right end of the decoding target block). This is a prediction parameter. The derived spatial merge candidate is stored in the merge candidate storage unit 303611.
　空間マージ候補導出部３０３６１３１では、空間マージ候補のVSPモードフラグVspModeFlagとして、隣接ブロックのVSPモードフラグVspModeFlagを継承する。すなわち、隣接ブロックのVSPモードフラグVspModeFlagが１の場合、対応する空間マージ候補のVSPモードフラグVspModeFlagは１、それ以外の場合はVSPモードフラグVspModeFlagを０とする。 The spatial merge candidate derivation unit 3036131 inherits the VSP mode flag VspModeFlag of the adjacent block as the VSP mode flag VspModeFlag of the spatial merge candidate. That is, when the VSP mode flag VspModeFlag of the adjacent block is 1, the VSP mode flag VspModeFlag of the corresponding spatial merge candidate is 1, and in other cases, the VSP mode flag VspModeFlag is 0.
　さらに、空間マージ候補導出部３０３６１３１は、隣接ブロックのVSPモードフラグVspModeFlagが１の場合には、隣接ブロックの変位ベクトルと変位ベクトルが示すレイヤのレイヤＩＤも継承する。すなわち、空間マージ候補導出部３０３６１３１は、空間マージ候補の変位ベクトルMvDispとレイヤＩＤRefViewIdxとして、隣接ブロックの変位ベクトルmvDispと、隣接ブロックの変位ベクトルが示すレイヤのレイヤＩＤrefViewIdxをそれぞれ設定する。 Furthermore, when the VSP mode flag VspModeFlag of the adjacent block is 1, the spatial merge candidate derivation unit 3036131 inherits the displacement vector of the adjacent block and the layer ID of the layer indicated by the displacement vector. That is, the spatial merge candidate derivation unit 3036131 sets the displacement vector MvDisp of the adjacent block and the layer ID refViewIdx of the layer indicated by the displacement vector of the adjacent block as the displacement vector MvDisp and the layer ID RefViewIdx of the spatial merge candidate, respectively.
　以下、時間マージ候補導出部３０３６１３２、結合マージ候補導出部３０３６１３３、ゼロマージ候補導出部３０３６１３４では、VSPモードフラグVspModeFlagは０に設定する。 Hereinafter, in the time merge candidate derivation unit 3036132, the merge merge candidate derivation unit 3036133, and the zero merge candidate derivation unit 3036134, the VSP mode flag VspModeFlag is set to 0.
　時間マージ候補導出部３０３６１３２は、復号対象ブロックの右下の座標を含む参照画像中のブロックの予測パラメータを予測パラメータメモリ３０７から読みだしマージ候補とする。参照画像の指定方法は、例えば、スライスヘッダに置いて指定された参照ピクチャインデックスrefIdxLXでも良いし、復号対象ブロックに隣接するブロックの参照ピクチャインデックスrefIdxLXのうち最小のものを用いて指定しても良い。導出されたマージ候補はマージ候補格納部３０３６１１に格納される。 The temporal merge candidate derivation unit 3036132 reads the prediction parameter of the block in the reference image including the lower right coordinate of the decoding target block from the prediction parameter memory 307 and sets it as a merge candidate. The reference picture designation method may be, for example, the reference picture index refIdxLX designated in the slice header, or may be designated using the smallest reference picture index refIdxLX of the block adjacent to the decoding target block. . The derived merge candidates are stored in the merge candidate storage unit 303611.
　結合マージ候補導出部３０３６１３３は、既に導出されマージ候補格納部３０３６１１に格納された２つの異なる導出済マージ候補のベクトルと参照ピクチャインデックスを、それぞれＬ０、Ｌ１のベクトルとして組み合わせることで結合マージ候補を導出する。導出されたマージ候補はマージ候補格納部３０３６１１に格納される。 The merge merge candidate derivation unit 3036133 derives merge merge candidates by combining two different derived merge candidate vectors and reference picture indexes already derived and stored in the merge candidate storage unit 303611 as L0 and L1 vectors, respectively. To do. The derived merge candidates are stored in the merge candidate storage unit 303611.
　ゼロマージ候補導出部３０３６１３４は、参照ピクチャインデックスrefIdxLXが０であり、ベクトルmvLXのＸ成分、Ｙ成分が共に０であるマージ候補を導出する。導出されたマージ候補はマージ候補格納部３０３６１１に格納される。 The zero merge candidate derivation unit 3036134 derives a merge candidate in which the reference picture index refIdxLX is 0 and both the X component and the Y component of the vector mvLX are 0. The derived merge candidates are stored in the merge candidate storage unit 303611.
　マージ候補選択部３０３６２は、マージ候補格納部３０３６１１に格納されているマージ候補のうち、インター予測パラメータ復号制御部３０３１から入力されたマージインデックスmerge_idxに対応するインデックスが割り当てられたマージ候補を、対象ＰＵのインター予測パラメータとして選択する。つまり、マージ候補リストをmergeCandListとするとmergeCandList[merge_idx]で示される予測パラメータを選択する。マージ候補選択部３０３６２は、選択したマージ候補を予測パラメータメモリ３０７（図５）に記憶するとともに、予測画像生成部３０８（図５）に出力する。 The merge candidate selection unit 30362 selects, from the merge candidates stored in the merge candidate storage unit 303611, a merge candidate to which an index corresponding to the merge index merge_idx input from the inter prediction parameter decoding control unit 3031 is assigned. As an inter prediction parameter. That is, when the merge candidate list is mergeCandList, the prediction parameter indicated by mergeCandList [merge_idx] is selected. The merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 (FIG. 5) and outputs it to the prediction image generation unit 308 (FIG. 5).
　図８は、本実施形態に係るAMVP予測パラメータ導出部３０３２の構成を示す概略図である。AMVP予測パラメータ導出部３０３２は、ベクトル候補導出部３０３３と予測ベクトル選択部３０３４を備える。ベクトル候補導出部３０３３は、参照ピクチャインデックスrefIdxに基づいて予測パラメータメモリ３０７（図５）が記憶するベクトル（動きベクトル又は変位ベクトル）をベクトル候補mvpLXとして読み出す。読み出されるベクトルは、復号対象ブロックから予め定めた範囲内にあるブロック（例えば、復号対象ブロックの左下端、左上端、右上端にそれぞれ接するブロックの全部又は一部）のそれぞれに係るベクトルである。 FIG. 8 is a schematic diagram showing the configuration of the AMVP prediction parameter derivation unit 3032 according to this embodiment. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033 and a prediction vector selection unit 3034. The vector candidate derivation unit 3033 reads a vector (motion vector or displacement vector) stored in the prediction parameter memory 307 (FIG. 5) as a vector candidate mvpLX based on the reference picture index refIdx. The vector to be read is a vector related to each of the blocks within a predetermined range from the decoding target block (for example, all or a part of the blocks in contact with the lower left end, the upper left upper end, and the upper right end of the decoding target block, respectively).
　予測ベクトル選択部３０３４は、ベクトル候補導出部３０３３が読み出したベクトル候補のうち、インター予測パラメータ復号制御部３０３１から入力されたベクトルインデックスmvp_LX_idxが示すベクトル候補を予測ベクトルmvpLXとして選択する。予測ベクトル選択部３０３４は、選択した予測ベクトルmvpLXを加算部３０３５に出力する。 The prediction vector selection unit 3034 selects a vector candidate indicated by the vector index mvp_LX_idx input from the inter prediction parameter decoding control unit 3031 among the vector candidates read by the vector candidate derivation unit 3033 as the prediction vector mvpLX. The prediction vector selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3035.
　図９は、ベクトル候補の一例を示す概念図である。図９に示す予測ベクトルリスト６０２は、ベクトル候補導出部３０３３において導出される複数のベクトル候補からなるリストである。予測ベクトルリスト６０２において、左右に一列に配列された５個の長方形は、それぞれ予測ベクトルを示す領域を示す。左端から２番目のmvp_LX_idxの真下の下向きの矢印とその下のmvpLXは、ベクトルインデックスmvp_LX_idxが、予測パラメータメモリ３０７においてベクトルmvpLXを参照するインデックスであることを示す。 FIG. 9 is a conceptual diagram showing an example of vector candidates. A predicted vector list 602 illustrated in FIG. 9 is a list including a plurality of vector candidates derived by the vector candidate deriving unit 3033. In the prediction vector list 602, five rectangles arranged in a line on the left and right indicate areas indicating prediction vectors, respectively. The downward arrow directly below the second mvp_LX_idx from the left end and mvpLX below the mvp_LX_idx indicate that the vector index mvp_LX_idx is an index referring to the vector mvpLX in the prediction parameter memory 307.
　候補ベクトルは、復号処理が完了したブロックであって、復号対象ブロックから予め定めた範囲のブロック（例えば、隣接ブロック）を参照し、参照したブロックに係るベクトルに基づいて生成される。なお、隣接ブロックには、対象ブロックに空間的に隣接するブロック、例えば、左ブロック、上ブロックの他、対象ブロックに時間的に隣接するブロック、例えば、対象ブロックと同じ位置で、表示時刻が異なるブロックから得られたブロックを含む。 The candidate vector is a block for which the decoding process has been completed, and is generated based on a vector related to the referenced block with reference to a block (for example, an adjacent block) in a predetermined range from the decoding target block. The adjacent block has a block that is spatially adjacent to the target block, for example, the left block and the upper block, and a block that is temporally adjacent to the target block, for example, the same position as the target block, and has a different display time. Contains blocks derived from blocks.
　加算部３０３５は、予測ベクトル選択部３０３４から入力された予測ベクトルmvpLXとインター予測パラメータ復号制御部から入力された差分ベクトルmvdLXを加算してベクトルmvLXを算出する。加算部３０３５は、算出したベクトルmvLXを予測画像生成部３０８（図５）に出力する。 The addition unit 3035 adds the prediction vector mvpLX input from the prediction vector selection unit 3034 and the difference vector mvdLX input from the inter prediction parameter decoding control unit to calculate a vector mvLX. The adding unit 3035 outputs the calculated vector mvLX to the predicted image generation unit 308 (FIG. 5).
　図１０は、第１の実施形態のインター予測パラメータ復号制御部３０３１の構成を示すブロック図である。図１０に示すように、インター予測パラメータ復号制御部３０３１は、残差予測重みインデックス復号部３０３１１、照度補償フラグ復号部３０３１２及び図示しない、分割モード復号部、マージフラグ復号部、マージインデックス復号部、インター予測フラグ復号部、参照ピクチャインデックス復号部、ベクトル候補インデックス復号部、ベクトル差分復号部を含んで構成される。分割モード復号部、マージフラグ復号部、マージインデックス復号部、インター予測フラグ復号部、参照ピクチャインデックス復号部、ベクトル候補インデックス復号部、ベクトル差分復号部は各々、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLXを復号する。 FIG. 10 is a block diagram illustrating a configuration of the inter prediction parameter decoding control unit 3031 according to the first embodiment. As shown in FIG. 10, the inter prediction parameter decoding control unit 3031 includes a residual prediction weight index decoding unit 30311, an illuminance compensation flag decoding unit 30312, and a split mode decoding unit, a merge flag decoding unit, a merge index decoding unit, not shown, An inter prediction flag decoding unit, a reference picture index decoding unit, a vector candidate index decoding unit, and a vector difference decoding unit are configured. The partition mode decoding unit, the merge flag decoding unit, the merge index decoding unit, the inter prediction flag decoding unit, the reference picture index decoding unit, the vector candidate index decoding unit, and the vector difference decoding unit are respectively divided mode part_mode, merge flag merge_flag, and merge index. The merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX are decoded.
　残差予測重みインデックス復号部３０３１１は、エントロピー復号部３０１を用いて、残差予測重みインデックスiv_res_pred_weight_idxを復号する。残差予測重みインデックス復号部３０３１１は、復号された残差予測重みインデックスiv_res_pred_weight_idxをマージモードパラメータ導出部３０３６とインター予測画像生成部３０９に出力する。 The residual prediction weight index decoding unit 30311 uses the entropy decoding unit 301 to decode the residual prediction weight index iv_res_pred_weight_idx. The residual prediction weight index decoding unit 30311 outputs the decoded residual prediction weight index iv_res_pred_weight_idx to the merge mode parameter derivation unit 3036 and the inter prediction image generation unit 309.
　照度補償フラグ復号部３０３１２は、エントロピー復号部３０１を用いて、照度補償フラグic_flagを復号する。照度補償フラグ復号部３０３１２は、復号された照度補償フラグic_flagをマージモードパラメータ導出部３０３６とインター予測画像生成部３０９に出力する。
　変位ベクトル取得部は、対象ＰＵに隣接するブロックが変位ベクトルを持つ場合には、その変位ベクトルを予測パラメータメモリ３０７から抽出し、予測パラメータメモリ３０７を参照し、対象ＰＵに隣接するブロックの予測フラグpredFlagLX、参照ピクチャインデックスrefIdxLXとベクトルmvLXを読み出す。変位ベクトル取得部は、内部にリファレンスレイヤ判定部３０３１１１を備える。変位ベクトル取得部は、対象ＰＵに隣接するブロックの予測パラメータを順に読み出し、リファレンスレイヤ判定部３０３１１１を用いて、隣接ブロックの参照ピクチャインデックスから隣接ブロックが変位ベクトルを備えるか否かを判定する。隣接ブロックが変位ベクトルを備える場合には、その変位ベクトルを出力する。隣接ブロックの予測パラメータに変位ベクトルが無い場合にはゼロベクトルを変位ベクトルとして出力する。 The illuminance compensation flag decoding unit 30312 uses the entropy decoding unit 301 to decode the illuminance compensation flag ic_flag. The illuminance compensation flag decoding unit 30312 outputs the decoded illuminance compensation flag ic_flag to the merge mode parameter derivation unit 3036 and the inter predicted image generation unit 309.
When the block adjacent to the target PU has a displacement vector, the displacement vector acquisition unit extracts the displacement vector from the prediction parameter memory 307, refers to the prediction parameter memory 307, and predicts the prediction flag of the block adjacent to the target PU. Read predFlagLX, reference picture index refIdxLX and vector mvLX. The displacement vector acquisition unit includes a reference layer determination unit 303111 therein. The displacement vector acquisition unit sequentially reads prediction parameters of blocks adjacent to the target PU, and determines whether the adjacent block has a displacement vector from the reference picture index of the adjacent block using the reference layer determination unit 303111. If the adjacent block has a displacement vector, the displacement vector is output. If there is no displacement vector in the prediction parameter of the adjacent block, the zero vector is output as the displacement vector.
　（リファレンスレイヤ判定部３０３１１１）
　リファレンスレイヤ判定部３０３１１１は、入力された参照ピクチャインデックスrefIdxLXに基づいて、参照ピクチャインデックスrefIdxLXが指す参照ピクチャと、対象ピクチャの関係を示すリファレンスレイヤ情報reference_layer_infoを定める。リファレンスレイヤ情報reference_layer_infoは、参照ピクチャへのベクトルmvLXが変位ベクトルであるか動きベクトルであるかを示す情報である。 (Reference layer determination unit 303111)
Based on the input reference picture index refIdxLX, the reference layer determination unit 303111 determines reference layer information reference_layer_info indicating a relationship between the reference picture indicated by the reference picture index refIdxLX and the target picture. Reference layer information reference_layer_info is information indicating whether the vector mvLX to the reference picture is a displacement vector or a motion vector.
　対象ピクチャのレイヤと参照ピクチャのレイヤが同じレイヤである場合の予測を、同一レイヤ予測と呼び、この場合に得られるベクトルは動きベクトルである。対象ピクチャのレイヤと、参照ピクチャのレイヤが異なるレイヤである場合の予測をレイヤ間予測と呼び、この場合に得られるベクトルは変位ベクトルである。 Prediction when the target picture layer and the reference picture layer are the same layer is called the same layer prediction, and the vector obtained in this case is a motion vector. Prediction when the target picture layer and the reference picture layer are different layers is called inter-layer prediction, and the vector obtained in this case is a displacement vector.
　　（デプスＤＶ導出部３５１）
　デプスＤＶ導出部３５１は、エントロピー復号部３０１において符号化データから復号されたデプスＤＶ変換テーブルDepthToDisparityBと、インター予測パラメータ復号部３０３から得られた、サブブロックの幅nSubBlkWと高さnSubBlkHと、分割フラグsplitFlagと、デプス画像refDepPelsと、デプス画像refDepPels上の対応ブロックの座標（ｘＴＬ、ｙＴＬ）と、レイヤＩＤrefViewIdxを用いて、以下の処理により、デプス由来の変位ベクトルの水平成分である視差配列disparitySamplesを導出する。 (Depth DV deriving unit 351)
The depth DV deriving unit 351 includes the depth DV conversion table DepthToDisparityB decoded from the encoded data by the entropy decoding unit 301, the width nSubBlkW and the height nSubBlkH of the subblock obtained from the inter prediction parameter decoding unit 303, and the division flag. Using splitFlag, depth image refDepPels, coordinates of corresponding blocks on depth image refDepPels (xTL, yTL), and layer ID refViewIdx, the following processing is used to derive disparity array disparitySamples that is the horizontal component of the depth-derived displacement vector To do.
　デプスＤＶ導出部３５１は、ブロックを分割したサブブロックをさらに分割したサブサブブロックごとに、サブサブブロックのコーナーおよびその付近の点を複数個用いて、デプスの代表値maxDepを導出する。 The depth DV deriving unit 351 derives a depth representative value maxDep by using a plurality of sub-sub-block corners and points in the vicinity thereof for each sub-sub-block obtained by further dividing the sub-block.
　具体的には、まず、デプスＤＶ導出部３５１は、サブサブブロックの幅nSubSubBlkWと高さnSubSubBlkHを決定する。分割フラグsplitFlagが１の場合には、サブブロックの幅nSubBlkWと高さnSubBlkHを半分にした値、もしくは、サブブロックの幅nSubBlkWを半分にした値と高さnSubBlkHを、サブサブブロックの幅nSubSubBlkWと高さnSubSubBlkHそれぞれに設定する。サブブロックの幅nSubBlkWと高さnSubBlkHのどちらを半分にするかは、サブブロックの左上端の座標のデプス画像の画素値をrefDepPelsP0、右上端の画素値をrefDepPelsP1、左下端の画素値をrefDepPelsP2、右下端の画素値をrefDepPelsP3とした場合、
　　( refDepPelsP0 > refDepPelsP3 ) == ( refDepPelsP1 > refDepPelsP2 )
の条件式が成立する場合には、高さを、成立しない場合には、幅を半分にする。また、分割フラグsplitFlagが０の場合には、サブサブブロックの幅nSubSubBlkWと高さnSubSubBlkHは、サブブロックと同じ幅nSubBlkWと高さnSubBlkHに設定する。なお、サブサブブロックの幅nSubSubBlkWと高さnSubSubBlkHの決定方法は上記の方法に限定されず、例えば、常にサブブロックの幅と高さを半分にした値を設定してもかまわないし、常にサブブロックと同じ幅と高さであってもかまわない。 Specifically, first, the depth DV deriving unit 351 determines the width nSubSubBlkW and the height nSubSubBlkH of the sub-subblock. When the split flag splitFlag is 1, the sub-block width nSubBlkW and the height nSubBlkH are halved, or the sub-block width nSubBlkW is halved and the height nSubBlkH is the sub-sub-block width nSubSubBlkW and high. Set each nSubSubBlkH. Whether the sub-block width nSubBlkW or height nSubBlkH is halved, the pixel value of the depth image at the upper left corner of the sub block is refDepPelsP0, the upper right pixel value is refDepPelsP1, the lower left pixel value is refDepPelsP2, If the pixel value at the bottom right is refDepPelsP3,
(refDepPelsP0> refDepPelsP3) == (refDepPelsP1> refDepPelsP2)
If the above conditional expression is satisfied, the height is halved. If not, the width is halved. When the split flag splitFlag is 0, the width nSubSubBlkW and the height nSubSubBlkH of the sub-sub block are set to the same width nSubBlkW and height nSubBlkH as the sub-block. The method for determining the width nSubSubBlkW and the height nSubSubBlkH of the sub-subblock is not limited to the above method.For example, a value obtained by halving the width and height of the sub-block may be set. The same width and height may be used.
　次に、デプスＤＶ導出部３５１は、サブサブブロックの左上の相対座標を（xSubB、ySubB）とした場合、サブサブブロックの左端のＸ座標xP0と、右端のＸ座標xP1と、上端のＹ座標yP0と、下端のＹ座標yP1を、
　　xP0 = Clip3( 0, pic_width_in_luma_samples - 1, xTL + xSubB )
　　yP0 = Clip3( 0, pic_height_in_luma_samples - 1, yTL + ySubB ) 
　　xP1 = Clip3( 0, pic_width_in_luma_samples - 1, xTL + xSubB + nSubSubBlkW - 1)
　　yP1 = Clip3( 0, pic_height_in_luma_samples - 1, yTL + ySubB + nSubSubBlkH - 1 )
の式により導出する。なお、pic_width_in_luma_samplesとpic_height_in_luma_samplesは、それぞれ画像の幅と高さを表す。 Next, the depth DV deriving unit 351 sets the left upper X coordinate xP0, the right end X coordinate xP1, and the upper end Y coordinate yP0 when the upper left relative coordinates of the sub sub block are (xSubB, ySubB). , Y coordinate yP1 of the lower end,
xP0 = Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB)
yP0 = Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB)
xP1 = Clip3 (0, pic_width_in_luma_samples-1, xTL + xSubB + nSubSubBlkW-1)
yP1 = Clip3 (0, pic_height_in_luma_samples-1, yTL + ySubB + nSubSubBlkH-1)
It is derived by the following formula. Note that pic_width_in_luma_samples and pic_height_in_luma_samples represent the width and height of the image, respectively.
　次に、デプスＤＶ導出部３５１は、サブサブブロックのデプスの代表値を導出する。具体的には、サブサブブロックのコーナーおよびその付近４点のデプス画像の画素値refDepPels[ xP0 ][ yP0 ]、refDepPels[ xP0 ][ yP1 ]、refDepPels[ xP1 ][ yP0 ]、refDepPels[ xP1 ][ yP1 ]の最大値である代表デプス値maxDepを、
　　maxDep = -1
　　maxDep = Max( maxDep, refDepPels[ xP0 ][ yP0 ] )
　　maxDep = Max( maxDep, refDepPels[ xP0 ][ yP1 ] )
　　maxDep = Max( maxDep, refDepPels[ xP1 ][ yP0 ] )
　　maxDep = Max( maxDep, refDepPels[ xP1 ][ yP1 ] )
の式を用いて導出する。なお、代表デプス値maxDepには初期値として-1が設定される。また、関数Max(x,y)は、第１引数ｘが第２引数ｙ以上であればｘを、そうでなければｙを返す関数である。 Next, the depth DV deriving unit 351 derives a representative value of the depth of the sub-subblock. Specifically, the pixel values refDepPels [xP0] [yP0], refDepPels [xP0], refDepPels [xP1] [yP0], refDepPels [xP1] [yP1] of the depth image at the corner of the sub-subblock and the four points in the vicinity thereof ] Representative depth value maxDep, which is the maximum value of
maxDep = -1
maxDep = Max (maxDep, refDepPels [xP0] [yP0])
maxDep = Max (maxDep, refDepPels [xP0] [yP1])
maxDep = Max (maxDep, refDepPels [xP1] [yP0])
maxDep = Max (maxDep, refDepPels [xP1] [yP1])
Derived using the following equation. Note that -1 is set as the initial value for the representative depth value maxDep. The function Max (x, y) is a function that returns x if the first argument x is greater than or equal to the second argument y, and returns y otherwise.
　デプスＤＶ導出部３５１は、代表デプス値maxDepとデプスＤＶ変換テーブルDepthToDisparityBと変位ベクトル(NBDV)MvDispが示すレイヤのレイヤＩＤrefViewIdxを用いて、デプス由来の変位ベクトルの水平成分である視差配列disparitySamplesを、サブサブブロック内の画素（ｘ、ｙ）（ｘは0からnSubSubBlkW-1、ｙは0からnSubSubBlkH-1の値を取る）ごとに、
　　disparitySamples[x][y] = DepthToDisparityB[refViewIdx][maxDep]・・（式Ａ）
の式を用いて導出する。
デプスＤＶ導出部３５１は、上記の処理を、サブブロック内の全てのサブサブブロックに対して実施する。デプスＤＶ導出部３５１は、導出した視差配列disparitySamplesを変位ベクトル取得部３０３６１２２及び視点合成予測部３０９４に出力する。 The depth DV deriving unit 351 uses the representative depth value maxDep, the depth DV conversion table DepthToDisparityB, and the layer ID refViewIdx of the layer indicated by the displacement vector (NBDV) MvDisp to subtract the disparity array disparitySamples that is the horizontal component of the displacement vector derived from the depth. For each pixel (x, y) in the block (where x is a value from 0 to nSubSubBlkW-1, y is a value from 0 to nSubSubBlkH-1),
disparitySamples [x] [y] = DepthToDisparityB [refViewIdx] [maxDep] ... (Formula A)
Derived using the following equation.
The depth DV deriving unit 351 performs the above processing on all sub-subblocks in the sub-block. The depth DV derivation unit 351 outputs the derived parallax array disparitySamples to the displacement vector acquisition unit 3036122 and the viewpoint synthesis prediction unit 3094.
　　（インター予測画像生成部３０９）
　図１１は、本実施形態に係るインター予測画像生成部３０９の構成を示す概略図である。インター予測画像生成部３０９は、動き変位補償部３０９１、残差予測部３０９２、照度補償部３０９３、視点合成予測部３０９４、インター予測画像生成制御部３０９６を含んで構成される。 (Inter prediction image generation unit 309)
FIG. 11 is a schematic diagram illustrating a configuration of the inter predicted image generation unit 309 according to the present embodiment. The inter prediction image generation unit 309 includes a motion displacement compensation unit 3091, a residual prediction unit 3092, an illuminance compensation unit 3093, a viewpoint synthesis prediction unit 3094, and an inter prediction image generation control unit 3096.
　インター予測画像生成制御部３０９６は、インター予測パラメータ復号部３０３から、VSPモードフラグVspModeFlagと予測パラメータを受け取る。インター予測画像生成制御部３０９６は、VSPモードフラグVspModeFlagが１の場合には、予測パラメータを視点合成予測部３０９４に出力する。また、インター予測画像生成制御部３０９６は、VSPモードフラグVspModeFlagが０の場合には、予測パラメータを動き変位補償部３０９１、残差予測部３０９２、照度補償部３０９３に出力する。また、インター予測画像生成制御部３０９６は、残差予測フラグiv_res_pred_weight_idxが０ではなく、かつ、対象ブロックが動き補償である場合に、動き変位補償部３０９１と残差予測実施フラグresPredFlagに残差予測を実行することを示す１を設定し、残差予測部３０９２に出力する。一方、残差予測フラグiv_res_pred_weight_idxが０である、又は、対象ブロックが動き補償でない場合（視差補償である場合）に、残差予測実施フラグresPredFlagに０を設定し、動き変位補償部３０９１と残差予測部３０９２に出力する。 The inter prediction image generation control unit 3096 receives the VSP mode flag VspModeFlag and the prediction parameter from the inter prediction parameter decoding unit 303. When the VSP mode flag VspModeFlag is 1, the inter prediction image generation control unit 3096 outputs the prediction parameter to the view synthesis prediction unit 3094. Further, when the VSP mode flag VspModeFlag is 0, the inter predicted image generation control unit 3096 outputs the prediction parameters to the motion displacement compensation unit 3091, the residual prediction unit 3092, and the illuminance compensation unit 3093. Also, the inter prediction image generation control unit 3096 performs residual prediction on the motion displacement compensation unit 3091 and the residual prediction execution flag resPredFlag when the residual prediction flag iv_res_pred_weight_idx is not 0 and the target block is motion compensation. 1 indicating execution is set and output to the residual prediction unit 3092. On the other hand, when the residual prediction flag iv_res_pred_weight_idx is 0, or when the target block is not motion compensation (in the case of disparity compensation), the residual prediction execution flag resPredFlag is set to 0, and the motion displacement compensation unit 3091 and the residual The result is output to the prediction unit 3092.
　　（動き変位補償）
　動き変位補償部３０９１は、インター予測画像生成制御部３０９６から入力された予測パラメータである、予測リスト利用フラグpredFlagLX、参照ピクチャインデックスrefIdxLX、ベクトルmvLX（動きベクトル、又は変位ベクトル）に基づいて、予測画像を生成する。動き変位補償部３０９１は、参照ピクチャメモリ３０６から、参照ピクチャインデックスrefIdxLXで指定された参照ピクチャの対象ブロックの位置を起点として、ベクトルmvLXだけずれた位置にあるブロックを読み出し補間することによって予測画像を生成する。ここで、ベクトルmvLXが整数ベクトルでない場合には、動き補償フィルタ（もしくは変位補償フィルタ）と呼ばれる小数位置の画素を生成するためのフィルタを施して、予測画像を生成する。一般に、ベクトルmvLXが動きベクトルの場合、上記処理を動き補償と呼び、変位ベクトルの場合は変位補償と呼ぶ。ここでは総称して動き変位補償と表現する。以下、Ｌ０予測の予測画像をpredSamplesL0、Ｌ１予測の予測画像をpredSamplesL1と呼ぶ。両者を区別しない場合predSamplesLXと呼ぶ。以下、動き変位補償部３０９１で得られた予測画像predSamplesLXに、さらに残差予測および照度補償が行われる例を説明するが、これらの出力画像もまた、予測画像predSamplesLXと呼ぶ。なお、以下の残差予測および照度補償において、入力画像と出力画像を区別する場合には、入力画像をpredSamplesLX、出力画像をpredSamplesLX´と表現する。 (Motion displacement compensation)
The motion displacement compensation unit 3091 is based on the prediction list input flag predFlagLX, the reference picture index refIdxLX, and the vector mvLX (motion vector or displacement vector) that are the prediction parameters input from the inter prediction image generation control unit 3096. Is generated. The motion displacement compensation unit 3091 reads out a block at a position shifted by the vector mvLX from the reference picture memory 306 from the position of the target block of the reference picture specified by the reference picture index refIdxLX, and interpolates the predicted image. Generate. Here, when the vector mvLX is not an integer vector, a prediction image is generated by applying a filter called a motion compensation filter (or displacement compensation filter) for generating a pixel at a decimal position. In general, when the vector mvLX is a motion vector, the above processing is called motion compensation, and when the vector mvLX is a displacement vector, it is called displacement compensation. Here, it is collectively referred to as motion displacement compensation. Hereinafter, the prediction image of L0 prediction is referred to as predSamplesL0, and the prediction image of L1 prediction is referred to as predSamplesL1. If the two are not distinguished, they are called predSamplesLX. Hereinafter, an example in which residual prediction and illuminance compensation are further performed on the prediction image predSamplesLX obtained by the motion displacement compensation unit 3091 will be described. These output images are also referred to as prediction images predSamplesLX. In the following residual prediction and illuminance compensation, when an input image and an output image are distinguished, the input image is expressed as predSamplesLX and the output image is expressed as predSamplesLX ′.
　　（残差予測）
　残差予測部３０９２は、残差予測実施フラグresPredFlagが１の場合に、インター予測画像生成制御部３０９６から入力された予測パラメータを用いて、残差予測を行う。残差予測部３０９２は、残差予測実施フラグresPredFlagが０の場合には、処理を行わない。refResSamples残差予測は、予測画像生成の対象とする対象レイヤ（第２のレイヤ画像）とは異なる参照レイヤ（第１のレイヤ画像）の残差を、対象レイヤの予測した画像である予測画像predSamplesLXに加えることにより行われる。すなわち、参照レイヤと同様の残差が対象レイヤにも生じると仮定して、既に導出された参照レイヤの残差を対象レイヤの残差の推定値として用いる。ベースレイヤ（ベースビュー）では同じレイヤの画像のみが参照画像となる。従って、参照レイヤ（第１のレイヤ画像）がベースレイヤ（ベースビュー）である場合には、参照レイヤの予測画像は動き補償による予測画像であることから、対象レイヤ（第２のレイヤ画像）による予測においても、動き補償による予測画像である場合に、残差予測は有効である。すなわち、残差予測は対象ブロックが動き補償の場合に有効であるという特性を持つ。 (Residual prediction)
When the residual prediction execution flag resPredFlag is 1, the residual prediction unit 3092 performs residual prediction using the prediction parameters input from the inter-prediction image generation control unit 3096. The residual prediction unit 3092 does not perform processing when the residual prediction execution flag resPredFlag is 0. The refResSamples residual prediction is a prediction image predSamplesLX that is an image obtained by predicting a residual of a reference layer (first layer image) different from a target layer (second layer image) that is a target of prediction image generation. This is done by adding to That is, assuming that the same residual as that of the reference layer also occurs in the target layer, the already derived residual of the reference layer is used as an estimated value of the residual of the target layer. In the base layer (base view), only the image of the same layer becomes the reference image. Therefore, when the reference layer (first layer image) is a base layer (base view), the predicted image of the reference layer is a predicted image by motion compensation, and thus depends on the target layer (second layer image). Also in prediction, residual prediction is effective in the case of a predicted image by motion compensation. That is, the residual prediction has a characteristic that it is effective when the target block is motion compensation.
　図１４は残差予測部３０９２の構成を示すブロック図である。残差予測部３０９２は、参照画像取得部３０９２２と、残差合成部３０９２３から構成される。 FIG. 14 is a block diagram showing a configuration of the residual prediction unit 3092. The residual prediction unit 3092 includes a reference image acquisition unit 30922 and a residual synthesis unit 30923.
　参照画像取得部３０９２２は、残差予測実施フラグresPredFlagが１の場合には、インター予測パラメータ復号部３０３から入力された動きベクトルmvLXと残差予測変位ベクトルmvDisp、及び参照ピクチャメモリ３０６に格納された対応ブロックcurrIvSamplesLXと対応ブロックの参照ブロックrefIvSamplesLXを読み出す。 When the residual prediction execution flag resPredFlag is 1, the reference image acquisition unit 30922 stores the motion vector mvLX and the residual prediction displacement vector mvDisp input from the inter prediction parameter decoding unit 303 and the reference picture memory 306. Read the corresponding block currIvSamplesLX and the reference block refIvSamplesLX of the corresponding block.
　図１５は、対応ブロックcurrIvSamplesLXを説明するための図である。図１５に示すように、対象レイヤ上の対象ブロックに対応する対応ブロックは、参照レイヤ上の画像の対象ブロックの位置を起点として、参照レイヤと対象レイヤの位置関係を示すベクトルである変位ベクトルmvDispだけずれた位置になるブロックに位置する。 FIG. 15 is a diagram for explaining the corresponding block currIvSamplesLX. As shown in FIG. 15, the corresponding block corresponding to the target block on the target layer is a displacement vector mvDisp that is a vector indicating the positional relationship between the reference layer and the target layer, starting from the position of the target block of the image on the reference layer. It is located in a block that is shifted by a distance.
　具体的には、参照画像取得部３０９２２は、対象ブロックの画素の座標（ｘ，ｙ）を、対象ブロックの変位ベクトルmvDispだけずらした位置の画素を導出する。変位ベクトルmvDispが１／４ペルの小数精度であることを考慮し参照画像取得部３０９２２は、対象ブロックの画素の座標が（ｘＰ、ｙＰ）である場合に対応する整数精度の画素Ｒ０のＸ座標xIntとＹ座標yInt、及び変位ベクトルmvDispのＸ成分の小数部分xFracとＹ成分の小数部分yFracを
　xInt = xPb + ( mvLX[ 0 ] >> 2 )
　yInt = yPb + ( mvLX[ 1 ] >> 2 )
　xFrac = mvLX[ 0 ] & 3 
　yFrac = mvLX[ 1 ] & 3 
の式により導出する。ここで、Ｘ & 3は、Ｘの下位２ビットのみを取り出す数式である。 Specifically, the reference image acquisition unit 30922 derives a pixel at a position where the coordinates (x, y) of the pixel of the target block are shifted by the displacement vector mvDisp of the target block. In consideration of the fact that the displacement vector mvDisp has a decimal precision of ¼ pel, the reference image acquisition unit 30922 uses the X coordinate of the pixel R0 with integer precision corresponding to the pixel coordinate of the target block being (xP, yP). xInt, Y coordinate yInt, and Xpart fraction xFrac and Y part fraction yFrac of displacement vector mvDisp xInt = xPb + (mvLX [0] >> 2)
yInt = yPb + (mvLX [1] >> 2)
xFrac = mvLX [0] & 3
yFrac = mvLX [1] & 3
It is derived by the following formula. Here, X & 3 is a mathematical expression for extracting only the lower 2 bits of X.
　次に、参照画像取得部３０９２２は、変位ベクトルmvDispが１／４ペルの小数精度であることを考慮し、補間画素predPartLX[ x ][ y ]を生成する。まず、整数画素Ａ(xA,yB)、Ｂ(xB,yB)、Ｃ(xC,yC)及びＤ(xD,yD)の座標を
　xA = Clip3( 0, picWidthInSamples 1, xInt )
　xB = Clip3( 0, picWidthInSamples 1, xInt + 1 )
　xC = Clip3( 0, picWidthInSamples 1, xInt )
　xD = Clip3( 0, picWidthInSamples 1, xInt + 1 ) 
　yA = Clip3( 0, picHeightInSamples 1, yInt )
　yB = Clip3( 0, picHeightInSamples 1, yInt )
　yC = Clip3( 0, picHeightInSamples 1, yInt + 1 ) 
　yD = Clip3( 0, picHeightInSamples 1, yInt + 1 )
の式により導出する。ここで、整数画素Ａは画素Ｒ０に対応した画素であり、整数画素Ｂ，Ｃ，Ｄはそれぞれ整数画素Ａの右、下、右下に隣接する整数精度の画素である。参照画像取得部３０９２２は、各整数画素Ａ、Ｂ、Ｃ、及びＤに対応する参照画素refPicLX[ xA][ yA ]、refPicLX[ xB ][ yB ]、refPicLX[ xC ][ yC ]、及びrefPicLX[ xD ][ yD ]を参照ピクチャメモリ３０６から読み出す。 Next, the reference image acquisition unit 30922 generates an interpolation pixel predPartLX [x] [y] in consideration of the fact that the displacement vector mvDisp has a pel resolution of 1/4 pel. First, the coordinates of integer pixels A (xA, yB), B (xB, yB), C (xC, yC) and D (xD, yD) are set to xA = Clip3 (0, picWidthInSamples 1, xInt)
xB = Clip3 (0, picWidthInSamples 1, xInt + 1)
xC = Clip3 (0, picWidthInSamples 1, xInt)
xD = Clip3 (0, picWidthInSamples 1, xInt + 1)
yA = Clip3 (0, picHeightInSamples 1, yInt)
yB = Clip3 (0, picHeightInSamples 1, yInt)
yC = Clip3 (0, picHeightInSamples 1, yInt + 1)
yD = Clip3 (0, picHeightInSamples 1, yInt + 1)
It is derived by the following formula. Here, the integer pixel A is a pixel corresponding to the pixel R0, and the integer pixels B, C, and D are integer precision pixels adjacent to the right, bottom, and bottom right of the integer pixel A, respectively. The reference image acquisition unit 30922 includes reference pixels refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], and refPicLX [corresponding to the integer pixels A, B, C, and D, respectively. xD] [yD] is read from the reference picture memory 306.
　そして、参照画像取得部３０９２２は、参照画素refPicLX[ xA ][ yA ]、refPicLX[ xB][ yB ]、refPicLX[ xC ][ yC ]、refPicLX[ xD ][ yD ]と変位ベクトルmvDispのＸ成分の小数部分xFracとＹ成分の小数部分yFracを用いて、画素Ｒ０から変位ベクトルmvDispの小数部分だけずらした位置の画素である補間画素predPartLX[ x ][ y ]を導出する。具体的には、
　predPartLX[ x ][ y ] = (refPicLX[ xA ][ yA ] * ( 8 xFrac ) * ( 8 yFrac )
+ refPicLX[ xB ][ yB ] * ( 8 yFrac ) * xFrac
+ refPicLX[ xC ][ yC ] * ( 8 xFrac ) * yFrac
+ refPicLX[ xD ][ yD ] * xFrac * yFrac ) >> 6
の式により導出する。 Then, the reference image acquisition unit 30922 includes the reference pixel refPicLX [xA] [yA], refPicLX [xB] [yB], refPicLX [xC] [yC], refPicLX [xD] [yD] and the X component of the displacement vector mvDisp. An interpolated pixel predPartLX [x] [y], which is a pixel shifted by the decimal part of the displacement vector mvDisp from the pixel R0, is derived using the fractional part xFrac and the fractional part yFrac of the Y component. In particular,
predPartLX [x] [y] = (refPicLX [xA] [yA] * (8 xFrac) * (8 yFrac)
+ refPicLX [xB] [yB] * (8 yFrac) * xFrac
+ refPicLX [xC] [yC] * (8 xFrac) * yFrac
+ refPicLX [xD] [yD] * xFrac * yFrac) >> 6
It is derived by the following formula.
　参照画像取得部３０９２２は、上記の補間画素導出処理を、対象ブロック内の各画素に対して行い、補間画素の集合を補間ブロックpredPartLXとする。参照画像取得部３０９２２は、導出した補間ブロックpredPartLXを、対応ブロックcurrIvSamplesLXとして、残差合成部３０９２３に出力する。 The reference image acquisition unit 30922 performs the above interpolation pixel derivation process on each pixel in the target block, and sets a set of interpolation pixels as an interpolation block predPartLX. The reference image acquisition unit 30922 outputs the derived interpolation block predPartLX to the residual synthesis unit 30923 as the corresponding block currIvSamplesLX.
　図１６は、参照ブロックrefIvSamplesLXを説明するための図である。図１６に示すように、参照レイヤ上の対応ブロックに対応する参照ブロックは、参照レイヤ上の参照画像の対応ブロックの位置を起点として、対象ブロックの動きベクトルmvLXだけずれた位置になるブロックに位置する。 FIG. 16 is a diagram for explaining the reference block refIvSamplesLX. As shown in FIG. 16, the reference block corresponding to the corresponding block on the reference layer is located at the block that is shifted by the motion vector mvLX of the target block, starting from the position of the corresponding block of the reference image on the reference layer. To do.
　参照画像取得部３０９２２は、対応ブロックcurrIvSamplesLXを導出した処理と、変位ベクトルmvDispをベクトル（mvDisp[ 0 ] + mvLX [ 0 ]、mvDisp[ 1 ] + mvLX [ 1 ]）に置き換えている点を除いて、同様の処理を行うことで、対応ブロックrefIvSamplesLXを導出する。参照画像取得部３０９２２は、対応ブロックrefIvSamplesLXを残差合成部３０９２３に出力する。 The reference image acquisition unit 30922 except for the process of deriving the corresponding block currIvSamplesLX and replacing the displacement vector mvDisp with a vector (mvDisp [0] + mvLX [0], mvDisp [1] + mvLX [1]) The corresponding block refIvSamplesLX is derived by performing the same processing. The reference image acquisition unit 30922 outputs the corresponding block refIvSamplesLX to the residual synthesis unit 30923.
　残差合成部３０９２３は、残差予測実施フラグresPredFlagが１の場合には、予測画像predSamplesLX、対応ブロックcurrIvSamplesLX、参照ブロックrefIvSamplesLX及び残差予測フラグiv_res_pred_weight_idxから、補正予測画像predSamplesLX´を導出する。補正予測画像predSamplesLX´は、
　predSamplesLX´ = predSamplesLX + 
((currIvSamplesLX - refIvSamplesLX) >> (iv_res_pred_weight_idx - 1))
の式を用いて求める。残差合成部３０９２３は、残差予測実施フラグresPredFlagが０の場合には、予測画像predSamplesLXをそのまま出力する。 When the residual prediction execution flag resPredFlag is 1, the residual synthesis unit 30923 derives a corrected predicted image predSamplesLX ′ from the predicted image predSamplesLX, the corresponding block currIvSamplesLX, the reference block refIvSamplesLX, and the residual prediction flag iv_res_pred_weight_idx. The corrected predicted image predSamplesLX´
predSamplesLX´ = predSamplesLX +
((currIvSamplesLX-refIvSamplesLX) >> (iv_res_pred_weight_idx-1))
It is calculated using the following formula. When the residual prediction implementation flag resPredFlag is 0, the residual synthesis unit 30923 outputs the predicted image predSamplesLX as it is.
　　（照度補償）
　照度補償部３０９３は、照度補償フラグic_flagが１の場合に、入力された予測画像predSamplesLXに対して、照度補償を行う。照度補償フラグic_flagが０の場合には、入力された予測画像predSamplesLXをそのまま出力する。照度補償部３０９３に入力される予測画像predSamplesLXは、残差予測実施フラグresPredFlagが０の場合には、動き変位補償部３０９１の出力画像であり、残差予測実施フラグresPredFlagが１の場合には、残差予測部３０９２の出力画像である。 (Illuminance compensation)
When the illumination compensation flag ic_flag is 1, the illumination compensation unit 3093 performs illumination compensation on the input predicted image predSamplesLX. When the illumination compensation flag ic_flag is 0, the input predicted image predSamplesLX is output as it is. The prediction image predSamplesLX input to the illuminance compensation unit 3093 is an output image of the motion displacement compensation unit 3091 when the residual prediction execution flag resPredFlag is 0, and when the residual prediction execution flag resPredFlag is 1, It is an output image of the residual prediction unit 3092.
　　（視点合成予測）
　視点合成予測部３０９４は、VSPモードフラグVspModeFlagが１の場合に、インター予測画像生成制御部３０９６から入力された予測パラメータを用いて、視点合成予測を行う。視点合成予測部３０９４は、VSPモードフラグVspModeFlagが０の場合には、処理を行わない。視点合成予測とは、対象ブロックを、サブブロックに分割し、サブブロック単位で、参照ピクチャメモリ３０６から、視差配列disparitySampleArray分だけずれた位置のブロックを読み出し補間することによって予測画像predSamplesを生成する処理である。 (Perspective synthesis prediction)
When the VSP mode flag VspModeFlag is 1, the view synthesis prediction unit 3094 performs view synthesis prediction using the prediction parameter input from the inter prediction image generation control unit 3096. The viewpoint synthesis prediction unit 3094 does not perform processing when the VSP mode flag VspModeFlag is 0. View synthesis prediction is a process of dividing a target block into sub-blocks, and generating predicted images predSamples by reading out and interpolating blocks at positions shifted by the disparity array disparitySampleArray from the reference picture memory 306 in sub-block units. It is.
　図１７は視点合成予測部３０９４の構成を示すブロック図である。視点合成予測部３０９４は、視差配列導出部３０９４１と、参照画像取得部３０９４２から構成される。 FIG. 17 is a block diagram showing a configuration of the viewpoint synthesis prediction unit 3094. The viewpoint synthesis prediction unit 3094 includes a parallax array derivation unit 30941 and a reference image acquisition unit 30942.
　視差配列導出部３０９４１は、VSPモードフラグVspModeFlagが１の場合に、サブブロック単位で、視差配列disparitySampleArrayを導出する。 When the VSP mode flag VspModeFlag is 1, the disparity array deriving unit 30941 derives a disparity array disparitySampleArray in units of sub blocks.
　具体的には、まず、視差配列導出部３０９４１は、参照ピクチャメモリ３０６から、復号対象ピクチャと同一ＰＯＣを持ち、なお且つ、変位ベクトルが示すレイヤ画像のレイヤＩＤRefViewIdxと同じレイヤＩＤであるデプス画像refDepPelsを読み出す。なお、読み出すデプス画像refDepPelsのレイヤは、参照ピクチャインデックスrefIdxLXが示す参照ピクチャと同じレイヤであってもよいし、復号対象の画像と同じレイヤであってもよい。 Specifically, first, the disparity array deriving unit 30941 has the same POC as the decoding target picture from the reference picture memory 306, and the depth image refDepPels having the same layer ID as the layer ID RefViewIdx of the layer image indicated by the displacement vector. Is read. The layer of the depth image refDepPels to be read may be the same layer as the reference picture indicated by the reference picture index refIdxLX, or may be the same layer as the image to be decoded.
　次に、視差配列導出部３０９４１は、対象ブロックの左上の座標（ｘＰ、ｙＰ）を変位ベクトルMvDispだけずらした座標（ｘＴＬ、ｙＴＬ）を、
　　xTL = xP + ( ( mvDisp[ 0 ] + 2 ) >> 2 )
　　yTL = yP + ( ( mvDisp[ 1 ] + 2 ) >> 2 )
の式より導出する。なお、mvDisp[ 0 ]、mvDisp[ 1 ]は、それぞれ変位ベクトルMvDispのＸ成分とＹ成分である。導出する座標（ｘＴＬ、ｙＴＬ）は、デプス画像refDepPels上の対象ブロックに対応するブロックの座標を示すものである。 Next, the parallax array deriving unit 30941 obtains coordinates (xTL, yTL) obtained by shifting the upper left coordinates (xP, yP) of the target block by the displacement vector MvDisp,
xTL = xP + ((mvDisp [0] + 2) >> 2)
yTL = yP + ((mvDisp [1] + 2) >> 2)
Derived from the equation Note that mvDisp [0] and mvDisp [1] are the X component and the Y component of the displacement vector MvDisp, respectively. The derived coordinates (xTL, yTL) indicate the coordinates of the block corresponding to the target block on the depth image refDepPels.
　次に、サブブロックの幅nSubBlkWと高さnSubBlkHを、
　　nSubBlkW = splitFlag ? 8 : nPSW
　　nSubBlkH = splitFlag ? 8 : nPSH
の式を用いて導出する。なお、分割フラグsplitFlagは対象ブロックの幅nPSWと高さnPSHが両方とも４より大きい場合には１に、そうでない場合には０となるフラグである。つまり、サブブロックの幅nSubBlkWと高さnSubBlkHには、対象ブロックの幅nPSWと高さnPSHが両方とも４より大きい場合には、両方とも８を設定し、対象ブロックの幅nPSWと高さnPSHのどちらかが４以下である場合には、対象ブロックの幅nPSWと高さnPSHをそれぞれ設定する。なお、サブブロックの幅nSubBlkWと高さnSubBlkHは上記の式による導出結果に限定されず、例えば16や32など、対象ブロックのサイズを超えない範囲内で値を設定するのはかまわない。 Next, subblock width nSubBlkW and height nSubBlkH,
nSubBlkW = splitFlag? 8: nPSW
nSubBlkH = splitFlag? 8: nPSH
Derived using the following equation. Note that the split flag splitFlag is a flag that is set to 1 when both the width nPSW and the height nPSH of the target block are larger than 4, and is set to 0 otherwise. That is, if the width nPSW and the height nPSH of the target block are both greater than 4, the width nSubBlkW and the height nSubBlkH of the subblock are both set to 8, and the width nPSW and the height nPSH of the target block are both set. When either is 4 or less, the width nPSW and the height nPSH of the target block are set. Note that the width nSubBlkW and the height nSubBlkH of the sub-block are not limited to the derivation result by the above formula, and values may be set within a range that does not exceed the size of the target block, such as 16 or 32, for example.
　次に、視差配列導出部３０９４１は、対象ブロック内の全てのサブブロックごとに、ブロックの左上画素を原点とした場合のサブブロックの幅nSubBlkWと高さnSubBlkHと、分割フラグsplitFlagと、デプス画像refDepPelsと、対応ブロックの座標（ｘＴＬ、ｙＴＬ）と、参照ピクチャインデックスrefIdxLXが示す参照ピクチャが属するレイヤのレイヤＩＤrefViewIdxと、をデプスＤＶ導出部３５１に出力することで、デプスＤＶ導出部３５１から視差配列disparitySampleArrayを得る。視差配列導出部３０９４１は、導出した視差配列disparitySampleArrayを参照画像取得部３０９４２に出力する。 Next, the disparity array deriving unit 30941, for every subblock in the target block, the width nSubBlkW and height nSubBlkH of the sub-block when the upper left pixel of the block is the origin, the split flag splitFlag, and the depth image refDepPels And the coordinates (xTL, yTL) of the corresponding block and the layer ID refViewIdx of the layer to which the reference picture indicated by the reference picture index refIdxLX belongs are output from the depth DV deriving unit 351 to the disparity array disparitySampleArray. Get. The parallax array derivation unit 30941 outputs the derived parallax array disparitySampleArray to the reference image acquisition unit 30942.
　参照画像取得部３０９４２は、VSPモードフラグVspModeFlagが１の場合に、視差配列導出部３０９４１から入力された視差配列disparitySampleArrayとインター予測パラメータ復号部３０３から入力された参照ピクチャインデックスrefIdxLXから、予測ブロックpredSamplesを導出する。 When the VSP mode flag VspModeFlag is 1, the reference image acquisition unit 30942 calculates a prediction block predSamples from the disparity array disparitySampleArray input from the disparity array deriving unit 30941 and the reference picture index refIdxLX input from the inter prediction parameter decoding unit 303. To derive.
　参照画像取得部３０９４２は、対象ブロック中の画素ごとに、その画素の座標から、対応する視差配列disparitySampleArrayの値だけＸ座標をずらした位置の画素を、参照ピクチャインデックスrefIdxLXが指定する参照ピクチャrefPicから抽出する。視差配列disparitySampleArrayが１／４ペルの小数精度であることを考慮し、参照画像取得部３０９４２は、対象ブロックの左上端の画素の座標が（ｘＰ，ｙＰ）であり、対象ブロック中の各画素の座標が（ｘＬ、ｙＬ）(ｘＬは0からnPbWｙＬは0からnPbHの値を取る)である場合に、参照ピクチャrefPicから抽出する画素の整数制度の座標（ｘＩｎｔ、ｙＩｎｔ）と、画素（ｘＬ、ｙＬ）に対応する視差配列disparitySampleArray[ｘＬ][ｙＬ]の小数部分ｘＦｒａｃと、ｙＦｒａｃを、
　xIntL = xP + xL+ disparitySamples[ xL ][ yL ]
　yIntL = yP + yL
　xFracL = disparitySamples[ xL ][ yL ] & 3
　xFracL = 0
の式より導出する。 For each pixel in the target block, the reference image acquisition unit 30942 extracts, from the reference picture refPic specified by the reference picture index refIdxLX, a pixel at a position where the X coordinate is shifted from the coordinates of the corresponding pixel by the value of the corresponding disparity array disparitySampleArray. Extract. In consideration of the fact that the disparity array disparitySampleArray has a decimal precision of ¼ pel, the reference image acquisition unit 30942 has the coordinates of the upper left pixel of the target block as (xP, yP), and each pixel in the target block When the coordinates are (xL, yL) (xL takes a value from 0 to nPbWyL takes a value from 0 to nPbH), the coordinates (xInt, yInt) of the pixel extracted from the reference picture refPic and the pixel (xL, yInt) yL), the decimal part xFrac and yFrac of the disparity array disparitySampleArray [xL] [yL] corresponding to
xIntL = xP + xL + disparitySamples [xL] [yL]
yIntL = yP + yL
xFracL = disparitySamples [xL] [yL] & 3
xFracL = 0
Derived from the equation
　次に、参照画像取得部３０９４２は、参照画像取得部３０９２２と同様の補間画素導出処理を対象ブロック内の各画素に対して行い、補間画素の集合を補間ブロックpredPartLXとする。参照画像取得部３０９４２は、導出した補間ブロックpredPartLXを予測ブロックpredSamplesとして、加算部３１２に出力する。 Next, the reference image acquisition unit 30942 performs an interpolation pixel derivation process similar to that of the reference image acquisition unit 30922 on each pixel in the target block, and sets a set of interpolation pixels as an interpolation block predPartLX. The reference image acquisition unit 30942 outputs the derived interpolation block predPartLX to the addition unit 312 as the prediction block predSamples.
　なお、図１２に示すように、インター予測画像生成部は、照度補償予測を行わない構成をとることも可能である。この場合、動き変位補償部３０９１´及び残差予測部３０９２´は、予測画像predSamplesLXを加算部３１２に出力する。 In addition, as shown in FIG. 12, the inter prediction image generation unit can take a configuration that does not perform illuminance compensation prediction. In this case, the motion displacement compensation unit 3091 ′ and the residual prediction unit 3092 ′ output the predicted image predSamplesLX to the addition unit 312.
　なお、図１３に示すように、インター予測画像生成部は、残差予測を行わない構成をとることも可能である。この場合、VSPモードフラグVspModeFlagが０の場合は、常に動き変位補償部３０９１´´が予測画像predSamplesLXを導出する。 Note that, as shown in FIG. 13, the inter-prediction image generation unit can be configured not to perform residual prediction. In this case, when the VSP mode flag VspModeFlag is 0, the motion displacement compensation unit 3091 ″ always derives the predicted image predSamplesLX.
　　（画像符号化装置の構成）
　次に、本実施形態に係る画像符号化装置１１の構成について説明する。図２０は、本実施形態に係る画像符号化装置１１の構成を示すブロック図である。画像符号化装置１１は、予測画像生成部１０１、減算部１０２、ＤＣＴ・量子化部１０３、エントロピー符号化部１０４、逆量子化・逆ＤＣＴ部１０５、加算部１０６、予測パラメータメモリ（予測パラメータ記憶部、フレームメモリ）１０８、参照ピクチャメモリ（参照画像記憶部、フレームメモリ）１０９、符号化パラメータ決定部１１０、予測パラメータ符号化部１１１、残差格納部３１３（残差記録部）を含んで構成される。予測パラメータ符号化部１１１は、インター予測パラメータ符号化部１１２及びイントラ予測パラメータ符号化部１１３を含んで構成される。 (Configuration of image encoding device)
Next, the configuration of the image encoding device 11 according to the present embodiment will be described. FIG. 20 is a block diagram illustrating a configuration of the image encoding device 11 according to the present embodiment. The image encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a prediction parameter memory (prediction parameter storage). Unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, prediction parameter coding unit 111, and residual storage unit 313 (residual recording unit). Is done. The prediction parameter encoding unit 111 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113.
　予測画像生成部１０１は、外部から入力されたレイヤ画像Ｔの視点毎の各ピクチャについて、そのピクチャを分割した領域であるブロック毎に予測ピクチャブロックpredSmaplesを生成する。ここで、予測画像生成部１０１は、予測パラメータ符号化部１１１から入力された予測パラメータに基づいて参照ピクチャメモリ１０９から参照ピクチャブロックを読み出す。予測パラメータ符号化部１１１から入力された予測パラメータとは、例えば、動きベクトル又は変位ベクトルである。予測画像生成部１０１は、符号化対象ブロックを起点として予測された動きベクトル又は変位ベクトルが示す位置にあるブロックの参照ピクチャブロックを読み出す。予測画像生成部１０１は、読み出した参照ピクチャブロックについて複数の予測方式のうちの１つの予測方式を用いて予測ピクチャブロックpredSmaplesを生成する。予測画像生成部１０１は、生成した予測ピクチャブロックpredSmaplesを減算部１０２と加算部１０６に出力する。なお、予測画像生成部１０１は、既に説明した予測画像生成部３０８と同じ動作であるため予測ピクチャブロックpredSmaplesの生成の詳細は省略する。 The predicted image generation unit 101 generates predicted picture blocks predSmaples for each block that is an area obtained by dividing the picture for each viewpoint of the layer image T input from the outside. Here, the predicted image generation unit 101 reads the reference picture block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter encoding unit 111. The prediction parameter input from the prediction parameter encoding unit 111 is, for example, a motion vector or a displacement vector. The predicted image generation unit 101 reads the reference picture block of the block at the position indicated by the motion vector or the displacement vector predicted from the encoding target block. The predicted image generation unit 101 generates predicted picture blocks predSmaples using one of the plurality of prediction schemes for the read reference picture block. The predicted image generation unit 101 outputs the generated predicted picture block predSmaples to the subtraction unit 102 and the addition unit 106. Note that since the predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 already described, details of generation of the predicted picture block predSmaples are omitted.
　予測画像生成部１０１は、予測方式を選択するために、例えば、レイヤ画像に含まれるブロックの画素毎の信号値と予測ピクチャブロックpredSmaplesの対応する画素毎の信号値との差分に基づく誤差値を最小にする予測方式を選択する。なお、予測方式を選択する方法は、これには限られない。 In order to select a prediction method, the predicted image generation unit 101, for example, calculates an error value based on the difference between the signal value for each pixel of the block included in the layer image and the signal value for each corresponding pixel of the predicted picture block predSmaples. Select the prediction method to minimize. Note that the method of selecting the prediction method is not limited to this.
　符号化対象のピクチャがベースビューのピクチャである場合には、複数の予測方式とは、イントラ予測、動き予測及びマージモードである。動き予測とは、上述のインター予測のうち、表示時刻間の予測である。マージモードとは、既に符号化されたブロックであって、符号化対象ブロックから予め定めた範囲内にあるブロックと同一の参照ピクチャブロック及び予測パラメータを用いる予測である。符号化対象のピクチャがベースビュー以外のピクチャである場合には、複数の予測方式とは、イントラ予測、動き予測、マージモード（視点合成予測を含む）、及び変位予測である。変位予測（視差予測）とは、上述のインター予測のうち、別レイヤ画像（別視点画像）間の予測である。変位予測（視差予測）に対して、追加予測（残差予測および照度補償）を行う場合と行わない場合の予測がある。 When the encoding target picture is a base view picture, the plurality of prediction methods are intra prediction, motion prediction, and merge mode. Motion prediction is prediction between display times among the above-mentioned inter predictions. The merge mode is a prediction that uses the same reference picture block and prediction parameter as a block that has already been encoded and is within a predetermined range from the encoding target block. When the encoding target picture is a picture other than the base view, the plurality of prediction methods are intra prediction, motion prediction, merge mode (including viewpoint synthesis prediction), and displacement prediction. The displacement prediction (disparity prediction) is prediction between different layer images (different viewpoint images) in the above-described inter prediction. For displacement prediction (disparity prediction), there are predictions with and without additional prediction (residual prediction and illuminance compensation).
　予測画像生成部１０１は、イントラ予測を選択した場合、予測ピクチャブロックpredSmaplesを生成する際に用いたイントラ予測モードを示す予測モードpredModeを予測パラメータ符号化部１１１に出力する。 When the intra prediction is selected, the prediction image generation unit 101 outputs a prediction mode predMode indicating the intra prediction mode used when generating the prediction picture block predSmaples to the prediction parameter encoding unit 111.
　予測画像生成部１０１は、動き予測を選択した場合、予測ピクチャブロックpredSmaplesを生成する際に用いた動きベクトルmvLXを予測パラメータメモリ１０８に記憶し、インター予測パラメータ符号化部１１２に出力する。動きベクトルmvLXは、符号化対象ブロックの位置から予測ピクチャブロックpredSmaplesを生成する際の参照ピクチャブロックの位置までのベクトルを示す。動きベクトルmvLXを示す情報には、参照ピクチャを示す情報（例えば、参照ピクチャインデックスrefIdxLX、ピクチャ順序番号ＰＯＣ）を含み、予測パラメータを表すものであっても良い。また、予測画像生成部１０１は、インター予測モードを示す予測モードpredModeを予測パラメータ符号化部１１１に出力する。 When the motion picture prediction is selected, the prediction image generation unit 101 stores the motion vector mvLX used when generating the prediction picture block predSmaples in the prediction parameter memory 108 and outputs the motion vector mvLX to the inter prediction parameter encoding unit 112. The motion vector mvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block predSmaples is generated. The information indicating the motion vector mvLX may include information indicating a reference picture (for example, a reference picture index refIdxLX, a picture order number POC), and may represent a prediction parameter. Further, the predicted image generation unit 101 outputs a prediction mode predMode indicating the inter prediction mode to the prediction parameter encoding unit 111.
　予測画像生成部１０１は、変位予測を選択した場合、予測ピクチャブロックpredSmaplesを生成する際に用いた変位ベクトルを予測パラメータメモリ１０８に記憶し、インター予測パラメータ符号化部１１２に出力する。変位ベクトルdvLXは、符号化対象ブロックの位置から予測ピクチャブロックpredSmaplesを生成する際の参照ピクチャブロックの位置までのベクトルを示す。変位ベクトルdvLXを示す情報には、参照ピクチャを示す情報（例えば、参照ピクチャインデックスrefIdxLX、ビューＩＤview_id）を含み、予測パラメータを表すものであっても良い。また、予測画像生成部１０１は、インター予測モードを示す予測モードpredModeを予測パラメータ符号化部１１１に出力する。 When the prediction image generation unit 101 selects the displacement prediction, the prediction image generation unit 101 stores the displacement vector used when generating the prediction picture block predSmaples in the prediction parameter memory 108 and outputs it to the inter prediction parameter encoding unit 112. The displacement vector dvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block predSmaples is generated. The information indicating the displacement vector dvLX may include information indicating a reference picture (for example, reference picture index refIdxLX, view IDview_id) and may represent a prediction parameter. Further, the predicted image generation unit 101 outputs a prediction mode predMode indicating the inter prediction mode to the prediction parameter encoding unit 111.
　予測画像生成部１０１は、マージモードを選択した場合、選択した参照ピクチャブロックを示すマージインデックスmerge_idxをインター予測パラメータ符号化部１１２に出力する。また、予測画像生成部１０１は、マージモードを示す予測モードpredModeを予測パラメータ符号化部１１１に出力する。 When the merge mode is selected, the predicted image generation unit 101 outputs a merge index merge_idx indicating the selected reference picture block to the inter prediction parameter encoding unit 112. Further, the predicted image generation unit 101 outputs a prediction mode predMode indicating the merge mode to the prediction parameter encoding unit 111.
　上記の、マージモードにおいて、予測画像生成部１０１は、VSPモードフラグVspModeFlagが視点合成予測を行うことを示す場合には、既に説明したように予測画像生成部１０１に含まれる視点合成予測部３０９４において視点合成予測を行う。また、動き予測、変位予測、マージモードにおいて、予測画像生成部１０１は、残差予測実施フラグresPredFlagが残差予測を行うことを示す場合には、既に説明したように予測画像生成部１０１に含まれる残差予測部３０９２において残差予測を行う。 In the merge mode described above, when the VSP mode flag VspModeFlag indicates that the viewpoint synthesis prediction is performed, the prediction image generation unit 101 performs the viewpoint synthesis prediction unit 3094 included in the prediction image generation unit 101 as described above. Perform viewpoint synthesis prediction. Further, in the motion prediction, displacement prediction, and merge mode, the prediction image generation unit 101 includes the prediction image generation unit 101 as described above when the residual prediction execution flag resPredFlag indicates that the residual prediction is performed. The residual prediction unit 3092 performs residual prediction.
　減算部１０２は、予測画像生成部１０１から入力された予測ピクチャブロックpredSmaplesの信号値を、外部から入力されたレイヤ画像Ｔの対応するブロックの信号値から画素毎に減算して、残差信号を生成する。減算部１０２は、生成した残差信号をＤＣＴ・量子化部１０３と符号化パラメータ決定部１１０に出力する。 The subtraction unit 102 subtracts the signal value of the predicted picture block predSmaples input from the predicted image generation unit 101 for each pixel from the signal value of the corresponding block of the layer image T input from the outside, and generates a residual signal. Generate. The subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103 and the encoding parameter determination unit 110.
　ＤＣＴ・量子化部１０３は、減算部１０２から入力された残差信号についてＤＣＴを行い、ＤＣＴ係数を算出する。ＤＣＴ・量子化部１０３は、算出したＤＣＴ係数を量子化して量子化係数を求める。ＤＣＴ・量子化部１０３は、求めた量子化係数をエントロピー符号化部１０４及び逆量子化・逆ＤＣＴ部１０５に出力する。 The DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102 and calculates a DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain a quantization coefficient. The DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy encoding unit 104 and the inverse quantization / inverse DCT unit 105.
　エントロピー符号化部１０４には、ＤＣＴ・量子化部１０３から量子化係数が入力され、符号化パラメータ決定部１１０から符号化パラメータが入力される。入力される符号化パラメータには、例えば、参照ピクチャインデックスrefIdxLX、ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLX、予測モードpredMode、マージインデックスmerge_idx、残差予測重みインデックスiv_res_pred_weight_idx、及び照度補償フラグic_flag等の符号がある。 The entropy coding unit 104 receives the quantization coefficient from the DCT / quantization unit 103 and the coding parameter from the coding parameter determination unit 110. The input encoding parameters include, for example, codes such as a reference picture index refIdxLX, a vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, a merge index merge_idx, a residual prediction weight index iv_res_pred_weight_idx, and an illumination compensation flag ic_flag.
　エントロピー符号化部１０４は、入力された量子化係数と符号化パラメータをエントロピー符号化して符号化ストリームＴｅを生成し、生成した符号化ストリームＴｅを外部に出力する。 The entropy encoding unit 104 generates an encoded stream Te by entropy encoding the input quantization coefficient and encoding parameter, and outputs the generated encoded stream Te to the outside.
　逆量子化・逆ＤＣＴ部１０５は、ＤＣＴ・量子化部１０３から入力された量子化係数を逆量子化してＤＣＴ係数を求める。逆量子化・逆ＤＣＴ部１０５は、求めたＤＣＴ係数について逆ＤＣＴを行い、復号残差信号を算出する。逆量子化・逆ＤＣＴ部１０５は、算出した復号残差信号を加算部１０６、残差格納部３１３、及び符号化パラメータ決定部１１０に出力する。 The inverse quantization / inverse DCT unit 105 inversely quantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 105 performs inverse DCT on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 105 outputs the calculated decoded residual signal to the addition unit 106, the residual storage unit 313, and the coding parameter determination unit 110.
　加算部１０６は、予測画像生成部１０１から入力された予測ピクチャブロックpredSmaplesの信号値と逆量子化・逆ＤＣＴ部１０５から入力された復号残差信号の信号値を画素毎に加算して、参照ピクチャブロックを生成する。加算部１０６は、生成した参照ピクチャブロックを参照ピクチャメモリ１０９に記憶する。 The addition unit 106 adds the signal value of the prediction picture block predSmaples input from the prediction image generation unit 101 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and refers to them. Generate a picture block. The adding unit 106 stores the generated reference picture block in the reference picture memory 109.
　予測パラメータメモリ１０８は、予測パラメータ符号化部１１１が生成した予測パラメータを、符号化対象のピクチャ及びブロック毎に予め定めた位置に記憶する。 The prediction parameter memory 108 stores the prediction parameter generated by the prediction parameter encoding unit 111 at a predetermined position for each picture and block to be encoded.
　参照ピクチャメモリ１０９は、加算部１０６が生成した参照ピクチャブロックを、符号化対象のピクチャ及びブロック毎に予め定めた位置に記憶する。 The reference picture memory 109 stores the reference picture block generated by the adding unit 106 at a predetermined position for each picture and block to be encoded.
　符号化パラメータ決定部１１０は、符号化パラメータの複数のセットのうち、１つのセットを選択する。符号化パラメータとは、上述した予測パラメータやこの予測パラメータに関連して生成される符号化の対象となるパラメータである。予測画像生成部１０１は、これらの符号化パラメータのセットの各々を用いて予測ピクチャブロックpredSmaplesを生成する。 The encoding parameter determination unit 110 selects one set from among a plurality of sets of encoding parameters. The encoding parameter is a parameter to be encoded that is generated in association with the above-described prediction parameter or the prediction parameter. The predicted image generation unit 101 generates predicted picture blocks predSmaples using each of these sets of encoding parameters.
　符号化パラメータ決定部１１０は、複数のセットの各々について情報量の大きさと符号化誤差を示すコスト値を算出する。コスト値は、例えば、符号量と二乗誤差に係数λを乗じた値との和である。符号量は、量子化誤差と符号化パラメータをエントロピー符号化して得られる符号化ストリームＴｅの情報量である。二乗誤差は、減算部１０２において算出された残差信号の残差値の二乗値についての画素間の総和である。係数λは、予め設定されたゼロよりも大きい実数である。符号化パラメータ決定部１１０は、算出したコスト値が最小となる符号化パラメータのセットを選択する。これにより、エントロピー符号化部１０４は、選択した符号化パラメータのセットを符号化ストリームＴｅとして外部に出力し、選択されなかった符号化パラメータのセットを出力しない。 The encoding parameter determination unit 110 calculates a cost value indicating the amount of information and the encoding error for each of a plurality of sets. The cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient λ. The code amount is the information amount of the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameter. The square error is the sum between pixels regarding the square value of the residual value of the residual signal calculated by the subtracting unit 102. The coefficient λ is a real number larger than a preset zero. The encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the set of unselected encoding parameters.
　予測パラメータ符号化部１１１は、予測画像生成部１０１から入力されたパラメータに基づいて予測ピクチャを生成する際に用いる予測パラメータを導出し、導出した予測パラメータを符号化して符号化パラメータのセットを生成する。予測パラメータ符号化部１１１は、生成した符号化パラメータのセットをエントロピー符号化部１０４に出力する。 The prediction parameter encoding unit 111 derives a prediction parameter used when generating a prediction picture based on the parameter input from the prediction image generation unit 101, and encodes the derived prediction parameter to generate a set of encoding parameters. To do. The prediction parameter encoding unit 111 outputs the generated set of encoding parameters to the entropy encoding unit 104.
　予測パラメータ符号化部１１１は、生成した符号化パラメータのセットのうち符号化パラメータ決定部１１０が選択したものに対応する予測パラメータを予測パラメータメモリ１０８に記憶する。 The prediction parameter encoding unit 111 stores, in the prediction parameter memory 108, a prediction parameter corresponding to the set of the generated encoding parameters selected by the encoding parameter determination unit 110.
　予測パラメータ符号化部１１１は、予測画像生成部１０１から入力された予測モードpredModeがインター予測モードを示す場合、インター予測パラメータ符号化部１１２を動作させる。予測パラメータ符号化部１１１は、予測モードpredModeがイントラ予測モードを示す場合、イントラ予測パラメータ符号化部１１３を動作させる。 The prediction parameter encoding unit 111 operates the inter prediction parameter encoding unit 112 when the prediction mode predMode input from the prediction image generation unit 101 indicates the inter prediction mode. The prediction parameter encoding unit 111 operates the intra prediction parameter encoding unit 113 when the prediction mode predMode indicates the intra prediction mode.
　インター予測パラメータ符号化部１１２は、符号化パラメータ決定部１１０から入力された予測パラメータに基づいてインター予測パラメータを導出する。インター予測パラメータ符号化部１１２は、インター予測パラメータを導出する構成として、インター予測パラメータ復号部３０３（図５等、参照）がインター予測パラメータを導出する構成と同一の構成を含む。インター予測パラメータ符号化部１１２の構成については、後述する。 The inter prediction parameter encoding unit 112 derives an inter prediction parameter based on the prediction parameter input from the encoding parameter determination unit 110. The inter prediction parameter encoding unit 112 includes the same configuration as the configuration in which the inter prediction parameter decoding unit 303 (see FIG. 5 and the like) derives the inter prediction parameter as a configuration for deriving the inter prediction parameter. The configuration of the inter prediction parameter encoding unit 112 will be described later.
　イントラ予測パラメータ符号化部１１３は、符号化パラメータ決定部１１０から入力された予測モードpredModeが示すイントラ予測モードIntraPredModeをインター予測パラメータのセットとして定める。 The intra prediction parameter encoding unit 113 determines the intra prediction mode IntraPredMode indicated by the prediction mode predMode input from the encoding parameter determination unit 110 as a set of inter prediction parameters.
　　（インター予測パラメータ符号化部の構成）
　次に、インター予測パラメータ符号化部１１２の構成について説明する。インター予測パラメータ符号化部１１２は、インター予測パラメータ復号部３０３に対応する手段である。 (Configuration of inter prediction parameter encoding unit)
Next, the configuration of the inter prediction parameter encoding unit 112 will be described. The inter prediction parameter encoding unit 112 is means corresponding to the inter prediction parameter decoding unit 303.
　図２１は、本実施形態に係るインター予測パラメータ符号化部１１２の構成を示す概略図である。 FIG. 21 is a schematic diagram illustrating a configuration of the inter prediction parameter encoding unit 112 according to the present embodiment.
　インター予測パラメータ符号化部１１２は、マージモードパラメータ導出部１１２１、AMVP予測パラメータ導出部１１２２、減算部１１２３、及びインター予測パラメータ符号化制御部１１２６を含んで構成される。 The inter prediction parameter encoding unit 112 includes a merge mode parameter deriving unit 1121, an AMVP prediction parameter deriving unit 1122, a subtracting unit 1123, and an inter prediction parameter encoding control unit 1126.
　マージモードパラメータ導出部１１２１は、上述のマージモードパラメータ導出部３０３６（図７参照）と同様な構成を有する。 The merge mode parameter deriving unit 1121 has the same configuration as the merge mode parameter deriving unit 3036 (see FIG. 7).
　AMVP予測パラメータ導出部１１２２は、上述のAMVP予測パラメータ導出部３０３２（図７参照）と同様な構成を有する。 The AMVP prediction parameter derivation unit 1122 has the same configuration as the AMVP prediction parameter derivation unit 3032 (see FIG. 7).
　減算部１１２３は、符号化パラメータ決定部１１０から入力されたベクトルmvLXからAMVP予測パラメータ導出部１１２２から入力された予測ベクトルmvpLXを減算して差分ベクトルmvdLXを生成する。差分ベクトルmvdLXはインター予測パラメータ符号化制御部１１２６に出力する。 The subtraction unit 1123 subtracts the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the vector mvLX input from the coding parameter determination unit 110 to generate a difference vector mvdLX. The difference vector mvdLX is output to the inter prediction parameter encoding control unit 1126.
　インター予測パラメータ符号化制御部１１２６は、インター予測に関連する符号（シンタックス要素の復号をエントロピー符号化部１０４に指示し、符号化データに含まれる符号（シンタックス要素）を例えば、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLXを符号化する。 The inter prediction parameter coding control unit 1126 instructs the entropy coding unit 104 to decode a code related to inter prediction (the syntax element) includes, for example, a code (syntax element) included in the coded data. , Merge flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX are encoded.
　インター予測パラメータ符号化制御部１１２６は、追加予測フラグ符号化部１０３１１、マージインデックス符号化部１０３１２、ベクトル候補インデックス符号化部１０３１３、及び、分割モード符号化部、マージフラグ符号化部、インター予測フラグ符号化部、参照ピクチャインデックス符号化部、ベクトル差分符号化部を含んで構成される。分割モード符号化部、マージフラグ符号化部、マージインデックス符号化部、インター予測フラグ符号化部、参照ピクチャインデックス符号化部、ベクトル候補インデックス符号化部１０３１３、ベクトル差分符号化部は各々、分割モードpart_mode、マージフラグmerge_flag、マージインデックスmerge_idx、インター予測フラグinter_pred_idc、参照ピクチャインデックスrefIdxLX、予測ベクトルインデックスmvp_LX_idx、差分ベクトルmvdLXを符号化する。 The inter prediction parameter encoding control unit 1126 includes an additional prediction flag encoding unit 10311, a merge index encoding unit 10312, a vector candidate index encoding unit 10313, a split mode encoding unit, a merge flag encoding unit, and an inter prediction flag. An encoding unit, a reference picture index encoding unit, and a vector difference encoding unit are configured. The division mode encoding unit, the merge flag encoding unit, the merge index encoding unit, the inter prediction flag encoding unit, the reference picture index encoding unit, the vector candidate index encoding unit 10313, and the vector difference encoding unit are respectively divided modes. Part_mode, merge flag merge_flag, merge index merge_idx, inter prediction flag inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX are encoded.
　追加予測フラグ符号化部１０３１１は、追加予測が行われるか否かを示すために、照度補償フラグic_flag、残差予測重みインデックスiv_res_pred_weight_idxを符号化する。 The additional prediction flag encoding unit 10311 encodes the illumination compensation flag ic_flag and the residual prediction weight index iv_res_pred_weight_idx to indicate whether or not additional prediction is performed.
　インター予測パラメータ符号化制御部１１２６は、予測画像生成部１０１から入力された予測モードpredModeがマージモードを示す場合には、符号化パラメータ決定部１１０から入力されたマージインデックスmerge_idxをエントロピー符号化部１０４に出力し、符号化させる。 When the prediction mode predMode input from the prediction image generation unit 101 indicates the merge mode, the inter prediction parameter encoding control unit 1126 uses the merge index merge_idx input from the encoding parameter determination unit 110 as the entropy encoding unit 104. To be encoded.
　また、インター予測パラメータ符号化制御部１１２６は、予測画像生成部１０１から入力された予測モードpredModeがインター予測モードを示す場合には、次の処理を行う。 In addition, when the prediction mode predMode input from the prediction image generation unit 101 indicates the inter prediction mode, the inter prediction parameter encoding control unit 1126 performs the following process.
　インター予測パラメータ符号化制御部１１２６は、符号化パラメータ決定部１１０から入力された参照ピクチャインデックスrefIdxLX及びベクトルインデックスmvp_LX_idx、減算部１１２３から入力された差分ベクトルmvdLXを統合する。インター予測パラメータ符号化制御部１１２６は、統合した符号をエントロピー符号化部１０４に出力し、符号化させる。 The inter prediction parameter encoding control unit 1126 integrates the reference picture index refIdxLX and the vector index mvp_LX_idx input from the encoding parameter determination unit 110, and the difference vector mvdLX input from the subtraction unit 1123. The inter prediction parameter encoding control unit 1126 outputs the integrated code to the entropy encoding unit 104 to be encoded.
　　（第２の実施形態）
　第１の実施形態の空間マージ候補導出部３０３６１３１においては、隣接ブロックからVSPモードフラグVspModeFlagや変位ベクトルと変位ベクトルが示すレイヤのレイヤＩＤを継承しているが、視点合成予測が実施されない場合には、これら継承されたパラメータは参照されることがない。 (Second Embodiment)
In the spatial merge candidate derivation unit 3036131 of the first embodiment, the VSP mode flag VspModeFlag and the layer ID of the layer indicated by the displacement vector and the displacement vector are inherited from adjacent blocks, but when the viewpoint synthesis prediction is not performed. These inherited parameters are never referenced.
　これに対し、本実施形態における空間マージ候補導出部３０３６１３１Ａでは、隣接ブロックからのVSPモードフラグVspModeFlagと変位ベクトルと変位ベクトルが示すレイヤのレイヤＩＤの継承は、残差予測重みインデックスiv_res_pred_weight_idxが０、かつ、照度補償フラグic_flagが０の場合のみ実施するようにする。逆に、空間マージ候補導出部３０３６１３１Ａは、残差予測重みインデックスiv_res_pred_weight_idxが０以外、又は、照度補償フラグic_flagが０以外の場合には、隣接ブロックからの継承を実施しない。 On the other hand, in the spatial merge candidate derivation unit 3036131A in the present embodiment, the inheritance of the layer ID of the layer indicated by the VSP mode flag VspModeFlag, the displacement vector, and the displacement vector from the adjacent block is zero in the residual prediction weight index iv_res_pred_weight_idx, and This is performed only when the illumination compensation flag ic_flag is 0. Conversely, the spatial merge candidate derivation unit 3036131A does not perform inheritance from adjacent blocks when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0.
　これにより、残差予測又は照度補償予測が実施される場合、即ち、視点合成予測が実施されない場合に、パラメータの継承をスキップすることで計算量が削減されるという効果を奏する。 Thereby, when residual prediction or illuminance compensation prediction is performed, that is, when viewpoint synthesis prediction is not performed, there is an effect that the calculation amount is reduced by skipping parameter inheritance.
　以下、図面を参照しながら本発明の実施形態について説明する。 Hereinafter, embodiments of the present invention will be described with reference to the drawings.
　本実施形態における画像伝送システム１Ａは、図１の画像伝送システム１に対して、画像符号化装置１１に代わり画像符号化装置１１Ａを、画像復号装置３１に代わり画像復号装置３１Ａを有する点が異なる。 The image transmission system 1A in the present embodiment is different from the image transmission system 1 in FIG. 1 in that an image encoding device 11A is provided instead of the image encoding device 11, and an image decoding device 31A is provided instead of the image decoding device 31. .
　画像復号装置３１Ａは、図５の画像復号装置３１に対して、インター予測パラメータ復号部３０３に代わりインター予測パラメータ復号部３０３Ａを有する点のみが異なる。 The image decoding device 31A differs from the image decoding device 31 of FIG. 5 only in that it has an inter prediction parameter decoding unit 303A instead of the inter prediction parameter decoding unit 303.
　インター予測パラメータ復号部３０３Ａは、図６のインター予測パラメータ復号部３０３に対して、マージモードパラメータ導出部３０３６に代わりマージモードパラメータ導出部３０３６Ａを有する点のみが異なる。 The inter prediction parameter decoding unit 303A is different from the inter prediction parameter decoding unit 303 in FIG. 6 only in that it includes a merge mode parameter deriving unit 3036A instead of the merge mode parameter deriving unit 3036.
　図２２は、第２の実施形態のマージモードパラメータ導出部３０３６Ａの構成を示すブロック図である。マージモードパラメータ導出部３０３６Ａは、図７のマージモードパラメータ導出部３０３６に対して、空間マージ候補導出部３０３６１３１に代わり空間マージ候補導出部３０３６１３１Ａを有する点のみが異なる。 FIG. 22 is a block diagram illustrating a configuration of the merge mode parameter deriving unit 3036A according to the second embodiment. The merge mode parameter deriving unit 3036A differs from the merge mode parameter deriving unit 3036 of FIG. 7 only in that it has a spatial merge candidate deriving unit 3036131A instead of the spatial merge candidate deriving unit 3036131.
　空間マージ候補導出部３０３６１３１Ａは、入力として残差予測重みインデックスiv_res_pred_weight_idxと、照度補償フラグic_flagを受け取る点が空間マージ候補導出部３０３６１３１と異なる。 The spatial merge candidate derivation unit 3036131A is different from the spatial merge candidate derivation unit 3036131 in that it receives the residual prediction weight index iv_res_pred_weight_idx and the illumination compensation flag ic_flag as inputs.
　空間マージ候補導出部３０３６１３１Ａは、空間マージ候補導出部３０３６１３１と同様の方法で、予測パラメータメモリ３０７から読み出した予測パラメータを空間マージ候補として導出する。 The spatial merge candidate derivation unit 3036131A derives a prediction parameter read from the prediction parameter memory 307 as a spatial merge candidate by the same method as the spatial merge candidate derivation unit 3036131.
　さらに、空間マージ候補導出部３０３６１３１Ａは、残差予測重みインデックスiv_res_pred_weight_idxが０、かつ、照度補償フラグic_flagが０の場合のみ、空間マージ候補のVSPモードフラグVspModeFlagと変位ベクトルMvDispとレイヤＩＤRefViewIdxとして、隣接ブロックのVSPモードフラグVspModeFlagと変位ベクトルと変位ベクトルが示すレイヤのレイヤＩＤをそれぞれ継承する。逆に、空間マージ候補導出部３０３６１３１Ａは、残差予測重みインデックスiv_res_pred_weight_idxが０以外、又は、照度補償フラグic_flagが０以外の場合には、空間マージ候補のVSPモードフラグVspModeFlagと変位ベクトルMvDispとレイヤＩＤRefViewIdxの継承を行わず、空間マージ候補のVSPモードフラグVspModeFlagとして０を設定する。 Furthermore, the spatial merge candidate derivation unit 3036131A sets the adjacent block as the VSP mode flag VspModeFlag, the displacement vector MvDisp, and the layer IDRefViewIdx of the spatial merge candidate only when the residual prediction weight index iv_res_pred_weight_idx is 0 and the illumination compensation flag ic_flag is 0. The VSP mode flag VspModeFlag, the displacement vector, and the layer ID of the layer indicated by the displacement vector are inherited. Conversely, when the residual prediction weight index iv_res_pred_weight_idx is other than 0 or the illumination compensation flag ic_flag is other than 0, the spatial merge candidate derivation unit 3036131A performs the spatial merge candidate VSP mode flag VspModeFlag, the displacement vector MvDisp, and the layer ID RefViewIdx. Is not carried out, and 0 is set as the VSP mode flag VspModeFlag of the spatial merge candidate.
　画像符号化装置１１Ａは、図２０の画像符号化装置１１に対して、マージモードパラメータ導出部１１２１に代わりマージモードパラメータ導出部１１２１Ａを有する点のみが異なる。 The image encoding device 11A differs from the image encoding device 11 of FIG. 20 only in that it has a merge mode parameter deriving unit 1121A instead of the merge mode parameter deriving unit 1121.
　マージモードパラメータ導出部１１２１Ａは、マージモードパラメータ導出部３０３６に対して、空間マージ候補導出部３０３６１３１に代わり空間マージ候補導出部３０３６１３１Ａを有する点のみが異なる。 The merge mode parameter deriving unit 1121A differs from the merge mode parameter deriving unit 3036 only in that it has a spatial merge candidate deriving unit 3036131A instead of the spatial merge candidate deriving unit 3036131.
　　（第３の実施形態）
　以下、図面を参照しながら本発明の実施形態について説明する。 (Third embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
　本実施形態における画像伝送システム１Ｂは、図１の画像伝送システム１に対して、画像符号化装置１１に代わり画像符号化装置１１Ｂを、画像復号装置３１に代わり画像復号装置３１Ｂを有する点が異なる。 The image transmission system 1B in this embodiment is different from the image transmission system 1 in FIG. 1 in that an image encoding device 11B is provided instead of the image encoding device 11, and an image decoding device 31B is provided instead of the image decoding device 31. .
　画像復号装置３１Ｂは、図５の画像復号装置３１に対して、インター予測パラメータ復号部３０３に代わりインター予測パラメータ復号部３０３Ｂを有する点のみが異なる。 The image decoding device 31B is different from the image decoding device 31 in FIG. 5 only in that an inter prediction parameter decoding unit 303B is provided instead of the inter prediction parameter decoding unit 303.
　インター予測パラメータ復号部３０３Ｂは、図６のインター予測パラメータ復号部３０３に対して、インター予測パラメータ復号制御部３０３１に代わりインター予測パラメータ復号制御部３０３１Ｂを有する点のみが異なる。 The inter prediction parameter decoding unit 303B is different from the inter prediction parameter decoding unit 303 in FIG. 6 only in that it has an inter prediction parameter decoding control unit 3031B instead of the inter prediction parameter decoding control unit 3031.
　図２３は、第３の実施形態のインター予測パラメータ復号制御部３０３１Ｂの構成を示すブロック図である。インター予測パラメータ復号制御部３０３１Ｂは、図１０のインター予測パラメータ復号制御部３０３１に対して、残差予測重みインデックス復号部３０３１１の代わりに残差予測重みインデックス復号部３０３１１Ｂを、照度補償フラグ復号部３０３１２の代わりに照度補償フラグ復号部３０３１２Ｂを有する点が異なる。 FIG. 23 is a block diagram illustrating a configuration of the inter prediction parameter decoding control unit 3031B according to the third embodiment. In contrast to the inter prediction parameter decoding control unit 3031 of FIG. 10, the inter prediction parameter decoding control unit 3031B replaces the residual prediction weight index decoding unit 30311 with a residual prediction weight index decoding unit 30311B, and an illuminance compensation flag decoding unit 30312. The difference is that an illumination compensation flag decoding unit 30312B is provided instead of.
　残差予測重みインデックス復号部３０３１１Ｂは、残差予測重みインデックス復号部３０３１１と同様に、エントロピー復号部３０１を用いて、残差予測重みインデックスiv_res_pred_weight_idxを復号するが、残差予測重みインデックスiv_res_pred_weight_idxの出力先に照度補償フラグ復号部３０３１２Ｂが含まれている点が異なる。 Similar to the residual prediction weight index decoding unit 30311, the residual prediction weight index decoding unit 30311B uses the entropy decoding unit 301 to decode the residual prediction weight index iv_res_pred_weight_idx, but the output destination of the residual prediction weight index iv_res_pred_weight_idx Is different in that an illuminance compensation flag decoding unit 30312B is included.
　照度補償フラグ復号部３０３１２Ｂは、入力として残差予測重みインデックス復号部３０３１１Ｂから残差予測重みインデックスiv_res_pred_weight_idxを受け取る。照度補償フラグ復号部３０３１２Ｂは、残差予測重みインデックスiv_res_pred_weight_idxが０の場合にのみ、照度補償フラグic_flagを復号する。一方、照度補償フラグ復号部３０３１２Ｂは、残差予測重みインデックスiv_res_pred_weight_idxが０以外の場合には、照度補償フラグic_flagを復号せず、０を設定した照度補償フラグic_flagを導出する。照度補償フラグ復号部３０３１２Ｂは、照度補償フラグic_flagをマージモードパラメータ導出部３０３６とインター予測画像生成部３０９に出力する。 The illumination compensation flag decoding unit 30312B receives the residual prediction weight index iv_res_pred_weight_idx from the residual prediction weight index decoding unit 30311B as an input. The illuminance compensation flag decoding unit 30312B decodes the illuminance compensation flag ic_flag only when the residual prediction weight index iv_res_pred_weight_idx is 0. On the other hand, when the residual prediction weight index iv_res_pred_weight_idx is other than 0, the illuminance compensation flag decoding unit 30312B does not decode the illuminance compensation flag ic_flag but derives the illuminance compensation flag ic_flag set to 0. The illuminance compensation flag decoding unit 30312B outputs the illuminance compensation flag ic_flag to the merge mode parameter derivation unit 3036 and the inter predicted image generation unit 309.
　図２４は、照度補償フラグ復号部３０３１２Ｂが復号対象とする符号化データの構成を示すシンタックス表の一部である。Ｓ１００１，Ｓ１００２で示す箇所が、照度補償フラグic_flagに関連する部分であり、照度補償フラグic_flagが符号化データ内に存在するには、少なくとも残差予測重みインデックスiv_res_pred_weight_idxが０である必要があることを示している。 FIG. 24 is a part of a syntax table showing a configuration of encoded data to be decoded by the illumination compensation flag decoding unit 30312B. The portions indicated by S1001 and S1002 are portions related to the illuminance compensation flag ic_flag, and at least the residual prediction weight index iv_res_pred_weight_idx needs to be 0 for the illuminance compensation flag ic_flag to exist in the encoded data. Show.
　画像符号化装置１１Ｂは、図２０の画像符号化装置１１に対して、符号化パラメータ決定部１１０に代わり符号化パラメータ決定部１１０Ｂを、インター予測パラメータ符号化部１１２に代わりインター予測パラメータ符号化部１１２Ｂを有する点のみが異なる。 The image encoding device 11B differs from the image encoding device 11 of FIG. 20 in that an encoding parameter determination unit 110B replaces the encoding parameter determination unit 110, and an inter prediction parameter encoding unit replaces the inter prediction parameter encoding unit 112. The only difference is that it has 112B.
　符号化パラメータ決定部１１０Ｂは、残差予測重みインデックスiv_res_pred_weight_idxに０以外を設定した場合には、照度補償フラグic_flagを必ず０に設定する点が符号化パラメータ決定部１１０と異なる。 The encoding parameter determination unit 110B is different from the encoding parameter determination unit 110 in that the illumination compensation flag ic_flag is always set to 0 when the residual prediction weight index iv_res_pred_weight_idx is set to other than 0.
　インター予測パラメータ符号化部１１２Ｂは、図２１のインター予測パラメータ符号化部１１２に対して、インター予測パラメータ符号化制御部１１２６に代わってインター予測パラメータ符号化制御１１２６Ｂを有する点のみが異なる。 The inter prediction parameter encoding unit 112B is different from the inter prediction parameter encoding unit 112 of FIG. 21 only in that it has an inter prediction parameter encoding control 1126B instead of the inter prediction parameter encoding control unit 1126.
　インター予測パラメータ符号化制御部１１２６Ｂは、残差予測重みインデックスiv_res_pred_weight_idxが０以外の場合には、照度補償フラグic_flagをエントロピー符号化部に出力しない点が、インター予測パラメータ符号化制御部１１２６とは異なる。 The inter prediction parameter coding control unit 1126B is different from the inter prediction parameter coding control unit 1126 in that the illuminance compensation flag ic_flag is not output to the entropy coding unit when the residual prediction weight index iv_res_pred_weight_idx is other than 0. .
　上記構成によれば、動き補償をする際に有効である残差予測が実施される場合に、視差補償をする際に有効である輝度補償予測を無効にすることで、符号化効率への影響を止めながら、残差予測が実施される場合に、輝度補償フラグの符号化を省略することが可能となり、符号化効率が向上するという効果を奏する。 According to the above configuration, when residual prediction that is effective when performing motion compensation is performed, the luminance compensation prediction that is effective when performing disparity compensation is disabled, thereby affecting the coding efficiency. When the residual prediction is performed while stopping the above, it is possible to omit the encoding of the luminance compensation flag, and the encoding efficiency is improved.
　なお、上述した実施形態における画像符号化装置１１、１１Ａ、１１Ｂ、画像復号装置３１、３１Ａ、３１Ｂの一部、例えば、エントロピー復号部３０１、予測パラメータ復号部３０２、予測画像生成部１０１、ＤＣＴ・量子化部１０３、エントロピー符号化部１０４、逆量子化・逆ＤＣＴ部１０５、符号化パラメータ決定部１１０、予測パラメータ符号化部１１１、エントロピー復号部３０１、予測パラメータ復号部３０２、予測画像生成部３０８、逆量子化・逆ＤＣＴ部３１１をコンピュータで実現するようにしても良い。その場合、この制御機能を実現するためのプログラムをコンピュータ読み取り可能な記録媒体に記録して、この記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行することによって実現しても良い。なお、ここでいう「コンピュータシステム」とは、画像符号化装置１１－１１Ｂ、画像復号装置３１－３１Ｂのいずれかに内蔵されたコンピュータシステムであって、ＯＳや周辺機器等のハードウェアを含むものとする。また、「コンピュータ読み取り可能な記録媒体」とは、フレキシブルディスク、光磁気ディスク、ＲＯＭ、ＣＤ－ＲＯＭ等の可搬媒体、コンピュータシステムに内蔵されるハードディスク等の記憶装置のことをいう。さらに「コンピュータ読み取り可能な記録媒体」とは、インターネット等のネットワークや電話回線等の通信回線を介してプログラムを送信する場合の通信線のように、短時間、動的にプログラムを保持するもの、その場合のサーバやクライアントとなるコンピュータシステム内部の揮発性メモリのように、一定時間プログラムを保持しているものも含んでも良い。また上記プログラムは、前述した機能の一部を実現するためのものであっても良く、さらに前述した機能をコンピュータシステムにすでに記録されているプログラムとの組み合わせで実現できるものであっても良い。 Note that a part of the image encoding devices 11, 11A, 11B and image decoding devices 31, 31A, 31B in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the predicted image generation unit 101, the DCT / Quantization unit 103, entropy encoding unit 104, inverse quantization / inverse DCT unit 105, encoding parameter determination unit 110, prediction parameter encoding unit 111, entropy decoding unit 301, prediction parameter decoding unit 302, and prediction image generation unit 308 The inverse quantization / inverse DCT unit 311 may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The “computer system” here is a computer system built in either the image encoding device 11-11B or the image decoding device 31-31B, and includes an OS and hardware such as peripheral devices. . The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In such a case, a volatile memory inside a computer system serving as a server or a client may be included and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
　また、上述した実施形態における画像符号化装置１１、１１Ａ、１１Ｂ、画像復号装置３１、３１Ａ、３１Ｂの一部、または全部を、ＬＳＩ（Ｌａｒｇｅ　Ｓｃａｌｅ　Ｉｎｔｅｇｒａｔｉｏｎ）等の集積回路として実現しても良い。画像符号化装置１１、１１Ａ、１１Ｂ、画像復号装置３１、３１Ａ、３１Ｂの各機能ブロックは個別にプロセッサ化しても良いし、一部、または全部を集積してプロセッサ化しても良い。また、集積回路化の手法はＬＳＩに限らず専用回路、または汎用プロセッサで実現しても良い。また、半導体技術の進歩によりＬＳＩに代替する集積回路化の技術が出現した場合、当該技術による集積回路を用いても良い。 Further, part or all of the image encoding devices 11, 11A, 11B and the image decoding devices 31, 31A, 31B in the above-described embodiments may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image encoding devices 11, 11A, 11B and the image decoding devices 31, 31A, 31B may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology may be used.
　以上、図面を参照してこの発明の一実施形態について詳しく説明してきたが、具体的な構成は上述のものに限られることはなく、この発明の要旨を逸脱しない範囲内において様々な設計変更等をすることが可能である。 As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to
　本発明は上述した各実施形態に限定されるものではなく、請求項に示した範囲で種々の変更が可能であり、異なる実施形態にそれぞれ開示された技術的手段を適宜組み合わせて得られる実施形態についても本発明の技術的範囲に含まれる。さらに、各実施形態にそれぞれ開示された技術的手段を組み合わせることにより、新しい技術的特徴を形成することができる。 The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.
　〔付記事項〕
　（１）この発明は上述した課題を解決するためになされたもので、本発明の一態様は、対象ブロックの予測画像を生成し復号する画像復号装置であって、残差予測を用いて予測画像を生成する残差予測部と、視点合成予測を用いて予測画像を生成する視点合成予測部と、残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、予測パラメータを導出するマージモードパラメータ導出部とを備え、前記マージモードパラメータ導出部は視点合成予測に用いる予測パラメータであるVSPマージ候補を導出する視点合成予測マージ候補導出部を備え、前記視点合成予測マージ候補導出部は、前記残差予測重みインデックスが０の場合のみ、前記VSPマージ候補を導出する。 [Additional Notes]
(1) The present invention has been made to solve the above-described problems, and one aspect of the present invention is an image decoding apparatus that generates and decodes a predicted image of a target block, and performs prediction using residual prediction. A residual prediction unit that generates an image, a viewpoint synthesis prediction unit that generates a predicted image using viewpoint synthesis prediction, and a residual prediction weight index that decodes a residual prediction weight index indicating whether or not to use residual prediction A decoding unit, and a merge mode parameter deriving unit for deriving a prediction parameter, the merge mode parameter deriving unit comprising a view synthesis prediction merge candidate deriving unit for deriving a VSP merge candidate that is a prediction parameter used for view synthesis prediction, The view synthesis prediction merge candidate derivation unit derives the VSP merge candidate only when the residual prediction weight index is 0.
　（２）また、本発明の他の態様は、対象ブロックの予測画像を生成し復号する画像復号装置であって、予測画像に照度補償を適用する照度補償部と、視点合成予測を用いて予測画像を生成する視点合成予測部と、照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部と、予測画像を生成するための予測パラメータを導出するマージモードパラメータ導出部とを備え、前記マージモードパラメータ導出部は視点合成予測に用いる予測パラメータであるVSPマージ候補を導出する視点合成予測マージ候補導出部を備え、前記視点合成予測マージ候補導出部は、前記照度補償フラグが０である場合のみ、前記VSPマージ候補を導出する。 (2) Moreover, the other aspect of this invention is an image decoding apparatus which produces | generates and decodes the estimated image of an object block, Comprising: The illumination intensity compensation part which applies illumination intensity compensation to a estimated image, and prediction using viewpoint synthetic | combination prediction Viewpoint synthesis prediction unit for generating an image, illuminance compensation flag decoding unit for decoding an illuminance compensation flag indicating whether to apply illuminance compensation, and a merge mode parameter deriving unit for deriving a prediction parameter for generating a predicted image The merge mode parameter derivation unit includes a view synthesis prediction merge candidate derivation unit that derives a VSP merge candidate that is a prediction parameter used for view synthesis prediction, and the view synthesis prediction merge candidate derivation unit includes the illuminance compensation flag. Only when V is 0, the VSP merge candidate is derived.
　（３）また、本発明の他の態様は、（１）に記載の画像復号装置であって、前記マージモードパラメータ導出部は、さらに、空間候補を導出し、なお且つ、空間候補を導出するために参照した隣接ブロックの復号に視点合成予測を用いていた場合には、該隣接ブロックの視点合成予測のための予測パラメータを継承する空間候補導出部を備え、前記空間候補導出部は、前記残差予測重みインデックスが０である場合に、該隣接ブロックの視点合成予測のための予測パラメータを継承する。 (3) According to another aspect of the present invention, there is provided the image decoding apparatus according to (1), wherein the merge mode parameter deriving unit further derives a space candidate and further derives a space candidate. When the viewpoint synthesis prediction is used for decoding the adjacent block referred to for this purpose, a spatial candidate derivation unit that inherits prediction parameters for the viewpoint synthesis prediction of the neighboring block is provided, and the spatial candidate derivation unit includes When the residual prediction weight index is 0, the prediction parameter for the viewpoint synthesis prediction of the adjacent block is inherited.
　（４）また、本発明の他の態様は、（２）に記載の画像復号装置であって、前記マージモードパラメータ導出部は、さらに、空間候補を導出し、なお且つ、空間候補を導出するために参照した隣接ブロックの復号に視点合成予測を用いていた場合には、該隣接ブロックの視点合成予測のための予測パラメータを継承する空間候補導出部を備え、前記空間候補導出部は、前記照度補償フラグが０である場合に、該隣接ブロックの視点合成予測のための予測パラメータを継承する。 (4) Further, another aspect of the present invention is the image decoding device according to (2), wherein the merge mode parameter deriving unit further derives a space candidate and further derives a space candidate. When the viewpoint synthesis prediction is used for decoding the adjacent block referred to for this purpose, a spatial candidate derivation unit that inherits prediction parameters for the viewpoint synthesis prediction of the neighboring block is provided, and the spatial candidate derivation unit includes When the illumination compensation flag is 0, the prediction parameter for the viewpoint synthesis prediction of the adjacent block is inherited.
　（５）また、本発明の他の態様は、（３）又は（４）に記載の画像復号装置であって、該隣接ブロックの視点合成予測のための予測パラメータは、該隣接ブロックの視点合成予測を行ったか否かを示す視点合成予測フラグである。 (5) According to another aspect of the present invention, there is provided the image decoding apparatus according to (3) or (4), wherein the prediction parameter for the viewpoint synthesis prediction of the adjacent block is the viewpoint synthesis of the adjacent block. This is a viewpoint synthesis prediction flag indicating whether or not prediction has been performed.
　（６）また、本発明の他の態様は、（３）又は（４）に記載の画像復号装置であって、該隣接ブロックの視点合成予測のための予測パラメータは、該隣接ブロックのベクトルとレイヤを示す情報であるビューインデックスである。 (6) According to another aspect of the present invention, there is provided the image decoding apparatus according to (3) or (4), wherein a prediction parameter for viewpoint synthesis prediction of the adjacent block includes a vector of the adjacent block. A view index that is information indicating a layer.
　（７）また、本発明の他の態様は、（１）に記載の画像復号装置であって、前記視点合成予測マージ候補導出部は、前記残差予測重みインデックスが０以外である場合には、前記VSPマージ候補の代わりに代替マージ候補を導出する。 (7) According to another aspect of the present invention, there is provided the image decoding device according to (1), wherein the viewpoint synthesis prediction merge candidate derivation unit is configured such that the residual prediction weight index is other than 0. The alternative merge candidate is derived instead of the VSP merge candidate.
　（８）また、本発明の他の態様は、（２）に記載の画像復号装置であって、前記視点合成予測マージ候補導出部は、前記照度補償フラグが０以外である場合には、前記VSPマージ候補の代わりに代替マージ候補を導出する。 (8) According to another aspect of the present invention, there is provided the image decoding device according to (2), wherein the viewpoint synthesis prediction merge candidate derivation unit includes the illuminance compensation flag other than 0, An alternative merge candidate is derived instead of the VSP merge candidate.
　（９）また、本発明の他の態様は、（７）又は（８）に記載の画像復号装置であって、前記視点合成予測マージ候補導出部は、前記代替マージ候補として、既に導出済みのマージ候補のベクトルのＸ，Ｙ成分の値を変更したマージ候補を導出する。 (9) Further, another aspect of the present invention is the image decoding device according to (7) or (8), in which the viewpoint synthesis prediction merge candidate derivation unit has already been derived as the alternative merge candidate A merge candidate is derived by changing the values of the X and Y components of the merge candidate vector.
　（１０）また、本発明の他の態様は、対象ブロックの予測画像を生成し復号する画像復号装置であって、視点合成予測を用いて予測画像を生成する視点合成予測部と、予測パラメータを導出するマージモードパラメータ導出部とを備え、前記マージモードパラメータ導出部は、視点合成予測に用いる予測パラメータであるVSPマージ候補を導出する視点合成予測マージ候補導出部を備え、前記視点合成予測マージ候補導出部は、前記対象ブロックのサイズが所定のサイズ以下の場合には、VSPマージ候補を導出しない。 (10) Another aspect of the present invention is an image decoding apparatus that generates and decodes a predicted image of a target block, and includes a viewpoint synthesis prediction unit that generates a predicted image using viewpoint synthesis prediction, and a prediction parameter. A merge mode parameter deriving unit for deriving, the merge mode parameter deriving unit comprising a view synthesis prediction merge candidate deriving unit for deriving a VSP merge candidate that is a prediction parameter used for view synthesis prediction, and the view synthesis prediction merge candidate The deriving unit does not derive a VSP merge candidate when the size of the target block is equal to or smaller than a predetermined size.
　（１１）また、本発明の他の態様は、対象ブロックの予測画像を生成し復号する画像復号装置であって、残差予測を用いて予測画像を生成する残差予測部と、予測画像に照度補償を適用する照度補償部と、残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部とを備え、前記照度補償フラグ復号部は、前記残差予測重みインデックスが０の場合にのみ、照度補償フラグを復号する。 (11) According to another aspect of the present invention, there is provided an image decoding apparatus that generates and decodes a predicted image of a target block, a residual prediction unit that generates a predicted image using residual prediction, and a prediction image An illuminance compensation unit that applies illuminance compensation, a residual prediction weight index decoding unit that decodes a residual prediction weight index that indicates whether to use residual prediction, and an illuminance compensation flag that indicates whether to apply illuminance compensation The illuminance compensation flag decoding unit decodes the illuminance compensation flag only when the residual prediction weight index is 0.
　本発明は、画像データが符号化された符号化データを復号する画像復号装置、および、画像データが符号化された符号化データを生成する画像符号化装置に好適に適用することができる。また、画像符号化装置によって生成され、画像復号装置によって参照される符号化データのデータ構造に好適に適用することができる。 The present invention can be suitably applied to an image decoding apparatus that decodes encoded data obtained by encoding image data and an image encoding apparatus that generates encoded data obtained by encoding image data. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.
１、１Ａ、１Ｂ…画像伝送システム
１１、１１Ａ、１１Ｂ…画像符号化装置
１０１…予測画像生成部
１０２…減算部
１０３…ＤＣＴ・量子化部
１０３１１…追加予測フラグ符号化部
１０３１２…マージインデックス符号化部
１０３１３…ベクトル候補インデックス符号化部
１０４…エントロピー符号化部
１０５…逆量子化・逆ＤＣＴ部
１０６…加算部
１０８…予測パラメータメモリ（フレームメモリ）
１０９…参照ピクチャメモリ（フレームメモリ）
１１０、１１０Ｂ…符号化パラメータ決定部
１１１…予測パラメータ符号化部
１１２、１１２Ｂ…インター予測パラメータ符号化部
１１２１、１１２１Ａ…マージモードパラメータ導出部
１１２２…AMVP予測パラメータ導出部
１１２３…減算部
１１２６、１１２６Ｂ…インター予測パラメータ符号化制御
１１３…イントラ予測パラメータ符号化部
２１…ネットワーク
３１、３１Ａ、３１Ｂ…画像復号装置
３０１…エントロピー復号部
３０２…予測パラメータ復号部
３０３、３０３Ａ、３０３Ｂ…インター予測パラメータ復号部
３０３１、３０３１Ｂ…インター予測パラメータ復号制御部
３０３１１、３０３１１Ｂ…残差予測重みインデックス復号部
３０３１１１…リファレンスレイヤ判定部
３０３１２、３０３１２Ｂ…照度補償フラグ復号部
３０３１３…ベクトル候補インデックス復号部
３０３２…AMVP予測パラメータ導出部
３０３３…ベクトル候補導出部
３０３４…予測ベクトル選択部
３０３５…加算部
３０３６、３０３６Ａ…マージモードパラメータ導出部
３０３６１…マージ候補導出部
３０３６１１…マージ候補格納部
３０３６１２…拡張マージ候補導出部
３０３６１２１…レイヤ間マージ候補導出部
３０３６１２２…変位ベクトル取得部
３０３６１２３…変位マージ候補導出部
３０３６１２４…ＶＳＰマージ候補導出部
３０３６１３…基本マージ候補導出部
３０３６１３１、３０３６１３１Ａ…空間マージ候補導出部
３０３６１３２…時間マージ候補導出部
３０３６１３３…結合マージ候補導出部
３０３６１３４…ゼロマージ候補導出部
３０３６２…マージ候補選択部
３０４…イントラ予測パラメータ復号部
３０６…参照ピクチャメモリ（フレームメモリ）
３０７…予測パラメータメモリ（フレームメモリ）
３０８…予測画像生成部
３０９…インター予測画像生成部
３０９１…変位補償部
３０９２…残差予測部
３０９２２…参照画像取得部
３０９２３…残差合成部
３０９３…照度補償部
３０９４…視点合成予測部
３０９４１…視差配列導出部
３０９４２…参照画像取得部
３０９６…インター予測画像生成制御部
３１０…イントラ予測画像生成部
３１１…逆量子化・逆ＤＣＴ部
３１２…加算部
３１３…残差格納部
３５１…デプスＤＶ導出部
４１…画像表示装置 DESCRIPTION OF SYMBOLS 1, 1A, 1B ... Image transmission system 11, 11A, 11B ... Image encoding apparatus 101 ... Prediction image generation part 102 ... Subtraction part 103 ... DCT / quantization part 10311 ... Additional prediction flag encoding part 10312 ... Merge index encoding Unit 10313 ... vector candidate index encoding unit 104 ... entropy encoding unit 105 ... inverse quantization / inverse DCT unit 106 ... addition unit 108 ... prediction parameter memory (frame memory)
109 ... Reference picture memory (frame memory)
110, 110B ... Coding parameter determination unit 111 ... Prediction parameter coding unit 112, 112B ... Inter prediction parameter coding unit 1121, 1121A ... Merge mode parameter derivation unit 1122 ... AMVP prediction parameter derivation unit 1123 ... Subtraction unit 1126, 1126B ... Inter prediction parameter encoding control 113 ... Intra prediction parameter encoding unit 21 ... Networks 31, 31A, 31B ... Image decoding device 301 ... Entropy decoding unit 302 ... Prediction parameter decoding units 303, 303A, 303B ... Inter prediction parameter decoding unit 3031 3031B: Inter prediction parameter decoding control unit 30311, 30311B ... Residual prediction weight index decoding unit 303111 ... Reference layer determination unit 30312, 30312B ... Illuminance compensation flag decoding unit 3 0313 ... Vector candidate index decoding unit 3032 ... AMVP prediction parameter derivation unit 3033 ... Vector candidate derivation unit 3034 ... Prediction vector selection unit 3035 ... Addition unit 3036, 3036A ... Merge mode parameter derivation unit 30361 ... Merge candidate derivation unit 303611 ... Merge candidate storage Unit 303612 ... extended merge candidate derivation unit 3036121 ... inter-layer merge candidate derivation unit 3036122 ... displacement vector acquisition unit 3036123 ... displacement merge candidate derivation unit 3036124 ... VSP merge candidate derivation unit 303613 ... basic merge candidate derivation units 3036131 and 3036131A ... spatial merge candidates Deriving unit 3036132 ... Time merge candidate deriving unit 3036133 ... Join merge candidate deriving unit 3036134 ... Zero merge candidate deriving unit 30362 ... Merge candidate selecting unit 304 ... Intra prediction parameter decoding unit 306... Reference picture memory (frame memory)
307 ... Prediction parameter memory (frame memory)
308 ... Prediction image generation unit 309 ... Inter prediction image generation unit 3091 ... Displacement compensation unit 3092 ... Residual prediction unit 30922 ... Reference image acquisition unit 30923 ... Residual synthesis unit 3093 ... Illuminance compensation unit 3094 ... Viewpoint synthesis prediction unit 30941 ... Parallax Array derivation unit 30942 ... reference image acquisition unit 3096 ... inter prediction image generation control unit 310 ... intra prediction image generation unit 311 ... inverse quantization / inverse DCT unit 312 ... addition unit 313 ... residual storage unit 351 ... depth DV derivation unit 41 ... Image display device

Claims (4)

  1. 　残差予測を用いて予測画像を生成する残差予測部と、
    　前記残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、
    　マージ候補リストを導出するマージモードパラメータ導出部とを備え、
    　前記マージモードパラメータ導出部は、
    　視点合成予測に用いるVSPマージ候補を導出し、前記残差予測重みインデックスが０の場合に、前記VSPマージ候補をマージ候補リストに追加することを特徴とする画像復号装置。     A residual prediction unit that generates a prediction image using residual prediction;
    A residual prediction weight index decoding unit that decodes a residual prediction weight index indicating whether to use the residual prediction;
    A merge mode parameter deriving unit for deriving a merge candidate list,
    The merge mode parameter derivation unit includes:
    An image decoding apparatus, wherein a VSP merge candidate used for view synthesis prediction is derived, and the VSP merge candidate is added to a merge candidate list when the residual prediction weight index is 0.
  2. 　予測画像に照度補償を適用する照度補償部と、
    　照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部と、
    　マージ候補リストを導出するマージモードパラメータ導出部とを備え、
    　前記マージモードパラメータ導出部は、
    　視点合成予測に用いる予測パラメータであるVSPマージ候補を導出し、前記照度補償フラグが０である場合に、前記VSPマージ候補をVSPマージ候補をマージ候補リストに追加することを特徴とする画像復号装置。     An illumination compensation unit that applies illumination compensation to the predicted image;
    An illuminance compensation flag decoding unit for decoding an illuminance compensation flag indicating whether or not to apply illuminance compensation;
    A merge mode parameter deriving unit for deriving a merge candidate list,
    The merge mode parameter derivation unit includes:
    An image decoding apparatus characterized by deriving a VSP merge candidate that is a prediction parameter used for viewpoint synthesis prediction and adding the VSP merge candidate to a merge candidate list when the illumination compensation flag is 0 .
  3. 　残差予測を用いて予測画像を生成する残差予測部と、
    　残差予測を用いるか否かを示す残差予測重みインデックスを復号する残差予測重みインデックス復号部と、
    　マージ候補リストを導出するマージモードパラメータ導出部と、
    　VSPモードフラグが１の場合に視点合成予測を行う視点合成予測部を備え、
    　前記マージモードパラメータ導出部は、前記残差予測重みインデックスが０以外の場合に、VSPモードフラグとして０を設定することを特徴とする画像復号装置。     A residual prediction unit that generates a prediction image using residual prediction;
    A residual prediction weight index decoding unit that decodes a residual prediction weight index indicating whether to use residual prediction;
    A merge mode parameter deriving unit for deriving a merge candidate list;
    A view synthesis prediction unit that performs view synthesis prediction when the VSP mode flag is 1,
    The merge mode parameter deriving unit sets 0 as a VSP mode flag when the residual prediction weight index is other than 0.
  4. 　予測画像に照度補償を適用する照度補償部と、
    　照度補償を適用するか否かを示す照度補償フラグを復号する照度補償フラグ復号部と、
    　予測パラメータを導出するマージモードパラメータ導出部と、
    　VSPモードフラグが１の場合に、視点合成予測を行う視点合成予測部を備え、
    前記マージモードパラメータ導出部は、前記照度補償フラグが０以外の場合に、VSPモードフラグとして０を設定することを特徴とする画像復号装置。     An illumination compensation unit that applies illumination compensation to the predicted image;
    An illuminance compensation flag decoding unit for decoding an illuminance compensation flag indicating whether or not to apply illuminance compensation;
    A merge mode parameter deriving unit for deriving a prediction parameter;
    A view synthesis prediction unit that performs view synthesis prediction when the VSP mode flag is 1,
    The merge mode parameter deriving unit sets 0 as a VSP mode flag when the illuminance compensation flag is other than 0.
PCT/JP2014/076977 2013-10-15 2014-10-08 Image decoding device and image coding device WO2015056620A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013-214423 2013-10-15
JP2013214423A JP2016213519A (en) 2013-10-15 2013-10-15 Image decoder and image encoder

Publications (1)

Publication Number Publication Date
WO2015056620A1 true WO2015056620A1 (en) 2015-04-23

Family

ID=52828066

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/076977 WO2015056620A1 (en) 2013-10-15 2014-10-08 Image decoding device and image coding device

Country Status (2)

Country Link
JP (1) JP2016213519A (en)
WO (1) WO2015056620A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016537869A (en) * 2013-10-18 2016-12-01 エルジー エレクトロニクス インコーポレイティド View synthesis prediction method and merge candidate list construction method using the same in multi-view video coding
CN110572666A (en) * 2018-06-05 2019-12-13 北京字节跳动网络技术有限公司 Interaction of asymmetrically weighted Merge with other coding tools
CN113170186A (en) * 2018-12-07 2021-07-23 夏普株式会社 Moving image decoding device

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019021947A1 (en) * 2017-07-24 2019-01-31 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Encoding device, decoding device, encoding method, and decoding method
US11388398B2 (en) 2018-01-11 2022-07-12 Qualcomm Incorporated Video coding using local illumination compensation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009532934A (en) * 2006-03-30 2009-09-10 エルジー エレクトロニクス インコーポレイティド Video signal decoding / encoding method and apparatus
JP2010021844A (en) * 2008-07-11 2010-01-28 Nippon Telegr & Teleph Corp <Ntt> Multi-viewpoint image encoding method, decoding method, encoding device, decoding device, encoding program, decoding program and computer-readable recording medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009532934A (en) * 2006-03-30 2009-09-10 エルジー エレクトロニクス インコーポレイティド Video signal decoding / encoding method and apparatus
JP2010021844A (en) * 2008-07-11 2010-01-28 Nippon Telegr & Teleph Corp <Ntt> Multi-viewpoint image encoding method, decoding method, encoding device, decoding device, encoding program, decoding program and computer-readable recording medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GERHARD TECH ET AL.: "3D-HEVC Draft Text 1", JOINT COLLABORATIVE TEAM ON 3D VIDEO CODING EXTENSION DEVELOPMENT OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 JCT3V-E1001-V3, ITU-T, pages 1 - 89 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016537869A (en) * 2013-10-18 2016-12-01 エルジー エレクトロニクス インコーポレイティド View synthesis prediction method and merge candidate list construction method using the same in multi-view video coding
CN110572666A (en) * 2018-06-05 2019-12-13 北京字节跳动网络技术有限公司 Interaction of asymmetrically weighted Merge with other coding tools
US11039165B2 (en) 2018-06-05 2021-06-15 Beijing Bytedance Network Technology Co., Ltd. Interaction of asymmetric weighted merges and other coding tools
CN110572666B (en) * 2018-06-05 2021-11-02 北京字节跳动网络技术有限公司 Interaction method, device and storage medium of asymmetric weighted Merge and other coding tools
US11706441B2 (en) 2018-06-05 2023-07-18 Beijing Bytedance Network Technology Co., Ltd Interaction of asymmetric weighted merges and other coding tools
CN113170186A (en) * 2018-12-07 2021-07-23 夏普株式会社 Moving image decoding device

Also Published As

Publication number Publication date
JP2016213519A (en) 2016-12-15

Similar Documents

Publication Publication Date Title
JP6469588B2 (en) Residual prediction device, image decoding device, image coding device, residual prediction method, image decoding method, and image coding method
JP6441236B2 (en) Image decoding apparatus and image encoding apparatus
WO2016125685A1 (en) Image decoding device, image encoding device, and prediction vector deriving device
WO2015056719A1 (en) Image decoding device and image coding device
JP6360053B2 (en) Illuminance compensation device, image decoding device, image coding device
JP6535673B2 (en) Video coding techniques using asymmetric motion segmentation
JP6225241B2 (en) Image decoding apparatus, image decoding method, image coding apparatus and image coding method
WO2015194669A1 (en) Image decoding apparatus, image coding apparatus, and prediction image generation device
WO2014103529A1 (en) Image decoding device and data structure
WO2015056620A1 (en) Image decoding device and image coding device
JP6473078B2 (en) Image decoding device
JP6118199B2 (en) Image decoding apparatus, image encoding apparatus, image decoding method, image encoding method, and computer-readable recording medium
WO2014103600A1 (en) Encoded data structure and image decoding device
WO2015141696A1 (en) Image decoding device, image encoding device, and prediction device
JP2016066864A (en) Image decoding device, image encoding device, and merge mode parameter derivation device
JP2017135432A (en) Viewpoint synthesis prediction apparatus, image decoding apparatus, and image encoding apparatus
WO2016056587A1 (en) Displacement arrangement derivation device, displacement vector derivation device, default reference view index derivation device, and depth lookup table derivation device
WO2015190510A1 (en) Viewpoint synthesis prediction device, image decoding device, and image coding device
JP6401707B2 (en) Image decoding apparatus, image decoding method, and recording medium
JP2015080053A (en) Image decoder and image encoder
JP2014204327A (en) Image decoding apparatus and image encoding apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14853353

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: JP

122 Ep: pct application non-entry in european phase

Ref document number: 14853353

Country of ref document: EP

Kind code of ref document: A1