TW201826793A

TW201826793A - Image coding apparatus, image decoding apparatus, and method

Info

Publication number: TW201826793A
Application number: TW106140328A
Authority: TW
Inventors: 安倍清史; 西孝啓; 遠間正真; 橋本隆
Original assignee: 美商松下電器（美國）知識產權公司
Priority date: 2016-11-22
Filing date: 2017-11-21
Publication date: 2018-07-16
Also published as: WO2018097079A1

Abstract

An encoding device is equipped with circuitry and memory. The circuitry uses the memory to: derive, from a motion vector used in the motion compensation of a processed block, a plurality of candidate motion vectors for a block to be processed included in a picture to be processed; select, without using the image region of the block to be processed, a final motion vector from among the plurality of candidate motion vectors by referring to two processed reference pictures that are both positioned either before or after the picture to be processed in terms of display time; and perform motion compensation for the block to be processed using the final motion vector.

Description

編碼裝置、解碼裝置、編碼方法及解碼方法Encoding device, decoding device, encoding method and decoding method

發明領域本揭示是有關於一種編碼裝置、解碼裝置、編碼方法及解碼方法。FIELD OF THE INVENTION The present disclosure relates to an encoding device, a decoding device, an encoding method, and a decoding method.

發明背景在作為最新的動態圖像編碼標準規格之HEVC(High Efficiency Video Coding(高效率視訊編碼))規格中，為了提升編碼效率而進行了各種的檢討(參照例如非專利文獻1)。此方式是以H.26x表示之ITU-T(國際電信聯盟電信標準化部門)規格、及以MPEG-x表示之ISO/IEC規格，並且已作為以H.264/AVC、或MPEG-4 AVC表示之規格的後續之影像編碼規格來檢討。先前技術文獻BACKGROUND OF THE INVENTION In the High Efficiency Video Coding (HEVC) standard, which is the latest standard for moving picture coding, various reviews have been conducted to improve coding efficiency (see, for example, Non-Patent Document 1). This method is based on the ITU-T (International Telecommunication Union Telecommunication Standardization Sector) specifications represented by H.26x and the ISO / IEC specifications represented by MPEG-x, and has been expressed as H.264 / AVC or MPEG-4 AVC The specifications of the subsequent image coding specifications are reviewed. Prior art literature

非專利文獻非專利文獻1：ITU-T Recommendation H.265「High efficiency video coding(高效率視訊編碼)」，2015年4月Non-Patent Document Non-Patent Document 1: ITU-T Recommendation H.265 "High efficiency video coding", April 2015

發明概要發明欲解決之課題在這種的編碼方法及解碼方法中，所期望的是可以提升編碼效率。SUMMARY OF THE INVENTION Problems to be Solved by the Invention In such a coding method and a decoding method, it is desired to improve coding efficiency.

本揭示之目的在於提供一種可以提升編碼效率之解碼裝置、編碼裝置、解碼方法或編碼方法。用以解決課題之手段The purpose of this disclosure is to provide a decoding device, a coding device, a decoding method, or a coding method that can improve coding efficiency. Means to solve the problem

本揭示之一態樣的編碼裝置具備電路、及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。An encoding device according to one aspect of the present disclosure includes a circuit and a memory. The circuit uses the memory to derive a processing target block included in a processing target image from a motion vector used for dynamic compensation of a processing completed block. Multiple candidate motion vectors, and does not use the image area of the processing target block, but refers to two processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture. A final motion vector is selected from the plurality of candidate motion vectors, and the final motion vector is used to perform dynamic compensation of the processing target block.

本揭示之一態樣的解碼裝置具備電路、及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。A decoding device according to one aspect of the present disclosure includes a circuit and a memory. The circuit uses the memory to derive a processing target block included in a processing target image from a motion vector used for dynamic compensation of a processing completed block. Multiple candidate motion vectors, and does not use the image area of the processing target block, but refers to two processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture. A final motion vector is selected from the plurality of candidate motion vectors, and the final motion vector is used to perform dynamic compensation of the processing target block.

再者，這些全面性的或具體的態樣可以藉由系統、裝置、方法、積體電路、電腦程式、或電腦可讀取的CD-ROM等非暫時的記錄媒體來實現，也可以藉由系統、裝置、方法、積體電路、電腦程式、及記錄媒體的任意的組合來實現。發明效果Furthermore, these comprehensive or specific aspects can be realized by a non-transitory recording medium such as a system, device, method, integrated circuit, computer program, or computer-readable CD-ROM, or by a non-transitory recording medium such as a computer-readable CD-ROM. The system, the device, the method, the integrated circuit, the computer program, and any combination of recording media can be implemented. Invention effect

本揭示可以提供一種可以提升編碼效率之解碼裝置、編碼裝置、解碼方法或編碼方法。The present disclosure can provide a decoding device, a coding device, a decoding method, or a coding method that can improve coding efficiency.

用以實施發明之形態 (成為本揭示之基礎的知識見解) 例如，編碼裝置是按每個區塊來對圖像進行編碼。編碼裝置在按每個區塊來對圖像進行編碼時，可利用畫面間預測，亦可利用畫面內預測。編碼裝置是在處理對象區塊(當前區塊)的編碼中利用畫面間預測的情況下，檢測處理對象區塊的運動向量，且利用已檢測的運動向量來生成處理對象區塊的預測圖像。並且，編碼裝置是藉由對處理對象區塊的預測圖像、及處理對象區塊的原圖像之差分圖像進行編碼，以減少編碼量。Forms for Implementing the Invention (Knowledge insights that form the basis of this disclosure) For example, the encoding device encodes an image for each block. When the encoding device encodes an image for each block, it may use inter-screen prediction or intra-screen prediction. The encoding device detects a motion vector of a processing target block and uses the detected motion vector to generate a predicted image of the processing target block when inter-frame prediction is used in encoding of the processing target block (current block). . In addition, the encoding device reduces the amount of encoding by encoding the prediction image of the processing target block and the difference image of the original image of the processing target block.

又，編碼裝置是對顯示運動向量的運動向量資訊進行編碼，且解碼裝置是對運動向量資訊進行解碼。此外，解碼裝置會對差分圖像進行解碼。並且，解碼裝置是利用藉由已解碼的運動向量資訊所顯示的運動向量，來生成處理對象區塊的預測圖像，並藉由對預測圖像及差分圖像進行加法運算，以再構成原圖像。藉此，解碼裝置可以將圖像解碼。In addition, the encoding device encodes the motion vector information showing the motion vector, and the decoding device decodes the motion vector information. In addition, the decoding device decodes the differential image. In addition, the decoding device uses the motion vector displayed by the decoded motion vector information to generate a predicted image of the processing target block, and adds the predicted image and the difference image to reconstruct the original image. image. Thereby, the decoding device can decode the image.

藉由編碼裝置對運動向量資訊進行編碼，且解碼裝置對運動向量資訊進行解碼，解碼裝置就可以利用在編碼裝置所使用的運動向量，來適當地生成處理對象區塊的預測圖像。另一方面，藉由將運動向量資訊編碼，而有編碼量增加的可能性。By encoding the motion vector information with the encoding device and decoding the vector information with the decoding device, the decoding device can appropriately use the motion vector used in the encoding device to appropriately generate a prediction image of the processing target block. On the other hand, by encoding the motion vector information, there is a possibility that the amount of encoding increases.

編碼裝置及解碼裝置亦可為了減少這種編碼量，而利用一種稱為FRUC(畫面頻率提升，Frame Rate Up-Conversion)的技術。在FRUC中，編碼裝置及解碼裝置是在不進行運動向量資訊的編碼及解碼的情形下，在編碼裝置及解碼裝置中以相同的方法來導出運動向量。The encoding device and the decoding device may also use a technology called FRUC (Frame Rate Up-Conversion) in order to reduce this amount of encoding. In FRUC, the encoding device and the decoding device derive the motion vector in the same way in the encoding device and the decoding device without encoding and decoding the motion vector information.

例如，在雙向FRUC方式中，編碼裝置及解碼裝置是參照空間上或時間上相鄰於處理對象區塊的編碼完成區塊之運動向量，以導出各自具有預測運動向量的複數個候選運動向量。接著，編碼裝置及解碼裝置會算出複數個候選運動向量的各自的評價值，並根據評價值來選擇1個最終運動向量。接著，編碼裝置及解碼裝置會利用最終運動向量來進行動態補償。For example, in the two-way FRUC method, the encoding device and the decoding device refer to the motion vectors of the coding completion blocks that are spatially or temporally adjacent to the processing target block to derive a plurality of candidate motion vectors each having a predicted motion vector. Next, the encoding device and the decoding device calculate respective evaluation values of the plurality of candidate motion vectors, and select one final motion vector based on the evaluation values. The encoding device and the decoding device then use the final motion vector to perform motion compensation.

又，評價值是藉由以候選運動向量所指示的參照圖片內之區域、以及與該參照圖片不同的規定之圖片內的區域之間的型樣匹配而算出的。例如，可利用在顯示順序上包夾處理對象圖片的2個參照圖片。亦即，會有下述之課題：作為可參照的圖片，並未存在有在顯示順序上包夾處理對象區塊的2個圖片的情況下，就不能利用這個方式。The evaluation value is calculated by pattern matching between a region in the reference picture indicated by the candidate motion vector and a region in a predetermined picture different from the reference picture. For example, it is possible to use two reference pictures that sandwich the processing target picture in the display order. That is, there is a problem that, as a reference picture, this method cannot be used when there are no two pictures that sandwich the processing target block in the display order.

再者，在P圖片中會有在參照圖片清單中僅登錄有1張參照圖片的情況，在此情況下，會有無法利用2張參照圖片的可能性。藉此，會有無法利用雙向FRUC方式的課題存在。Furthermore, there may be a case where only one reference picture is registered in the reference picture list in the P picture, and in this case, there is a possibility that two reference pictures cannot be used. As a result, there is a problem that the bidirectional FRUC method cannot be used.

藉此，例如，即使在可參照的圖片中，未存在有在顯示順序上包夾處理對象區塊的2個圖片之情況下，仍可以利用雙向FRUC。據此，由於可以在更多的情況中利用雙向FRUC方式，因此可以改善編碼效率。With this, for example, even if there are no two pictures that sandwich the processing target block in the display order among the reference pictures, the bidirectional FRUC can be used. Accordingly, since the bidirectional FRUC method can be used in more cases, the coding efficiency can be improved.

例如，即使前述處理對象圖片為P圖片亦可。For example, the picture to be processed may be a P picture.

藉此，由於可以對P圖片利用雙向FRUC，因此可以在更多的情況中利用雙向FRUC方式。據此，可以改善編碼效率。Thereby, since the bidirectional FRUC can be used for the P picture, the bidirectional FRUC method can be used in more cases. Accordingly, encoding efficiency can be improved.

例如，在用於前述處理對象圖片的參照圖片清單中顯示2張參照圖片的情況下，前述2張處理完成的參照圖片亦可為顯示於前述參照圖片清單中的前述2張參照圖片。For example, when two reference pictures are displayed in the reference picture list used for the processing target picture, the two reference pictures that have been processed may also be the two reference pictures displayed in the reference picture list.

例如，亦可為在用於前述處理對象圖片的參照圖片清單中顯示1張參照圖片的情況下，前述2張處理完成的參照圖片的其中一張是顯示於前述參照圖片清單中的前述1張參照圖片，而前述2張處理完成的參照圖片之另一張是未顯示在前述參照圖片清單中的其他處理完成的參照圖片。For example, when one reference picture is displayed in the reference picture list used for the processing target picture, one of the two reference pictures that have been processed is the one displayed in the reference picture list. The reference picture, and the other of the two processed reference pictures is a reference picture that is not yet displayed in the reference picture list.

藉此，即使在P圖片的參照圖片清單中僅顯示1張參照圖片的情況下，仍可以利用雙向FRUC。據此，由於可以在更多的情況中利用雙向FRUC方式，因此可以改善編碼效率。This allows bidirectional FRUC to be used even when only one reference picture is displayed in the reference picture list of the P picture. Accordingly, since the bidirectional FRUC method can be used in more cases, the coding efficiency can be improved.

例如，前述電路亦可進一步地將顯示於前述參照圖片清單的前述1張參照圖片、及前述其他處理完成的參照圖片保存於圖片記憶體。For example, the circuit may further store the one reference picture displayed in the reference picture list and the reference picture completed by the other processing in a picture memory.

例如，亦可為在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述2張處理完成參照圖片的前述其中一張，是前述複數個候選運動向量當中以處理對象的候選運動向量來表示的參照圖片，且前述2張處理完成參照圖片的前述另一張，是可參照的圖片當中，顯示時間最接近於前述處理對象圖片的圖片。For example, in the selection of the final motion vector, the respective evaluation values of the plurality of candidate motion vectors may be calculated by referring to the two reference pictures that have been processed, and the final value may be selected based on the calculated evaluation values. The motion vector, the one of the two processed reference pictures, is a reference picture represented by the candidate motion vector of the processing target among the plurality of candidate motion vectors, and the two processed pictures complete the other one of the reference pictures. Is the picture with the closest display time to the processing target picture among the pictures that can be referred to.

例如，即使前述處理對象圖片為B圖片亦可。For example, the picture to be processed may be a B picture.

藉此，在B圖片中，即使在可參照的圖片中，未存在有在顯示順序上包夾處理對象區塊的2個圖片之情況下，仍可以利用雙向FRUC。據此，由於可以在更多的情況中利用雙向FRUC方式，因此可以改善編碼效率。Thus, in the B picture, even if there are no two pictures that sandwich the processing target block in the display order among the reference pictures, the bidirectional FRUC can be used. Accordingly, since the bidirectional FRUC method can be used in more cases, the coding efficiency can be improved.

例如，亦可為在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述處理對象圖片具有第1參照圖片清單及第2參照圖片清單，前述2張處理完成參照圖片的其中一張是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且是屬於前述第1參照圖片清單的參照圖片，前述2張處理完成參照圖片的另一張是屬於前述第2參照圖片清單的參照圖片當中，顯示時間最接近於前述處理對象圖片的圖片。For example, in the selection of the final motion vector, the respective evaluation values of the plurality of candidate motion vectors may be calculated by referring to the two reference pictures that have been processed, and the final value may be selected based on the calculated evaluation values. A motion vector, the processing target picture has a first reference picture list and a second reference picture list, and one of the two processed reference pictures is a reference picture represented by the candidate motion vector of the processing target among the plurality of candidate motion vectors Is a reference picture belonging to the first reference picture list, and the other two reference pictures that have been processed are pictures that are closest to the processing target picture among the reference pictures belonging to the second reference picture list.

例如，前述處理對象圖片亦可為P圖片。For example, the processing target picture may be a P picture.

例如，亦可為在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述2張處理完成參照圖片的其中一張，是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且前述2張處理完成參照圖片的另一張，是可參照的圖片當中，顯示時間最接近於前述處理對象圖片的圖片。For example, in the selection of the final motion vector, the respective evaluation values of the plurality of candidate motion vectors may be calculated by referring to the two reference pictures that have been processed, and the final value may be selected based on the calculated evaluation values. A motion vector, one of the aforementioned two processed reference pictures is a reference picture represented by the candidate motion vector of the processing target among the plurality of candidate motion vectors, and the other two of the aforementioned reference pictures are processed. Among the reference pictures, the picture whose display time is closest to the aforementioned processing target picture.

例如，前述處理對象圖片亦可為B圖片。For example, the processing target picture may be a B picture.

本揭示之一態樣的編碼方法，亦可從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。In one aspect of the present disclosure, the encoding method can also derive a plurality of candidate motion vectors for the processing target block included in the processing target picture from the motion vectors used for the dynamic compensation of the processing block, without using the foregoing processing. The image area of the target block refers to two processed reference pictures whose display time is located in the front and rear of the processing target picture in the same direction to select the final motion vector from the plurality of candidate motion vectors. The aforementioned final motion vector is used to perform dynamic compensation of the aforementioned processing target block.

本揭示之一態樣的解碼方法，亦可從處理完成區塊的動態補償所用的運動向量中導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。In one aspect of the present disclosure, the decoding method can also derive a plurality of candidate motion vectors for the processing target block included in the processing target picture from the motion vectors used for the dynamic compensation of the processed block, without using the foregoing processing target. The image area of the block refers to two processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture, to select the final motion vector from the plurality of candidate motion vectors, and Dynamic compensation of the processing target block is performed using the final motion vector.

此外，這些全面性的或具體的態樣可以藉由系統、裝置、方法、積體電路、電腦程式、或電腦可讀取的CD-ROM等非暫時的記錄媒體來實現，也可以藉由系統、裝置、方法、積體電路、電腦程式、及記錄媒體的任意組合來實現。In addition, these comprehensive or specific aspects can be realized by a non-transitory recording medium such as a system, device, method, integrated circuit, computer program, or computer-readable CD-ROM, or by a system , Devices, methods, integrated circuits, computer programs, and any combination of recording media.

又，在雙向FRUC方式中，必須算出複數個候選運動向量之各自的評價值。據此，會有候選運動向量的數量變得越多，處理量越增加的課題。Further, in the bidirectional FRUC method, it is necessary to calculate respective evaluation values of a plurality of candidate motion vectors. According to this, there is a problem that as the number of candidate motion vectors becomes larger, the processing amount increases.

本揭示之一態樣的編碼裝置具備電路、及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，其中該處理對象圖片是具有2個參照圖片清單的B圖片，且不使用前述處理對象區塊的圖像區域，而是算出前述複數個候選運動向量當中表示參照圖片的候選運動向量之評價值，其中該參照圖片是在前述2個參照圖片清單的其中一個即選擇參照圖片清單中所顯示的參照圖片，並根據已算出的評價值，從表示前述選擇參照圖片清單中所顯示的參照圖片之前述候選運動向量中，選擇最終運動向量，而利用前述最終運動向量來進行前述處理對象區塊的動態補償。An encoding device according to one aspect of the present disclosure includes a circuit and a memory. The circuit uses the memory to derive a processing target block included in a processing target image from a motion vector used for dynamic compensation of a processing completed block. Multiple candidate motion vectors, where the processing target picture is a B picture with 2 reference picture lists, and does not use the image area of the processing target block, but calculates the reference picture among the multiple candidate motion vectors. The evaluation value of the candidate motion vector, where the reference picture is one of the two reference picture lists, that is, the reference picture displayed in the reference picture list is selected, and according to the calculated evaluation value, the selected reference picture list is displayed. Among the candidate motion vectors of the displayed reference picture, a final motion vector is selected, and the final motion vector is used to perform dynamic compensation of the processing target block.

藉此，由於可以使算出評價值的對象之候選運動向量變少，因此可以減低處理量。Thereby, the number of candidate motion vectors of the object for which the evaluation value is calculated can be reduced, so that the amount of processing can be reduced.

例如，亦可為前述電路利用前述記憶體，並進一步地將前述2個參照圖片清單當中登錄於開頭的參照圖片之顯示時間較接近於前述處理對象圖片的顯示時間之參照圖片清單，決定為前述選擇參照圖片清單。For example, the aforementioned memory may be used for the aforementioned circuit, and the display time of the reference picture registered at the beginning of the two reference picture lists may be further determined to be the reference picture list whose display time is closer to the display time of the processing target picture. Select a reference picture list.

藉此，可以擷取更有用的候選運動向量。Thereby, more useful candidate motion vectors can be captured.

例如，亦可為前述電路利用前述記憶體，並進一步地將前述2個參照圖片清單當中將登錄於開頭的參照圖片設定成更優先來解碼的參照圖片清單，決定為前述選擇參照圖片清單。For example, the memory may be used for the circuit, and the reference picture registered at the beginning among the two reference picture lists may be set as a reference picture list to be decoded with higher priority, and the selected reference picture list may be determined.

例如，亦可為前述電路利用前述記憶體，並進一步地將前述2個參照圖片清單當中，登錄於開頭的參照圖片之量化寬度較小的參照圖片清單，決定為前述選擇參照圖片清單。For example, the memory may be used for the circuit, and the reference picture list with the smaller quantization width of the first reference picture registered among the two reference picture lists may be further determined as the selected reference picture list.

例如，亦可為前述電路利用前述記憶體，並進一步地生成編碼位元流，其中該編碼位元流包含用於特定前述選擇參照圖片清單的資訊。For example, the aforementioned memory may also be used for the aforementioned circuit, and the encoded bit stream may be further generated, wherein the encoded bit stream contains information for specifying the aforementioned selected reference picture list.

藉此，由於在解碼裝置中，亦可不進行決定選擇運動向量的處理，因此可以減低解碼裝置的處理量。This makes it possible to reduce the processing amount of the decoding device because the decoding device does not need to perform the process of determining the selection of the motion vector.

本揭示之一態樣的解碼裝置具備電路、及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，其中該處理對象圖片是具有2個參照圖片清單的B圖片，且不使用前述處理對象區塊的圖像區域，而是算出前述複數個候選運動向量當中表示參照圖片的候選運動向量之評價值，其中該參照圖片是在前述2個參照圖片清單的其中一個即選擇參照圖片清單中所顯示的參照圖片，並根據已算出的評價值，從表示前述選擇參照圖片清單中所顯示的參照圖片之前述候選運動向量中，選擇最終運動向量，而利用前述最終運動向量來進行前述處理對象區塊的動態補償。A decoding device according to one aspect of the present disclosure includes a circuit and a memory. The circuit uses the memory to derive a processing target block included in a processing target image from a motion vector used for dynamic compensation of a processing completed block. Multiple candidate motion vectors, where the processing target picture is a B picture with 2 reference picture lists, and does not use the image area of the processing target block, but calculates the reference picture among the multiple candidate motion vectors. The evaluation value of the candidate motion vector, where the reference picture is one of the two reference picture lists, that is, the reference picture displayed in the reference picture list is selected, and according to the calculated evaluation value, the selected reference picture list is displayed. Among the candidate motion vectors of the displayed reference picture, a final motion vector is selected, and the final motion vector is used to perform dynamic compensation of the processing target block.

例如，亦可為前述電路利用前述記憶體，並進一步地取得用於特定包含於編碼位元流的前述選擇參照圖片清單之資訊，以利用前述資訊來特定前述選擇參照圖片清單。For example, it is also possible to use the memory for the circuit and further obtain information for specifying the selected reference picture list included in the encoded bit stream, so as to use the information to specify the selected reference picture list.

本揭示之一態樣的編碼方法，是從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，其中該處理對象圖片是具有2個參照圖片清單的B圖片，且不使用前述處理對象區塊的圖像區域，而是算出前述複數個候選運動向量當中表示參照圖片的候選運動向量之評價值，其中該參照圖片是在前述2個參照圖片清單的其中一個即選擇參照圖片清單中所顯示的參照圖片，並根據已算出的評價值，從表示前述選擇參照圖片清單中所顯示的參照圖片之前述候選運動向量中，選擇最終運動向量，而利用前述最終運動向量來進行前述處理對象區塊的動態補償。One aspect of the present disclosure is an encoding method that derives a plurality of candidate motion vectors for a processing target block included in a processing target picture from the motion vectors used for dynamic compensation of the processing completed block, where the processing target picture is The B picture with two reference picture lists does not use the image area of the processing target block, but calculates the evaluation value of the candidate motion vector representing the reference picture among the plurality of candidate motion vectors, where the reference picture is in One of the two reference picture lists is to select a reference picture displayed in the reference picture list, and select from the candidate motion vectors representing the reference pictures displayed in the selected reference picture list according to the calculated evaluation value. The final motion vector, and use the final motion vector to perform the dynamic compensation of the processing target block.

本揭示之一態樣的解碼方法，是從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，其中該處理對象圖片是具有2個參照圖片清單的B圖片，且不使用前述處理對象區塊的圖像區域，而是算出前述複數個候選運動向量當中表示參照圖片的候選運動向量之評價值，其中該參照圖片是在前述2個參照圖片清單的其中一個即選擇參照圖片清單中所顯示的參照圖片，並根據已算出的評價值，從表示前述選擇參照圖片清單中所顯示的參照圖片之前述候選運動向量中，選擇最終運動向量，而利用前述最終運動向量來進行前述處理對象區塊的動態補償。One aspect of the present disclosure is a decoding method that derives a plurality of candidate motion vectors for a processing target block included in a processing target picture from the motion vectors used for dynamic compensation of the processing completed block, where the processing target picture is The B picture with two reference picture lists does not use the image area of the processing target block, but calculates the evaluation value of the candidate motion vector representing the reference picture among the plurality of candidate motion vectors, where the reference picture is in One of the two reference picture lists is to select a reference picture displayed in the reference picture list, and select from the candidate motion vectors representing the reference pictures displayed in the selected reference picture list according to the calculated evaluation value. The final motion vector, and use the final motion vector to perform the dynamic compensation of the processing target block.

以下，參照圖式來具體地說明實施形態。Hereinafter, embodiments will be specifically described with reference to the drawings.

再者，在以下所說明的實施形態都是顯示全面性的或具體的例子之實施形態。以下實施形態所示的數值、形狀、材料、構成要件、構成要件的配置位置及連接形態、步驟、步驟的順序等，都只是一個例子，並非用來限定請求的範圍的主旨。又，以下的實施形態的構成要件之中，針對沒有記載在表示最上位概念之獨立請求項中的構成要件，是作為任意之構成要件來說明。 (實施形態1)It should be noted that the embodiments described below are all embodiments showing comprehensive or specific examples. The numerical values, shapes, materials, constituent elements, arrangement positions and connection forms, steps, and order of the steps shown in the following embodiments are merely examples, and are not intended to limit the scope of the request. In addition, among the constituent elements of the following embodiments, constituent elements that are not described in the independent request item indicating the highest-level concept are described as arbitrary constituent elements. (Embodiment 1)

首先，說明實施形態1的概要，來作為可適用後述之本揭示的各態樣中說明的處理及/或構成的編碼裝置及解碼裝置之一例。但是，實施形態1只不過是可適用本揭示的各態樣中說明的處理及/或構成之編碼裝置及解碼裝置的一例，本揭示的各態樣中說明的處理及/或構成，亦可在與實施形態1不同的編碼裝置及解碼裝置中實施。First, the outline of the first embodiment will be described as an example of an encoding device and a decoding device to which the processing and / or configuration described in the aspects of the present disclosure described later can be applied. However, the first embodiment is merely an example of an encoding device and a decoding device to which the processing and / or configuration described in each aspect of the present disclosure can be applied, and the processing and / or configuration described in each aspect of the present disclosure may also be applied. It is implemented in an encoding device and a decoding device different from those in the first embodiment.

對實施形態1適用在本揭示的各態樣中所說明的處理及/或構成的情況下，亦可進行例如以下的任一項。 (1)對於實施形態1的編碼裝置或解碼裝置，可將構成該編碼裝置或解碼裝置的複數個構成要件當中，與本揭示的各態樣中所說明的構成要件相對應的構成要件，替換為本揭示的各態樣中所說明的構成要件。 (2)對於實施形態1的編碼裝置或解碼裝置，針對構成該編碼裝置或解碼裝置的複數個構成要件當中一部分的構成要件，施加了功能或實施之處理的追加、替換、刪除等任意的變更後，將與本揭示的各態樣中所說明的構成要件相對應的構成要件，替換為本揭示的各態樣中所說明的構成要件。 (3)對於實施形態1的編碼裝置或解碼裝置所實施的方法，可於施加了處理之追加、及/或針對該方法所包含的複數個處理當中的一部分之處理進行替換、刪除等任意的變更後，將與本揭示的各態樣中說明的處理相對應的處理，替換為本揭示的各態樣中所說明的處理。 (4)可將構成實施形態1的編碼裝置或解碼裝置之複數個構成要件當中的一部分之構成要件，與下述構成要件組合而實施：本揭示的各態樣中所說明的構成要件、具備有本揭示的各態樣中所說明的構成要件所具備的功能之一部分的構成要件、或實施本揭示的各態樣中所說明的構成要件所實施的處理之一部分的構成要件。 (5)可將具備構成實施形態1的編碼裝置或解碼裝置之複數個構成要件當中的一部分之構成要件所具備的功能之一部分的構成要件、或實施構成實施形態1的編碼裝置或解碼裝置之複數個構成要件當中的一部分之構成要件所實施的處理之一部分的構成要件，與下述構成要件組合而實施：本揭示的各態樣中所說明的構成要件、具備本揭示的各態樣中所說明的構成要件所具備的功能之一部分的構成要件、或實施本揭示的各態樣中所說明的構成要件所實施的處理之一部分的構成要件。 (6)對於實施形態1的編碼裝置或解碼裝置所實施的方法，可將在該方法所包含的複數個處理當中，與本揭示的各態樣中所說明的處理相對應的處理，替換為本揭示的各態樣中所說明的處理。 (7)可將實施形態1的編碼裝置或解碼裝置所實施的方法所包含的複數個處理當中之一部分的處理，與本揭示的各態樣中所說明的處理組合而實施。When the process and / or configuration described in each aspect of the present disclosure is applied to the first embodiment, for example, any of the following may be performed. (1) With regard to the encoding device or the decoding device according to the first embodiment, among the plurality of components constituting the encoding device or the decoding device, the components corresponding to the components described in the aspects of the present disclosure may be replaced. These are constituent elements described in each aspect of the present disclosure. (2) With regard to the encoding device or the decoding device according to the first embodiment, arbitrary changes such as additions, replacements, and deletions of functions or implemented processes are added to a part of a plurality of the components constituting the encoding device or the decoding device. Then, the constituent elements corresponding to the constituent elements described in the aspects of the present disclosure are replaced with the constituent elements described in the aspects of the present disclosure. (3) The method implemented by the encoding device or the decoding device according to the first embodiment may be arbitrarily changed, such as adding processing, and / or replacing or deleting a part of a plurality of processing included in the method. After the change, the processes corresponding to the processes described in the aspects of the present disclosure are replaced with the processes explained in the aspects of the present disclosure. (4) A part of the plurality of constituent elements constituting the encoding apparatus or the decoding apparatus according to the first embodiment may be implemented in combination with the following constituent elements: the constituent elements described in each aspect of the present disclosure, including There are constituent elements that are part of the functions possessed by the constituent elements described in the aspects of the present disclosure, or constituent elements that are part of the processing performed by the constituent elements described in the aspects of the present disclosure. (5) It is possible to implement the constituent elements having a part of the functions provided by the constituent elements constituting a part of a plurality of constituent elements constituting the encoding device or the decoding apparatus according to the first embodiment, or to implement the encoding apparatus or the decoding apparatus constituting the first embodiment. A part of the plurality of constituent elements is a combination of the constituent elements that are part of the processing performed by the constituent elements, and are implemented in combination with the constituent elements described in the aspects of the present disclosure, and the aspects provided with the aspects of the present disclosure. The constituent elements that are part of the functions provided by the constituent elements that are described, or the constituent elements that are part of the processes performed by implementing the constituent elements described in the aspects of the present disclosure. (6) For the method implemented by the encoding device or decoding device according to the first embodiment, among the plurality of processes included in the method, the processes corresponding to the processes described in the aspects of the present disclosure can be replaced with The processing described in each aspect of the present disclosure. (7) A part of the plurality of processes included in the method implemented by the encoding device or the decoding device according to the first embodiment may be combined with the processing described in each aspect of the present disclosure.

再者，在本揭示的各態樣中所說明的處理及/或構成的實施之方式，並不限定於上述的例子。例如，亦可在以和實施形態1中揭示的動態圖像/圖像編碼裝置或動態圖像/圖像解碼裝置不同之目的來利用的裝置中實施，亦可單獨地來實施各態樣中所說明的處理及/或構成。又，也可以將不同的態樣中說明的處理及/或構成組合而實施。 [編碼裝置之概要]It should be noted that the implementation of the processing and / or configuration described in each aspect of the present disclosure is not limited to the examples described above. For example, it may be implemented in a device which is used for a different purpose from the moving image / image encoding device or the moving image / image decoding device disclosed in the first embodiment, or may be implemented separately in each aspect. Process and / or composition described. The processes and / or configurations described in different aspects may be combined and implemented. [Outline of encoding device]

首先，說明實施形態1之編碼裝置的概要。圖1是顯示實施形態1之編碼裝置100的功能構成之方塊圖。編碼裝置100是以區塊單位對動態圖像/圖像進行編碼的動態圖像/圖像編碼裝置。First, the outline of the encoding device according to the first embodiment will be described. FIG. 1 is a block diagram showing a functional configuration of an encoding device 100 according to the first embodiment. The encoding device 100 is a moving image / image encoding device that encodes moving images / images in units of blocks.

如圖1所示，編碼裝置100是以區塊單位對圖像進行編碼的裝置，並具備分割部102、減法部104、轉換部106、量化部108、熵編碼部110、逆量化部112、逆轉換部114、加法部116、區塊記憶體118、迴路濾波部120、框記憶體122、框內預測部124、框間預測部126、及預測控制部128。As shown in FIG. 1, the encoding device 100 is a device that encodes an image in units of blocks, and includes a division unit 102, a subtraction unit 104, a conversion unit 106, a quantization unit 108, an entropy encoding unit 110, an inverse quantization unit 112, The inverse conversion unit 114, the addition unit 116, the block memory 118, the loop filtering unit 120, the frame memory 122, the in-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128.

編碼裝置100可藉由例如通用處理器及記憶體來實現。在此情況下，藉由處理器執行保存在記憶體的軟體程式時，處理器是作為分割部102、減法部104、轉換部106、量化部108、熵編碼部110、逆量化部112、逆轉換部114、加法部116、迴路濾波部120、框內預測部124、框間預測部126及預測控制部128而發揮功能。又，編碼裝置100亦可作為對應於分割部102、減法部104、轉換部106、量化部108、熵編碼部110、逆量化部112、逆轉換部114、加法部116、迴路濾波部120、框內預測部124、框間預測部126及預測控制部128的1個以上之專用的電子電路來實現。The encoding device 100 can be implemented by, for example, a general-purpose processor and a memory. In this case, when the software program stored in the memory is executed by the processor, the processor functions as the division unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy encoding unit 110, the inverse quantization unit 112, and the inverse The conversion unit 114, the addition unit 116, the loop filtering unit 120, the intra-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128 function. The encoding device 100 may also correspond to the division unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy encoding unit 110, the inverse quantization unit 112, the inverse conversion unit 114, the addition unit 116, the loop filter unit 120, One or more dedicated electronic circuits of the intra-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128 are implemented.

以下，針對包含在編碼裝置100的各構成要件來進行說明。 [分割部]Hereinafter, each constituent element included in the encoding device 100 will be described. [Division]

分割部102是將包含在輸入動態圖像的各圖片分割成複數個區塊，且將各區塊輸出至減法部104。例如，分割部102首先將圖片分割為固定尺寸(例如128x128)的區塊。此固定尺寸的區塊被稱為編碼樹單元(CTU)。並且，分割部102會根據遞迴的四元樹(quadtree)及/或二元樹(binary tree)區塊分割，而將各個固定尺寸的區塊分割成可變尺寸(例如64x64以下)的區塊。此可變尺寸的區塊有時被稱為編碼單元(CU)、預測單元(PU)或轉換單元(TU)。再者，在本實施形態中，亦可不需要區別CU、PU及TU，而使圖片內的一部分或全部的區塊成為CU、PU、TU的處理單位。The division unit 102 divides each picture included in the input moving image into a plurality of blocks, and outputs each block to the subtraction unit 104. For example, the segmentation unit 102 first divides a picture into blocks of a fixed size (for example, 128 × 128). This fixed-size block is called a coding tree unit (CTU). In addition, the dividing unit 102 divides the quadtree and / or binary tree blocks based on the recursion, and divides each fixed-size block into regions of variable size (for example, 64x64 or less). Piece. This variable-sized block is sometimes called a coding unit (CU), a prediction unit (PU), or a conversion unit (TU). Furthermore, in this embodiment, it is not necessary to distinguish between CU, PU, and TU, and a part or all of the blocks in the picture may be processed by CU, PU, and TU.

圖2是顯示實施形態1中的區塊分割的一例之圖。在圖2中，實線是表示藉由四元樹區塊分割的區塊交界，而虛線是表示藉由二元樹區塊分割的區塊交界。FIG. 2 is a diagram showing an example of block division in the first embodiment. In FIG. 2, the solid line indicates a block boundary divided by a quaternary tree block, and the dotted line indicates a block boundary divided by a quaternary tree block.

在此，區塊10是128x128像素的正方形區塊(128x128區塊)。此128x128區塊10首先被分割成4個正方形的64x64區塊(四元樹區塊分割)。Here, the block 10 is a square block of 128 × 128 pixels (128 × 128 block). This 128x128 block 10 is first partitioned into 4 square 64x64 blocks (quaternary tree block partitioning).

左上的64x64區塊進一步被垂直地分割成2個矩形的32x64區塊，且左邊的32x64區塊進一步被垂直地分割成2個矩形的16x64區塊(二元樹區塊分割)。其結果，左上的64x64區塊被分割成2個16x64區塊11、12、以及32x64區塊13。The upper left 64x64 block is further vertically divided into 2 rectangular 32x64 blocks, and the left 32x64 block is further vertically divided into 2 rectangular 16x64 blocks (binary tree block division). As a result, the upper left 64x64 block is divided into two 16x64 blocks 11, 12 and 32x64 block 13.

右上的64x64區塊被水平地分割為2個矩形的64x32區塊14、15(二元樹區塊分割)。The upper right 64x64 block is horizontally divided into two rectangular 64x32 blocks 14, 15 (binary tree block partition).

左下的64x64區塊被分割為4個正方形的32x32區塊(四元樹區塊分割)。4個32x32區塊當中，將左上的區塊及右下的區塊進一步地分割。左上的32x32區塊被垂直地分割成2個矩形的16x32區塊，且將右邊的16x32區塊進一步水平地分割為2個16x16區塊(二元樹區塊分割)。右下的32x32區塊被水平地分割成2個32x16區塊(二元樹區塊分割)。其結果，可將左下的64x64區塊分割成：16x32區塊16、2個16x16區塊17、18、2個32x32區塊19、20、與2個32x16區塊21、22。The 64x64 block at the bottom left is divided into 4 square 32x32 blocks (quaternary tree block partition). Among the four 32x32 blocks, the upper left block and the lower right block are further divided. The upper left 32x32 block is vertically divided into two rectangular 16x32 blocks, and the right 16x32 block is further horizontally divided into two 16x16 blocks (binary tree block division). The lower right 32x32 block is horizontally divided into two 32x16 blocks (binary tree block partition). As a result, the 64x64 block at the bottom left can be divided into: 16x32 block 16, two 16x16 blocks 17, 18, two 32x32 blocks 19, 20, and two 32x16 blocks 21, 22.

右下的64x64區塊23未被分割。The lower right 64x64 block 23 is not divided.

如以上，在圖2中，區塊10是根據遞迴的四元樹及二元樹區塊分割，而被分割成13個可變尺寸的區塊11~23。這種分割被稱為QTBT(四元樹加二元樹區塊結構(quad-tree plus binary tree))分割。As above, in FIG. 2, the block 10 is divided according to the recursive quaternary and binary tree blocks, and is divided into 13 variable-size blocks 11 to 23. This segmentation is called QTBT (quad-tree plus binary tree) segmentation.

再者，在圖2中，雖然是將1個區塊分割成4個或2個區塊(四元樹或二元樹區塊分割)，但分割並不限定於此。例如，亦可將1個區塊分割成3個區塊(三元樹區塊分割)。這種包含了三元樹區塊分割的分割，被稱為MBT(多類型樹(multi type tree))分割。 [減法部]Moreover, in FIG. 2, although one block is divided into four or two blocks (quaternary tree or binary tree block division), the division is not limited to this. For example, one block may be divided into three blocks (ternary tree block division). This type of segmentation including ternary tree block segmentation is called MBT (multi type tree) segmentation. [Subtraction Division]

減法部104是以由分割部102所分割的區塊單位來從原訊號(原樣本)中減去預測訊號(預測樣本)。也就是說，減法部104會算出編碼對象區塊(以下，稱為當前區塊)的預測誤差(也可稱為殘差)。而且，減法部104會將算出的預測誤差輸出至轉換部106。The subtraction unit 104 subtracts the prediction signal (prediction sample) from the original signal (original sample) in units of blocks divided by the division unit 102. That is, the subtraction unit 104 calculates a prediction error (also referred to as a residual) of a coding target block (hereinafter, referred to as a current block). Then, the subtraction unit 104 outputs the calculated prediction error to the conversion unit 106.

原訊號是編碼裝置100的輸入訊號，且是表示構成動態圖像的各圖片之圖像的訊號(例如亮度(luma)訊號及2個色差(chroma)訊號)。在以下，有時也會將表示圖像的訊號稱為樣本。 [轉換部]The original signal is an input signal of the encoding device 100 and is a signal (for example, a luma signal and two chroma signals) indicating an image of each picture constituting a moving image. Hereinafter, a signal representing an image may be referred to as a sample. [Conversion Department]

轉換部106是將空間區域的預測誤差轉換成頻率區域的轉換係數，並將轉換係數輸出至量化部108。具體來說，轉換部106是例如對空間區域的預測誤差進行預定之離散餘弦轉換(DCT)或離散正弦轉換(DST)。The conversion unit 106 converts a prediction error in a spatial region into a conversion coefficient in a frequency region, and outputs the conversion coefficient to the quantization unit 108. Specifically, the conversion unit 106 performs, for example, a predetermined discrete cosine transform (DCT) or a discrete sine transform (DST) on a prediction error of a spatial region.

再者，轉換部106亦可從複數個轉換類型之中自適應地選擇轉換類型，且使用與所選擇的轉換類型相對應之轉換基底函數(transform basis function)，來將預測誤差轉換成轉換係數。有時將這種轉換稱為EMT(外顯性多重核心轉換(explicit multiple core transform))或AMT(適應性多重轉換(adaptive multiple transform))。In addition, the conversion unit 106 may also adaptively select a conversion type from a plurality of conversion types, and use a transform basis function corresponding to the selected conversion type to convert the prediction error into a conversion coefficient. . This conversion is sometimes referred to as EMT (explicit multiple core transform) or AMT (adaptive multiple transform).

複數個轉換類型包含例如DCT-II、DCT-V、DCT-VIII、DST-I、及DST-VII。圖3是顯示對應於各轉換類型的轉換基底函數之表格。在圖3中N是表示輸入像素的數量。從這些複數個轉換類型之中的轉換類型之選擇，可依例如預測的種類(框內預測(intra-prediction)及框間預測(inter-prediction))而定，亦可依框內預測模式而定。The plurality of conversion types include, for example, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII. FIG. 3 is a table showing conversion basis functions corresponding to each conversion type. In FIG. 3, N is the number of input pixels. The selection of the conversion type from among the plurality of conversion types may depend on, for example, the type of prediction (intra-prediction and inter-prediction), or it may depend on the in-frame prediction mode. set.

這種顯示是否適用EMT或AMT的資訊(可稱為例如AMT旗標(AMT flag))及顯示所選擇的轉換類型的資訊，可在CU層級被訊號化。再者，這些資訊的訊號化並不需限定於CU層級，也可以是其他的層級(例如，序列層級(sequence level)、圖片層級(picture level)、片段層級(slice level)、圖塊層級(tile level)或CTU層級)。This type of information showing whether EMT or AMT is applicable (may be called, for example, AMT flag) and information showing the selected conversion type can be signaled at the CU level. Furthermore, the signalization of this information does not need to be limited to the CU level, but may be other levels (e.g., sequence level, picture level, slice level, tile level ( tile level) or CTU level).

又，轉換部106也可以將轉換係數(轉換結果)再轉換。有時將這種再轉換稱為AST(適應性二次轉換(adaptive secondary transform))或NSST(不可分的二次轉換(non-separable secondary transform))。例如，轉換部106會按對應於框內預測誤差的轉換係數之區塊所包含的每個子區塊(例如4x4子區塊)進行再轉換。顯示是否適用NSST的資訊以及與NSST所使用的轉換矩陣相關的資訊，是在CU層級被訊號化。再者，這些資訊的訊號化並不需限定於CU層級，也可以是其他的層級(例如，序列層級(sequence level)、圖片層級(picture level)、片段層級(slice level)、圖塊層級(tile level)或CTU層級)。The conversion unit 106 may convert the conversion coefficient (conversion result) again. Such retransforms are sometimes referred to as AST (adaptive secondary transform) or NSST (non-separable secondary transform). For example, the conversion unit 106 performs re-conversion for each sub-block (for example, a 4 × 4 sub-block) included in the block corresponding to the conversion coefficient of the in-frame prediction error. Information showing whether NSST is applicable and information related to the conversion matrix used by NSST is signaled at the CU level. Furthermore, the signalization of this information does not need to be limited to the CU level, but may be other levels (e.g., sequence level, picture level, slice level, tile level ( tile level) or CTU level).

在此，可分離之(Separable)轉換是指以相當於輸入的維度之數來按各個方向分離並進行複數次轉換的方式，不可分離之(Non-Separable)轉換是指當輸入為多維時將2個以上的維度匯總而視為1維，且一次進行轉換的方式。Here, the separable transformation refers to a method of separating and performing a plurality of transformations in each direction with the number of dimensions corresponding to the input, and the non-separable transformation refers to transforming the input when the input is multidimensional. A method of summing two or more dimensions as one dimension and converting them at one time.

例如，作為不可分離之轉換的一例，可列舉如下之轉換：在輸入為4x4的區塊之情況下，將其視為具有16個要件的一種排列，而對該排列以16x16的轉換矩陣來進行轉換處理。For example, as an example of an inseparable transformation, the following transformations can be enumerated: when the input is a 4x4 block, it is regarded as an arrangement with 16 requirements, and the arrangement is performed using a 16x16 transformation matrix. Conversion processing.

又，同樣地，於將4x4的輸入區塊視為具有16個要件的一種排列之後，對該排列進行複數次吉文斯旋轉(Givens rotation)之類的轉換(超立方吉文斯轉換，Hypercube Givens Transform)也是不可分離之轉換的例子。 [量化部]Similarly, after the 4x4 input block is regarded as an arrangement with 16 requirements, the arrangement is subjected to transformations such as Givens rotation (Hypercube Givens Transform, Hypercube Givens Transform). ) Is also an example of an inseparable transformation. [Quantization Department]

量化部108是對從轉換部106輸出的轉換係數進行量化。具體來說，量化部108是以規定的掃描順序掃描當前區塊的轉換係數，且根據與所掃描的轉換係數相對應之量化參數(QP)來對該轉換係數進行量化。並且，量化部108會將當前區塊之已量化的轉換係數(以下，稱為量化係數)輸出到熵編碼部110及逆量化部112。The quantization unit 108 quantizes the conversion coefficients output from the conversion unit 106. Specifically, the quantization unit 108 scans the conversion coefficients of the current block in a predetermined scanning order, and quantizes the conversion coefficients based on a quantization parameter (QP) corresponding to the scanned conversion coefficients. In addition, the quantization unit 108 outputs the quantized conversion coefficients (hereinafter, referred to as quantization coefficients) of the current block to the entropy encoding unit 110 and the inverse quantization unit 112.

規定的順序是用於轉換係數的量化/逆量化之順序。例如，規定的掃描順序是以頻率的遞升順序(從低頻到高頻的順序)或遞降順序(從高頻到低頻的順序)來定義。The prescribed order is an order for quantization / inverse quantization of conversion coefficients. For example, the prescribed scanning order is defined in ascending order of frequency (order from low frequency to high frequency) or descending order (order from high frequency to low frequency).

所謂量化參數是定義量化步距(量化寬度)的參數。例如，量化參數的值增加的話，會使量化步距也增加。也就是說，量化參數的值增加的話，會使量化誤差増大。 [熵編碼部]The quantization parameter is a parameter that defines a quantization step (quantization width). For example, increasing the value of the quantization parameter will increase the quantization step. In other words, if the value of the quantization parameter is increased, the quantization error will increase. [Entropy coding section]

熵編碼部110是藉由對從量化部108輸入之量化係數進行可變長度編碼，來生成編碼訊號(編碼位元流(bit stream))。具體來說，熵編碼部110是例如將量化係數二值化，而對二值訊號進行算術編碼。 [逆量化部]The entropy coding unit 110 generates a coding signal (encoded bit stream) by variable-length coding the quantization coefficient input from the quantization unit 108. Specifically, for example, the entropy coding unit 110 binarizes the quantization coefficient and performs arithmetic coding on the binary signal. [Inverse quantization section]

逆量化部112是對來自量化部108的輸入即量化係數進行逆量化。具體來說，逆量化部112是以規定的掃描順序對當前區塊的量化係數進行逆量化。並且，逆量化部112會將當前區塊之已逆量化的轉換係數輸出到逆轉換部114。 [逆轉換部]The inverse quantization unit 112 performs inverse quantization on a quantization coefficient that is an input from the quantization unit 108. Specifically, the inverse quantization unit 112 performs inverse quantization on the quantization coefficients of the current block in a predetermined scanning order. Then, the inverse quantization section 112 outputs the inverse quantized conversion coefficients of the current block to the inverse conversion section 114. [Inverse Conversion Department]

逆轉換部114是藉由對來自逆量化部112的輸入即轉換係數進行逆轉換，以復原預測誤差。具體來說，逆轉換部114是藉由對轉換係數進行與由轉換部106所進行的轉換相對應之逆轉換，來復原當前區塊的預測誤差。並且，逆轉換部114會將復原的預測誤差輸出至加法部116。The inverse conversion unit 114 restores the prediction error by inversely converting the conversion coefficient that is an input from the inverse quantization unit 112. Specifically, the inverse conversion unit 114 restores the prediction error of the current block by performing inverse conversion on the conversion coefficient corresponding to the conversion performed by the conversion unit 106. Then, the inverse conversion unit 114 outputs the restored prediction error to the addition unit 116.

再者，由於復原的預測誤差會因量化而失去資訊，因此和減法部104算出的預測誤差並不一致。亦即，復原的預測誤差中包含有量化誤差。 [加法部]Furthermore, since the restored prediction error loses information due to quantization, it does not agree with the prediction error calculated by the subtraction unit 104. That is, the restored prediction error includes a quantization error. [Addition Department]

加法部116會對來自逆轉換部114的輸入即預測誤差、及來自預測控制部128的輸入即預測樣本進行加法運算，藉此再構成當前區塊。而且，加法部116會將再構成的區塊輸出到區塊記憶體118及迴路濾波部120。有時也將再構成區塊稱為局部解碼區塊(local decoding block)。 [區塊記憶體]The addition unit 116 adds the prediction error that is an input from the inverse conversion unit 114 and the prediction sample that is an input from the prediction control unit 128 to construct a current block. Then, the adding unit 116 outputs the reconstructed block to the block memory 118 and the loop filtering unit 120. The reconstructed block is sometimes called a local decoding block. [Block memory]

區塊記憶體118是用於保存在框內預測中所參照的區塊且也是編碼對象圖片(以下，稱為當前圖片)內的區塊之儲存部。具體來說，區塊記憶體118會保存從加法部116輸出的再構成區塊。 [迴路濾波部]The block memory 118 is a storage unit for storing the blocks referred to in the in-frame prediction and also the blocks in the encoding target picture (hereinafter, referred to as the current picture). Specifically, the block memory 118 stores the reconstructed blocks output from the adding unit 116. [Loop Filtering Division]

迴路濾波部120會對藉由加法部116再構成的區塊施行迴路濾波，且將已進行濾波的再構成區塊輸出到框記憶體122。所謂迴路濾波器是在編碼迴路內使用的濾波器(內嵌式迴路濾波器(In-loop filter))，且包含例如去區塊濾波器(Deblocking Filter，DF)、取樣自適應偏移(Sample Adaptive Offset，SAO)及自適應迴路濾波器(Adaptive Loop Filter，ALF)等。The loop filtering unit 120 performs loop filtering on the blocks reconstructed by the adding unit 116, and outputs the filtered reconstructed blocks to the frame memory 122. The so-called loop filter is a filter (in-loop filter) used in the coding loop, and includes, for example, a deblocking filter (DF) and a sample adaptive offset (Sample). Adaptive Offset (SAO) and Adaptive Loop Filter (ALF).

在ALF中，可適用去除編碼失真用的最小平方誤差濾波器，例如可適用按當前區塊內的2x2子區塊的每一個，根據局部的梯度(gradient)的方向及活動性(activity)來從複數個濾波器之中選擇的1個濾波器。In ALF, a least square error filter for removing coding distortion can be applied. For example, it can be applied to each of the 2x2 sub-blocks in the current block according to the direction of the local gradient and activity. One filter selected from a plurality of filters.

具體來說，首先，可將子區塊(例如2x2子區塊)分類成複數個類別(class)(例如15個或25個類別)。子區塊的分類是根據梯度的方向及活動性來進行。例如，可利用梯度的方向值D(例如0~2或0~4)與梯度的活性值A(例如0~4)來算出分類值C(例如C=5D＋A)。而且，根據分類值C，來將子區塊分類成複數個類別(例如15個或25個類別)。Specifically, first, a sub-block (for example, a 2 × 2 sub-block) may be classified into a plurality of classes (for example, 15 or 25 classes). The classification of sub-blocks is based on the direction and activity of the gradient. For example, the classification value C (for example, C = 5D + A) can be calculated by using the direction value D (for example, 0 ~ 2 or 0 ~ 4) of the gradient and the activity value A (for example, 0 ~ 4) of the gradient. Furthermore, the sub-block is classified into a plurality of categories (for example, 15 or 25 categories) according to the classification value C.

梯度的方向值D可藉由例如比較複數個方向(例如水平、垂直及2個對角方向)的梯度來導出。又，梯度的活性值A是藉由例如對複數個方向的梯度作加法運算，並對加法結果進行量化來導出。The direction value D of the gradient can be derived by, for example, comparing gradients in a plurality of directions (for example, horizontal, vertical, and two diagonal directions). The activity value A of the gradient is derived by, for example, adding gradients in a plurality of directions and quantifying the addition result.

根據這種分類的結果，即可從複數個濾波器之中決定子區塊用的濾波器。Based on the results of this classification, a filter for a sub-block can be determined from a plurality of filters.

作為在ALF中所用的濾波器之形狀，可利用的有例如圓對稱形狀。圖4A~圖4C是顯示在ALF中所用的濾波器之形狀的複數個例子之圖。圖4A是顯示5x5菱形(diamond)形狀濾波器，圖4B是顯示7x7菱形形狀濾波器，圖4C是顯示9x9菱形形狀濾波器。顯示濾波器的形狀之資訊，是在圖片層級被訊號化。再者，顯示濾波器的形狀之資訊的訊號化並不需要限定於圖片層級，亦可為其他的層級(例如，序列層級、片段層級、圖塊層級、CTU層級或CU層級)。As the shape of the filter used in ALF, for example, a circularly symmetric shape can be used. 4A to 4C are diagrams showing plural examples of the shape of a filter used in the ALF. FIG. 4A shows a 5x5 diamond shape filter, FIG. 4B shows a 7x7 diamond shape filter, and FIG. 4C shows a 9x9 diamond shape filter. Information about the shape of the display filter is signaled at the picture level. Furthermore, the signalization of the information showing the shape of the filter does not need to be limited to the picture level, but may be other levels (for example, sequence level, slice level, tile level, CTU level, or CU level).

ALF的開啟/關閉(on/off)是在例如圖片層級或CU層級決定的。例如，針對亮度是在CU層級決定是否適用ALF，而針對色差則是在圖片層級決定是否適用ALF。顯示ALF的開啟/關閉之資訊，可在圖片層級或CU層級被訊號化。再者，顯示ALF的開啟/關閉之資訊的訊號化並不需要限定於圖片層級或CU層級，亦可為其他的層級(例如，序列層級、片段層級、圖塊層級或CTU層級)。The on / off of ALF is determined at the picture level or the CU level, for example. For example, whether to apply ALF is determined at the CU level for brightness, and whether or not ALF is applied at the picture level for color difference. Display ALF on / off information, which can be signaled at the picture level or the CU level. Furthermore, the signalization of the information showing the on / off of ALF need not be limited to the picture level or the CU level, but may be other levels (for example, the sequence level, the fragment level, the tile level, or the CTU level).

可選擇的複數個濾波器(例如到15個或25個為止的濾波器)之係數集合(set)，是在圖片層級被訊號化。再者，係數集合的訊號化並不需要限定於圖片層級，也可以是其他的層級(例如序列層級、片段層級、圖塊層級、CTU層級、CU層級或子區塊層級)。 [框記憶體]A set of coefficients of a plurality of selectable filters (for example, filters up to 15 or 25) is signalized at the picture level. Moreover, the signalization of the coefficient set does not need to be limited to the picture level, but may be other levels (such as sequence level, fragment level, tile level, CTU level, CU level, or sub-block level). [Frame memory]

框記憶體122是用於保存框間預測所用的參照圖片之儲存部，有時也被稱為框緩衝器(frame buffer)。具體來說，框記憶體122會保存已藉由迴路濾波部120而被濾波的再構成區塊。 [框內預測部]The frame memory 122 is a storage unit for storing reference pictures used for inter-frame prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 122 stores the reconstructed blocks that have been filtered by the loop filtering unit 120. [In-frame prediction section]

框內預測部124是參照已保存於區塊記憶體118的當前圖片內之區塊來進行當前區塊的框內預測(也稱為畫面內預測)，藉此生成預測訊號(框內預測訊號)。具體來說，框內預測部124是參照與當前區塊相鄰的區塊之樣本(例如亮度值、色差值)來進行框內預測，藉此生成框內預測訊號，並將框內預測訊號輸出至預測控制部128。The in-frame prediction section 124 refers to the blocks already stored in the current picture of the block memory 118 to perform in-frame prediction (also called in-frame prediction) of the current block, thereby generating a prediction signal (in-frame prediction signal) ). Specifically, the in-frame prediction unit 124 performs in-frame prediction with reference to samples (e.g., luminance values and color difference values) of blocks adjacent to the current block, thereby generating an in-frame prediction signal, and performs in-frame prediction. The signal is output to the prediction control unit 128.

例如，框內預測部124會利用事先規定的複數個框內預測模式之中的1個來進行框內預測。複數個框內預測模式包含1個以上之非方向性預測模式、以及複數個方向性預測模式。For example, the intra-frame prediction unit 124 performs intra-frame prediction using one of a plurality of predetermined intra-frame prediction modes. The plurality of in-frame prediction modes include one or more non-directional prediction modes and a plurality of directional prediction modes.

1個以上的非方向性預測模式包含例如H.265/HEVC(高效率視訊編碼(High-Efficiency Video Coding))規格(非專利文獻1)所規定的平面(Planar)預測模式及DC預測模式。The one or more non-directional prediction modes include, for example, a planar prediction mode and a DC prediction mode defined by the H.265 / HEVC (High-Efficiency Video Coding) specification (Non-Patent Document 1).

複數個方向性預測模式包含例如H.265/HEVC規格所規定的33個方向之預測模式。再者，複數個方向性預測模式亦可除了33個方向以外，更進一步地包含32個方向的預測模式(合計65個方向性預測模式)。圖5是顯示框內預測中的67個框內預測模式(2個非方向性預測模式及65個方向性預測模式)之圖。實線箭頭是表示H.265/HEVC規格所規定的33個方向，虛線箭頭是表示追加的32個方向。The plurality of directional prediction modes include, for example, prediction modes in 33 directions prescribed by the H.265 / HEVC standard. Furthermore, the plurality of directional prediction modes may include, in addition to 33 directions, prediction modes in 32 directions (a total of 65 directional prediction modes). FIG. 5 is a diagram showing 67 intra-frame prediction modes (2 non-directional prediction modes and 65 directional prediction modes) in the intra-frame prediction. The solid arrows indicate the 33 directions defined by the H.265 / HEVC standard, and the dotted arrows indicate the 32 additional directions.

再者，在色差區塊的框內預測中，亦可參照亮度區塊。也就是說，也可以根據當前區塊的亮度成分，來預測當前區塊的色差成分。有時可將這種框內預測稱為CCLM(交叉成分線性模型，cross-component linear model)預測。這種參照亮度區塊的色差區塊之框內預測模式(例如被稱為CCLM模式)，也可以作為1種色差區塊的框內預測模式來加入。Furthermore, in the frame prediction of the color difference block, a luminance block may also be referred to. That is, the color difference component of the current block can also be predicted based on the brightness component of the current block. Such in-frame prediction may be referred to as CCLM (cross-component linear model) prediction. Such an in-frame prediction mode (for example, referred to as a CCLM mode) of a color-difference block that refers to a luminance block may also be added as an in-frame prediction mode of one color-difference block.

框內預測部124亦可根據水平/垂直方向的參照像素之梯度來補正框內預測後的像素值。這種伴隨補正的框內預測有時被稱為PDPC(獨立位置框內預測組合，position dependent intra prediction combination)。顯示有無適用PDPC之資訊(被稱為例如PDPC旗標)，是在例如CU層級被訊號化。再者，此資訊的訊號化並不需要限定於CU層級，也可以是其他的層級(例如序列層級、圖片層級、片段層級、圖塊層級或CTU層級)。 [框間預測部]The intra-frame prediction unit 124 may also correct the pixel values after intra-frame prediction based on the gradient of the reference pixels in the horizontal / vertical direction. Such intra-frame prediction accompanied by correction is sometimes referred to as PDPC (position-dependent intra prediction combination). Information indicating whether or not PDPC is applicable (called, for example, the PDPC flag) is signaled at, for example, the CU level. Furthermore, the signalization of this information does not need to be limited to the CU level, but may be other levels (such as sequence level, picture level, fragment level, tile level, or CTU level). [Inter-frame prediction section]

框間預測部126會參照保存在框記憶體122的參照圖片且也是與當前圖片不同的參照圖片，來進行當前區塊的框間預測(也稱為畫面間預測)，藉此生成預測訊號(框間預測訊號)。框間預測是以當前區塊或當前區塊內的子區塊(例如4x4區塊)之單位來進行。例如，框間預測部126是針對當前區塊或子區塊而在參照圖片內進行運動搜尋(運動估計(motion estimation))。而且，框間預測部126是利用以運動搜尋所得到的運動資訊(例如運動向量)來進行動態補償，藉此生成當前區塊或子區塊的框間預測訊號。而且，框間預測部126會將生成的框間預測訊號輸出至預測控制部128。The inter-frame prediction unit 126 refers to the reference picture stored in the frame memory 122 and is also a reference picture different from the current picture to perform inter-frame prediction (also referred to as inter-frame prediction) of the current block, thereby generating a prediction signal ( Inter-frame prediction signal). Inter-frame prediction is performed in units of a current block or a sub-block (for example, a 4x4 block) within the current block. For example, the inter-frame prediction unit 126 performs motion search (motion estimation) in the reference picture for the current block or subblock. Furthermore, the inter-frame prediction unit 126 performs motion compensation by using motion information (for example, a motion vector) obtained by motion search, thereby generating an inter-frame prediction signal of a current block or a sub-block. Then, the inter-frame prediction unit 126 outputs the generated inter-frame prediction signal to the prediction control unit 128.

使用於動態補償的運動資訊會被訊號化。在運動向量的訊號化中，亦可使用運動向量預測子(motion vector predictor)。也就是說，亦可將運動向量與運動向量預測子之間的差分訊號化。The motion information used for motion compensation will be signaled. In signaling a motion vector, a motion vector predictor may be used. That is, the difference signal between the motion vector and the motion vector predictor may be converted into a signal.

再者，不只是由運動搜尋得到的當前區塊之運動資訊，亦可連相鄰區塊的運動資訊也利用，來生成框間預測訊號。具體來說，亦可將根據由運動搜尋得到的運動資訊之預測訊號、以及根據相鄰區塊的運動資訊之預測訊號作加權相加，藉此以當前區塊內的子區塊單位來生成框間預測訊號。這種框間預測(動態補償)有時被稱為OBMC(重疊區塊動態補償，overlapped block motion compensation)。Furthermore, not only the motion information of the current block obtained from the motion search, but also the motion information of adjacent blocks can also be used to generate the inter-frame prediction signal. Specifically, the prediction signals based on the motion information obtained from the motion search and the prediction signals based on the motion information of the neighboring blocks can also be weighted and added to generate the sub-block units in the current block. Inter-frame prediction signal. Such inter-frame prediction (dynamic compensation) is sometimes referred to as OBMC (overlapped block motion compensation).

在這種OBMC模式中，顯示OBMC用的子區塊之尺寸的資訊(例如被稱為OBMC區塊尺寸)，是在序列層級被訊號化。又，顯示是否適用OBMC模式的資訊(例如被稱為OBMC旗標)，是在CU層級被訊號化。再者，這些資訊的訊號化之層級並不需要限定於序列層級及CU層級，亦可為其他的層級(例如圖片層級、片段層級、圖塊層級、CTU層級或子區塊層級)。In this OBMC mode, information (for example, called OBMC block size) that displays the size of subblocks for OBMC is signaled at the sequence level. The information indicating whether the OBMC mode is applicable (for example, called the OBMC flag) is signaled at the CU level. Furthermore, the level of signalization of these information does not need to be limited to the sequence level and the CU level, but may be other levels (such as the picture level, the fragment level, the tile level, the CTU level, or the sub-block level).

再者，也可以不將運動資訊訊號化，而在解碼裝置側導出。例如，也可以使用H.265/HEVC規格所規定的合併模式(merge mode)。又，亦可藉由例如在解碼裝置側進行運動搜尋來導出運動資訊。在此情況下，可在不使用當前區塊的像素值的情形下進行運動搜尋。In addition, the motion information may not be signaled, but may be derived on the decoding device side. For example, a merge mode defined by the H.265 / HEVC standard may be used. In addition, the motion information may be derived by performing a motion search on the decoding device side, for example. In this case, motion search can be performed without using the pixel values of the current block.

在此，針對在解碼裝置側進行運動搜尋的模式進行說明。有時將該在解碼裝置側進行運動搜尋的模式稱為PMMVD(型樣匹配運動向量導出，pattern matched motion vector derivation)模式、或FRUC(畫面頻率提升，frame rate up-conversion)模式。Here, a mode of motion search on the decoding device side will be described. This mode of motion search on the decoding device side is sometimes referred to as a PMMVD (pattern matched motion vector derivation) mode or a FRUC (frame rate up-conversion) mode.

首先，可參照空間上或時間上與當前區塊相鄰之編碼完成的區塊的運動向量，而生成各自具有運動向量預測子的複數個候選之清單(與合併清單共通亦可)。而且，算出候選清單所包含的各候選之評價值，並根據評價值來選擇1個候選。First, a motion vector of an encoded block adjacent to the current block in space or time may be referred to, and a list of a plurality of candidates each having a motion vector predictor may be generated (commonly with the merged list). Then, an evaluation value of each candidate included in the candidate list is calculated, and one candidate is selected based on the evaluation value.

而且，可根據所選擇的候選之運動向量，來導出當前區塊用的運動向量。具體來說，是例如，將所選擇的候選之運動向量原樣導出作為當前區塊用的運動向量。又，亦可例如，在與所選擇的候選之運動向量相對應的參照圖片內的位置之周邊區域中，藉由進行型樣匹配，來導出當前區塊用的運動向量。Moreover, the motion vector for the current block can be derived based on the selected candidate's motion vector. Specifically, for example, the selected candidate motion vector is derived as it is as a motion vector for the current block. In addition, for example, in a peripheral region of a position in a reference picture corresponding to the selected candidate motion vector, pattern matching is performed to derive a motion vector for the current block.

再者，評價值是藉由與運動向量相對應的參照圖片內的區域、與規定的區域之間的型樣匹配來算出的。The evaluation value is calculated by pattern matching between a region in the reference picture corresponding to the motion vector and a predetermined region.

作為型樣匹配，可使用第1型樣匹配或第2型樣匹配。有時會將第1型樣匹配及第2型樣匹配分別稱為雙向匹配(bilateral matching)及模板匹配(template matching)。As the pattern matching, the first pattern matching or the second pattern matching can be used. The first pattern matching and the second pattern matching are sometimes referred to as bilateral matching and template matching, respectively.

在第1型樣匹配中，是在為不同的2個參照圖片內的2個區塊且也是沿著當前區塊的運動軌跡(motion trajectory)的2個區塊之間進行型樣匹配。從而，在第1型樣匹配中，是使用沿著當前區塊的運動軌跡之其他參照圖片內的區域，來作為上述候選的評價值的算出用之規定的區域。In the first pattern matching, pattern matching is performed between two blocks that are two blocks in two different reference pictures and are also along the motion trajectory of the current block. Therefore, in the first pattern matching, a region in another reference picture along the motion trajectory of the current block is used as a predetermined region for calculating the candidate evaluation value.

圖6是用於說明沿著運動軌跡的2個區塊間的型樣匹配(雙向匹配)之圖。如圖6所示，在第1型樣匹配中，是在為沿著當前區塊(Cur block)的運動軌跡之2個區塊且也是不同的2個參照圖片(Ref0、Ref1)內的2個區塊的配對中，搜尋最匹配的配對，藉此導出2個運動向量(MV0、MV1)。FIG. 6 is a diagram for explaining pattern matching (two-way matching) between two blocks along a motion trajectory. As shown in FIG. 6, in the first type matching, 2 in 2 reference pictures (Ref0, Ref1) that are two blocks along the motion trajectory of the current block (Cur block) are also different. Among the pairs of blocks, the best matching pair is searched to derive 2 motion vectors (MV0, MV1).

在連續的運動軌跡的假設之下，指示2個參照區塊的運動向量(MV0、MV1)會相對於當前圖片(Cur Pic)與2個參照圖片(Ref0、Ref1)之間的時間上之距離(TD0、TD1)而成比例。例如，當前圖片在時間上位於2個參照圖片之間，且從當前圖片到2個參照圖片的時間上之距離為相等的情況下，在第1型樣匹配中，會導出鏡像對稱的雙向之運動向量。Under the assumption of continuous motion trajectories, the motion vectors (MV0, MV1) indicating the two reference blocks will be relative to the time distance between the current picture (Cur Pic) and the two reference pictures (Ref0, Ref1). (TD0, TD1). For example, if the current picture is located between two reference pictures in time, and the time distance from the current picture to the two reference pictures is equal, in the first type matching, a mirror-symmetric two-way Motion vector.

在第2型樣匹配中，是在當前圖片內的模板(在當前圖片內與當前區塊相鄰的區塊(例如上及/或左的相鄰區塊))與參照圖片內的區塊之間進行型樣匹配。從而，在第2型樣匹配中，是使用在當前圖片內之與當前區塊相鄰的區塊，來作為上述候選的評價值的算出用之規定的區域。In the second type matching, the template in the current picture (the block adjacent to the current block in the current picture (such as the upper and / or left adjacent blocks)) and the block in the reference picture Pattern matching between. Therefore, in the second pattern matching, a block adjacent to the current block in the current picture is used as a predetermined area for calculating the candidate evaluation value.

圖7是用於說明在當前圖片內的模板與參照圖片內的區塊之間的型樣匹配(模板匹配)之圖。如圖7所示，在第2型樣匹配中，是藉由在參照圖片(Ref0)內搜尋：與在當前圖片(Cur Pic)內相鄰於當前區塊(Cur block)的區塊最匹配的區塊，以導出當前區塊的運動向量。FIG. 7 is a diagram for explaining pattern matching (template matching) between a template in a current picture and a block in a reference picture. As shown in FIG. 7, in the second type of matching, the search is performed in the reference picture (Ref0): the best match is with the block adjacent to the current block in the current picture (Cur Pic). To derive the motion vector of the current block.

這種顯示是否適用FRUC模式的資訊(例如可稱為FRUC旗標)，是在CU層級被訊號化。又，在適用FRUC模式的情況下(例如FRUC旗標為真的情況下)，可將顯示型樣匹配的方法(第1型樣匹配或第2型樣匹配)之資訊(例如可稱為FRUC模式旗標)在CU層級訊號化。再者，這些資訊的訊號化並不需要限定於CU層級，亦可為其他的層級(例如，序列層級、圖片層級、片段層級、圖塊層級、CTU層級或子區塊層級)。This type of information (for example, the FRUC flag) indicating whether the FRUC mode is applicable is signaled at the CU level. In addition, when the FRUC mode is applied (for example, when the FRUC flag is true), information on the method of pattern matching (first pattern matching or second pattern matching) can be displayed (for example, it can be called FRUC Mode flag) is signaled at the CU level. Furthermore, the signalization of such information does not need to be limited to the CU level, but may be other levels (for example, sequence level, picture level, fragment level, tile level, CTU level, or sub-block level).

再者，也可以藉由與運動搜尋不同的方法，在解碼裝置側導出運動資訊。例如，亦可根據假設了等速直線運動的模型，以像素單位使用周邊像素值來算出運動向量的補正量。Furthermore, the motion information can also be derived on the decoding device side by a method different from the motion search. For example, based on a model that assumes constant-speed linear motion, the correction amount of the motion vector may be calculated using the surrounding pixel values in pixel units.

在此，針對根據假設了等速直線運動的模型來導出運動向量的模式進行說明。有時將此模式稱為BIO(雙向光流，bi-directional optical flow)模式。Here, a mode for deriving a motion vector from a model that assumes a constant velocity linear motion will be described. This mode is sometimes called a BIO (bi-directional optical flow) mode.

圖8是用於說明假設了等速直線運動的模型之圖。在圖8中，(v_x ，v_y )表示速度向量，而τ₀ 、τ₁ 分別表示當前圖片(Cur Pic)與2個參照圖片(Ref₀ ，Ref₁ )之間的時間上之距離。(MVx₀ ，MVy₀ )是表示對應於參照圖片Ref₀ 的運動向量，(MVx₁ ，MVy₁ )是表示對應於參照圖片Ref₁ 的運動向量。FIG. 8 is a diagram for explaining a model in which constant-speed linear motion is assumed. In FIG. 8, (v _x , v _y ) represents the velocity vector, and τ ₀ and τ ₁ represent the temporal distance between the current picture (Cur Pic) and the two reference pictures (Ref ₀ , Ref ₁ ). (MVx ₀ , MVy ₀ ) is a motion vector corresponding to the reference picture Ref ₀ , and (MVx ₁ , MVy ₁ ) is a motion vector corresponding to the reference picture Ref ₁ .

此時，在速度向量(v_x ，v_y )的等速直線運動的假設之下，(MVx₀ ，MVy₀ )及(MVx₁ ，MVy₁ )各自被表示成(v_x τ₀ ，v_y τ₀ )及(-v_x τ₁ ，-v_y τ₁ )，且以下的光流等式(1)成立。 [數學式1] At this time, under the assumption of constant velocity linear motion of the velocity vector (v _x , v _y ), (MVx ₀ , MVy ₀ ) and (MVx ₁ , MVy ₁ ) are each expressed as (v _x τ ₀ , v _y τ ₀ ) and (-v _x τ ₁ , -v _y τ ₁ ), and the following optical flow equation (1) holds. [Mathematical formula 1]

在此，I^(k) 是表示動態補償後的參照圖像k(k=0，1)之亮度值。此光流等式是表示下述的(i)、(ii)、及(iii)之和等於零：(i)亮度值的時間微分、(ii)水平方向的速度及參照圖像的空間梯度之水平成分的積、及(iii)垂直方向的速度及參照圖像的空間梯度之垂直成分的積。根據此光流等式與赫米內插法公式(Hermite interpolation)的組合，可將從合併清單等得到的區塊單位之運動向量以像素單位進行補正。Here, I ^(k) is a luminance value indicating a reference image k (k = 0, 1) after motion compensation. This optical flow equation represents the sum of (i), (ii), and (iii) equal to zero: (i) the time differentiation of the luminance value, (ii) the speed in the horizontal direction, and the spatial gradient of the reference image The product of the horizontal component and (iii) the product of the vertical component of the vertical velocity and the spatial gradient of the reference image. According to the combination of this optical flow equation and Hermite interpolation, the motion vector of the block unit obtained from the combined list and the like can be corrected in pixel units.

再者，亦可藉由與根據假設了等速直線運動的模型之運動向量的導出不同之方法，在解碼裝置側導出運動向量。例如，亦可根據複數個相鄰區塊的運動向量而以子區塊單位來導出運動向量。Furthermore, the motion vector may be derived on the decoding device side by a method different from that of deriving a motion vector from a model that assumes a constant-speed linear motion. For example, the motion vector may be derived in units of sub-blocks based on the motion vectors of a plurality of adjacent blocks.

在此，針對根據複數個相鄰區塊的運動向量而以子區塊單位來導出運動向量的模式進行說明。此模式被稱為仿射動態補償預測(affine motion compensation prediction)模式。Here, a mode for deriving a motion vector in units of sub-blocks based on motion vectors of a plurality of adjacent blocks will be described. This mode is called the affine motion compensation prediction mode.

圖9是用於說明根據複數個相鄰區塊的運動向量之子區塊單位的運動向量的導出之圖。在圖9中，當前區塊包含16個4x4子區塊。在此，是根據相鄰區塊的運動向量來導出當前區塊的左上角控制點之運動向量v₀ ，且根據相鄰子區塊的運動向量來導出當前區塊的右上角控制點之運動向量v₁ 。而且，利用2個運動向量v₀ 及v₁ ，並藉由以下的式(2)，來導出當前區塊內的各子區塊之運動向量(v_x ，v_y )。 [數學式2] FIG. 9 is a diagram for explaining derivation of a motion vector based on a sub-block unit of a motion vector of a plurality of adjacent blocks. In Figure 9, the current block contains 16 4x4 sub-blocks. Here, the motion vector v ₀ of the upper-left corner control point of the current block is derived based on the motion vector of the neighboring block, and the motion of the upper-right corner control point of the current block is derived based on the motion vector of the neighboring sub-block. Vector v ₁ . Furthermore, the motion vectors (v _x , v _y ) of each sub-block in the current block are derived using the two motion vectors v ₀ and v ₁ and the following formula (2). [Mathematical formula 2]

在此，x及y各自表示子區塊的水平位置及垂直位置，且w是表示預定的加權係數。Here, x and y each indicate a horizontal position and a vertical position of a sub-block, and w is a predetermined weighting factor.

在這種仿射動態補償預測模式中，左上及右上角控制點的運動向量之導出方法也可以包含幾個不同的模式。這種顯示仿射動態補償預測模式的資訊(可稱為例如仿射旗標)，是在CU層級被訊號化。再者，顯示此仿射動態補償預測模式的資訊之訊號化並不需要限定於CU層級，也可以是其他的層級(例如序列層級、圖片層級、片段層級、圖塊層級、CTU層級或子區塊層級)。 [預測控制部]In this affine motion compensation prediction mode, the method of deriving the motion vectors of the upper left and upper right control points can also include several different modes. This information (may be called, for example, the affine flag) showing the affine motion-compensated prediction mode is signaled at the CU level. Furthermore, the signalization of the information showing this affine dynamic compensation prediction mode does not need to be limited to the CU level, but may be other levels (such as sequence level, picture level, fragment level, tile level, CTU level, or sub-regions). Block level). [Predictive Control Department]

預測控制部128會選擇框內預測訊號及框間預測訊號的任一個，且將所選擇的訊號作為預測訊號而輸出至減法部104及加法部116。 [解碼裝置的概要]The prediction control unit 128 selects any one of the intra-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal to the subtraction unit 104 and the addition unit 116 as a prediction signal. [Outline of Decoding Device]

接著，針對可對從上述編碼裝置100輸出的編碼訊號(編碼位元流)進行解碼之解碼裝置的概要進行說明。圖10是顯示實施形態1之解碼裝置200的功能構成之方塊圖。解碼裝置200是以區塊單位對動態圖像/圖像進行解碼的動態圖像/圖像解碼裝置。Next, an outline of a decoding device capable of decoding an encoded signal (encoded bit stream) output from the encoding device 100 will be described. FIG. 10 is a block diagram showing a functional configuration of the decoding device 200 according to the first embodiment. The decoding device 200 is a moving image / image decoding device that decodes moving images / images in units of blocks.

如圖10所示，解碼裝置200具備熵解碼部202、逆量化部204、逆轉換部206、加法部208、區塊記憶體210、迴路濾波部212、框記憶體214、框內預測部216、框間預測部218、及預測控制部220。As shown in FIG. 10, the decoding device 200 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse conversion unit 206, an addition unit 208, a block memory 210, a loop filtering unit 212, a frame memory 214, and an in-frame prediction unit 216. , Inter-frame prediction unit 218, and prediction control unit 220.

解碼裝置200可藉由例如通用處理器及記憶體來實現。在此情況下，藉由處理器執行保存在記憶體的軟體程式時，處理器是作為熵解碼部202、逆量化部204、逆轉換部206、加法部208、迴路濾波部212、框內預測部216、框間預測部218、及預測控制部220而發揮功能。又，解碼裝置200也可以是作為對應於熵解碼部202、逆量化部204、逆轉換部206、加法部208、迴路濾波部212、框內預測部216、框間預測部218、及預測控制部220的1個以上之專用的電子電路來實現。The decoding device 200 may be implemented by, for example, a general-purpose processor and a memory. In this case, when the software program stored in the memory is executed by the processor, the processor functions as the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filtering unit 212, and the frame prediction The unit 216, the inter-frame prediction unit 218, and the prediction control unit 220 function. Further, the decoding device 200 may correspond to the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filtering unit 212, the intra-frame prediction unit 216, the inter-frame prediction unit 218, and prediction control The unit 220 is implemented by one or more dedicated electronic circuits.

以下，針對包含在解碼裝置200的各構成要件來進行說明。 [熵解碼部]Hereinafter, each constituent element included in the decoding device 200 will be described. [Entropy decoding section]

熵解碼部202是對編碼位元流進行熵解碼。具體來說，熵解碼部202是例如從編碼位元流對二值訊號進行算術解碼。而且，熵解碼部202會對二值訊號進行多值化(debinarize)。藉此，熵解碼部202會以區塊單位將量化係數輸出至逆量化部204。 [逆量化部]The entropy decoding unit 202 performs entropy decoding on a coded bit stream. Specifically, the entropy decoding unit 202 performs arithmetic decoding on the binary signal from the encoded bit stream, for example. The entropy decoding unit 202 debinarizes the binary signal. As a result, the entropy decoding unit 202 outputs the quantization coefficient to the inverse quantization unit 204 in units of blocks. [Inverse quantization section]

逆量化部204是對來自熵解碼部202的輸入即解碼對象區塊(以下，稱為當前區塊)的量化係數進行逆量化。具體來說，逆量化部204是針對當前區塊的量化係數的每一個，根據對應於該量化係數的量化參數，來對該量化係數進行逆量化。並且，逆量化部204會將當前區塊之已進行逆量化的量化係數(也就是轉換係數)輸出至逆轉換部206。 [逆轉換部]The inverse quantization unit 204 performs inverse quantization on the quantization coefficient of a decoding target block (hereinafter, referred to as a current block) that is an input from the entropy decoding unit 202. Specifically, the inverse quantization unit 204 performs inverse quantization on each quantization coefficient of the current block based on a quantization parameter corresponding to the quantization coefficient. In addition, the inverse quantization unit 204 outputs the quantized coefficients (that is, conversion coefficients) of the current block that have been inversely quantized to the inverse conversion unit 206. [Inverse Conversion Department]

逆轉換部206是藉由對來自逆量化部204的輸入即轉換係數進行逆轉換，以復原預測誤差。The inverse conversion unit 206 restores the prediction error by inversely converting the conversion coefficient that is an input from the inverse quantization unit 204.

在例如已從編碼位元流中解讀出的資訊顯示的是適用EMT或AMT的情況下(例如AMT旗標為真)，逆轉換部206會根據顯示已解讀的轉換類型之資訊，來對當前區塊的轉換係數進行逆轉換。For example, if the information that has been decoded from the encoded bit stream shows that EMT or AMT is applicable (for example, the AMT flag is true), the inverse conversion unit 206 will perform the The conversion coefficient of the block is inversely converted.

又，在例如已從編碼位元流中解讀出的資訊顯示的是適用NSST的情況下，逆轉換部206會對轉換係數適用逆再轉換。 [加法部]If, for example, the information that has been decoded from the encoded bit stream indicates that NSST is applicable, the inverse conversion unit 206 applies inverse reconversion to the conversion coefficient. [Addition Department]

加法部208會對來自逆轉換部206的輸入即預測誤差、及來自預測控制部220的輸入即預測樣本進行加法運算，藉此再構成當前區塊。而且，加法部208會將再構成的區塊輸出到區塊記憶體210及迴路濾波部212。 [區塊記憶體]The addition unit 208 adds the prediction error, which is an input from the inverse conversion unit 206, and the prediction sample, which is an input from the prediction control unit 220, to reconstruct the current block. The addition unit 208 outputs the reconstructed blocks to the block memory 210 and the loop filter unit 212. [Block memory]

區塊記憶體210是用於保存在框內預測中所參照的區塊且也是解碼對象圖片(以下，稱為當前圖片)內的區塊之儲存部。具體來說，區塊記憶體210會保存從加法部208輸出的再構成區塊。 [迴路濾波部]The block memory 210 is a storage unit for storing the blocks referred to in the in-frame prediction and is also a block in a decoding target picture (hereinafter, referred to as a current picture). Specifically, the block memory 210 stores the reconstructed blocks output from the adding unit 208. [Loop Filtering Division]

迴路濾波部212會對藉由加法部208再構成的區塊施行迴路濾波，且將已進行濾波的再構成區塊輸出到框記憶體214及顯示裝置等。The loop filtering unit 212 performs loop filtering on the blocks reconstructed by the adding unit 208, and outputs the filtered reconstructed blocks to the frame memory 214, the display device, and the like.

當從編碼位元流中解讀出的ALF之開啟/關閉的資訊顯示的是ALF之開啟的情況下，可根據局部的梯度之方向及活動性而從複數個濾波器之中選擇1個濾波器，且將所選擇的濾波器適用於再構成區塊。 [框記憶體]When the ALF on / off information read from the encoded bit stream shows that the ALF is on, one filter can be selected from a plurality of filters according to the direction and activity of the local gradient. And apply the selected filter to the reconstructed block. [Frame memory]

框記憶體214是用於保存框間預測所用的參照圖片之儲存部，有時也被稱為框緩衝器(frame buffer)。具體來說，框記憶體214會保存已藉由迴路濾波部212而被濾波的再構成區塊。 [框內預測部]The frame memory 214 is a storage unit for storing reference pictures used for inter-frame prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 214 stores the reconstructed blocks that have been filtered by the loop filtering unit 212. [In-frame prediction section]

框內預測部216是根據已從編碼位元流中解讀出的框內預測模式，並參照保存於區塊記憶體210的當前圖片內之區塊來進行框內預測，藉此生成預測訊號(框內預測訊號)。具體來說，框內預測部216是參照與當前區塊相鄰的區塊之樣本(例如亮度值、色差值)來進行框內預測，藉此生成框內預測訊號，並將框內預測訊號輸出至預測控制部220。The in-frame prediction unit 216 performs in-frame prediction based on the in-frame prediction mode that has been decoded from the encoded bit stream and refers to the blocks stored in the current picture of the block memory 210, thereby generating a prediction signal ( Frame prediction signal). Specifically, the in-frame prediction unit 216 performs in-frame prediction with reference to samples (e.g., luminance values and color difference values) of blocks adjacent to the current block, thereby generating an in-frame prediction signal, and performs intra-frame prediction. The signal is output to the prediction control unit 220.

再者，在色差區塊的框內預測中選擇參照亮度區塊的框內預測模式之情況下，框內預測部216也可以根據當前區塊的亮度成分，來預測當前區塊的色差成分。When the intra-frame prediction mode that refers to the luminance block is selected in the intra-frame prediction of the color difference block, the intra-frame prediction unit 216 may predict the color difference component of the current block based on the luminance component of the current block.

又，當已從編碼位元流中解讀出的資訊顯示的是適用PDPC的情況下，框內預測部216會根據水平/垂直方向的參照像素之梯度來補正框內預測後的像素值。 [框間預測部]In addition, when the information that has been decoded from the encoded bit stream shows that PDPC is applicable, the in-frame prediction unit 216 corrects the pixel values after the in-frame prediction based on the gradient of the reference pixels in the horizontal / vertical direction. [Inter-frame prediction section]

框間預測部218是參照保存於框記憶體214的參照圖片，來預測當前區塊。預測是以當前區塊或當前區塊內的子區塊(例如4x4區塊)之單位來進行。例如，框間預測部218會利用從編碼位元流中解讀出的運動資訊(例如運動向量)來進行動態補償，藉此生成當前區塊或子區塊的框間預測訊號，並將框間預測訊號輸出至預測控制部220。The inter-frame prediction unit 218 refers to a reference picture stored in the frame memory 214 to predict a current block. The prediction is performed in units of the current block or a sub-block (for example, a 4x4 block) within the current block. For example, the inter-frame prediction unit 218 uses motion information (such as a motion vector) decoded from the encoded bit stream to perform dynamic compensation, thereby generating an inter-frame prediction signal for the current block or sub-block, and The prediction signal is output to the prediction control unit 220.

再者，從編碼位元流中解讀出的資訊顯示的是適用OBMC模式的情況下，框間預測部218會使用的不只有藉由運動搜尋所得到的當前區塊之運動資訊，還有相鄰區塊的運動資訊，以生成框間預測訊號。Furthermore, when the information decoded from the coded bit stream shows that the OBMC mode is applicable, the inter-frame prediction unit 218 will use not only the motion information of the current block obtained through motion search, but also the relative information. Motion information of neighboring blocks to generate inter-frame prediction signals.

又，從編碼位元流中解讀出的資訊顯示的是適用FRUC模式的情況下，框間預測部218會依照從編碼流中解讀出的型樣匹配之方法(雙向匹配或模板匹配)來進行運動搜尋，藉此導出運動資訊。並且，框間預測部218會利用已導出的運動資訊來進行動態補償。In addition, when the information decoded from the encoded bit stream shows that the FRUC mode is applied, the inter-frame prediction unit 218 performs the pattern matching method (two-way matching or template matching) decoded from the encoded stream. Sport search to export sport information. Then, the inter-frame prediction unit 218 performs motion compensation using the derived motion information.

又，在適用BIO模式的情況下，框間預測部218會根據假設了等速直線運動的模型來導出運動向量。又，在從編碼位元流中解讀出的資訊顯示的是適用仿射動態補償預測模式的情況下，框間預測部218會根據複數個相鄰區塊的運動向量，以子區塊單位來導出運動向量。 [預測控制部]When the BIO mode is applied, the inter-frame prediction unit 218 derives a motion vector from a model that assumes a constant-speed linear motion. In addition, when the information decoded from the coded bit stream shows that the affine motion compensation prediction mode is applied, the inter-frame prediction unit 218 uses sub-block units based on the motion vectors of a plurality of adjacent blocks. Export the motion vector. [Predictive Control Department]

預測控制部220會選擇框內預測訊號及框間預測訊號的任一個，且將所選擇的訊號作為預測訊號而輸出至加法部208。 [畫面間預測處理]The prediction control unit 220 selects one of the intra-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal to the addition unit 208 as a prediction signal. [Inter-screen prediction processing]

圖11是本實施形態之編碼方法中的畫面間預測處理的流程圖。圖1所示的編碼裝置100在對以複數個圖片所構成的動態圖像進行編碼時，編碼裝置100的框間預測部126等是執行圖11所示的處理。FIG. 11 is a flowchart of inter-frame prediction processing in the encoding method according to this embodiment. When the encoding device 100 shown in FIG. 1 encodes a moving image composed of a plurality of pictures, the inter-frame prediction unit 126 and the like of the encoding device 100 execute the processing shown in FIG. 11.

框間預測部126會對包含於處理對象圖片的複數個區塊之每一個區塊，重複進行步驟S101~S108的處理。在此，區塊是指對象圖片被分割而成的處理單位，且是稱為例如編碼單元、預測區塊、或預測單元的圖像區域。再者，區塊也可以是這些處理單位被分割而成的子區塊。The inter-frame prediction unit 126 repeats the processing of steps S101 to S108 for each of the plurality of blocks included in the processing target picture. Here, a block refers to a processing unit into which a target picture is divided, and is an image region called, for example, a coding unit, a prediction block, or a prediction unit. Furthermore, a block may be a sub-block into which these processing units are divided.

首先，框間預測部126是從處理完成區塊的運動向量中導出複數個候選運動向量(S101)。具體來說，框間預測部126是從處理完成區塊的動態補償所用之運動向量中導出處理對象區塊的候選運動向量。First, the inter-frame prediction unit 126 derives a plurality of candidate motion vectors from the motion vectors of the processed block (S101). Specifically, the inter-frame prediction unit 126 derives a candidate motion vector of a processing target block from a motion vector used for dynamic compensation of a processing-completed block.

更具體來說，框間預測部126亦可將於處理完成區塊的動態補償所用的運動向量導出作為處理對象區塊的候選運動向量。又，框間預測部126亦可依規定的比率來將於處理完成區塊的動態補償所用的運動向量縮放(scaling)，且將已縮放的運動向量導出作為處理對象區塊的候選運動向量。More specifically, the inter-frame prediction unit 126 may also derive, as a candidate motion vector of the processing target block, the motion vector used for the dynamic compensation of the processed block. In addition, the inter-frame prediction unit 126 may scale the motion vector used for dynamic compensation of the processed block according to a predetermined ratio, and derive the scaled motion vector as a candidate motion vector of the processing target block.

在此，處理完成區塊是在處理對象區塊被處理之前已被處理的子區塊，且可以表示為先行區塊。例如，處理完成子區塊亦可為已進行動態補償的區塊，亦可為編碼完成或解碼完成區塊。Here, the processing-completed block is a sub-block that has been processed before the processing-target block is processed, and can be expressed as a preceding block. For example, the processed sub-block may also be a block that has been dynamically compensated, and may also be a coding-completed or decoded-completed block.

又，例如，用於導出處理對象區塊的候選運動向量之處理完成區塊，是藉由處理對象區塊的位置來特定。具體來說，用於導出處理對象區塊的候選運動向量之處理完成區塊，亦可為在空間上或時間上相鄰於處理對象區塊的處理完成區塊。In addition, for example, the processing completion block for deriving the candidate motion vector of the processing target block is specified by the position of the processing target block. Specifically, the processing completion block for deriving the candidate motion vector of the processing target block may also be a processing completion block that is spatially or temporally adjacent to the processing target block.

又，框間預測部126亦可從複數個處理完成區塊的複數個運動向量中，導出處理對象區塊的候選運動向量。也就是說，框間預測部126亦可藉由組合複數個處理完成區塊的複數個運動向量，以導出處理對象區塊的候選運動向量。又，框間預測部126亦可從複數個處理完成區塊的複數個運動向量中導出處理對象區塊的複數個候選運動向量。Further, the inter-frame prediction unit 126 may also derive a candidate motion vector of the processing target block from the plurality of motion vectors of the plurality of processed blocks. That is, the inter-frame prediction unit 126 can also derive a candidate motion vector of a processing target block by combining a plurality of motion vectors of a plurality of processed completed blocks. In addition, the inter-frame prediction unit 126 may also derive a plurality of candidate motion vectors of the processing target block from a plurality of motion vectors of the plurality of processed blocks.

接著，框間預測部126是從複數個候選運動向量之中選擇最終運動向量。在此，框間預測部126在處理對象區域與處理完成區域當中是僅參照處理完成區域。也就是說，框間預測部126並不參照處理對象區塊中的處理對象區域之圖像，而是參照複數個處理完成區塊中的處理完成區域之再構成圖像，來選擇處理對象區塊的最終運動向量。Next, the inter-frame prediction unit 126 selects a final motion vector from among a plurality of candidate motion vectors. Here, the inter-frame prediction unit 126 refers to only the processing-completed region among the processing target region and the processing-completed region. That is, the inter-frame prediction unit 126 does not refer to the images of the processing target area in the processing target block, but refers to the reconstructed images of the processing completed areas in the plurality of processing completed blocks to select the processing target area. The final motion vector of the block.

具體來說，框間預測部126是利用雙向FRUC方式，並參照藉由複數個候選運動向量所指示的複數個候選區域之再構成圖像等，而按每個候選運動向量來算出評價值。接著，框間預測部126會從複數個候選運動向量當中選擇評價值為最高的候選運動向量作為最終運動向量。Specifically, the inter-frame prediction unit 126 calculates an evaluation value for each candidate motion vector by using a bidirectional FRUC method and referring to the reconstructed images of the plurality of candidate regions indicated by the plurality of candidate motion vectors. Next, the inter-frame prediction unit 126 selects the candidate motion vector with the highest evaluation value from the plurality of candidate motion vectors as the final motion vector.

首先，說明處理對象圖片為B圖片的情況(S102的B圖片)的動作。圖12是用於說明此情況的評價值之算出方法的圖。First, the operation when the processing target picture is a B picture (B picture in S102) will be described. FIG. 12 is a diagram for explaining a method of calculating an evaluation value in this case.

在此例子中，處理對象圖片為B圖片，且具有第1參照圖片清單與第2參照圖片清單之2個參照圖片清單。在第1參照圖片清單及第2參照圖片清單的每一清單中，均登錄有2張處理完成圖片。又，評價對象之候選運動向量所指示的是第1參照圖片清單的0號之參照圖片(第1參照圖片)。In this example, the processing target picture is a B picture and has two reference picture lists of a first reference picture list and a second reference picture list. In each of the first reference picture list and the second reference picture list, two processed pictures are registered. The candidate motion vector of the evaluation target indicates the reference picture No. 0 (the first reference picture) in the first reference picture list.

框間預測部126是利用第1參照圖片、及第2參照圖片來算出評價值。在此，第2參照圖片是包含在圖片記憶體(例如框記憶體122或214)內之可參照的圖片中的登錄於與第1參照圖片不同的圖片清單(第2參照圖片清單)的圖片當中，顯示時間最接近於處理對象圖片的圖片。又，第1參照圖片及第2參照圖片雙方都是顯示時間比處理對象圖片更前面的圖片。The inter-frame prediction unit 126 calculates an evaluation value using the first reference picture and the second reference picture. Here, the second reference picture is a picture registered in a picture list (second reference picture list) different from the first reference picture among the reference pictures included in the picture memory (for example, the frame memory 122 or 214). Among them, the picture whose display time is closest to the processing target picture. Both the first reference picture and the second reference picture are pictures whose display time is earlier than the processing target picture.

框間預測部126是相對於處理對象區塊而導出候選區域與對稱區域的差分值，其中該候選區域是以評價對象之候選運動向量所指定的第1參照圖片之同等位置上的再構成圖像，該對稱區域是以對稱運動向量所指定的第2參照圖片之同等位置上的再構成圖像。例如，處理對象區塊的形狀及大小，與候選區域及對稱區域的形狀及大小是相同的。The inter-frame prediction unit 126 derives a difference value between a candidate region and a symmetric region with respect to a processing target block, where the candidate region is a reconstruction map at an equivalent position of the first reference picture specified by the candidate motion vector of the evaluation target. This symmetrical area is a reconstructed image at the same position as the second reference picture specified by the symmetrical motion vector. For example, the shape and size of the processing target block are the same as those of the candidate area and the symmetrical area.

又，框間預測部126是藉由以規定的比率來縮放候選運動向量，以導出對稱運動向量。在此，規定的比率亦可為從處理對象圖片的顯示順序(顯示時間)減去第2參照圖片的顯示順序所得到的值，相對於從處理對象圖片的顯示順序減去第1參照圖片的顯示順序所得到的值之比率。顯示順序可以由POC(圖片順序計數，Picture Order Count)來表示。The inter-frame prediction unit 126 scales the candidate motion vector by a predetermined ratio to derive a symmetric motion vector. Here, the predetermined ratio may be a value obtained by subtracting the display order of the second reference picture from the display order (display time) of the processing target picture, as compared to the value obtained by subtracting the first reference picture from the display order of the processing target picture. Displays the ratio of the values in the sequence. The display order can be represented by a POC (Picture Order Count).

又，框間預測部126可藉由差分絕對值和(SAD)或差分平方和(SSD)等，來導出候選區域的再構成圖像及對稱區域的再構成圖像之差分值。Further, the inter-frame prediction unit 126 may derive a difference value between the reconstructed image of the candidate region and the reconstructed image of the symmetric region by using a difference absolute value sum (SAD) or a difference squared sum (SSD).

框間預測部126是利用已得到的差分值來算出評價值。例如，若差分值越小，框間預測部126會算出越高的評價值。再者，框間預測部126除了差分值以外，亦可還利用其以外的資訊來算出評價值。The inter-frame prediction unit 126 calculates an evaluation value using the obtained difference value. For example, if the difference value is smaller, the inter-frame prediction unit 126 calculates a higher evaluation value. In addition, the inter-frame prediction unit 126 may calculate the evaluation value using information other than the difference value.

圖13是顯示另外的動作例之圖。再者，參照圖片清單的構成等與圖12是同樣的。在圖13所示的例子中，評價對象之候選運動向量所指示的是第2參照圖片清單的0號之參照圖片(第1參照圖片)。FIG. 13 is a diagram showing another operation example. The configuration and the like of the reference picture list are the same as those of FIG. 12. In the example shown in FIG. 13, the candidate motion vector of the evaluation target indicates the reference picture No. 0 (the first reference picture) of the second reference picture list.

與上述同樣地，框間預測部126是將包含在圖片記憶體內之可參照的圖片中的登錄於與第1參照圖片不同的圖片清單(第1參照圖片清單)的圖片當中，顯示時間最接近於處理對象圖片的圖片，選擇作為第2參照圖片。據此，在此情況下，是將在顯示時間順序上位於處理對象圖片與第1參照圖片之間的第1參照圖片清單之0號的參照圖片，選擇作為第2參照圖片。又，與上述同樣地，第1參照圖片及第2參照圖片雙方都是顯示時間比處理對象圖片更前面的圖片。In the same way as above, the inter-frame prediction unit 126 registers the reference pictures included in the picture memory in a picture list different from the first reference picture (the first reference picture list), and the display time is closest. The picture to be processed is selected as the second reference picture. Accordingly, in this case, the reference picture No. 0 of the first reference picture list located between the processing target picture and the first reference picture in the display time order is selected as the second reference picture. In the same manner as described above, both the first reference picture and the second reference picture are pictures whose display time is earlier than the processing target picture.

又，藉由與上述同樣的手法，框間預測部126可特定出候選區域、對稱運動向量、及對稱區域，並根據候選區域及對稱區域的差分值來算出評價值。再者，在圖13所示的例子中，由於第2參照圖片在顯示時間順序上是位於比第1參照圖片更後面，因此對稱運動向量是指示候選運動向量的反方向。In addition, by the same method as described above, the inter-frame prediction unit 126 can identify a candidate region, a symmetrical motion vector, and a symmetrical region, and calculate an evaluation value based on the difference between the candidate region and the symmetrical region. Furthermore, in the example shown in FIG. 13, since the second reference picture is located later than the first reference picture in the display time sequence, the symmetric motion vector indicates the opposite direction of the candidate motion vector.

藉由這種處理，框間預測部126可算出複數個候選運動向量的每一個的評價值。With this processing, the inter-frame prediction unit 126 can calculate an evaluation value for each of the plurality of candidate motion vectors.

接著，框間預測部126會將複數個候選運動向量之中評價值成為最佳的值(例如評價值最高)的候選運動向量選擇作為最終運動向量。Next, the inter-frame prediction unit 126 selects the candidate motion vector whose evaluation value is the best value (for example, the evaluation value is the highest) among the plurality of candidate motion vectors as the final motion vector.

像這樣，框間預測部126是利用在第1參照圖片清單及第2參照圖片清單中顯示的2張參照圖片，來選擇最終運動向量(S104)。再者，在圖12及圖13所示的例子中，雖然是利用參照圖片清單的0號之參照圖片來作為第1參照圖片及第2參照圖片，但亦可利用0號以外的參照圖片。In this way, the inter-frame prediction unit 126 selects the final motion vector using the two reference pictures displayed in the first reference picture list and the second reference picture list (S104). In the examples shown in FIGS. 12 and 13, although the reference picture No. 0 of the reference picture list is used as the first reference picture and the second reference picture, reference pictures other than 0 may be used.

接著，框間預測部126是利用候選區域與對稱區域之雙方來進行動態補償以生成預測圖像，其中該候選區域是最終運動向量所指示的第1參照圖片之區域，該對稱區域是對稱運動向量所指示的第2參照圖片之區域(S105)。Next, the inter-frame prediction unit 126 performs dynamic compensation using both candidate regions and symmetrical regions to generate a predicted image, where the candidate regions are regions of the first reference picture indicated by the final motion vector, and the symmetrical regions are symmetrical motions. The area of the second reference picture indicated by the vector (S105).

接著，說明處理對象圖片為P圖片的情況之動作。圖14是顯示處理對象圖片為P圖片(S102的P圖片)，且在處理對象圖片的第1參照圖片清單中登錄有2張處理完成圖片的情況(S103的是)的動作例之圖。評價對象之候選運動向量所指示的是第1參照圖片清單的1號之參照圖片(第1參照圖片)。Next, an operation when the processing target picture is a P picture will be described. FIG. 14 is a diagram showing an example of the operation when the processing target picture is a P picture (P picture of S102), and two processed pictures are registered in the first reference picture list of the processing target picture (Yes in S103). The candidate motion vector of the evaluation target indicates the reference picture No. 1 (the first reference picture) in the first reference picture list.

在此情況下，框間預測部126也是將包含在圖片記憶體內之可參照的圖片中的與第1參照圖片不同的圖片當中，顯示時間最接近於處理對象圖片的圖片，選擇作為第2參照圖片。據此，在此情況下，是將第1參照圖片清單的0號之參照圖片選擇作為第2參照圖片。又，與上述同樣地，第1參照圖片及第2參照圖片雙方都是顯示時間比處理對象圖片更前面的圖片。In this case, the inter-frame prediction unit 126 also selects a picture whose display time is closest to the processing target picture among pictures that are different from the first reference picture in the reference pictures included in the picture memory. image. Accordingly, in this case, the reference picture No. 0 of the first reference picture list is selected as the second reference picture. In the same manner as described above, both the first reference picture and the second reference picture are pictures whose display time is earlier than the processing target picture.

再者，在圖14中，在圖片記憶體內之可參照圖片中，雖然包含有第1參照圖片清單中所包含的圖片以外的圖片，但這些圖片亦可不包含在可參照圖片中。又，在圖片記憶體內之可參照圖片中，包含有第1參照圖片清單中所包含的圖片以外之圖片的情況下，框間預測部126亦可將包含在第1參照圖片清單中的與第1參照圖片不同的圖片當中，顯示時間最接近於處理對象圖片的圖片，選擇作為第2參照圖片。亦即，框間預測部126亦可在圖片記憶體內之可參照圖片當中，從包含於第1參照圖片清單的圖片中選擇第2參照圖片。In addition, in FIG. 14, the reference pictures in the picture memory include pictures other than the pictures included in the first reference picture list, but these pictures may not be included in the reference pictures. In addition, when the reference pictures in the picture memory include pictures other than the pictures included in the first reference picture list, the inter-frame prediction unit 126 may compare the first reference picture list with the first reference picture list. 1 Among the different reference pictures, the picture whose display time is closest to the processing target picture is selected as the second reference picture. That is, the inter-frame prediction unit 126 may select the second reference picture from the pictures included in the first reference picture list among the reference pictures in the picture memory.

又，藉由與上述同樣的手法，框間預測部126可特定出候選區域、對稱運動向量、及對稱區域，並根據候選區域及對稱區域的差分值來算出評價值。In addition, by the same method as described above, the inter-frame prediction unit 126 can identify a candidate region, a symmetrical motion vector, and a symmetrical region, and calculate an evaluation value based on the difference between the candidate region and the symmetrical region.

藉由這種處理，框間預測部126可算出複數個候選運動向量的每一個的評價值。接著，框間預測部126會將複數個候選運動向量之中評價值成為最佳的值(例如評價值最高)的候選運動向量選擇作為最終運動向量。With this processing, the inter-frame prediction unit 126 can calculate an evaluation value for each of the plurality of candidate motion vectors. Next, the inter-frame prediction unit 126 selects the candidate motion vector whose evaluation value is the best value (for example, the evaluation value is the highest) among the plurality of candidate motion vectors as the final motion vector.

像這樣，框間預測部126是利用第1參照圖片清單中所顯示的2張參照圖片，來選擇最終運動向量(S106)。再者，在圖14所示的例子中，雖然是利用第1參照圖片清單的1號之參照圖片來作為第1參照圖片，但亦可利用1號以外的參照圖片。In this way, the inter-frame prediction unit 126 selects the final motion vector using the two reference pictures displayed in the first reference picture list (S106). In the example shown in FIG. 14, although the reference picture No. 1 in the first reference picture list is used as the first reference picture, reference pictures other than No. 1 may be used.

接著，框間預測部126是利用最終運動向量所指示的第1參照圖片的候選區域來進行動態補償，以生成預測圖像(S107)。亦即，框間預測部126是進行不利用對稱區域的動態補償。再者，框間預測部126亦可與B圖片同樣地，藉由利用候選區域與對稱區域之雙方來進行動態補償，以生成預測圖像。Next, the inter-frame prediction unit 126 performs motion compensation using the candidate region of the first reference picture indicated by the final motion vector to generate a predicted image (S107). That is, the inter-frame prediction unit 126 performs motion compensation without using a symmetric region. In addition, the inter-frame prediction unit 126 may generate a predicted image by performing motion compensation using both the candidate region and the symmetrical region in the same manner as the B picture.

接著，說明處理對象圖片為P圖片(S102的P圖片)，且在處理對象圖片的第1參照圖片清單中僅登錄有1張處理完成圖片的情況(S103的否)之動作。圖15是顯示此情況的動作例之圖。在此例子中，評價對象之候選運動向量所指示的是第1參照圖片清單的0號之參照圖片(第1參照圖片)。Next, an operation in a case where the processing target picture is a P picture (P picture in S102) and only one processed picture is registered in the first reference picture list of the processing target picture (No in S103). FIG. 15 is a diagram showing an operation example in this case. In this example, the candidate motion vector of the evaluation target indicates the reference picture No. 0 (the first reference picture) of the first reference picture list.

在此情況下，框間預測部126也是將包含在圖片記憶體內之可參照的圖片中的與第1參照圖片不同的圖片當中，顯示時間最接近於處理對象圖片的圖片，選擇作為第2參照圖片。又，與上述同樣地，第1參照圖片及第2參照圖片雙方都是顯示時間比處理對象圖片更前面的圖片。In this case, the inter-frame prediction unit 126 also selects a picture whose display time is closest to the processing target picture among pictures that are different from the first reference picture in the reference pictures included in the picture memory. image. In the same manner as described above, both the first reference picture and the second reference picture are pictures whose display time is earlier than the processing target picture.

但是，在此情況下，第2參照圖片是未登錄於第1參照圖片清單的圖片。據此，編碼裝置100及解碼裝置200會將此第2參照圖片作為可參照的圖片而保存在圖片記憶體中。例如，編碼裝置100會生成包含指定此第2參照圖片的資訊之編碼位元流。解碼裝置200是藉由參照該資訊，而將第2參照圖片作為可參照的圖片來保存於圖片記憶體中。例如，上述資訊是作為與參照圖片清單不同的資訊，而保存於編碼位元流的標頭區域等。例如，可將此資訊保存於序列層、圖片層、及片段層的任一個的標頭區域中。However, in this case, the second reference picture is a picture that is not registered in the first reference picture list. Accordingly, the encoding device 100 and the decoding device 200 store the second reference picture in the picture memory as a referenceable picture. For example, the encoding device 100 generates an encoded bit stream containing information specifying the second reference picture. The decoding device 200 refers to this information, and stores the second reference picture as a reference picture in the picture memory. For example, the above information is stored in a header area of the encoded bit stream as information different from the reference picture list. For example, this information can be stored in the header area of any of the sequence layer, the picture layer, and the clip layer.

再者，在圖15中，雖然在圖片記憶體內之可參照圖片中，包含有第1參照圖片清單中所包含的圖片以外的複數張圖片，但只要包含有第1參照圖片清單中所包含的圖片以外的1張以上的圖片即可。例如，此1張以上的圖片是包含與第1參照圖片不同且顯示時間比處理對象圖片更前面的圖片圖片當中，顯示時間最接近於處理對象圖片的圖片。Furthermore, in FIG. 15, although the reference pictures in the picture memory include a plurality of pictures other than the pictures included in the first reference picture list, as long as the pictures included in the first reference picture list are included, One or more images other than the image may be sufficient. For example, the one or more pictures include pictures that are different from the first reference picture and that are displayed earlier than the processing target picture, and whose display time is closest to the processing target picture.

又，藉由與上述同樣的手法，框間預測部126可特定出候選區域、對稱運動向量、及對稱區域，而根據候選區域及對稱區域的差分值來算出評價值。In addition, by the same method as described above, the inter-frame prediction unit 126 can identify the candidate region, the symmetric motion vector, and the symmetric region, and calculate an evaluation value based on the difference between the candidate region and the symmetric region.

像這樣，框間預測部126是利用在第1參照圖片清單所顯示的1張參照圖片、及未顯示在第1參照圖片清單中的其他圖片，來選擇最終運動向量(S108)。In this manner, the inter-frame prediction unit 126 selects the final motion vector using one reference picture displayed in the first reference picture list and other pictures not displayed in the first reference picture list (S108).

接著，框間預測部126是利用最終運動向量所指示的第1參照圖片的候選區域來進行動態補償，以生成預測圖像(S107)。亦即，框間預測部126是進行不利用對稱區域的動態補償。再者，框間預測部126亦可與B圖片同樣地，藉由利用候選區域與對稱區域之雙方來進行動態補償，以生成測圖像。Next, the inter-frame prediction unit 126 performs motion compensation using the candidate region of the first reference picture indicated by the final motion vector to generate a predicted image (S107). That is, the inter-frame prediction unit 126 performs motion compensation without using a symmetric region. In addition, the inter-frame prediction unit 126 may generate motion images by performing motion compensation using both candidate regions and symmetrical regions in the same manner as the B picture.

再者，選擇了上述所說明的最終運動向量之後，框間預測部126亦可進一步地利用同樣的方法來對以最終運動向量表示的區域之周邊區域仔細地搜尋，將最終運動向量補正成可指示出評價值成為更佳的值之區域，並利用補正後的最終運動向量來進行動態補償。In addition, after the final motion vector described above is selected, the inter-frame prediction unit 126 can further use the same method to carefully search the surrounding area of the area represented by the final motion vector, and correct the final motion vector to be possible. A region where the evaluation value becomes a better value is indicated, and the final motion vector after correction is used for dynamic compensation.

又，圖11所示的處理亦可不以區塊單位，而是以將區塊進一步地分割而成的子區塊單位來進行。又，框間預測部126亦可組合區塊單位的處理與子區塊單位的處理，而以2階段的方式來進行處理。The processing shown in FIG. 11 may be performed not in block units, but in sub-block units in which blocks are further divided. Further, the inter-frame prediction unit 126 may combine the processing in a block unit and the processing in a sub-block unit, and perform the processing in a two-stage manner.

又，在上述中，雖然顯示了第1參照圖片及第2參照圖片的顯示時間都比處理對象圖片的顯示時間更前面的例子，但是第1參照圖片及第2參照圖片的顯示時間亦可都是比處理對象圖片的顯示時間更後面。In the above, although the examples in which the display time of the first reference picture and the second reference picture are longer than the display time of the processing target picture are shown, the display time of the first reference picture and the second reference picture may be both. It is later than the display time of the processing target picture.

又，圖11所示的處理是在編碼裝置100及解碼裝置200之雙方來進行。The processing shown in FIG. 11 is performed by both the encoding device 100 and the decoding device 200.

如以上，本實施形態之編碼裝置100或解碼裝置200，是從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量(S101)，且不使用處理對象區塊的圖像區域，而是參照顯示時間位於比處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從複數個候選運動向量中選擇最終運動向量(S104、S106或S108)，並利用最終運動向量來進行處理對象區塊的動態補償(S105或S107)。As described above, the encoding device 100 or the decoding device 200 of this embodiment derives a plurality of candidate motion vectors for the processing target block included in the processing target picture from the motion vectors used for the dynamic compensation of the processing completed block (S101 ), And does not use the image area of the processing target block, but selects a plurality of candidate motion vectors by referring to 2 processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture. The final motion vector (S104, S106, or S108), and the final motion vector is used to perform dynamic compensation of the processing target block (S105 or S107).

又，處理對象圖片為P圖片(S102的P圖片)。藉此，由於可以對P圖片利用雙向FRUC，因此可以在更多的情況中利用雙向FRUC方式。據此，可以改善編碼效率。The processing target picture is a P picture (P picture in S102). Thereby, since the bidirectional FRUC can be used for the P picture, the bidirectional FRUC method can be used in more cases. Accordingly, encoding efficiency can be improved.

又，在用於處理對象圖片的參照圖片清單中顯示有2張參照圖片的情況下(S103的是)，2張處理完成的參照圖片是顯示在參照圖片清單中的2張參照圖片。When two reference pictures are displayed in the reference picture list for the processing target picture (YES in S103), the two processed reference pictures are two reference pictures displayed in the reference picture list.

又，在用於處理對象圖片的參照圖片清單中顯示有1張參照圖片的情況下(S103中的否)，2張處理完成的參照圖片的其中一張是顯示於參照圖片清單中的1張參照圖片，2張處理完成的參照圖片之另一張是未顯示在參照圖片清單中的其他處理完成的參照圖片。When one reference picture is displayed in the reference picture list for the processing target picture (No in S103), one of the two processed reference pictures is one displayed in the reference picture list. The reference picture, the other of the two processed reference pictures is another processed reference picture that is not displayed in the reference picture list.

又，編碼裝置100或解碼裝置200是將顯示於參照圖片清單的1張參照圖片、及其他處理完成的參照圖片保存於圖片記憶體中。In addition, the encoding device 100 or the decoding device 200 stores one reference picture displayed in the reference picture list and other processed reference pictures in a picture memory.

又，在最終運動向量的選擇(S108)中，是參照2張處理完成的參照圖片，來算出複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇最終運動向量，2張處理完成參照圖片的其中一張是複數個候選運動向量當中以處理對象的候選運動向量所指示的參照圖片，2張處理完成參照圖片的另一張是可參照的圖片當中，顯示時間最接近於處理對象圖片的圖片。Moreover, in the selection of the final motion vector (S108), two reference pictures that have been processed are used to calculate the respective evaluation values of the plurality of candidate motion vectors, and the final motion vector is selected based on the calculated evaluation values. 2 One of the processed reference pictures is the reference picture indicated by the candidate motion vector of the processing target among the plurality of candidate motion vectors, and the other of the processed reference pictures is the reference picture with the closest display time Used to process the target picture.

又，處理對象圖片為B圖片(S102的B圖片)。藉此，在B圖片中，即使在可參照的圖片中，未存在有在顯示順序上包夾處理對象區塊的2個圖片之情況下，仍可以利用雙向FRUC。據此，由於可以在更多的情況中利用雙向FRUC方式，因此可以改善編碼效率。The processing target picture is a B picture (B picture in S102). Thus, in the B picture, even if there are no two pictures that sandwich the processing target block in the display order among the reference pictures, the bidirectional FRUC can be used. Accordingly, since the bidirectional FRUC method can be used in more cases, the coding efficiency can be improved.

又，在最終運動向量的選擇(S104)中，是參照2張處理完成的參照圖片，來算出複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇最終運動向量。處理對象圖片(B圖片)具有第1參照圖片清單及第2參照圖片清單。2張處理完成參照圖片的其中一張是複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且是屬於第1參照圖片清單的參照圖片。2張處理完成參照圖片的另一張是在屬於第2參照圖片清單的參照圖片當中，顯示時間最接近於處理對象圖片的圖片。 (實施形態2)Moreover, in the selection of the final motion vector (S104), the evaluation values of the plurality of candidate motion vectors are calculated by referring to the two reference pictures that have been processed, and the final motion vector is selected based on the calculated evaluation values. The processing target picture (B picture) includes a first reference picture list and a second reference picture list. One of the two processed reference pictures is a reference picture represented by the candidate motion vector of the processing target among the plurality of candidate motion vectors, and is a reference picture belonging to the first reference picture list. The other of the two processed reference pictures is a picture whose display time is closest to the processing target picture among the reference pictures belonging to the second reference picture list. (Embodiment 2)

在本實施形態中，是針對畫面間預測處理的另外的形態來說明。在本實施形態中，是對算出評價值的候選運動向量進行篩減。藉此，可以減低處理量。In this embodiment, another aspect of the inter-screen prediction process will be described. In this embodiment, the candidate motion vector for which the evaluation value is calculated is filtered. This can reduce the amount of processing.

圖16是本實施形態之編碼方法中的畫面間預測處理的流程圖。圖1所示的編碼裝置100在對以複數個圖片所構成的動態圖像進行編碼時，編碼裝置100的框間預測部126等是執行圖16所示的處理。FIG. 16 is a flowchart of inter-frame prediction processing in the encoding method according to this embodiment. When the encoding device 100 shown in FIG. 1 encodes a moving image composed of a plurality of pictures, the inter-frame prediction unit 126 and the like of the encoding device 100 execute the processing shown in FIG. 16.

又，處理對象圖片是B圖片，且具有第1參照圖片清單及第2參照圖片清單。The processing target picture is a B picture and includes a first reference picture list and a second reference picture list.

首先，框間預測部126是將第1參照圖片清單、及第2參照圖片清單的其中一個選擇作為選擇參照圖片清單(S201)。再者，此處理的詳細內容容後描述。First, the inter-frame prediction unit 126 selects one of the first reference picture list and the second reference picture list as a selected reference picture list (S201). The details of this processing will be described later.

接著，編碼裝置100(例如熵編碼部110)是對用於特定選擇參照圖片清單之選擇參照圖片清單資訊進行編碼(S202)。亦即，編碼裝置100是生成包含選擇參照圖片清單資訊的編碼位元流。例如，可將選擇參照圖片清單資訊保存於序列層、圖片層、及片段層之任一個的標頭區域中。例如，選擇參照圖片清單資訊是1位元的旗標，且第1值表示第1參照圖片清單為選擇參照圖片清單，第2值表示第2參照圖片清單為選擇參照圖片清單。Next, the encoding device 100 (for example, the entropy encoding unit 110) encodes the selected reference picture list information for specifying the selected reference picture list (S202). That is, the encoding device 100 generates an encoded bit stream including the information of the selected reference picture list. For example, the selection reference picture list information may be stored in a header area of any of a sequence layer, a picture layer, and a clip layer. For example, the selected reference picture list information is a one-bit flag, and the first value indicates that the first reference picture list is a selected reference picture list, and the second value indicates that the second reference picture list is a selected reference picture list.

接著，框間預測部126會以區塊單位來進行以下的處理。首先，框間預測部126是從處理完成區塊的運動向量中導出複數個候選運動向量(S203)。再者，此處理的詳細內容與實施形態1的步驟S101是同樣的。Next, the inter-frame prediction unit 126 performs the following processing on a block basis. First, the inter-frame prediction unit 126 derives a plurality of candidate motion vectors from the motion vectors of the processed block (S203). The details of this process are the same as those of step S101 in the first embodiment.

接著，框間預測部126是藉由進行複數個候選運動向量的篩減，來決定1個以上的評價對象運動向量(S204)。接著，框間預測部126是算出評價對象運動向量的各自的評價值，且將評價值成為最佳的值之評價對象運動向量選擇作為最終運動向量(S205)。再者，作為評價值的算出方法，可以利用例如與實施形態1同樣的手法。但是，在本實施形態中，框間預測部126亦可利用顯示時間比處理對象圖片更前面的圖片及更後面的圖片之2張處理完成圖片來算出評價值。Next, the inter-frame prediction unit 126 determines the one or more evaluation target motion vectors by filtering out a plurality of candidate motion vectors (S204). Next, the inter-frame prediction unit 126 calculates each evaluation value of the evaluation target motion vector, and selects the evaluation target motion vector whose evaluation value becomes the optimal value as the final motion vector (S205). As a method of calculating the evaluation value, for example, the same method as in the first embodiment can be used. However, in the present embodiment, the inter-frame prediction unit 126 may calculate the evaluation value by using two processed pictures that are displayed earlier than the picture to be processed and the later pictures.

最後，框間預測部126是利用已選擇的最終運動向量來進行動態補償，以生成預測圖像(S206)。Finally, the inter-frame prediction unit 126 performs motion compensation using the selected final motion vector to generate a predicted image (S206).

在複數個候選運動向量的篩減(S204)中，框間預測部126是對複數個候選運動向量的每一個，判定包含該候選運動向量所指示的參照圖片之參照圖片清單，是否與在步驟S201中所決定的選擇參照圖片清單一致(S204A)。亦即，框間預測部126是對判定對象之候選運動向量所表示的參照圖片是否包含於選擇參照圖片清單中進行判定。框間預測部126在該等清單並不一致的情況下(S204A的否)，會將判定對象之候選運動向量從評價對象運動向量中排除(S204B)。又，框間預測部126在該等清單為一致的情況下(S204A的是)，會將判定對象之候選運動向量決定為評價對象運動向量。亦即，框間預測部126是將複數個候選運動向量當中，可指示包含於選擇參照圖片清單中的參照圖片之1個以上的候選運動向量，決定為評價對象運動向量。In the reduction of the plurality of candidate motion vectors (S204), the inter-frame prediction unit 126 determines, for each of the plurality of candidate motion vectors, whether a reference picture list including a reference picture indicated by the candidate motion vector is the same as in step The selection reference pictures determined in S201 agree (S204A). That is, the inter-frame prediction unit 126 determines whether the reference picture indicated by the candidate motion vector of the determination target is included in the selected reference picture list. When the inter-frame prediction unit 126 does not agree with these lists (No in S204A), the candidate motion vector of the determination target is excluded from the evaluation target motion vector (S204B). When the lists match (S204A: YES), the inter-frame prediction unit 126 determines the candidate motion vector of the determination target as the evaluation target motion vector. That is, the inter-frame prediction unit 126 determines one or more candidate motion vectors among a plurality of candidate motion vectors that can be included in the reference picture list in the selected reference picture list, and determines them as the evaluation target motion vectors.

藉由以上，由於可以抑制為了選擇最終運動向量而必須進行評價的候選運動向量的數量，因此可以減少處理量。As described above, the number of candidate motion vectors that must be evaluated in order to select the final motion vector can be suppressed, so that the amount of processing can be reduced.

接著，針對決定選擇參照圖片清單的方法(S201)進行說明。圖17是對選擇參照圖片清單進行決定處理(S201)之流程圖。Next, a method (S201) for determining a selection of a reference picture list will be described. FIG. 17 is a flowchart of a process of determining a selection reference picture list (S201).

首先，框間預測部126是擷取圖片Pic1與圖片Pic2作為代表圖片，其中該圖片Pic1是登錄於第1參照圖片清單的開頭(0號)的圖片，該圖片Pic2是登錄於第2參照圖片清單的開頭(0號)的圖片(S211)。First, the inter-frame prediction unit 126 captures a picture Pic1 and a picture Pic2 as representative pictures. The picture Pic1 is a picture registered at the beginning (No. 0) of the first reference picture list, and the picture Pic2 is registered in the second reference picture Picture of the beginning (No. 0) of the list (S211).

接著，框間預測部126是比較處理對象圖片與第1代表圖片(Pic1)的顯示時間之差、及處理對象圖片與第2代表圖片(Pic2)的顯示時間之差，而將包含與處理對象圖片的顯示時間之差較少的代表圖片之參照圖片清單決定為選擇參照圖片清單(S212、S213及S214)。Next, the inter-frame prediction unit 126 compares the display time difference between the processing target picture and the first representative picture (Pic1) and the display time difference between the processing target picture and the second representative picture (Pic2), and includes the processing target picture and the processing target picture. The reference picture list of the representative pictures with a small difference in the display time of the pictures is determined as the selected reference picture list (S212, S213, and S214).

在顯示時間之差為相同的情況下，框間預測部126會比較第1代表圖片與第2代表圖片在解碼時的優先順序，並將包含優先順序較高的代表圖片之參照圖片清單決定為選擇參照圖片清單(S215、S216及S217)。在此，解碼時的優先順序是指例如時域可調能力(temporal scalability)編號。又，顯示時間之差為相同的情況，是指例如第1代表圖片及第2代表圖片存在於處理對象圖片的前後，且顯示時間之差為相同的情況。When the difference in display time is the same, the inter-frame prediction unit 126 compares the priority order of the first representative picture and the second representative picture when decoding, and determines the reference picture list including the representative picture with the higher priority as Select a reference picture list (S215, S216, and S217). Here, the priority at the time of decoding refers to, for example, a temporal scalability number. The case where the difference in display time is the same means, for example, the case where the first representative picture and the second representative picture exist before and after the processing target picture, and the difference in display time is the same.

在優先順序為相同的情況下，框間預測部126會比較第1代表圖片與第2代表圖片的量化寬度，且將包含量化寬度較小的代表圖片之參照圖片清單決定為選擇參照圖片清單(S218~S220)。When the priority order is the same, the inter-frame prediction unit 126 compares the quantization widths of the first representative picture and the second representative picture, and determines the reference picture list including the representative picture with a smaller quantization width as the selected reference picture list ( S218 ~ S220).

再者，框間預測部126亦可在此處已說明的3個判定當中僅進行一部分的判定，亦可更換這些判定的順序。又，框間預測部126亦可添加上述以外的判定來進行判定。In addition, the inter-frame prediction unit 126 may perform only a part of the three determinations described here, and may change the order of these determinations. In addition, the inter-frame prediction unit 126 may add a determination other than the above to perform the determination.

接著，說明本實施形態之解碼裝置200的動作。圖18是本實施形態之解碼方法中的畫面間預測處理的流程圖。再者，步驟S302~S305的處理與圖16所示之步驟S203~S206的處理是同樣的。Next, the operation of the decoding device 200 according to this embodiment will be described. FIG. 18 is a flowchart of inter-frame prediction processing in the decoding method according to this embodiment. The processing in steps S302 to S305 is the same as the processing in steps S203 to S206 shown in FIG. 16.

解碼裝置200的框間預測部218是將第1參照圖片清單、及第2參照圖片清單的其中一個選擇作為選擇參照圖片清單(S301)。The inter-frame prediction unit 218 of the decoding device 200 selects one of the first reference picture list and the second reference picture list as a selected reference picture list (S301).

圖19是對解碼處理中的選擇參照圖片清單進行決定處理(S301)之流程圖。FIG. 19 is a flowchart of a determination process (S301) of the selection reference picture list in the decoding process.

首先，解碼裝置200(例如熵解碼部202)是從編碼位元流中取得選擇參照圖片清單資訊(S311)。例如，解碼裝置200可從編碼位元流的序列層、圖片層、及片段層的任一個的標頭區域中，將選擇參照圖片清單資訊解碼。First, the decoding device 200 (for example, the entropy decoding unit 202) obtains the selected reference picture list information from the encoded bit stream (S311). For example, the decoding device 200 may decode the selected reference picture list information from a header area of any of a sequence layer, a picture layer, and a slice layer of the encoded bit stream.

框間預測部218是因應於選擇參照圖片清單資訊的值，而將第1參照圖片清單與第2參照圖片清單的其中一個決定為選擇參照圖片清單。例如，框間預測部218在選擇參照圖片清單資訊顯示第1值的情況下(S312的第1值)，是將第1參照圖片清單決定為選擇參照圖片清單(S313)。框間預測部218在選擇參照圖片清單資訊顯示第2值的情況下(S312的第2值)，是將第2參照圖片清單決定為選擇參照圖片清單(S314)。The inter-frame prediction unit 218 determines one of the first reference picture list and the second reference picture list as the selected reference picture list in response to the value of the selected reference picture list information. For example, when the inter-frame prediction unit 218 displays the first reference picture list information (the first value in S312), the inter-frame prediction unit 218 determines the first reference picture list as the selected reference picture list (S313). When the inter-frame prediction unit 218 displays the second value in the selected reference picture list information (the second value in S312), it determines the second reference picture list as the selected reference picture list (S314).

再者，解碼裝置200在步驟S301中，亦可取代圖19所示的處理，而進行與在編碼裝置100所進行的圖17所示之處理相同的處理。在此情況下，編碼裝置100亦可不將選擇參照圖片清單資訊包含於編碼位元流中。藉此，可以減少編碼位元流的資料量。In addition, the decoding device 200 may perform the same processing as that shown in FIG. 17 performed by the encoding device 100 instead of the processing shown in FIG. 19 in step S301. In this case, the encoding device 100 may not include the selection reference picture list information in the encoding bit stream. This can reduce the amount of data in the encoded bit stream.

如以上，本實施形態之編碼裝置100或解碼裝置200，是從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，其中該處理對象圖片是具有2個參照圖片清單的B圖片(S203或S302)，且不使用處理對象區塊的圖像區域，而是算出複數個候選運動向量當中表示參照圖片的候選運動向量之評價值，其中該參照圖片是在2個參照圖片清單的其中一個即選擇參照圖片清單中所顯示的參照圖片，並根據已算出的評價值，從表示前述選擇參照圖片清單中所顯示的參照圖片之候選運動向量中，選擇最終運動向量(S204及S205、或S303及S304)，而利用最終運動向量來進行處理對象區塊的動態補償(S206或S305)。As described above, the encoding device 100 or the decoding device 200 of this embodiment derives a plurality of candidate motion vectors for the processing target block included in the processing target picture from the motion vectors used for the dynamic compensation of the processed block. The processing target picture is a B picture (S203 or S302) with two reference picture lists, and does not use the image area of the processing target block, but calculates an evaluation of the candidate motion vectors representing the reference picture among the plurality of candidate motion vectors. Value, where the reference picture is one of the two reference picture lists, that is, the reference picture displayed in the reference picture list is selected, and according to the calculated evaluation value, the reference picture displayed in the selected reference picture list is displayed. Among the candidate motion vectors, the final motion vector (S204 and S205, or S303 and S304) is selected, and the final motion vector is used to perform dynamic compensation of the processing target block (S206 or S305).

藉此，由於可以使算出評價值的對象之候選運動向量變少，因此可以減低處理量。又，在2個參照圖片清單中所顯示的參照圖片大多是對稱的情況。據此，即使在將評價對象篩減為僅指示在1個參照圖片清單中所顯示的參照圖片之候選運動向量的情況下，也不會有精度大幅下降的情形。亦即，本實施形態的手法可以抑制精度的降低，並且減低處理量。Thereby, the number of candidate motion vectors of the object for which the evaluation value is calculated can be reduced, so that the amount of processing can be reduced. In addition, the reference pictures displayed in the two reference picture lists are often symmetrical. Accordingly, even when the evaluation target is narrowed down to the candidate motion vector indicating only the reference picture displayed in the one reference picture list, the accuracy may not be significantly reduced. That is, the method of this embodiment can suppress a decrease in accuracy and reduce a processing amount.

又，編碼裝置100或解碼裝置200可更進一步地將2個參照圖片清單當中，登錄於開頭的參照圖片的顯示時間較接近於處理對象圖片的顯示時間之參照圖片清單，決定為選擇參照圖片清單(S212~S214)。藉此，可以擷取更有用的候選運動向量。In addition, the encoding device 100 or the decoding device 200 may further select the reference picture list in which the display time of the reference picture registered at the beginning of the two reference picture lists is closer to the display time of the processing target picture, and select the reference picture list. (S212 ~ S214). Thereby, more useful candidate motion vectors can be captured.

又，編碼裝置100或解碼裝置200可更進一步地將2個參照圖片清單當中，登錄於開頭的參照圖片被設定成更優先地解碼的參照圖片清單，決定為選擇參照圖片清單(S215~S217)。藉此，可以擷取更有用的候選運動向量。In addition, the encoding device 100 or the decoding device 200 may further set the reference picture list that is registered at the top of the two reference picture lists as the reference picture list to be decoded with higher priority, and decide to select the reference picture list (S215 to S217). . Thereby, more useful candidate motion vectors can be captured.

又，編碼裝置100或解碼裝置200可更進一步地將2個參照圖片清單當中，登錄於開頭的參照圖片之量化寬度較小的參照圖片清單，決定為前述選擇參照圖片清單(S218~S220)。藉此，可以擷取更有用的候選運動向量。In addition, the encoding device 100 or the decoding device 200 may further determine the reference picture list having the smaller quantization width of the first reference picture among the two reference picture lists as the selected reference picture list (S218 to S220). Thereby, more useful candidate motion vectors can be captured.

又，編碼裝置100更進一步地生成包含用於特定選擇參照圖片清單的資訊之編碼位元流(S202)。又，解碼裝置200更進一步地取得用於特定包含於編碼位元流中的選擇參照圖片清單之資訊(S311)，並利用該資訊來特定選擇參照圖片清單(S312~S314)。藉此，由於在解碼裝置200中，亦可不進行決定選擇運動向量的處理，因此可以減低解碼裝置200的處理量。 [編碼裝置的組裝例]Furthermore, the encoding device 100 further generates an encoded bit stream including information for specifying a reference picture list (S202). Further, the decoding device 200 further obtains information for specifying a selected reference picture list included in the encoded bit stream (S311), and uses the information to specify the selected reference picture list (S312 to S314). This makes it possible to reduce the processing amount of the decoding device 200 because the decoding device 200 does not need to perform the process of determining the selection of the motion vector. [Assembly example of encoding device]

圖20是顯示實施形態1或實施形態2之編碼裝置100的組裝例之方塊圖。編碼裝置100具備電路160及記憶體162。例如，圖1所示之編碼裝置100的複數個構成要件是藉由圖20所示之電路160及記憶體162來組裝。FIG. 20 is a block diagram showing an assembly example of the encoding device 100 according to the first embodiment or the second embodiment. The encoding device 100 includes a circuit 160 and a memory 162. For example, a plurality of constituent elements of the encoding device 100 shown in FIG. 1 are assembled by the circuit 160 and the memory 162 shown in FIG. 20.

電路160是進行資訊處理的電路，且是可對記憶體162進行存取的電路。例如，電路160是對圖像資訊進行編碼之專用或通用的電子電路。電路160亦可為CPU之類的處理器。又，電路160亦可為複數個電子電路的集合體。又，例如，電路160亦可在圖1所示之編碼裝置100的複數個構成要件當中，發揮除了用於儲存資訊的構成要件之外的複數個構成要件之作用。The circuit 160 is a circuit that performs information processing, and is a circuit that can access the memory 162. For example, the circuit 160 is a dedicated or general-purpose electronic circuit that encodes image information. The circuit 160 may also be a processor such as a CPU. The circuit 160 may be an assembly of a plurality of electronic circuits. In addition, for example, the circuit 160 may also play a role of plural constituent elements other than the constituent elements for storing information among the plural constituent elements of the encoding device 100 shown in FIG. 1.

記憶體162是可儲存電路160用於對圖像資訊進行編碼的資訊之通用或專用的記憶體。記憶體162亦可為電子電路，亦可連接至電路160。又，亦可將記憶體162包含在電路160中。又，記憶體162亦可為複數個電子電路的集合體。又，記憶體162亦可為磁碟或光碟等，亦可表現為儲存器或記錄媒體等。又，記憶體162亦可為非揮發性記憶體，亦可為揮發性記憶體。The memory 162 is a general-purpose or dedicated memory that can store information used by the circuit 160 to encode image information. The memory 162 may also be an electronic circuit, and may also be connected to the circuit 160. The memory 162 may be included in the circuit 160. The memory 162 may be an aggregate of a plurality of electronic circuits. In addition, the memory 162 may be a magnetic disk, an optical disk, or the like, and may also be expressed as a storage or a recording medium. The memory 162 may be a non-volatile memory or a volatile memory.

例如，在記憶體162中亦可儲存欲編碼的圖像資訊，亦可儲存對應於已編碼的圖像資訊之位元串。又，在記憶體162中亦可儲存有電路160對圖像資訊進行編碼用的程式。For example, the memory 162 may also store image information to be encoded, and a bit string corresponding to the encoded image information may also be stored. A program for encoding the image information by the circuit 160 may be stored in the memory 162.

又，例如，電路160亦可在圖1所示之編碼裝置100的複數個構成要件當中，發揮用於儲存資訊的構成要件之作用。具體來說，記憶體162亦可發揮圖1所示之區塊記憶體118及框記憶體122的作用。In addition, for example, the circuit 160 may also play a role of a constituent element for storing information among a plurality of constituent elements of the encoding device 100 shown in FIG. 1. Specifically, the memory 162 can also function as the block memory 118 and the frame memory 122 shown in FIG. 1.

再者，在編碼裝置100中，亦可不組裝圖1等所示之複數個構成要件的全部，且亦可不進行上述複數個處理的全部。圖1等所示之複數個構成要件的一部分亦可包含在其他的裝置中，且上述複數個處理的一部分亦可藉由其他的裝置來執行。並且，在編碼裝置100中，能夠組裝圖1等所示之複數個構成要件當中的一部分，且進行上述複數個處理的一部分，藉此抑制處理的延遲。 [解碼裝置的組裝例]Further, in the encoding device 100, all of the plurality of constituent elements shown in FIG. 1 and the like may not be assembled, and all of the plurality of processes described above may not be performed. A part of the plurality of constituent elements shown in FIG. 1 and the like may be included in other devices, and a part of the plurality of processes described above may be executed by other devices. In addition, in the encoding device 100, a part of the plurality of constituent elements shown in FIG. 1 and the like can be assembled, and a part of the plurality of processes described above can be performed, thereby suppressing processing delay. [Assembly example of decoding device]

圖21是顯示實施形態1或實施形態2之解碼裝置200的組裝例之方塊圖。解碼裝置200具備電路260及記憶體262。例如，圖10所示之解碼裝置200的複數個構成要件是藉由圖21所示之電路260及記憶體262來組裝。FIG. 21 is a block diagram showing an assembly example of the decoding device 200 according to the first embodiment or the second embodiment. The decoding device 200 includes a circuit 260 and a memory 262. For example, a plurality of constituent elements of the decoding device 200 shown in FIG. 10 are assembled by a circuit 260 and a memory 262 shown in FIG. 21.

電路260是進行資訊處理的電路，且是可對記憶體262進行存取的電路。例如，電路260是對圖像資訊進行解碼之通用或專用的電子電路。電路260亦可為CPU之類的處理器。又，電路260亦可為複數個電子電路的集合體。又，例如，電路260亦可在圖10所示之解碼裝置200的複數個構成要件當中，發揮除了用於儲存資訊的構成要件之外的複數個構成要件之作用。The circuit 260 is a circuit that performs information processing, and is a circuit that can access the memory 262. For example, the circuit 260 is a general-purpose or dedicated electronic circuit that decodes image information. The circuit 260 may also be a processor such as a CPU. The circuit 260 may be an assembly of a plurality of electronic circuits. In addition, for example, the circuit 260 may play a role of a plurality of constituent elements other than the constituent elements for storing information among the plurality of constituent elements of the decoding device 200 shown in FIG. 10.

記憶體262是儲存電路260用於對圖像資訊進行解碼的資訊之通用或專用的記憶體。記憶體262亦可為電子電路，亦可連接至電路260。又，亦可將記憶體262包含在電路260中。又，記憶體262亦可為複數個電子電路的集合體。又，記憶體262亦可為磁碟或光碟等，亦可表現為儲存器或記錄媒體等。又，記憶體262亦可為非揮發性記憶體，亦可為揮發性記憶體。The memory 262 is a general-purpose or dedicated memory storing information used by the circuit 260 to decode image information. The memory 262 may also be an electronic circuit, and may also be connected to the circuit 260. The memory 262 may be included in the circuit 260. The memory 262 may be an aggregate of a plurality of electronic circuits. In addition, the memory 262 may be a magnetic disk, an optical disk, or the like, and may also be expressed as a memory or a recording medium. The memory 262 may be a non-volatile memory or a volatile memory.

例如，在記憶體262中亦可儲存對應於已編碼的圖像資訊之位元串，亦可儲存對應於已解碼的位元串之圖像資訊。又，在記憶體262中亦可儲存有電路260用於對圖像資訊進行解碼的程式。For example, a bit string corresponding to the encoded image information may also be stored in the memory 262, and image information corresponding to the decoded bit string may also be stored. In addition, a program for the circuit 260 to decode image information may be stored in the memory 262.

又，例如，電路260亦可在圖10所示之解碼裝置200的複數個構成要件當中，發揮用於儲存資訊的構成要件之作用。具體來說，記憶體262亦可發揮圖10所示之區塊記憶體210及框記憶體214的作用。In addition, for example, the circuit 260 may also play a role of a constituent element for storing information among a plurality of constituent elements of the decoding device 200 shown in FIG. 10. Specifically, the memory 262 can also function as the block memory 210 and the frame memory 214 shown in FIG. 10.

再者，在解碼裝置200中，亦可不組裝圖10等所示之複數個構成要件的全部，且亦可不進行上述複數個處理的全部。圖10等所示之複數個構成要件的一部分亦可包含在其他的裝置中，且上述複數個處理的一部分亦可藉由其他的裝置來執行。並且，在解碼裝置200中，能夠組裝圖10等所示之複數個構成要件當中的一部分，且進行上述複數個處理的一部分，藉此抑制處理的延遲。In addition, in the decoding device 200, all of the plurality of constituent elements shown in FIG. 10 and the like may not be assembled, and all of the plurality of processes described above may not be performed. A part of the plurality of constituent elements shown in FIG. 10 and the like may be included in other devices, and a part of the plurality of processes described above may be executed by other devices. In addition, in the decoding device 200, a part of the plurality of constituent elements shown in FIG. 10 and the like can be assembled, and a part of the plurality of processes described above can be performed, thereby suppressing processing delay.

以上，雖然說明了本實施形態之編碼裝置及解碼裝置，但本揭示並非限定於此實施形態。Although the encoding device and the decoding device according to this embodiment have been described above, the present disclosure is not limited to this embodiment.

又，包含於上述實施形態之編碼裝置及解碼裝置的各處理部典型上是作為積體電路即LSI而實現。這些可以個別地集成為1個晶片，亦可以由包含一部分或全部的方式來集成為1個晶片。Each processing unit included in the encoding device and the decoding device of the embodiment is typically implemented as an LSI, which is an integrated circuit. These can be integrated into one chip individually, or can be integrated into one chip by including a part or all of them.

又，積體電路化並不限於LSI，亦可利用專用電路或通用處理器來實現。亦可利用在LSI製造後可程式設計的FPGA(Field Programmable Gate Array(現場可程式閘陣列))、或可再構成LSI內部之電路電池的連接或設定之可重組態處理器(reconfigurable processor)。The integrated circuit is not limited to LSI, and can be realized by a dedicated circuit or a general-purpose processor. Field Programmable Gate Array (FPGA), which can be programmed after LSI manufacturing, or a reconfigurable processor (reconfigurable processor) that can reconnect or configure the circuit battery inside the LSI .

在上述各實施形態中，各構成要件可由專用之硬體來構成，亦可藉由執行適合於各構成要件之軟體程式來實現。各構成要件亦可藉由CPU或處理器等程式執行部將已記錄於硬碟或半導體記憶體等記錄媒體的軟體程式讀取並執行來實現。In each of the embodiments described above, each constituent element may be constituted by dedicated hardware, or may be implemented by executing a software program suitable for each constituent element. Each constituent element can also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

換言之，編碼裝置及解碼裝置具備處理電路(processing circuitry)、及電連接於該處理電路之(可由該處理電路存取的)儲存裝置(storage)。處理電路包含專用的硬體及程式執行部之至少其中一種。又，在處理電路包含程式執行部的情形下，儲存裝置會儲存可藉由該程式執行部執行之軟體程式。處理電路是利用儲存裝置而於上述實施形態中執行編碼方法或解碼方法。In other words, the encoding device and the decoding device include processing circuitry and a storage device (accessible by the processing circuit) electrically connected to the processing circuit. The processing circuit includes at least one of dedicated hardware and a program execution unit. When the processing circuit includes a program execution unit, the storage device stores a software program that can be executed by the program execution unit. The processing circuit executes an encoding method or a decoding method in the above-mentioned embodiment using a storage device.

此外，本揭示亦可為上述軟體程式，亦可為已記錄有上述程式之非暫時的電腦可讀取的記錄媒體。又，上述程式當然可以透過網際網路等傳送媒體而流通。In addition, the present disclosure may also be the aforementioned software program or a non-transitory computer-readable recording medium on which the aforementioned program has been recorded. It is needless to say that the above-mentioned programs can be distributed through transmission media such as the Internet.

又，上述所使用之數字，全部都是為了具體地說明本揭示而例示的數字，本揭示並不受例示之數字所限制。In addition, the numbers used above are all numbers exemplified in order to specifically explain the present disclosure, and the present disclosure is not limited by the numbers exemplified.

又，方塊圖中的功能方塊之分割僅為一例，亦可將複數個功能方塊作為一個功能方塊來實現、或將一個功能方塊分割為複數個、或將一部分之功能移至其他的功能方塊。又，亦可以並列或時間分割的方式使單一的硬體或軟體處理具有類似的功能的複數個功能方塊之功能。In addition, the division of the functional blocks in the block diagram is only an example, and a plurality of functional blocks may be implemented as one functional block, or one functional block may be divided into a plurality, or a part of the functions may be moved to other functional blocks. In addition, a single hardware or software can also be used to process the functions of multiple function blocks with similar functions in a parallel or time division manner.

又，執行包含於上述編碼方法或解碼方法之步驟的順序，是為了具體地說明本揭示而例示之順序，為上述以外的順序亦可。又，上述步驟之一部分亦可與其他的步驟同時(並列)執行。The order in which the steps included in the encoding method or the decoding method are executed is the order exemplified in order to specifically explain the present disclosure, and the order other than the above may be used. In addition, part of the above steps may be performed simultaneously (parallel) with other steps.

以上，針對本揭示的一個或複數個態樣之編碼裝置、解碼裝置、編碼方法及解碼方法，雖然根據實施形態進行了說明，但本揭示並不限定於該等實施形態。只要不脫離本揭示之主旨，將本發明所屬技術領域中具有通常知識者可設想得到之各種變形施行於本實施形態者、或組合不同的實施形態中的構成要件而建構之形態，均可包含於本揭示的一個或複數個態樣的範圍內。 (實施形態3)Although the encoding device, decoding device, encoding method, and decoding method according to one or more aspects of the present disclosure have been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as it does not deviate from the gist of the present disclosure, various modifications that can be conceived by a person having ordinary skill in the technical field to which the present invention pertains can be implemented in this embodiment, or a structure constructed by combining the constituent elements in different embodiments, can include Within the scope of one or more aspects of the present disclosure. (Embodiment 3)

在以上之各實施形態中，功能方塊的每一個通常可藉由MPU及記憶體等來實現。又，功能方塊的每一個所進行之處理，通常是藉由使處理器等程式執行部將已記錄於ROM等記錄媒體的軟體(程式)讀出並執行來實現。該軟體可藉由下載等來發布，亦可記錄於半導體記憶體等記錄媒體來發布。再者，當然也可以藉由硬體(專用電路)來實現各功能方塊。In each of the above embodiments, each of the functional blocks can usually be realized by an MPU, a memory, or the like. In addition, the processing performed by each of the functional blocks is usually realized by causing a program execution unit such as a processor to read out and execute software (program) recorded in a recording medium such as a ROM. The software can be distributed by downloading or the like, and can also be distributed by recording on a recording medium such as a semiconductor memory. Moreover, of course, each functional block can also be realized by hardware (dedicated circuit).

又，在各實施形態中所說明的處理，亦可藉由利用單一的裝置(系統)而集中處理來實現、或者亦可藉由利用複數個裝置而分散處理來實現。又，執行上述程式的處理器可為單個，亦可為複數個。亦即，可進行集中處理、或者亦可進行分散處理。In addition, the processing described in each embodiment may be realized by centralized processing using a single device (system), or may be realized by distributed processing using a plurality of devices. In addition, the processor executing the program may be a single processor or a plurality of processors. That is, centralized processing may be performed, or distributed processing may be performed.

本發明不受以上之實施例所限定，可進行各種的變更，且該等變更亦包含於本發明之範圍內。The present invention is not limited by the above embodiments, and various changes can be made, and these changes are also included in the scope of the present invention.

在此，更進一步地說明上述各實施形態所示之動態圖像編碼方法(圖像編碼方法)或動態圖像解碼方法(圖像解碼方法)的應用例及利用其之系統。該系統之特徵在於具有利用圖像編碼方法之圖像編碼裝置、利用圖像解碼方法之圖像解碼裝置、及具備兩者之圖像編碼解碼裝置。針對系統中的其他構成，可以因應於情況而適當地變更。 [使用例]Here, application examples of the moving image encoding method (image encoding method) or the moving image decoding method (image decoding method) shown in the above-mentioned embodiments and the system using the same will be further described. This system is characterized by having an image encoding device using an image encoding method, an image decoding device using an image decoding method, and an image encoding and decoding device having both. The other components in the system can be appropriately changed in accordance with the situation. [Example of use]

圖22是顯示實現內容發送服務(content delivery service)的內容供給系統ex100之整體構成的圖。將通訊服務的提供地區分割成所期望的大小，且在各格區(cell)內分別設置固定無線電台即基地台ex106、ex107、ex108、ex109、ex110。FIG. 22 is a diagram showing the overall configuration of a content supply system ex100 that implements a content delivery service. The area where communication services are provided is divided into desired sizes, and fixed radio stations, ie base stations ex106, ex107, ex108, ex109, ex110, are respectively set in each cell.

在此內容供給系統ex100中，可透過網際網路服務提供者ex102或通訊網ex104、及基地台ex106~ex110，將電腦ex111、遊戲機ex112、相機ex113、家電ex114、及智慧型手機ex115等各機器連接到網際網路ex101。該內容供給系統ex100亦可形成為組合並連接上述任一要件。亦可在不透過作為固定無線電台之基地台ex106~ex110的情況下，將各機器透過電話網或近距離無線等直接或間接地相互連接。又，串流伺服器(streaming server)ex103，是透過網際網路ex101等而與電腦ex111、遊戲機ex112、相機ex113、家電ex114、及智慧型手機ex115等各機器相連接。又，串流伺服器ex103是透過衛星ex116而與飛機ex117內之熱點(hot spot)內的終端等連接。In this content supply system ex100, computers ex111, game consoles ex112, cameras ex113, home appliances ex114, and smart phones ex115 can be connected to each other through the Internet service provider ex102 or the communication network ex104, and the base stations ex106 to ex110. Connected to the internet ex101. The content supply system ex100 may be formed to combine and connect any of the above-mentioned requirements. The devices can also be directly or indirectly connected to each other through a telephone network or near-field wireless without passing through base stations ex106 to ex110, which are fixed radio stations. The streaming server ex103 is connected to various devices such as a computer ex111, a game machine ex112, a camera ex113, a home appliance ex114, and a smartphone ex115 through the Internet ex101 and the like. The streaming server ex103 is connected to a terminal or the like in a hot spot in the aircraft ex117 through the satellite ex116.

再者，亦可取代基地台ex106~ex110 ，而使用無線存取點或熱點等。又，串流伺服器ex103可在不透過網際網路ex101或網際網路服務提供者ex102的情形下直接與通訊網ex104連接，亦可在不透過衛星ex116的情形下直接與飛機ex117連接。Furthermore, it can also replace the base stations ex106 ~ ex110, and use wireless access points or hotspots. In addition, the streaming server ex103 may be directly connected to the communication network ex104 without using the Internet ex101 or the Internet service provider ex102, or may be directly connected to the aircraft ex117 without using the satellite ex116.

相機ex113是數位相機等可進行靜態圖攝影及動態圖攝影之機器。又，智慧型手機ex115可為對應於一般稱作2G、3G、3.9G、4G、還有今後被稱為5G的移動通訊系統之方式的智慧型電話機、便攜電話機、或者PHS(Personal Handyphone System(個人手持電話系統))等。The camera ex113 is a digital camera and the like that can perform still image photography and motion image photography. The smart phone ex115 may be a smart phone, a portable phone, or a PHS (Personal Handyphone System (PHS)) that corresponds to a method commonly referred to as a 2G, 3G, 3.9G, 4G, and 5G mobile communication system. Personal Handy Phone System)) etc.

家電ex118可為冰箱、或包含於家庭用燃料電池汽電共生系統(cogeneration system)之機器等。The home appliance ex118 may be a refrigerator or a device included in a domestic fuel cell gas-electricity co-generation system.

在內容供給系統ex100中，具有攝影功能之終端是透過基地台ex106等而連接到串流伺服器ex103，藉此使實況(live)即時傳遞等變得可行。在實況即時傳遞中，終端(電腦ex111、遊戲機ex112、相機ex113、家電ex114、智慧型手機ex115、及飛機ex117內的終端等)會對使用者利用該終端所攝影之靜態圖或動態圖內容進行在上述各實施形態所說明的編碼處理，並對藉由編碼而得到的影像資料、及對應於影像的聲音被編碼而成的聲音資料進行多工化，來將所獲得的資料傳送至串流伺服器ex103。亦即，各終端是作為本發明的一個態樣的圖像編碼裝置而發揮功能。In the content supply system ex100, a terminal having a photographing function is connected to the streaming server ex103 through a base station ex106 or the like, thereby making live transmission and the like feasible. In live transmission, the terminal (computer ex111, game console ex112, camera ex113, home appliances ex114, smart phone ex115, and terminal in aircraft ex117, etc.) will display the content of the static or dynamic pictures taken by the user using the terminal The encoding processing described in each of the above embodiments is performed, and the image data obtained by encoding and the audio data obtained by encoding the sound corresponding to the image are multiplexed to transmit the obtained data to the string. Stream server ex103. That is, each terminal functions as an image encoding device according to one aspect of the present invention.

另一方面，串流伺服器ex103會進行內容資料之串流發送，該內容資料即是對有要求之客戶端(client)傳送的內容資料。客戶端是指可將已經過上述編碼處理之資料解碼的電腦ex111、遊戲機ex112、相機ex113、家電ex114、智慧型手機ex115、及飛機ex117內之終端等。已接收到所發送之資料的各機器會將所接收到之資料解碼處理並播放。亦即，各機器是作為本發明之一個態樣的圖像解碼裝置而發揮功能。 [分散處理]On the other hand, the streaming server ex103 performs streaming transmission of content data, which is content data transmitted to a requesting client. The client refers to a computer ex111, a game machine ex112, a camera ex113, a home appliance ex114, a smart phone ex115, and a terminal in an aircraft ex117 that can decode the data that has been encoded as described above. Each machine that has received the transmitted data will decode the received data and play it. That is, each device functions as an image decoding device according to one aspect of the present invention. [Decentralized processing]

又，串流伺服器ex103亦可為複數個伺服器或複數台電腦，且將資料分散並處理或記錄以進行發送。例如，串流伺服器ex103可藉由CDN(內容傳遞網路，Contents Delivery Network)來實現，亦可藉由分散於全世界的多數個邊緣伺服器(edge server)與連接邊緣伺服器之間的網路來實現內容發送。在CDN上，會因應於客戶來動態地分配在物理上相近之邊緣伺服器。並且，可以藉由將內容快取(cache)及發送至該邊緣伺服器來減少延遲。又，由於可以在發生某種錯誤時或因流量之增加等而改變通訊狀態時，以複數個邊緣伺服器將處理分散、或將發送主體切換為其他的邊緣伺服器，來繞過已發生障礙的網路的部分以持續發送，因此可以實現高速且穩定的發送。In addition, the streaming server ex103 may be a plurality of servers or a plurality of computers, and the data may be distributed and processed or recorded for transmission. For example, the streaming server ex103 can be implemented by a CDN (Contents Delivery Network), or by a number of edge servers distributed around the world and connected to the edge server. Internet for content delivery. On the CDN, the edge servers that are physically close to each other are dynamically allocated according to customers. In addition, latency can be reduced by caching and sending content to the edge server. In addition, when a certain error occurs or the communication status is changed due to an increase in traffic, etc., it is possible to use a plurality of edge servers to distribute the processing or switch the sending subject to another edge server to bypass the obstacle that has occurred. The part of the network is continuously transmitted, so high-speed and stable transmission can be achieved.

又，不僅是發送本身的分散處理，已攝影的資料之編碼處理亦可在各終端進行，且也可在伺服器側進行，亦可互相分擔來進行。作為一例，一般在編碼處理中，會進行2次處理循環。在第1次的循環中是檢測在框或場景單位下之圖像的複雜度或編碼量。又，在第2次的循環中是進行維持畫質並提升編碼效率的處理。例如，藉由使終端進行第1次的編碼處理，且使接收內容之伺服器側進行第2次的編碼處理，可以減少在各終端之處理負荷並且提升內容的質與效率。此時，只要有以近乎即時的方式來進行接收並解碼的要求，也可以用其他終端來接收並播放終端已進行的第一次之編碼完成資料，因此也可做到更靈活的即時發送。In addition, not only the distributed processing of the transmission itself, but also the encoding processing of the photographed data may be performed at each terminal, and may also be performed at the server side, or may be performed by sharing each other. As an example, in the encoding process, two processing cycles are generally performed. In the first loop, the complexity or encoding amount of the image in the frame or scene unit is detected. In the second loop, processing is performed to maintain image quality and improve coding efficiency. For example, by enabling the terminal to perform the first encoding process and the server receiving the content to perform the second encoding process, it is possible to reduce the processing load on each terminal and improve the quality and efficiency of the content. At this time, as long as there is a request for receiving and decoding in a near-instant manner, other terminals can also be used to receive and play the first encoded completion data that the terminal has performed, so more flexible real-time transmission can also be achieved.

作為其他的例子，相機ex113等會從圖像中進行特徵量擷取，並將與特徵量相關之資料作為元資料(meta data)來壓縮並傳送至伺服器。伺服器會進行例如從特徵量判斷目標(object)之重要性並切換量化精度等的因應圖像之意義的壓縮。特徵量資料對於在伺服器之再度壓縮時的運動向量預測之精度及效率提升特別有效。又，亦可在終端進行VLC(可變長度編碼)等簡易的編碼，並在伺服器進行CABAC(上下文參考之適應性二值算術編碼方式)等處理負荷較大的編碼。As another example, the camera ex113 and the like will extract feature quantities from the image, and compress and transmit the data related to the feature quantities as meta data to the server. The server performs compression corresponding to the meaning of the image, such as judging the importance of an object from a feature amount and switching the quantization accuracy. The feature quantity data is particularly effective for improving the accuracy and efficiency of motion vector prediction when the server is recompressed. In addition, simple coding such as VLC (Variable Length Coding) may be performed on the terminal, and coding with a large processing load such as CABAC (Adaptive Binary Arithmetic Coding Method of Context Reference) may be performed on the server.

此外，作為其他的例子，在運動場、購物商場、或工廠等中，會有藉由複數個終端拍攝幾乎相同的場景之複數個影像資料存在的情況。此時，可利用已進行攝影之複數個終端、與因應需要而沒有進行攝影之其他終端及伺服器，以例如GOP(圖片群組，Group of Picture)單位、圖片單位、或將圖片分割而成之圖塊(tile)單位等來各自分配編碼處理而進行分散處理。藉此，可以減少延遲，而更加能夠實現即時性(real-time)。In addition, as another example, in a sports field, a shopping mall, a factory, or the like, there may be cases where a plurality of image data are captured by a plurality of terminals to capture almost the same scene. At this time, a plurality of terminals that have been photographed, and other terminals and servers that have not been photographed according to needs may be used, for example, by GOP (Group of Picture) units, picture units, or divided into pictures. Each tile unit or the like is assigned an encoding process and distributed processing is performed. Thereby, the delay can be reduced, and real-time can be realized more.

又，由於複數個影像資料幾乎為相同的場景，因此亦可利用伺服器進行管理及/或指示，以將在各終端所攝影之影像資料互相地配合參照。或者，亦可使伺服器接收來自各終端之編碼完成資料，並在複數個資料間變更參照關係、或者補正或更換圖片本身並重新編碼。藉此，可以生成已提高一個個資料之質與效率的串流(stream)。In addition, since the plurality of image data are almost the same scene, a server can also be used for management and / or instruction, so that the image data photographed at each terminal can be coordinated and referred to each other. Alternatively, the server may receive the coding completion data from each terminal, and change the reference relationship between the plurality of data, or correct or replace the picture itself and re-encode. Thereby, a stream which has improved the quality and efficiency of each piece of data can be generated.

又，伺服器亦可在進行變更影像資料之編碼方式的轉碼(transcode)後再發送影像資料。例如，伺服器亦可將MPEG類之編碼方式轉換為VP類，亦可將H.264轉換為H.265。In addition, the server may transmit the image data after transcoding the encoding method of the image data. For example, the server can also convert MPEG encoding to VP encoding, and it can also convert H.264 to H.265.

如此，即可藉由終端或1個以上的伺服器來進行編碼處理。因此，以下雖然使用「伺服器」或「終端」等記載來作為進行處理之主體，但亦可在終端進行在伺服器進行之處理的一部分或全部，且亦可在伺服器進行在終端進行之處理的一部分或全部。又，有關於上述內容，針對解碼處理也是同樣的。 [3D、多角度]In this way, encoding processing can be performed by a terminal or one or more servers. Therefore, although descriptions such as "server" or "terminal" are used as the main body of the processing below, some or all of the processing performed by the server may be performed at the terminal, and it may also be performed by the server at the terminal. Part or all of the processing. The above description is the same for the decoding process. [3D, multi-angle]

近年來，以下作法也在逐漸增加中，即，將彼此幾乎同步的複數台相機ex113及/或智慧型手機ex115等終端所攝影到的不同場景、或者將從不同的角度攝影相同的場景之圖像或影像加以整合並利用。各終端所攝影到之影像會根據另外取得的終端間之相對的位置關係、或者包含於影像之特徵點為一致的區域等而被整合。In recent years, the following methods have also been gradually increased, that is, pictures of different scenes photographed by terminals such as multiple cameras ex113 and / or smart phones ex115 that are almost synchronized with each other, or pictures of the same scene from different angles Image or image to integrate and use. The images captured by each terminal are integrated based on the relative positional relationship between the terminals obtained separately, or the areas where the feature points of the images are consistent.

伺服器不僅對二維的動態圖像進行編碼，亦可根據動態圖像的場景解析等而自動地、或者在使用者所指定的時刻中，對靜態圖進行編碼並傳送至接收終端。此外，伺服器在可以取得攝影終端間之相對的位置關係的情況下，不僅是二維動態圖像，還可以根據相同場景從不同的角度所攝影之影像，來生成該場景之三維形狀。再者，伺服器亦可將藉由點雲(point cloud)而生成之三維的資料另外編碼，亦可根據使用三維資料來辨識或追蹤人物或目標的結果，而從複數個終端所攝影的影像中選擇、或再構成並生成欲傳送至接收終端的影像。The server not only encodes the two-dimensional moving image, but also automatically encodes the static image at the time specified by the user based on scene analysis of the moving image, etc., and transmits it to the receiving terminal. In addition, when the server can obtain the relative positional relationship between the shooting terminals, it is not only a two-dimensional moving image, but also a three-dimensional shape of the scene based on the images captured from different angles of the same scene. In addition, the server can additionally encode the three-dimensional data generated by the point cloud, and can also use the three-dimensional data to identify or track the results of people or targets, and capture images from multiple terminals. Select, or reconstruct and generate an image to be transmitted to the receiving terminal.

如此，使用者可以任意選擇對應於各攝影終端之各影像來享受場景，也可以享受從利用複數個圖像或影像再構成之三維資料中切出任意視點而成的影像之內容。此外，與影像同樣地，聲音也可從複數個不同的角度進行收音，且伺服器亦可配合影像，將來自特定之角度或空間的聲音與影像進行多工化並傳送。In this way, the user can arbitrarily select each image corresponding to each photography terminal to enjoy the scene, and can also enjoy the content of an image obtained by cutting out arbitrary viewpoints from three-dimensional data reconstructed from a plurality of images or images. In addition, like the video, the sound can be received from a plurality of different angles, and the server can also cooperate with the video to multiplex and transmit the sound and video from a specific angle or space.

又，近年來，Virtual Reality(虛擬實境，VR)及Augmented Reality(擴增虛擬實境，AR)等將現實世界與虛擬世界建立對應之內容也逐漸普及。在VR圖像的情形下，伺服器亦可分別製作右眼用及左眼用之視點圖像，並藉由Multi-View Coding(多視圖編碼，MVC)等在各視點影像間進行容許參照之編碼，亦可不互相參照而作為不同的串流來進行編碼。在不同的串流之解碼時，可使其互相同步來播放，以因應使用者之視點來重現虛擬的三維空間。In addition, in recent years, contents that establish a correspondence between the real world and the virtual world, such as Virtual Reality (VR) and Augmented Reality (AR), have also gradually spread. In the case of VR images, the server can also create right-view and left-view viewpoint images, and use Multi-View Coding (Multi-View Coding, MVC) to allow permissible reference between the viewpoint images. The encoding may be performed as a different stream without referring to each other. When decoding different streams, they can be played in synchronization with each other to reproduce the virtual three-dimensional space according to the user's viewpoint.

在AR圖像的情形下，伺服器會根據三維之位置或使用者之視點的移動，將虛擬空間上之虛擬物體資訊重疊於現實空間之相機資訊。解碼裝置亦可取得或保持虛擬物體資訊及三維資料，並因應使用者之視點的移動而生成二維圖像並順暢地連結，藉以製作重疊資料。或者，亦可為解碼裝置除了虛擬物體資訊之委託之外還將使用者的視點之移動也傳送至伺服器，且伺服器配合從保持於伺服器之三維資料中所接收到的視點的移動來製作重疊資料，並將重疊資料編碼且發送至解碼裝置。再者，亦可為重疊資料除了RGB以外還具有顯示穿透度的α值，伺服器將從三維資料所製作出之目標以外的部分之α值設定為0等，並在該部分為穿透狀態下進行編碼。或者，伺服器亦可如色度鍵(chroma key)的形式，將規定之值的RGB值設定為背景，而生成目標以外之部分是形成為背景色之資料。In the case of AR images, the server will superimpose the virtual object information in the virtual space with the camera information in the real space according to the three-dimensional position or the movement of the user's viewpoint. The decoding device can also obtain or maintain virtual object information and three-dimensional data, and generate a two-dimensional image and smoothly link it according to the movement of the user's viewpoint, thereby making overlapping data. Alternatively, in addition to the request of the virtual object information, the decoding device can also transmit the movement of the user's viewpoint to the server, and the server cooperates with the movement of the viewpoint received from the three-dimensional data held on the server. Make overlapping data, encode and send the overlapping data to the decoding device. In addition, the superimposed data may have an alpha value that shows penetration in addition to RGB. The alpha value of the part other than the target created from the three-dimensional data is set to 0, etc., and the part is penetrated. Coded in the status. Alternatively, the server may set a RGB value of a predetermined value as a background, such as a form of a chroma key, and generate data other than the target to be formed as a background color.

同樣地，被發送之資料的解碼處理可在客戶端即各終端進行，亦可在伺服器側進行，亦可互相分擔而進行。作為一例，亦可使某個終端暫時將接收要求傳送至伺服器，並在其他終端接收因應該要求之內容且進行解碼處理，再將解碼完成之訊號傳送至具有顯示器的裝置。藉由不依靠可通訊之終端本身的性能而將處理分散並選擇適當之內容的作法，可以播放畫質良好的資料。又，作為其他的例子，亦可用TV等接收大尺寸之圖像資料，並將圖片分割後之圖塊等一部分的區域解碼並顯示於鑑賞者之個人終端。藉此，可以將整體圖片共有化，並且可以就近確認自己負責的領域或想要更詳細地確認之區域。Similarly, the decoding process of the transmitted data can be performed on the client side, that is, on each terminal, or on the server side, or they can be shared with each other. As an example, a terminal may temporarily transmit a reception request to a server, and receive the content in response to the request at another terminal and perform a decoding process, and then transmit the decoded signal to a device with a display. By not relying on the performance of the communicable terminal itself, the method of decentralizing the processing and selecting appropriate content can play back good-quality data. In addition, as another example, a TV or the like may be used to receive large-sized image data, and a part of an area such as a tile after the picture is divided may be decoded and displayed on the personal terminal of the viewer. In this way, you can share the overall picture, and you can confirm the area you are responsible for or the area you want to confirm in more detail.

又，今後可預想到下述情形：不論屋內外，在近距離、中距離、或長距離之無線通訊為可複數使用的狀況下，利用MPEG-DASH等之發送系統規格，一邊對連接中的通訊切換適當的資料一邊無縫地接收內容。藉此，使用者不僅對本身之終端，連設置於屋內外之顯示器等的解碼裝置或顯示裝置都可自由地選擇並且即時切換。又，可以做到根據本身的位置資訊等，一邊切換要進行解碼之終端及要進行顯示之終端並一邊進行解碼。藉此，也可在往目的地之移動中，一邊在埋入有可顯示之元件的鄰近建築物的牆面或地面的一部分顯示地圖資訊，一邊移動。又，也可做到如下情形，即，令編碼資料快取到可以在短時間內從接收終端進行存取之伺服器、或者複製到內容傳遞伺服器(content delivery server)中的邊緣伺服器等，根據在網路上對編碼資料的存取容易性，來切換接收資料之位元率(bit-rate)。 [可調式編碼]In addition, in the future, the following situations can be expected: regardless of indoor and outdoor conditions, in the case of close-range, middle-range, or long-range wireless communication that can be used multiple times, using the transmission system specifications such as MPEG-DASH, Communication switches the appropriate data while receiving content seamlessly. Thereby, the user can freely select not only his own terminal, but also a decoding device or a display device such as a display installed inside or outside the house, and switch instantly. In addition, it is possible to perform decoding while switching between a terminal to be decoded and a terminal to be displayed based on its own location information and the like. Thereby, it is also possible to move while moving to the destination while displaying map information on a part of the wall or the ground of an adjacent building in which displayable elements are embedded. In addition, it is also possible to cache the encoded data to a server that can be accessed from the receiving terminal in a short time, or to copy it to an edge server in a content delivery server. According to the ease of access to the encoded data on the network, the bit-rate of the received data is switched. [Adjustable coding]

關於內容之切換，是利用圖23所示之可調整的串流來進行說明，該可調整的串流應用了上述各實施形態中所示之動態圖像編碼方法，並進行壓縮編碼。雖然伺服器具有複數個內容相同而質卻不同的串流來作為個別的串流也無妨，但亦可如圖示般構成為藉由分層來進行編碼，而實現時間上/空間上可調整之串流，並活用該串流的特徵來切換內容。亦即，藉由使解碼側因應性能這種內在要因與通訊頻帶之狀態等的外在要因來決定要解碼至哪一層，解碼側即可自由地切換低解析度之內容與高解析度之內容來解碼。例如，當想在回家後以網路電視等機器收看於移動中以智慧型手機ex115收看之影像的後續時，該機器只要將相同的串流解碼至不同的層即可，因此可以減輕伺服器側的負擔。The content switching is described by using an adjustable stream shown in FIG. 23, which applies the moving image coding method shown in each of the above embodiments and performs compression coding. Although the server has multiple streams with the same content but different qualities as individual streams, it is not a problem, but it can also be structured as shown in the figure to encode by layering to achieve time / space adjustment. Stream, and use the characteristics of the stream to switch the content. That is, the decoding side can decide which layer to decode according to the internal factors such as performance and internal factors such as the state of the communication band. The decoding side can freely switch between low-resolution content and high-resolution content. To decode. For example, when you want to watch the follow-up of an image watched on a mobile phone ex115 with a smartphone ex115 after you return home, the machine only needs to decode the same stream to different layers, so the servo can be reduced. Load on the device side.

此外，如上述地，除了實現按每一層將圖片編碼、且在基本層之上位存在增強層(enhancement layer)之具可調整性(scalability)的構成以外，亦可使增強層包含有根據圖像之統計資訊等的元資訊，且使解碼側根據元資訊對基本層之圖片進行超解析，藉此來生成高畫質化之內容。所謂超解析可以是相同解析度中的SN比之提升、以及解析度之擴大的任一種。元資訊包含：用於特定超解析處理中使用之線形或非線形的濾波係數之資訊、或者特定超解析處理中使用之濾波處理、機械學習或最小平方運算中的參數值之資訊等。In addition, as described above, in addition to implementing a scalable structure in which a picture is coded for each layer and an enhancement layer is provided above the base layer, the enhancement layer can also include an image based on the image. Metadata, such as statistical information, and make the decoding side super-parse the pictures in the base layer based on the meta-information to generate high-quality content. The so-called super-resolution may be any of an increase in the SN ratio and an increase in the resolution in the same resolution. Meta-information includes: information for linear or non-linear filter coefficients used in specific super-analytic processing, or information about filter values used in specific super-analytic processing, mechanical learning, or parameter values in least square operations.

或者，亦可構成為因應圖像內之目標等的含義而將圖片分割為圖塊等，且使解碼側選擇欲解碼之圖塊，藉此僅將一部分之區域解碼。又，藉由將目標之屬性(人物、車、球等)與影像內之位置(同一圖像中的座標位置等)作為元資訊加以保存，解碼側即可根據元資訊來特定所期望之目標的位置，並決定包含該目標之圖塊。例如，如圖24所示，可使用HEVC中的SEI訊息等與像素資料為不同之資料保存構造來保存元資訊。此元資訊是表示例如主目標之位置、尺寸、或色彩等。Alternatively, it may be configured to divide a picture into tiles or the like according to the meaning of an object or the like in the image, and cause the decoding side to select a tile to be decoded, thereby decoding only a part of the area. In addition, by storing the attributes of the target (person, car, ball, etc.) and the position in the image (coordinate position in the same image, etc.) as meta-information, the decoding side can specify the desired target based on the meta-information. And determine the tile that contains the target. For example, as shown in FIG. 24, the meta-information may be stored using a data storage structure different from the pixel data, such as the SEI message in HEVC. This meta information indicates, for example, the position, size, or color of the main target.

又，亦可以串流、序列或隨機存取單位等由複數個圖片構成之單位來保存元資訊。藉此，解碼側可以取得特定人物出現在影像內之時刻等，且藉由與圖片單位之資訊對照，可以特定出目標存在之圖片、以及目標在圖片內的位置。 [網頁之最佳化]In addition, the meta information may be stored in a unit composed of a plurality of pictures, such as a stream, a sequence, or a random access unit. With this, the decoding side can obtain the moment when a specific person appears in the image, etc., and by comparing with the information of the picture unit, the picture where the target exists and the position of the target in the picture can be specified. [Optimization of web pages]

圖25是顯示電腦ex111等中的網頁的顯示畫面例之圖。圖26是顯示智慧型手機ex115等中的網頁的顯示畫面例之圖。如圖25及圖26所示，在網頁包含複數個對圖像內容之鏈接即鏈接圖像的情況下，其外觀會依閱覽之元件而不同。在畫面上可看到複數個鏈接圖像的情況下，直至使用者明確地選擇鏈接圖像、或者鏈接圖像接近畫面之中央附近或鏈接圖像之整體進入畫面內為止，顯示裝置(解碼裝置)都是顯示具有各內容之靜態圖或I圖片(框內編碼畫面，Intra Picture)作為鏈接圖像、或者以複數個靜態圖或I圖片等來顯示gif動畫形式的影像、或者僅接收基本層來將影像解碼及顯示。FIG. 25 is a diagram showing an example of a display screen displaying a web page on a computer ex111 or the like. FIG. 26 is a diagram showing an example of a display screen showing a web page in a smartphone ex115 or the like. As shown in FIG. 25 and FIG. 26, when a web page includes a plurality of links to image content, that is, a linked image, its appearance will vary depending on the elements viewed. When a plurality of linked images can be seen on the screen, the display device (decoding device) is displayed until the user explicitly selects the linked image, or the linked image approaches the center of the screen or the entire linked image enters the screen. ) All display static pictures or I pictures (intra picture) with linked content as linked images, or display multiple gifs in the form of gif animation, or only receive the basic layer To decode and display the image.

在已由使用者選擇出鏈接圖像的情況下，顯示裝置會將基本層設為最優先來解碼。再者，只要在構成網頁之HTML中具有表示可調整之內容的資訊，亦可使顯示裝置解碼至增強層。又，為了擔保即時性，在選擇之前或通訊頻帶非常吃緊的情況下，顯示裝置可以藉由僅解碼及顯示前向參照(forward reference)之圖片(I圖片(框內編碼畫面)、P圖片(預測畫面，Predictive Picture)、僅前向參照之B圖片(雙向預估編碼畫面，Bi-directionally Predictive Picture))，來減低開頭圖片之解碼時刻與顯示時刻之間的延遲(從內容之解碼開始到顯示開始之間的延遲)。又，顯示裝置亦可特意無視圖片之參照關係，而將所有的B圖片及P圖片設成前向參照來粗略地解碼，並隨著時間經過使接收之圖片增加來進行正常的解碼。 [自動行駛]When the linked image has been selected by the user, the display device sets the base layer as the highest priority for decoding. Furthermore, as long as the HTML constituting the webpage has information indicating adjustable content, the display device can also be decoded to an enhancement layer. In addition, in order to guarantee the timeliness, the display device can decode and display only forward reference pictures (I pictures (frame coded pictures), P pictures ( (Predictive Picture), forward-referenced B picture (Bi-directionally Predictive Picture)), to reduce the delay between the decoding time and the display time of the beginning picture (from the beginning of content decoding to the Show the delay between starts). In addition, the display device may deliberately ignore the reference relationship of pictures, and set all B pictures and P pictures to be forward reference for rough decoding, and increase the received pictures over time to perform normal decoding. [Automatic driving]

又，在為了汽車之自動行駛或行駛支援而傳送接收二維或三維之地圖資訊等的靜態圖或影像資料的情況下，除了屬於1個以上的層的圖像資料之外，接收終端亦可將天候或施工之資訊等也都接收作為元資訊，並對應於這些來解碼。再者，元資訊可以屬於層，亦可單純與圖像資料進行多工化。In addition, in the case of transmitting and receiving two-dimensional or three-dimensional map information and other static pictures or video data for automatic driving or driving support of a car, the receiving terminal may be in addition to image data belonging to one or more layers. The weather and construction information are also received as meta-information and decoded in accordance with these. Furthermore, meta-information can belong to layers, or it can simply be multiplexed with image data.

此時，由於包含接收終端之車、無人機(drone)或飛機等會移動，因此藉由接收終端在接收要求時會傳送該接收終端之位置資訊之作法，即可一邊切換基地台ex106~ex110一邊實現無縫的接收及解碼。又，接收終端會因應於使用者之選擇、使用者之狀況、或通訊頻帶的狀態，而變得可動態地切換要將元資訊接收到何種程度、或要將地圖資訊更新至何種程度。At this time, since the vehicle including the receiving terminal, drone, or airplane will move, the base station ex106 ~ ex110 can be switched by the receiving terminal by transmitting the location information of the receiving terminal when receiving the request. Achieve seamless reception and decoding. In addition, the receiving terminal may dynamically switch the degree to which the meta-information is received or the degree to which the map information is to be updated according to the user's selection, the user's condition, or the state of the communication band. .

如以上，在內容供給系統ex100中，客戶端可即時地接收使用者所傳送之已編碼的資訊，並將其進行解碼、播放。 [個人內容之發送]As described above, in the content supply system ex100, the client can immediately receive the encoded information transmitted by the user, and decode and play it. [Send personal content]

又，在內容供給系統ex100中，不僅是來自影像發送業者之高畫質且長時間的內容，來自個人之低畫質且短時間的內容的單播(unicast)、或多播(multicast)發送也是可做到的。又，這種個人的內容被認為今後也會持續增加下去。為了將個人內容作成更優良之內容，伺服器亦可在進行編輯處理之後進行編碼處理。這可藉由例如以下之構成來實現。In addition, in the content supply system ex100, not only high-quality and long-term content from video distribution companies, but also unicast or multicast transmission of low-quality and short-term content from individuals. It can be done. It is thought that such personal content will continue to increase in the future. In order to make personal content better, the server can also perform encoding processing after editing processing. This can be achieved by, for example, the following configuration.

伺服器會在攝影時即時或累積於攝影後，從原圖或編碼完成資料中進行攝影錯誤、場景搜尋、意義解析、及目標檢測等辨識處理。而且，伺服器會根據辨識結果以手動或自動方式進行下述編輯：補正失焦或手震等、刪除亮度較其他圖片低或未聚焦之場景等重要性低的場景、強調目標之邊緣、變化色調等。伺服器會根據編輯結果來將編輯後之資料編碼。又，當攝影時刻太長時會導致收視率下降的情況也是眾所皆知的，伺服器會根據圖像處理結果而以自動的方式，如上述地不僅對重要性低之場景還有動態較少的場景等進行剪輯，以使其因應攝影時間成為特定之時間範圍內的內容。或者，伺服器亦可根據場景之意義解析的結果來生成摘錄(digest)並進行編碼。The server performs recognition processing such as photography errors, scene search, meaning analysis, and target detection from the original image or the coded completed data in real time or after the photography. Moreover, the server will perform the following manual or automatic editing based on the recognition results: correction of out-of-focus or camera shake, deletion of less important scenes such as scenes with lower brightness or unfocused scenes, emphasis on the edges of the target, changes Hue, etc. The server will encode the edited data according to the edit result. In addition, it is well known that when the shooting time is too long, the ratings will decrease. The server will automatically perform the process according to the image processing results. Edit a small number of scenes, etc., so that it is within a specific time range according to the shooting time. Alternatively, the server may generate a digest based on the result of the meaning analysis of the scene and encode the digest.

再者，在個人內容中，也有照原樣的話會有侵害著作權、著作人格權、或肖像權等之內容攝入的案例，也有當共享的範圍超過所欲共享之範圍等對個人來說不方便的情況。據此，例如，伺服器亦可將畫面周邊部之人臉、或房子內部等特意變更為未聚焦之圖像並編碼。又，伺服器亦可辨識編碼對象圖像內是否拍到與事先登錄之人物不同的人物的臉，並在拍到的情況下，進行將臉的部分打上馬賽克等之處理。或者，作為編碼之前處理或後處理，使用者亦可基於著作權等之觀點而於圖像中指定想要加工之人物或背景區域後，令伺服器進行將所指定之區域替換為另外的影像、或者使焦點模糊等處理。如果是人物，可以在動態圖像中一邊追蹤人物一邊替換臉的部分的影像。In addition, there are cases where personal content infringes copyright, personality rights, or portrait rights, as it is, and it is inconvenient for individuals when the scope of sharing exceeds the scope of sharing. Case. According to this, for example, the server may also intentionally change the face of the screen periphery or the interior of the house into an unfocused image and encode it. In addition, the server can also recognize whether a face of a person different from the person registered in advance is captured in the encoding target image, and if it is captured, perform processing such as mosaicing the face portion. Alternatively, as a pre- or post-encoding process, the user can also specify the person or background area to be processed in the image based on the viewpoint of copyright, etc., and cause the server to replace the specified area with another image, Or blur the focus. If it is a person, you can replace the face part of the face while tracking the person in the moving image.

又，由於資料量較小之個人內容的視聽對即時性的要求較強，因此，雖然也會取決於頻帶寬，但解碼裝置首先會最優先地接收基本層再進行解碼及播放。解碼裝置亦可在這段期間內接收增強層，且於循環播放等播放2次以上的情形下，將增強層也包含在內來播放高畫質的影像。像這樣，只要是進行可調整之編碼的串流，就可以提供一種雖然在未選擇時或初次看到的階段是粗略的動態圖，但串流會逐漸智能化(smart)而使圖像變好的體驗。除了可調式編碼以外，即使將第1次播放之粗略的串流、與參照第1次之動態圖而編碼之第2次的串流構成為1個串流，也可以提供同樣的體驗。 [其他之使用例]In addition, since the viewing and listening of personal content with a small amount of data has strong requirements for immediacy, although it also depends on the frequency bandwidth, the decoding device first receives the base layer first and then decodes and plays it. The decoding device can also receive the enhancement layer during this period, and in the case of loop playback, such as playing twice or more, the enhancement layer is also included to play the high-quality image. As such, as long as it is an adjustable encoded stream, it can provide a rough dynamic picture when it is not selected or the first time it is seen, but the stream will gradually be smart and the image will be changed. Good experience. In addition to the adjustable encoding, the same experience can be provided even if the rough stream of the first playback and the second stream of the encoding encoded with reference to the first motion picture are configured as one stream. [Other use cases]

又，這些編碼或解碼處理一般是在各終端所具有之LSIex500中處理。LSIex500可為單晶片(one chip)，亦可為由複數個晶片形成之構成。再者，亦可將動態圖像編碼或解碼用之軟體安裝到可以在電腦ex111等讀取之某種記錄媒體(CD-ROM、軟式磁碟(flexible disk)、或硬碟等)，並使用該軟體來進行編碼或解碼處理。此外，在智慧型手機ex115為附有相機的情況下，亦可傳送以該相機取得之動態圖資料。此時的動態圖資料是以智慧型手機ex115所具有的LSIex500來編碼處理而成之資料。In addition, these encoding or decoding processes are generally processed in the LSIex500 that each terminal has. The LSIex500 may be a single chip, or may be formed of a plurality of chips. In addition, you can also install software for encoding or decoding moving images on a recording medium (CD-ROM, flexible disk, or hard disk) that can be read on a computer such as ex111, and use it. This software performs encoding or decoding processing. In addition, when the smart phone ex115 is equipped with a camera, it can also transmit dynamic image data obtained by the camera. The dynamic image data at this time is data encoded and processed by the LSI ex500 of the smart phone ex115.

再者，LSIex500亦可為將應用軟體下載並啟動(activate)之構成。此時，終端首先會判定該終端是否對應於內容之編碼方式、或者是否具有特定服務之執行能力。在終端沒有對應於內容之編碼方式時、或者不具有特定服務之執行能力的情況下，終端會下載編碼解碼器或應用軟體，然後，取得及播放內容。Furthermore, the LSIex500 may be configured to download and activate application software. At this time, the terminal first determines whether the terminal corresponds to the encoding method of the content, or whether it has the ability to execute specific services. When the terminal does not have a coding method corresponding to the content, or does not have the ability to execute specific services, the terminal downloads a codec or application software, and then obtains and plays the content.

又，不限於透過網際網路ex101之內容供給系統ex100，在數位播放用系統中也可以安裝上述各實施形態之至少動態圖像編碼裝置(圖像編碼裝置)或動態圖像解碼裝置(圖像解碼裝置)之任一個。由於是利用衛星等來將已使影像與聲音被多工化之多工資料乘載於播放用之電波來進行傳送接收，因此會有相對於內容供給系統ex100之容易形成單播的構成更適合多播的差別，但有關於編碼處理及解碼處理仍可為同樣之應用。 [硬體構成]In addition, the content supply system ex100 is not limited to the Internet ex101. At least a moving image encoding device (image encoding device) or a moving image decoding device (image) Decoding device). Since satellites are used to transmit and receive images and sounds that have been multiplexed with multiplexed data on radio waves for transmission and reception, it will be more suitable for the content supply system ex100 which is easy to form a unicast. The multicast is different, but the encoding and decoding processes can still be used for the same application. [Hardware composition]

圖27是顯示智慧型手機ex115之圖。又，圖28是顯示智慧型手機ex115的構成例之圖。智慧型手機ex115具備：用於在與基地台ex110之間傳送接收電波的天線ex450、可拍攝影像及靜態圖之相機部ex465、顯示已將以相機部ex465所拍攝到之影像以及以天線ex450所接收到之影像等解碼之資料的顯示部ex458。智慧型手機ex115更具備：觸控面板等之操作部ex466、用於輸出聲音或音響之揚聲器等即聲音輸出部ex457、用於輸入聲音之麥克風等即聲音輸入部ex456、可保存所攝影之影像或靜態圖、錄音之聲音、接收之影像或靜態圖、郵件等已編碼之資料、或已解碼之資料的記憶體部ex467、及作為與SIMex468之間的介面部即插槽部ex464，該SIMex468是用於特定使用者，且以網路為首進行對各種資料的存取之認證。再者，取代記憶體部ex467而使用外接記憶體亦可。FIG. 27 is a diagram showing a smart phone ex115. FIG. 28 is a diagram showing a configuration example of the smartphone ex115. The smartphone ex115 includes an antenna ex450 for transmitting and receiving radio waves to and from the base station ex110, a camera section ex465 capable of capturing images and still images, a display showing images captured by the camera section ex465, and an antenna ex450. Display section ex458 of decoded data such as received images. The smart phone ex115 further includes an operation unit ex466 such as a touch panel, a sound output unit ex457 such as a speaker for outputting sound or sound, a sound input unit ex456 such as a microphone for inputting a sound, and a recorded image can be stored. Or still image, recorded sound, received image or still image, mail and other encoded data, or decoded data memory part ex467, and the slot part ex464 as the interface between SIMex468 It is used for specific users and authenticates access to various data, including the Internet. Alternatively, an external memory may be used instead of the memory unit ex467.

又，統合地控制顯示部ex458及操作部ex466等主控制部ex460，是透過匯流排ex470而連接於電源電路部ex461、操作輸入控制部ex462、影像訊號處理部ex455、相機介面部ex463、顯示器控制部ex459、調變/解調部ex452、多工/分離部ex453、聲音訊號處理部ex454、插槽部ex464、及記憶體部ex467。In addition, the main control units ex460, such as the integrated display control unit ex458 and the operation unit ex466, are connected to the power supply circuit unit ex461, the operation input control unit ex462, the image signal processing unit ex455, the camera interface portion ex463, and the display control via a bus ex470 The unit ex459, the modulation / demodulation unit ex452, the multiplexing / demultiplexing unit ex453, the audio signal processing unit ex454, the slot unit ex464, and the memory unit ex467.

電源電路部ex461在藉由使用者之操作而將電源鍵設成開啟狀態時，會藉由從電池組(battery pack)對各部供給電力而將智慧型手機ex115起動為可運作之狀態。When the power supply circuit section ex461 sets the power key to the on state by a user's operation, the smart phone ex115 is activated to operate by supplying power to each section from a battery pack.

智慧型手機ex115會根據具有CPU、ROM及RAM等之主控制部ex460的控制，進行通話及資料通訊等處理。通話時，是以聲音訊號處理部ex454將以聲音輸入部ex456所收音之聲音訊號轉換為數位聲音訊號，並以調變/解調部ex452對其進行展頻處理，接著以傳送/接收部ex451施行數位類比轉換處理及頻率轉換處理後，透過天線ex450傳送。又，將接收資料放大且施行頻率轉換處理及類比數位轉換處理，並以調變/解調部ex452進行解展頻處理，接著以聲音訊號處理部ex454轉換為類比聲音訊號後，由聲音輸出部ex457將其輸出。資料通訊模式時，是藉由本體部之操作部ex466等的操作而透過操作輸入控制部ex462將正文(text)、靜態圖、或影像資料送出至主控制部ex460，而同樣地進行傳送接收處理。在資料通訊模式時傳送影像、靜態圖、或影像與聲音的情形下，影像訊號處理部ex455是藉由在上述各實施形態中所示的動態圖像編碼方法，將保存於記憶體部ex467之影像訊號或從相機部ex465輸入之影像訊號壓縮編碼，並將已編碼之影像資料送出至多工/分離部ex453。又，聲音訊號處理部ex454是將以相機部ex465拍攝影像或靜態圖等時被聲音輸入部ex456所收音之聲音訊號編碼，並將已編碼之聲音資料送出至多工/分離部ex453。多工/分離部ex453是以規定之方式對編碼完成影像資料與編碼完成聲音資料進行多工化，並以調變/解調部(調變/解調電路部)ex452、及傳送/接收部ex451施行調變處理及轉換處理，並透過天線ex450來傳送。The smart phone ex115 performs processing such as calling and data communication according to the control of the main control unit ex460 including the CPU, ROM, and RAM. During a call, the sound signal processing unit ex454 converts the sound signal received by the sound input unit ex456 into a digital sound signal, and performs spread spectrum processing on the modulation / demodulation unit ex452, and then the transmission / reception unit ex451 After digital analog conversion processing and frequency conversion processing are performed, transmission is performed through the antenna ex450. In addition, the received data is amplified and subjected to frequency conversion processing and analog digital conversion processing, and the demodulation processing is performed by the modulation / demodulation unit ex452, and then converted to the analog sound signal by the sound signal processing unit ex454, and then the sound output unit ex457 outputs it. In the data communication mode, the main body part's operation part ex466 and other operations are used to send the text (text), still pictures, or image data to the main control part ex460 through the operation input control part ex462, and the same transmission and reception processing is performed. . In the case of transmitting images, still images, or images and sounds in the data communication mode, the image signal processing unit ex455 is stored in the memory portion ex467 by the moving image encoding method shown in the above embodiments. The image signal or the image signal input from the camera part ex465 is compression-encoded, and the encoded image data is sent to the multiplex / separation part ex453. The audio signal processing unit ex454 encodes the audio signals received by the audio input unit ex456 when shooting images or still images with the camera unit ex465, and sends the encoded audio data to the multiplexing / separation unit ex453. The multiplexing / separating section ex453 multiplexes the encoded image data and encoded audio data in a prescribed manner, and uses a modulation / demodulation section (modulation / demodulation circuit section) ex452, and a transmission / reception section. ex451 performs modulation processing and conversion processing, and transmits it through the antenna ex450.

在已接收附加於電子郵件或網路聊天之影像、或鏈接至網頁等之影像的情形下，為了對已透過天線ex450接收之多工資料進行解碼，多工/分離部ex453是藉由分離多工資料，而將多工資料分成影像資料之位元流與聲音資料之位元流，再透過同步匯流排ex470將已編碼之影像資料供給至影像訊號處理部ex455，並且將已編碼之聲音資料供給至聲音訊號處理部ex454。影像訊號處理部ex455是藉由對應於上述各實施形態所示之動態圖像編碼方法的動態圖像解碼方法來解碼影像訊號，並透過顯示器控制部ex459而由顯示部ex458顯示被鏈接之動態圖像檔案中所含的影像或靜態圖。又，聲音訊號處理部ex454是將聲音訊號解碼，並由聲音輸出部ex457輸出聲音。再者，由於即時串流(real time streaming)已普及，因此依據使用者的狀況，也可能在社會上不適合發出聲音的場所發生聲音的播放。因此，作為初始值，較理想的構成是，在不使聲音訊號播放的情形下僅播放影像資料。亦可僅在使用者進行點選影像資料等操作的情形下才將聲音同步播放。In the case where an image attached to an e-mail or a web chat or an image linked to a web page has been received, in order to decode the multiplexed data that has been received through the antenna ex450, the multiplex / separation section ex453 The multi-tasking data is divided into a bit stream of image data and a bit stream of sound data, and then the encoded image data is supplied to the image signal processing unit ex455 through the synchronous bus ex470, and the encoded sound data It is supplied to the sound signal processing unit ex454. The image signal processing unit ex455 decodes the image signal by a moving image decoding method corresponding to the moving image encoding method shown in each of the above embodiments, and displays the linked dynamic image through the display control unit ex459 through the display control unit ex459. Like images or still images contained in files. The audio signal processing unit ex454 decodes the audio signal and outputs the audio from the audio output unit ex457. Furthermore, since real-time streaming has become popular, depending on the user's situation, sound playback may occur in places that are not suitable for sound generation in society. Therefore, as an initial value, an ideal configuration is to play only the video data without playing the audio signal. The sound can also be played synchronously only when the user performs operations such as clicking on image data.

又，在此雖然以智慧型手機ex115為例進行了說明，但可作為終端而被考慮的有下述3種組裝形式：除了具有編碼器及解碼器兩者之傳送接收型終端以外，還有僅具有編碼器之傳送終端、以及僅具有解碼器之接收終端。此外，在數位播送用系統中，雖然是設成接收或傳送已在影像資料中將聲音資料等多工化之多工資料來進行說明，但在多工資料中，除了聲音資料以外，亦可將與影像有關聯之文字資料等多工化，且亦可接收或傳送影像資料本身而非多工資料。In addition, although the smart phone ex115 has been described as an example, there are three types of assembly that can be considered as a terminal: In addition to a transmission and reception terminal having both an encoder and a decoder, there are also A transmitting terminal having only an encoder and a receiving terminal having only a decoder. In addition, in the digital broadcasting system, although it is set to receive or transmit multiplexed data that has been multiplexed with audio data in the video data for explanation, in the multiplexed data, in addition to audio data, Multiplex the text data associated with the image, and also receive or send the image data itself instead of the multiplexed data.

再者，雖然是設為使包含CPU之主控制部ex460控制編碼或解碼處理並進行了說明，但終端具備GPU的情況也很多。因此，也可以構成為藉由在CPU與GPU上已共通的記憶體、或將位址管理成可以共通地使用的記憶體，來活用GPU之性能而將較寬廣區域一併處理。藉此可以縮短編碼時間，確保即時性，而可以實現低延遲。特別是在不利用CPU的情形下，利用GPU並以圖片等單位來一併進行運動搜尋、解塊濾波方法(deblock filter)、SAO(取樣自適應偏移，Sample Adaptive Offset)、及轉換、量化之處理時，是有效率的。産業上之可利用性In addition, although the main control unit ex460 including a CPU has been described as controlling encoding or decoding processing, the terminal may include a GPU in many cases. Therefore, it can also be configured to use the performance of the GPU to process a wider area together by using memory that is common to the CPU and GPU, or managing addresses to memory that can be used in common. This can reduce encoding time, ensure immediateness, and achieve low latency. Especially without using the CPU, use the GPU to perform motion search, deblock filter, SAO (Sample Adaptive Offset), and conversion and quantization in units of pictures and other units. It is efficient in processing. Industrial availability

本揭示可在例如電視機、數位錄影機、汽車導航系統、行動電話、數位相機、數位攝影機、視訊會議系統或電子鏡子等方面利用。The present disclosure can be used in, for example, televisions, digital video recorders, car navigation systems, mobile phones, digital cameras, digital video cameras, video conference systems, or electronic mirrors.

10~23‧‧‧區塊10 ~ 23‧‧‧block

100‧‧‧編碼裝置100‧‧‧ encoding device

102‧‧‧分割部102‧‧‧Division

104‧‧‧減法部104‧‧‧Subtraction Division

106‧‧‧轉換部106‧‧‧ Conversion Department

108‧‧‧量化部108‧‧‧Quantitative Department

110‧‧‧熵編碼部110‧‧‧Entropy coding department

112、204‧‧‧逆量化部112, 204‧‧‧ Inverse quantification department

114、206‧‧‧逆轉換部114, 206‧‧‧ Inverse Conversion Department

116、208‧‧‧加法部116, 208‧‧‧Addition Department

118、210‧‧‧區塊記憶體118, 210‧‧‧ block memory

120、212‧‧‧迴路濾波部120, 212‧‧‧loop filtering department

122、214‧‧‧框記憶體122, 214‧‧‧ frame memory

124、216‧‧‧框內預測部124, 216‧‧‧ Frame prediction department

126、218‧‧‧框間預測部126, 218‧‧‧‧ Inter-frame prediction department

128、220‧‧‧預測控制部128, 220‧‧‧ Predictive Control Department

160、260‧‧‧電路160, 260‧‧‧circuit

162、262‧‧‧記憶體162, 262‧‧‧Memory

200‧‧‧解碼裝置200‧‧‧ decoding device

202‧‧‧熵解碼部202‧‧‧Entropy Decoding Department

ex100‧‧‧內容供給系統ex100‧‧‧Content Supply System

ex101‧‧‧網際網路ex101‧‧‧Internet

ex102‧‧‧網際網路服務提供者ex102‧‧‧Internet Service Provider

ex103‧‧‧串流伺服器ex103‧‧‧streaming server

ex104‧‧‧通訊網ex104‧‧‧Communication Network

ex106、ex107、ex108、ex109、ex110‧‧‧基地台ex106, ex107, ex108, ex109, ex110‧‧‧ base station

ex111‧‧‧電腦ex111‧‧‧Computer

ex112‧‧‧遊戲機ex112‧‧‧Game console

ex113‧‧‧相機ex113‧‧‧Camera

ex114‧‧‧家電ex114‧‧‧Household appliances

ex115‧‧‧智慧型手機ex115‧‧‧Smartphone

ex116‧‧‧衛星ex116‧‧‧ satellite

ex117‧‧‧飛機ex117‧‧‧plane

ex450‧‧‧天線ex450‧‧‧antenna

ex451‧‧‧傳送/接收部ex451‧‧‧Transmission / Reception Department

ex452‧‧‧調變/解調部ex452‧‧‧Modulation / Demodulation Department

ex453‧‧‧多工/分離部ex453‧‧‧Multiplexing / Separation Department

ex454‧‧‧聲音訊號處理部ex454‧‧‧Sound Signal Processing Department

ex455‧‧‧影像訊號處理部ex455‧‧‧Image Signal Processing Department

ex456‧‧‧聲音輸入部ex456‧‧‧Voice input section

ex457‧‧‧聲音輸出部ex457‧‧‧Sound output

ex458‧‧‧顯示部ex458‧‧‧Display

ex459‧‧‧顯示器控制部ex459‧‧‧Display Control

ex460‧‧‧主控制部ex460‧‧‧Main Control Department

ex461‧‧‧電源電路部ex461‧‧‧Power circuit department

ex462‧‧‧操作輸入控制部ex462‧‧‧Operation input control section

ex463‧‧‧相機介面部ex463‧‧‧camera face

ex464‧‧‧插槽部ex464‧‧‧Slot

ex465‧‧‧相機部ex465‧‧‧Camera Department

ex466‧‧‧操作部ex466‧‧‧Operation Department

ex467‧‧‧記憶體部ex467‧‧‧Memory

ex468‧‧‧SIMex468‧‧‧SIM

ex470‧‧‧匯流排ex470‧‧‧Bus

MV0、MV1、MVx0、MVy0、MVx1、MVy1、v0、v1‧‧‧運動向量MV0, MV1, MVx0, MVy0, MVx1, MVy1, v0, v1‧‧‧ motion vectors

Ref0、Ref1‧‧‧參照圖片Ref0, Ref1‧‧‧ reference pictures

S101~S108、S201~S206、S204A、S204B、S211~S220、S301~S305、S303A、S303B、S311~S314‧‧‧步驟S101 ~ S108, S201 ~ S206, S204A, S204B, S211 ~ S220, S301 ~ S305, S303A, S303B, S311 ~ S314‧‧‧Steps

TD0、TD1‧‧‧距離TD0, TD1‧‧‧ Distance

圖1是顯示實施形態1之編碼裝置的功能構成之方塊圖。FIG. 1 is a block diagram showing a functional configuration of an encoding device according to the first embodiment.

圖2是顯示實施形態1中的區塊分割的一例之圖。FIG. 2 is a diagram showing an example of block division in the first embodiment.

圖3是顯示對應於各轉換類型的轉換基底函數之表格。FIG. 3 is a table showing conversion basis functions corresponding to each conversion type.

圖4A是顯示在ALF中所用的濾波器之形狀的一例之圖。FIG. 4A is a diagram showing an example of the shape of a filter used in ALF.

圖4B是顯示在ALF中所用的濾波器之形狀的其他的一例之圖。FIG. 4B is a diagram showing another example of the shape of the filter used in the ALF.

圖4C是顯示在ALF中所用的濾波器之形狀的其他的一例之圖。FIG. 4C is a diagram showing another example of the shape of the filter used in the ALF.

圖5是顯示框內預測中的67個框內預測模式之圖。FIG. 5 is a diagram showing 67 intra-frame prediction modes in the intra-frame prediction.

圖6是用於說明沿著運動軌跡的2個區塊間的型樣匹配(雙向匹配)之圖。FIG. 6 is a diagram for explaining pattern matching (two-way matching) between two blocks along a motion trajectory.

圖7是用於說明在當前圖片內的模板與參照圖片內的區塊之間的型樣匹配(模板匹配)之圖。FIG. 7 is a diagram for explaining pattern matching (template matching) between a template in a current picture and a block in a reference picture.

圖8是用於說明假設了等速直線運動的模型之圖。FIG. 8 is a diagram for explaining a model in which constant-speed linear motion is assumed.

圖9是用於說明根據複數個相鄰區塊的運動向量之子區塊單位的運動向量的導出之圖。FIG. 9 is a diagram for explaining derivation of a motion vector based on a sub-block unit of a motion vector of a plurality of adjacent blocks.

圖10是顯示實施形態1之解碼裝置的功能構成之方塊圖。Fig. 10 is a block diagram showing a functional configuration of a decoding device according to the first embodiment.

圖11是實施形態1之畫面間預測處理的流程圖。11 is a flowchart of an inter-screen prediction process according to the first embodiment.

圖12是用於說明實施形態1之評價值的算出處理之一例的圖。FIG. 12 is a diagram for explaining an example of an evaluation value calculation process according to the first embodiment.

圖13是用於說明實施形態1之評價值的算出處理之一例的圖。FIG. 13 is a diagram for explaining an example of an evaluation value calculation process according to the first embodiment.

圖14是用於說明實施形態1之評價值的算出處理之一例的圖。FIG. 14 is a diagram for explaining an example of an evaluation value calculation process according to the first embodiment.

圖15是用於說明實施形態1之評價值的算出處理之一例的圖。15 is a diagram for explaining an example of an evaluation value calculation process according to the first embodiment.

圖16是實施形態2之編碼裝置中的畫面間預測處理的流程圖。Fig. 16 is a flowchart of inter-frame prediction processing in the encoding device of the second embodiment.

圖17是實施形態2之編碼裝置中的選擇參照圖片清單決定處理的流程圖。Fig. 17 is a flowchart of a process of determining a reference picture list in the encoding device of the second embodiment.

圖18是實施形態2之解碼裝置中的畫面間預測處理的流程圖。18 is a flowchart of an inter-frame prediction process in a decoding device according to the second embodiment.

圖19是實施形態2之解碼裝置中的選擇參照圖片清單決定處理的流程圖。19 is a flowchart of a process of determining a reference picture list in a decoding device according to the second embodiment.

圖20是顯示編碼裝置之組裝例的方塊圖。FIG. 20 is a block diagram showing an assembly example of the encoding device.

圖21是顯示解碼裝置之組裝例的方塊圖。FIG. 21 is a block diagram showing an assembly example of the decoding device.

圖22是實現內容發送服務(content delivery service)的內容供給系統之整體構成圖。FIG. 22 is an overall configuration diagram of a content supply system that implements a content delivery service.

圖23是顯示可調式編碼時之編碼構造的一例之圖。FIG. 23 is a diagram showing an example of a coding structure during adjustable coding.

圖24是顯示可調式編碼時之編碼構造的一例之圖。FIG. 24 is a diagram showing an example of a coding structure during adjustable coding.

圖25是顯示網頁的顯示畫面例之圖。FIG. 25 is a diagram showing an example of a display screen displaying a web page.

圖26是顯示網頁的顯示畫面例之圖。FIG. 26 is a diagram showing an example of a display screen for displaying a web page.

圖27是顯示智慧型手機的一例之圖。FIG. 27 is a diagram showing an example of a smartphone.

圖28是顯示智慧型手機的構成例之方塊圖。FIG. 28 is a block diagram showing a configuration example of a smartphone.

Claims

一種編碼裝置，具備：電路；及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。An encoding device includes: a circuit; and a memory, wherein the circuit uses the memory to derive a plurality of candidates for a processing target block included in a processing target image from a motion vector used for dynamic compensation of a processing completed block. The motion vector does not use the image area of the processing target block, but refers to two processed reference pictures whose display time is located in front and rear of the processing target picture in the same direction from the plurality of candidates. A final motion vector is selected from the motion vectors, and the foregoing motion vector is used to perform dynamic compensation of the processing target block.

如請求項1之編碼裝置，其中，前述處理對象圖片為P圖片。The encoding device according to claim 1, wherein the processing target picture is a P picture.

如請求項2之編碼裝置，其中，在用於前述處理對象圖片的參照圖片清單中顯示2張參照圖片的情況下，前述2張處理完成的參照圖片是顯示於前述參照圖片清單中的前述2張參照圖片。For example, the encoding device of claim 2, wherein when two reference pictures are displayed in the reference picture list used for the processing target picture, the two reference pictures that have been processed are displayed in the two reference pictures in the reference picture list Reference pictures.

如請求項2之編碼裝置，其中，在用於前述處理對象圖片的參照圖片清單中顯示1張參照圖片的情況下，前述2張處理完成的參照圖片的其中一張是顯示於前述參照圖片清單中的前述1張參照圖片，而前述2張處理完成的參照圖片之另一張是未顯示在前述參照圖片清單中的其他處理完成的參照圖片。For example, the encoding device of claim 2, wherein when one reference picture is displayed in the reference picture list for the processing target picture, one of the two reference pictures that have been processed is displayed in the reference picture list One of the aforementioned reference pictures in the above, and the other of the aforementioned two processed reference pictures is a reference picture of other processes not shown in the aforementioned reference picture list.

如請求項4之編碼裝置，其中，前述電路是進一步地將顯示於前述參照圖片清單的前述1張參照圖片、及前述其他處理完成的參照圖片保存於圖片記憶體。According to the encoding device of claim 4, wherein the circuit further saves the one reference picture displayed in the reference picture list and the reference picture completed by other processing in a picture memory.

如請求項4之編碼裝置，其中，在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述2張處理完成參照圖片的前述其中一張，是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且前述2張處理完成參照圖片的前述另一張，是可參照的圖片當中，顯示時間最接近於前述處理對象圖片的圖片。For example, the encoding device of claim 4, wherein in the selection of the final motion vector, the respective evaluation values of the plurality of candidate motion vectors are calculated by referring to the two reference pictures that have been processed, and based on the calculated evaluations. Value to select the final motion vector, one of the two processed reference pictures is a reference picture represented by the candidate motion vector of the plurality of candidate motion vectors, and the two processed reference pictures The other one is a picture that can be referred to, and whose display time is closest to the processing target picture.

如請求項1之編碼裝置，其中，前述處理對象圖片為B圖片。The encoding device according to claim 1, wherein the processing target picture is a B picture.

如請求項7之編碼裝置，其中，在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述處理對象圖片具有第1參照圖片清單及第2參照圖片清單，前述2張處理完成參照圖片的其中一張是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且是屬於前述第1參照圖片清單的參照圖片，前述2張處理完成參照圖片的另一張是屬於前述第2參照圖片清單的參照圖片當中，顯示時間最接近於前述處理對象圖片的圖片。For example, the encoding device of claim 7, wherein in the selection of the final motion vector, the respective evaluation values of the plurality of candidate motion vectors are calculated by referring to the two reference pictures that have been processed, and based on the calculated evaluations Value to select the final motion vector, the processing target picture has a first reference picture list and a second reference picture list, and one of the two processed reference pictures is a candidate motion among the plurality of candidate motion vectors to be processed. The reference picture represented by the vector is a reference picture belonging to the first reference picture list, and the other two reference pictures that have been processed are the reference pictures belonging to the second reference picture list, and the display time is closest to the processing. Picture of object picture.

一種解碼裝置，具備：電路；及記憶體，前述電路是利用前述記憶體，從處理完成區塊的動態補償所用的運動向量中，導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。A decoding device includes: a circuit; and a memory, wherein the circuit uses the memory to derive a plurality of candidates for a processing target block included in a processing target picture from a motion vector used for dynamic compensation of a processing completed block. The motion vector does not use the image area of the processing target block, but refers to two processed reference pictures whose display time is located in front and rear of the processing target picture in the same direction from the plurality of candidates. A final motion vector is selected from the motion vectors, and the foregoing motion vector is used to perform dynamic compensation of the processing target block.

如請求項9之解碼裝置，其中，前述處理對象圖片為P圖片。The decoding device according to claim 9, wherein the processing target picture is a P picture.

如請求項10之解碼裝置，其中，在用於前述處理對象圖片的參照圖片清單中顯示2張參照圖片的情況下，前述2張處理完成的參照圖片是顯示於前述參照圖片清單中的前述2張參照圖片。The decoding device according to claim 10, wherein when two reference pictures are displayed in the reference picture list used for the processing target picture, the two reference pictures that have been processed are displayed in the two reference pictures in the reference picture list Reference pictures.

如請求項10之解碼裝置，其中，在用於前述處理對象圖片的參照圖片清單中顯示1張參照圖片的情況下，前述2張處理完成的參照圖片的其中一張是顯示於前述參照圖片清單中的前述1張參照圖片，而前述2張處理完成的參照圖片之另一張是未顯示在前述參照圖片清單中的其他處理完成的參照圖片。The decoding device according to claim 10, wherein when one reference picture is displayed in the reference picture list used for the processing target picture, one of the two reference pictures that have been processed is displayed in the reference picture list One of the aforementioned reference pictures in the above, and the other of the aforementioned two processed reference pictures is a reference picture of other processes not shown in the aforementioned reference picture list.

如請求項12之解碼裝置，其中，前述電路是進一步地將顯示於前述參照圖片清單的前述1張參照圖片、及前述其他處理完成的參照圖片保存於圖片記憶體。According to the decoding device of claim 12, wherein the circuit further saves the one reference picture displayed in the reference picture list and the other reference pictures that have been processed in the picture memory.

如請求項10之解碼裝置，其中，在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述2張處理完成參照圖片的其中一張，是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且前述2張處理完成參照圖片的另一張，是可參照的圖片當中，顯示時間最接近於前述處理對象圖片的圖片。The decoding device according to claim 10, wherein in the selection of the final motion vector, each of the plurality of candidate motion vectors is evaluated by referring to the two reference pictures that have been processed, and based on the calculated evaluations Value to select the final motion vector, one of the two processed reference pictures is a reference picture represented by the candidate motion vector of the processing target among the plurality of candidate motion vectors, and the other two processed complete reference pictures are another One of the pictures that can be referred to is the picture whose display time is closest to the picture to be processed.

如請求項9之解碼裝置，其中，前述處理對象圖片為B圖片。The decoding device according to claim 9, wherein the processing target picture is a B picture.

如請求項15之解碼裝置，其中，在前述最終運動向量的選擇中，是參照前述2張處理完成的參照圖片，來算出前述複數個候選運動向量之各自的評價值，且根據已算出的評價值來選擇前述最終運動向量，前述處理對象圖片具有第1參照圖片清單及第2參照圖片清單，前述2張處理完成參照圖片的其中一張是前述複數個候選運動向量當中以處理對象的候選運動向量表示的參照圖片，且是屬於前述第1參照圖片清單的參照圖片，前述2張處理完成參照圖片的另一張是屬於前述第2參照圖片清單的參照圖片當中，顯示時間最接近於前述處理對象圖片的圖片。The decoding device according to claim 15, wherein, in the selection of the final motion vector, each of the plurality of candidate motion vectors is calculated by referring to the two reference pictures that have been processed, and based on the calculated evaluations. Value to select the final motion vector, the processing target picture has a first reference picture list and a second reference picture list, and one of the two processed reference pictures is a candidate motion among the plurality of candidate motion vectors to be processed. The reference picture represented by the vector is a reference picture belonging to the first reference picture list, and the other two reference pictures that have been processed are the reference pictures belonging to the second reference picture list, and the display time is closest to the processing. Picture of object picture.

一種編碼方法，是從處理完成區塊的動態補償所用的運動向量中導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。An encoding method is to derive a plurality of candidate motion vectors for a processing target block included in a processing target picture from a motion vector used for dynamic compensation of a processing completed block, without using an image area of the processing target block, Instead, reference is made to two processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture to select the final motion vector from the plurality of candidate motion vectors, and use the final motion vector to perform Dynamic compensation of the aforementioned processing target block.

一種解碼方法，是從處理完成區塊的動態補償所用的運動向量中導出包含於處理對象圖片的處理對象區塊用之複數個候選運動向量，且不使用前述處理對象區塊的圖像區域，而是參照顯示時間位於比前述處理對象圖片更前方及更後方當中相同方向的2張處理完成的參照圖片，來從前述複數個候選運動向量中選擇最終運動向量，並利用前述最終運動向量來進行前述處理對象區塊的動態補償。A decoding method is to derive a plurality of candidate motion vectors for a processing target block included in a processing target picture from a motion vector used for dynamic compensation of a processing completed block, without using the image area of the processing target block, Instead, reference is made to two processed reference pictures whose display time is in the same direction as the front and rear of the processing target picture to select the final motion vector from the plurality of candidate motion vectors, and use the final motion vector to perform Dynamic compensation of the aforementioned processing target block.