WO2020183859A1 - Image coding device, image decoding device, image coding method, image decoding method, and program - Google Patents

Image coding device, image decoding device, image coding method, image decoding method, and program Download PDF

Info

Publication number
WO2020183859A1
WO2020183859A1 PCT/JP2019/050359 JP2019050359W WO2020183859A1 WO 2020183859 A1 WO2020183859 A1 WO 2020183859A1 JP 2019050359 W JP2019050359 W JP 2019050359W WO 2020183859 A1 WO2020183859 A1 WO 2020183859A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
image
quantization
prediction
interest
Prior art date
Application number
PCT/JP2019/050359
Other languages
French (fr)
Japanese (ja)
Inventor
真悟 志摩
Original Assignee
キヤノン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by キヤノン株式会社 filed Critical キヤノン株式会社
Publication of WO2020183859A1 publication Critical patent/WO2020183859A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • the present invention relates to an image coding device, an image decoding device, an image coding method, an image decoding method, and a program.
  • HEVC High Efficiency Video Coding
  • a basic block having a size larger than that of a conventional macroblock (16 ⁇ 16 pixels) has been adopted.
  • This large size basic block is called a CTU (Coding Tree Unit), and its size is a maximum of 64 x 64 pixels.
  • the CTU is further divided into sub-blocks that serve as units for prediction and conversion.
  • Patent Document 1 discloses a technique for encoding such a quantization matrix.
  • the introduction of a quantization matrix is being considered in the same way as HEVC.
  • the quantization matrix in HEVC is premised on the conventional prediction methods such as intra-prediction and inter-prediction, and cannot correspond to the current picture reference prediction which is a new prediction method. Therefore, the quantization control according to the frequency component cannot be performed for the error of the current picture reference prediction, and the improvement of the subjective image quality cannot be expected.
  • the present invention has been made to solve the above-mentioned problems, and a technique for improving subjective image quality by enabling a quantization process using a quantization matrix corresponding to the current picture reference prediction of the above definition. It is what we are trying to provide.
  • the image coding apparatus of the present invention has the following configuration. That is, An image coding device that divides an image into multiple blocks and encodes them in block units.
  • a prediction means for generating a prediction image of a block of interest to be encoded by using a block-shaped pixel group in a coded region of a frame to which the block of interest belongs.
  • a conversion means that generates a conversion coefficient by orthogonally converting the error between the pixel of the block of interest and the predicted image. It has a quantization means for quantizing the conversion coefficient generated by the conversion means by using a quantization parameter and a quantization matrix.
  • the figure which shows the example of the sub-block division used in an embodiment The figure which shows the example of the sub-block division used in an embodiment.
  • the figure which shows the example of the sub-block division used in an embodiment The figure which shows the example of the sub-block division used in an embodiment.
  • the figure which shows the example of the sub-block division used in an embodiment The figure which shows the example of the sub-block division used in an embodiment.
  • the figure which shows the example of the sub-block division used in an embodiment. which shows the example of the quantization matrix used in embodiment.
  • the figure which shows the example of the quantization matrix used in embodiment. The figure which shows the scanning method of the element of the quantization matrix used in embodiment.
  • the figure which shows the difference value matrix of the quantization matrix generated by embodiment The figure which shows the difference value matrix of the quantization matrix generated by embodiment.
  • FIG. 1 is a block diagram showing an image coding device to which the present embodiment is applied.
  • the image coding device according to the embodiment will be described as being mounted on a digital video camera having an image sensor that generates image data to be encoded. However, it should be noted that this is just for ease of understanding, and the source of the image to be encoded does not matter.
  • the control unit 150 is composed of a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM that is used as a work area of the CPU, and controls each component described below. It controls the entire image encoding device.
  • the block division unit 102 inputs the image data captured by the imaging unit via the input terminal 101. Then, the block division unit 102 divides the input image into a plurality of basic blocks, and outputs the image data of the basic block unit to the prediction unit 104 in the subsequent stage.
  • the quantization matrix holding unit 103 generates and holds a quantization matrix.
  • the method of generating the quantization matrix is not particularly limited, but the user may input the quantization matrix, calculate from the characteristics of the input image, or use a predetermined value as an initial value.
  • the quantization matrix holding unit 103 of the present embodiment generates and holds a two-dimensional quantization matrix corresponding to the orthogonal conversion of three types of 8 ⁇ 8 pixels shown in FIGS. 8A to 8C.
  • the quantization matrix is for weighting the quantization process with respect to the conversion coefficient according to the frequency component.
  • the quantization step for each transformation factor is weighted by multiplying the reference value by the value of each element in the quantization matrix.
  • the reference value of the quantization step is defined by the quantization parameter.
  • the prediction unit 104 determines the sub-block division method for the image data in the basic block unit received from the block division unit 102, and performs intra-frame prediction, which is intra-frame prediction, inter-frame prediction, which is inter-frame prediction, and the like in sub-block units. And generate the predicted image data. Further, the prediction unit 104 calculates and outputs a prediction error which is a difference between the image data of the subblock and the predicted image data. In addition, the prediction unit 104 also outputs information necessary for prediction, for example, information such as a sub-block division method, a prediction mode, and a motion vector, together with a prediction error. Hereinafter, the information necessary for this prediction is referred to as prediction information.
  • the conversion / quantization unit 105 generates a conversion coefficient by orthogonally converting the prediction error in subblock units from the prediction unit 104. Further, the conversion / quantization unit 105 uses the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameter to quantize the generated conversion coefficient, so that the conversion coefficient after quantization is performed. (Hereinafter, simply the quantization coefficient) is obtained.
  • the function of performing orthogonal conversion and the function of performing quantization may be configured separately.
  • the inverse quantization / inverse conversion unit 106 dequantizes the quantization coefficient output from the conversion / quantization unit 105 using the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameters. By doing so, the conversion coefficient is reproduced. Further, the inverse quantization / inverse conversion unit 106 reproduces the prediction error by performing inverse orthogonal conversion with respect to the reproduced conversion coefficient.
  • the function of performing inverse quantization and the function of performing inverse orthogonal transformation may be configured separately.
  • the image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information output from the prediction unit 104, and generates the prediction image data for the block of interest to be encoded. Then, the image reproduction unit 107 generates the reproduction image data by adding the prediction error input from the inverse quantization / inverse conversion unit 106 to the predicted image data, and stores the reproduced image data in the frame memory 108.
  • the in-loop filter unit 109 performs in-loop filter processing such as a deblocking filter and a sample adaptive offset on the reproduced image stored in the frame memory 108, and stores the filtered image data in the frame memory 108 again.
  • the coding unit 110 encodes the quantization coefficient output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104, generates coded data, and outputs the coded data.
  • the information for the image decoding device to derive the quantization parameter is also encoded in the bit stream by the coding unit 110.
  • the quantization matrix coding unit 113 encodes the quantization matrix held in the quantization matrix holding unit 103, generates the coding data of the quantization matrix, and outputs it.
  • the integrated coding unit 111 generates header data using the coding data of the quantization matrix which is the output from the quantization matrix coding unit 113. Further, the integrated coding unit 111 forms a bit stream together with the header data and the coded data output from the coding unit 110, and outputs the bit stream to the outside via the output terminal 112.
  • the output target is a recording medium such as a memory card, but the output target may be a network or the like, and the type thereof is not particularly limited.
  • the coding target is a moving image, and it is configured to be input in frame units.
  • the block division unit 102 will be described as being divided into basic blocks of 8 ⁇ 8 pixels, but it should be understood that this is only for facilitating understanding.
  • a block larger than the basic block for example, a block having 64 ⁇ 64 pixels or a block having 128 ⁇ 128 pixels
  • the quantization matrix Prior to image coding, the quantization matrix is generated and coded.
  • the control unit 150 controls the quantization matrix holding unit 103 to generate a quantization matrix.
  • the quantization matrix holding unit 103 generates a quantization matrix according to the size of the subblock to be encoded and the type of prediction method.
  • a quantization matrix for a subblock having a size of 8 ⁇ 8 pixels without division is generated for the basic block of 8 ⁇ 8 pixels shown in FIG. 7A.
  • the generated quantization matrix is not limited to this, and a quantization matrix corresponding to the shape of the subblock such as 4 ⁇ 8 or 8 ⁇ 4, 4 ⁇ 4 may be generated.
  • the method for determining each element constituting the quantization matrix is not particularly limited. For example, a predetermined initial value may be used, or may be set individually. Further, it may be generated according to the characteristics of the image.
  • the quantization matrix holding unit 103 holds the quantization matrix generated in this way. Examples of the quantization matrix generated by the quantization matrix holding unit 103 are shown in FIGS. 8A to 8C.
  • FIG. 8A is an example of a quantization matrix corresponding to intra-prediction
  • FIG. 8B is for inter-prediction
  • FIG. 8C is for current picture reference prediction.
  • the thick frame represents the quantization matrix. For the sake of simplicity, it is assumed that each square is composed of 64 pixels of 8 ⁇ 8, and each square in the thick frame represents each element value constituting the quantization matrix. In the present embodiment, it is assumed that the three types of quantization matrices 800 to 802 shown in FIGS.
  • each element in the quantization matrix 800 to 802 is of course included in this. Not limited. It is also possible to hold a plurality of quantization matrices for the same prediction method depending on the size of the subblock or whether the coding target is a luminance block or a color difference block. In general, since the quantization matrix realizes the quantization process according to the human visual characteristics, as shown in FIGS. 8A to 8C, the element values of the low frequency portion corresponding to the upper left portion of the quantization matrices 800 to 802 are set. It is small and the element value of the high frequency part corresponding to the lower right part is large.
  • the quantization matrix coding unit 113 reads out the quantization matrices 800 to 802 stored in the two-dimensional shape as shown in FIGS. 8A to 8C in order from the quantization matrix holding unit 106, and scans (scans) each element. Calculate the difference and place it in a one-dimensional matrix.
  • the quantization matrix coding unit 113 of the present embodiment uses each of the quantization matrices 800 to 802 shown in FIGS. 8A to 8C by the scanning method shown in FIG. 9, and sets each element as the immediately preceding element in the scanning order. Calculate the difference between. For example, the 8 ⁇ 8 pixel quantization matrix 802 shown in FIG. 8C is scanned by the scanning method shown in FIG.
  • the difference from a predetermined initial value for example, “8”
  • the difference from an arbitrary value or the value of the first element itself may be used.
  • the quantization matrices 800 to 802 of FIGS. 8A to 8C use the scanning method of FIG. 9, and the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C are generated.
  • the quantization matrix coding unit 113 further encodes the difference matrices 1000 to 1002 to generate the coding data of the quantization matrix.
  • coding is performed using the coding table shown in FIG. 11A, but the coding table is not limited to this, and for example, the coding table shown in FIG. 11B may be used.
  • the quantization matrix coding unit 113 outputs the coded data of the quantization matrices 800 to 802 shown in FIGS. 8A to 8C thus generated to the integrated coding unit 111 in the subsequent stage.
  • the integrated coding unit 111 encodes the header information necessary for encoding the image data and integrates the encoded data of the quantization matrix.
  • the image data is encoded.
  • the image data for one frame input from the terminal 101 is supplied to the block dividing unit 102.
  • the block division unit 102 divides the image data of the input frame into a plurality of basic blocks, and outputs the image data of each basic block unit to the prediction unit 104.
  • the size of the basic block in this embodiment is 8 ⁇ 8 pixels.
  • the prediction unit 104 executes prediction processing on the image data of the basic block input from the block division unit 102. Specifically, a subblock division method for dividing a basic block into finer subblocks is determined, and a prediction mode such as intra prediction, inter prediction, or current picture reference prediction is determined for each subblock.
  • FIGS. 7C to 7F show examples of subblock division methods.
  • Reference numerals 700 to 705 representing the thick frame each represent a basic block, and for the sake of simplicity, the configuration is 8 ⁇ 8 pixels, and each quadrangle in the thick frame represents a subblock.
  • FIG. 7B shows an example of the conventional square subblock division, in which the 8 ⁇ 8 pixel basic block 701 is divided into four 4 ⁇ 4 pixel subblocks.
  • FIGS. 7C to 7F show that the basic blocks 702 to 705 are composed of two or three rectangular subblocks.
  • FIG. 7C shows a subblock division in which the basic block 702 is vertically elongated into two 4 ⁇ 8 pixels.
  • FIG. 7D shows the subblock division of the basic block 703 into two horizontally long 8 ⁇ 4 pixels.
  • FIG. 7E shows three rectangular sub-block divisions in which the basic block 704 is vertically longitudinal and has a width of 1: 2: 1.
  • FIG. 7F shows three rectangular subblock divisions in which the basic block 705 is horizontally longitudinal and has a width of 1: 2: 1. In this way, not only squares but also rectangular sub-blocks are used for encoding processing.
  • FIG. 7A which does not divide the basic block of 8 ⁇ 8 pixels, is used, but the sub-block division method is not limited to this.
  • a quadtree division as shown in FIG. 7B, a ternary division as shown in FIGS. 7E and 7F, or a binary tree division as shown in FIGS. 7C and 7D may be used.
  • the subblock division other than FIG. 7A is also used, the quantization matrix corresponding to the subblock used in the quantization matrix holding unit 103 is generated. Further, the generated quantization matrix will be encoded by the quantization matrix coding unit 113.
  • prediction method used in this embodiment will be described again.
  • three types of prediction methods are used: intra prediction, inter prediction, and current picture reference prediction.
  • Intra-prediction is to generate predicted pixels of a coded block by using coded pixels located spatially around the coded block.
  • Prediction pixel generation methods include Angular prediction for 33 directions such as horizontal prediction and vertical prediction, DC prediction with the average value of the decoded pixel values of adjacent blocks as the predicted value, and plane approximation from the decoded pixels of the adjacent block. Planar prediction and the like for generating pixel values are included, and information indicating which one is used is also generated.
  • Inter-prediction generates predicted pixels of a coded target block using encoded pixels of a frame that is different in time from the coded target block, and also generates motion information indicating a frame to be referred to, a motion vector, and the like.
  • a similar block-shaped pixel group is generated as a prediction block of the coding target block from within the coded region of the frame to which the coding target block belongs. For example, if there is a block-shaped pixel group similar to the coded block at a position separated from the coded block by x pixels in the left direction and y pixels in the upward direction, this "x pixel on the left and y pixel on the top”. Is generated as displacement information. In this way, in the current picture reference prediction, the prediction block of the coded block is generated, and the displacement information used for generating the prediction pixel is also generated.
  • This current picture reference prediction is a technique suitable for improving the coding efficiency, especially in an artificial image such as a computer screen in which the same characters and textures repeatedly appear in a frame to be coded / decoded. It can be said that.
  • the prediction unit 104 determines the prediction mode that minimizes the error from any of the above prediction modes. Then, the prediction unit 104 generates prediction image data from the determined prediction mode and encoded pixels, further generates prediction error from the input image data and prediction image data, and converts and quantizes the prediction error. Output to 105. Further, the prediction unit 104 outputs information such as subblock division and prediction mode as prediction information to the conversion / quantization unit 105, the coding unit 110, the image reproduction unit 107, and the inverse quantization / inverse conversion unit 106.
  • the conversion / quantization unit 105 performs orthogonal conversion on the prediction error input from the prediction unit 104 to obtain a conversion coefficient. Then, the conversion / quantization unit 105 performs quantization with respect to the conversion coefficient using the quantization matrix according to the prediction mode and the quantization parameter, and generates the quantization coefficient. Specifically, the conversion / quantization unit 105 performs orthogonal conversion processing corresponding to the size of the subblock to generate an orthogonal conversion coefficient, and then stores the conversion coefficient in the quantization matrix holding unit 103 according to the prediction mode. Quantization is performed using the quantization matrix and the quantization parameters, and the quantization coefficient is generated.
  • the conversion / quantization unit 105 of the present embodiment quantizes the conversion coefficient of the subblock predicted by the intra prediction using the quantization matrix 800 of FIG. 8A. Further, the conversion coefficient of the sub-block in which the inter-prediction is performed is quantized using the quantization matrix 801 of FIG. 8B. Then, the conversion coefficient of the sub-block for which the current picture reference prediction has been performed is quantized using the quantization matrix 802 of FIG. 8C. However, the quantization matrix used is not limited to this. Then, the conversion / quantization unit 105 outputs the quantization coefficient obtained in the quantization process to the coding unit 110 and the inverse quantization / inverse conversion unit 106.
  • the inverse quantization / inverse conversion unit 106 inversely quantizes the input quantization coefficient using the quantization matrix held in the quantization matrix holding unit 103 and the quantization parameter, and reproduces the conversion coefficient. In this way, the process of reproducing (deriving) the conversion coefficient using the quantization matrix and the quantization parameter is referred to as inverse quantization. Further, the inverse quantization / inverse conversion unit 106 performs inverse orthogonal conversion of the reproduced conversion coefficient to reproduce the prediction error. As the quantization matrix used in the inverse quantization process by the inverse quantization / inverse conversion unit 106, the same one as the quantization matrix used by the conversion / quantization unit 105 for the conversion coefficient of the coded block is used.
  • the inverse quantization / inverse conversion unit 106 determines the quantization matrix to be used based on the prediction information from the prediction unit 104.
  • the inverse quantization / inverse conversion unit 106 outputs the reproduced prediction error to the image reproduction unit 107.
  • the image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104, and reproduces the predicted image of the subblock. Then, the image data is reproduced from the reproduced predicted image and the prediction error of the subblock input from the inverse quantization / inverse conversion unit 106, and stored in the frame memory 108.
  • the in-loop filter unit 109 reads the reproduced image from the frame memory 108 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 109 re-stores the image data after the filter processing in the frame memory 108.
  • the coding unit 110 entropy-encodes the quantization coefficient generated by the conversion / quantization unit 105 and the prediction information input from the prediction unit 104 in block units to generate coded data.
  • the method of entropy coding is not particularly specified, but Golomb coding, arithmetic coding, Huffman coding and the like can be used.
  • the generated coded data is output to the integrated coding unit 111.
  • the integrated coding unit 111 forms a bit stream by multiplexing the above-mentioned header coding data (including the coding data of the quantization matrix) and the coding data input from the coding unit 110. Then, the integrated coding unit 111 outputs the formed bit stream from the terminal 112 to the outside of the apparatus (for example, a recording medium).
  • FIG. 6A is an example of a bit stream output in the first embodiment.
  • the sequence header contains the coded data of the quantization matrix, and is composed of the coded results of each element.
  • the encoded position is not limited to this, and of course, a configuration in which the image header portion and other header portions are encoded may be used.
  • the quantization matrix when the quantization matrix is changed in one sequence, it can be updated by newly encoding the quantization matrix. At this time, all the quantization matrices may be rewritten, or a part of them may be changed by specifying the prediction mode of the quantization matrix corresponding to the rewritten quantization matrix.
  • FIG. 3 is a flowchart showing the coding process for one frame under the control of the control unit 150 in the image coding device of the first embodiment.
  • the quantization matrix holding unit 103 Prior to image coding, the quantization matrix holding unit 103 generates and holds a two-dimensional quantization matrix in S301.
  • the quantization matrix holding unit 103 of the present embodiment is a quantization matrix 800 to 802 for intra prediction, inter prediction, and current picture reference prediction corresponding to a subblock of 8 ⁇ 8 pixels shown in FIGS. 8A to 8C. Will be generated and retained.
  • the quantization matrix coding unit 113 scans each of the quantization matrices 800 to 802 generated and held in S301, calculates the difference of each element, and generates the difference matrix 1000 to 1002. That is, in the present embodiment, the quantization matrices 800 to 802 shown in FIGS. 8A to 8C are scanned by the scanning method of FIG. 9, and the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C are generated. Become. The quantization matrix coding unit 113 encodes the difference matrices 1000 to 1002 thus generated, and generates the coded data of the quantization matrix.
  • the integrated coding unit 111 encodes and outputs the header information necessary for coding the image data, including the coded data of the generated quantization matrix.
  • the block division unit 102 divides the input image in frame units into basic block units.
  • the prediction unit 104 executes prediction processing on the image data in basic block units generated in S304, and generates prediction information such as sub-block division information and prediction mode and prediction image data.
  • the prediction unit 104 of the present embodiment tries three types of prediction methods, intra prediction, inter prediction, and current picture reference prediction, and determines one prediction method that minimizes the prediction error.
  • the method for determining the prediction method is not limited to this, and the coding cost may be calculated from the magnitude of the prediction error and the amount of code generated to determine the prediction method that minimizes the coding cost.
  • the prediction unit 104 calculates a prediction error from the input image data and the prediction image data, and outputs the prediction error together with the prediction information.
  • the conversion / quantization unit 105 orthogonally converts the prediction error calculated in S305 to generate a conversion coefficient. Further, the conversion / quantization unit 105 uses any of the corresponding quantization matrices 800 to 802 held in the quantization matrix holding unit 103 and the quantization parameter based on the prediction information to generate a quantum of the conversion coefficient. Is performed to generate a quantization coefficient.
  • the sub-block in which the intra-prediction is used is the quantization matrix 800 of FIG. 8A
  • the sub-block in which the inter-prediction is used is the quantization matrix 801 of FIG.
  • the quantization matrix 802 of FIG. 8C will be used for the block.
  • the inverse quantization / inverse conversion unit 106 inversely quantizes the quantization coefficient generated in S306 using the corresponding quantization matrix held in the quantization matrix holding unit 103, and obtains the conversion coefficient. Reproduce. Further, the inverse quantization / inverse conversion unit 106 performs inverse orthogonal conversion with respect to the reproduced conversion coefficient and reproduces the prediction error. In this step, the quantization matrix used in the inverse quantization is the same as the quantization matrix used in S306.
  • the image reproduction unit 107 reproduces the predicted image based on the prediction information generated in S305. Further, the image reproduction unit 107 reproduces the image data from the reproduced predicted image and the prediction error generated in S307.
  • the coding unit 110 encodes the prediction information generated in S305 and the quantization coefficient generated in S306 to generate coded data for the image data. In addition, the coding unit 110 generates a bit stream including other coded data.
  • control unit 150 determines whether or not the coding of all the basic blocks in the frame is completed, and if it is determined that the coding is completed, the processing is performed in S311 and the uncoded basic blocks remain. If it is determined that the process is returned to S304.
  • the in-loop filter unit 109 performs in-loop filter processing on the image data reproduced in S308, generates the filtered image data, and ends the processing.
  • the quantization matrix is individually defined for the three types of prediction methods of intra-prediction, inter-prediction, and current picture reference prediction, and all three types of quantization matrices are encoded. However, some of them may be shared.
  • the sub-block using the current picture reference prediction is also quantized using the quantization matrix 801 (FIG. 8B) for the inter-prediction image, and the coding of the quantization matrix 802 in FIG. 8C is omitted. Is also possible. As a result, it is possible to reduce the amount of code equivalent to 802 minutes of the quantization matrix of FIG. 8C, and reduce the image quality deterioration caused by an error due to prediction using a block-shaped pixel group such as a jerky motion.
  • the image coding device may be able to select which quantization matrix is used in the subblock of the current picture reference prediction in response to an instruction from a user from an operation unit (not shown).
  • the information indicating which is selected is stored in the header of the stream as the quantization matrix setting information.
  • FIG. 6B An example of the stream configuration in this case is shown in FIG. 6B.
  • the quantization matrix setting information is "0"
  • the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 800 (FIG. 8A) for intra prediction. ..
  • the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 801 (FIG. 8B) for inter-prediction.
  • the quantization matrix setting information is "2”
  • the quantization matrix used for the subblock using the current picture reference prediction is the dedicated quantization matrix 802 for the current picture reference prediction (FIG. 8C). It is assumed that. This makes it possible to selectively realize the reduction of the quantization matrix code amount and the original quantization control for the subblock using the current picture reference prediction.
  • FIG. 2 shows the configuration of the image decoding device according to the second embodiment.
  • This image decoding device will be described as decoding the coded bit stream generated by the image coding device of the first embodiment.
  • the control unit 250 is composed of a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM that is used as a work area of the CPU, and controls each component described below. It controls the entire image decoding device.
  • the separation / decoding unit 202 separates the coded bit stream input from the input terminal 201 into coded data related to information related to decoding processing and coefficients, and decodes the coded data existing in the header part of the coded bit stream. Specifically, the separation / decoding unit 202 separates the coded data of the quantization matrix from the coded bit stream, and outputs the coded data of the separated quantization matrix to the quantization matrix decoding unit 209. Further, the separation / decoding unit 202 separates the coded data of the image from the coded bit stream, and outputs the coded data of the separated image to the decoding unit 203. That is, the separation / decoding unit 202 performs the reverse processing of the integrated coding unit 111 of FIG.
  • the quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and holds the quantization matrices 800 to 802.
  • the decoding unit 203 decodes the encoded data of the image input from the separation decoding unit 202, and reproduces the quantization coefficient and the prediction information. Then, the decoding unit 203 supplies the quantization coefficient to the inverse quantization / inverse conversion unit 204, and outputs the prediction information to the inverse quantization / inverse conversion unit 204 and the image reproduction unit 205.
  • the information for deriving the quantization parameter is also decoded from the bit stream by the decoding unit 203.
  • the inverse quantization / inverse conversion unit 204 inputs the quantization coefficient from the decoding unit 203. Then, the inverse quantization / inverse conversion unit 204 uses the quantization matrix corresponding to the prediction mode indicated by the prediction information and the quantization parameter, similarly to the inverse quantization / inverse conversion unit 106 of FIG. The conversion coefficient is obtained by performing the inverse quantization process of the quantization coefficient. Further, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion on the obtained conversion coefficient, reproduces the prediction error, and outputs the prediction error to the image reproduction unit 205.
  • the image reproduction unit 205 generates the predicted image data by appropriately referring to the frame memory 206 (which stores the decoded image data) based on the prediction information input from the prediction unit 203. Then, the image reproduction unit 205 generates the reproduction image data from the predicted image data and the prediction error reproduced by the inverse quantization / inverse conversion unit 204. Then, the image reproduction unit 205 stores the generated reproduced image in the frame memory 206.
  • the in-loop filter unit 207 is the same as the in-loop filter unit 109 in FIG. 1, and performs an in-loop filter process such as a deblocking filter on the reproduced image stored in the frame memory 206 to obtain the filtered image. Restore in frame memory 206. As a result, since the image data that can be displayed after the filter processing is stored in the frame memory, it is output to an external device (a display device is a typical example thereof) via the output terminal 208.
  • an external device a display device is a typical example thereof
  • the image decoding operation in the image decoding device of the second embodiment will be described in more detail below.
  • the bit stream generated by the image coding apparatus of the first embodiment described above is input in frame units.
  • the separation / decoding unit 202 inputs a bitstream via the input terminal 201, separates it into information related to decoding processing and coded data related to coefficients, and decodes the coded data existing in the header part of the bitstream. More specifically, the separation / decoding unit 202 separates the coded data of the quantization matrix stored in the header unit of the bitstream and outputs it to the quantization matrix decoding unit 209. The separation / decoding unit 202 of the present embodiment extracts the coded data of the quantization matrix from the sequence header of the bitstream shown in FIG. 6A and outputs the coded data to the quantization matrix decoding unit 209. As a result, in the present embodiment, the coded data of the quantized matrices 801 to 802 shown in FIGS. 8A to 8C is output to the quantized matrix decoding unit 209. Further, the separation / decoding unit 202 separates the coded data of the image following the header unit from the bit stream and outputs it to the decoding unit 203.
  • the quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and reproduces the one-dimensional difference matrices 1000 to 1002 shown in FIGS. 10A to 10C.
  • decoding is performed using the coding table shown in FIG. 11A, but the coding table is not limited to this, as long as the same one as the coding device is used. May use other coding tables.
  • the quantization matrix decoding unit 209 scans the reproduced one-dimensional difference matrix and reproduces the two-dimensional quantization matrix.
  • the operation opposite to the operation of the quantization matrix coding unit 113 of the first embodiment is performed.
  • the quantization matrix decoding unit 209 uses the scanning methods shown in FIGS. 9A to 9 from the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C, and the three types of quantization matrices 800 shown in FIGS. 8A to 8C, respectively. 802 is regenerated and retained.
  • the decoding unit 203 decodes the coded data input from the separation decoding unit 202 and reproduces the quantization coefficient and the prediction information.
  • the decoding unit 203 outputs the reproduced quantization coefficient to the inverse quantization / inverse conversion unit 204, and outputs the reproduced prediction information to the inverse quantization / inverse conversion unit 204 and the image reproduction unit 205.
  • the inverse quantization / inverse conversion unit 204 generates a conversion coefficient by performing inverse quantization on the input quantization coefficient using the quantization matrix and the quantization parameter.
  • the inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion of the obtained conversion coefficient to reproduce the prediction error. Then, the inverse quantization / inverse conversion unit 204 outputs the reproduced prediction error to the image reproduction unit 205.
  • the inverse quantization / inverse conversion unit 204 of the present embodiment determines the quantization matrix used in the inverse quantization process according to the prediction mode of the decoding target block determined by the prediction information reproduced by the decoding unit 203. .. That is, the sub-block in which the intra-prediction is used is the quantization matrix 800 of FIG.
  • the sub-block in which the inter-prediction is used is the quantization matrix 801 of FIG. 8B, and the sub-block in which the current picture reference prediction is used.
  • the quantization matrix 802 of FIG. 8C is used for the block.
  • the quantization matrix used is not limited to this, as long as it is the same as the quantization matrix used in the conversion / quantization unit 105 and the inverse quantization / inverse conversion unit 106 of the first embodiment. good.
  • the image reproduction unit 205 reproduces the predicted image by appropriately referring to the frame memory 206 based on the prediction information input from the decoding unit 203. Similar to the prediction unit 104 in the first embodiment, the image reproduction unit 205 of the second embodiment uses three types of prediction methods: intra prediction, inter prediction, and current picture reference prediction. Since the specific prediction process is the same as that of the prediction unit 104 of the first embodiment, the description thereof will be omitted.
  • the image reproduction unit 205 reproduces the image data by adding the predicted image and the prediction error input from the inverse quantization / inverse conversion unit 204, and stores the image data in the frame memory 206. The stored image data is used as a reference when making a prediction.
  • the in-loop filter unit 207 reads the reproduced image from the frame memory 206 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 207 stores the filtered image in the frame memory 206 again.
  • the reproduced image stored in the frame memory 206 is finally output to the outside from the output terminal 208.
  • FIG. 4 is a flowchart showing the coding process for one frame under the control of the control unit 250 in the image decoding apparatus of the second embodiment.
  • the separation / decoding unit 202 separates the bit stream into coded data related to information related to decoding processing and coefficients, and decodes the coded data of the header portion. As a result, the coded data of the quantization matrix is obtained.
  • the separation decoding unit 202 outputs the coded data of the obtained quantization matrix to the quantization matrix decoding unit 209, and outputs the coded data of the subsequent image data to the decoding unit 203.
  • the quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and reproduces the one-dimensional difference matrices 1000 to 1002 shown in FIGS. 10A to 10C. Further, the quantization matrix decoding unit 209 scans each of the reproduced one-dimensional difference matrices 1000 to 1002, and reproduces the two-dimensional quantization matrices 800 to 802. That is, in the present embodiment, the quantization matrix decoding unit 209 scans the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C by using the scanning method shown in FIG. 9, and is shown in FIGS. 8A to 8C. The three types of quantization matrices 800 to 802 are regenerated and retained.
  • the decoding unit 203 decodes the coded data input from the separation decoding unit 202 and reproduces the quantization coefficient and the prediction information.
  • the inverse quantization / inverse conversion unit 204 has a quantization parameter with any of the quantization matrices 800 to 802 held by the quantization matrix decoding unit 209 with respect to the quantization coefficient input from the decoding unit 203. Inverse quantization is performed using and to obtain the conversion coefficient. Specifically, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion of the obtained conversion coefficient and reproduces the prediction error. The inverse quantization / inverse conversion unit 204 of the present embodiment selects the quantization matrix used in the inverse quantization process according to the prediction mode determined by the prediction information reproduced in S403. That is, the inverse quantization / inverse conversion unit 204 uses the quantization matrix 800 of FIG.
  • the quantization matrix 801 of FIG. 8B for the subblock in which the inter prediction is used.
  • the quantization matrix 802 of FIG. 8C is selected for the subblock in which the current picture reference prediction is used, and the quantization matrix selected from the quantization matrix decoding unit 209 is acquired.
  • the quantization matrix used is not limited to this, and may be the same as the quantization matrix used in S306 and S307 of the first embodiment.
  • the image reproduction unit 205 reproduces the predicted image by referring to the frame memory 207 using the prediction information input from the decoding unit 203.
  • three types of prediction methods are used: intra prediction, inter prediction, and current picture reference prediction.
  • the image reproduction unit 205 reproduces the image data by further adding the reproduced predicted image and the prediction error generated by the inverse quantization / inverse conversion unit 204.
  • the image reproduction unit 205 stores the reproduced image data in the frame memory 207.
  • control unit 250 determines whether or not the decoding of all the basic blocks in the frame of interest has been completed, and if so, proceeds to S407, and if not, proceeds to the next basic block. The process returns to S403 for decoding.
  • the in-loop filter unit 207 performs in-loop filter processing on the image data stored in the frame memory 207, generates a filtered image, re-stores it in the frame memory 207, and performs this processing. To finish.
  • the quantization is controlled for each frequency component to decode the bit stream with improved subjective image quality. be able to.
  • the quantization matrix is individually defined for the three types of prediction methods of intra prediction, inter prediction, and current picture reference prediction, and all three types of quantization matrices are decoded.
  • some of them may be shared.
  • the sub-block using the current picture reference prediction is also dequantized using the quantization matrix 800 of FIG. 8A in the same manner as the sub-block using the intra prediction, and the decoding of the quantization matrix 802 of FIG. 8C is omitted. It is also possible to do.
  • the sub-block using the current picture reference prediction is also dequantized using the quantization matrix 801 of FIG. 8B in the same manner as the sub-block using the inter-prediction, and the quantization matrix 802 of FIG. 8C is decoded. It can be omitted.
  • the code amount of 802 minutes of the quantization matrix of FIG. 8C it is also possible to decode a bit stream in which image quality deterioration caused by an error due to prediction using a block-shaped pixel group such as jerky motion is reduced. it can.
  • the quantization matrix for the sub-block using the current picture reference prediction is uniquely determined, but it may be configured to be selectable by introducing an identifier.
  • quantization matrix setting information is provided in the header portion as shown in FIG. 6B.
  • the quantization matrix setting information is "0"
  • the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 800 (FIG. 8A) for intra prediction. ..
  • the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 801 (FIG. 8B) for inter-prediction.
  • the quantization matrix setting information is "2”
  • the quantization matrix used for the subblock using the current picture reference prediction is the dedicated quantization matrix 802 for the current picture reference prediction (FIG. 8C). It is assumed that. This makes it possible to decode a bitstream that selectively realizes the reduction of the quantization matrix code amount and the original quantization control for the subblock using the current picture reference prediction.
  • each of the processing units shown in FIGS. 1 and 2 has been described as being configured by hardware. However, the processing performed by each processing unit shown in these figures may be configured by a computer program.
  • FIG. 5 is a block diagram showing a configuration example of computer hardware applicable to the image coding device and the decoding device according to each of the above embodiments.
  • the CPU 501 controls the entire computer by using the computer programs and data stored in the RAM 502 and the ROM 503, and also executes each of the above-described processes as performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 501 functions as each processing unit shown in FIGS. 1 and 2.
  • the RAM 502 has an area for temporarily storing computer programs and data loaded from the external storage device 506, data acquired from the outside via the I / F (interface) 507, and the like. Further, the RAM 502 has a work area used by the CPU 501 when executing various processes. That is, the RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as appropriate.
  • the ROM 503 stores the setting data of this computer, the boot program, and the like.
  • the operation unit 504 is composed of a keyboard, a mouse, and the like, and can be operated by a user of the computer to input various instructions to the CPU 501.
  • the output unit 505 outputs the processing result by the CPU 501.
  • the output unit 505 is composed of, for example, a liquid crystal display.
  • the external storage device 506 is a large-capacity information storage device represented by a hard disk drive device.
  • the external storage device 506 stores an OS (operating system) and a computer program for realizing the functions of the respective parts shown in FIGS. 1 and 2 in the CPU 501. Further, each image data as a processing target may be stored in the external storage device 506.
  • the computer programs and data stored in the external storage device 506 are appropriately loaded into the RAM 502 according to the control by the CPU 501, and are processed by the CPU 501.
  • a network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 507, and the computer acquires and sends various information via the I / F 507. Can be done.
  • Reference numeral 508 is a bus connecting the above-mentioned parts.
  • the CPU 501 executes the boot program of the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and starts the OS.
  • the present device enables communication via the interface 507 and functions as an information processing device.
  • the CPU 501 loads an application related to image coding (corresponding to FIG. 3) from the external storage device 506 into the RAM 502 and executes it, so that the CPU 501 functions as various processing units shown in FIG.
  • This device will function as an image coding device.
  • the CPU 501 loads an application related to image decoding (corresponding to FIG. 4) from the external storage device 506 into the RAM 502 and executes the CPU 501
  • the CPU 501 functions as various processing units shown in FIG. It will function as an image decoding device.
  • the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
  • a circuit for example, ASIC
  • the present invention is used in a coding device / decoding device that encodes / decodes still images / moving images.
  • it can be applied to a coding method and a decoding method using a quantization matrix.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The purpose of the present invention is to improve subjective image quality by performing quantization using a quantization matrix with respect also to an error in the case of a prediction method in which a block-shaped prediction pixel group of a coded picture is used. To this end, this image coding device, which divides an image into a plurality of blocks and performs coding in units of blocks, comprises: a prediction unit that generates a prediction image of a block of interest to be coded by using a block-shaped pixel group in a coded region of a frame to which the block of interest belongs; a transforming unit that performs orthogonal transform on an error between the pixels of the block of interest and the prediction image, and generates a transform coefficient; and a quantization unit that quantizes the transform coefficient generated by the transforming unit, using a quantization parameter and a quantization matrix.

Description

画像符号化装置、画像復号装置、画像符号化方法、画像復号方法、及びプログラムImage coding device, image decoding device, image coding method, image decoding method, and program
　本発明は画像符号化装置、画像復号装置、画像符号化方法、画像復号方法、及びプログラムに関するものである。 The present invention relates to an image coding device, an image decoding device, an image coding method, an image decoding method, and a program.
　動画像の圧縮記録の符号化方式として、ＨＥＶＣ（High Efficiency Video Coding）符号化方式が知られている。ＨＥＶＣでは符号化効率向上のため、従来のマクロブロック（１６×１６画素）より大きなサイズの基本ブロックが採用された。この大きなサイズの基本ブロックはＣＴＵ（Coding Tree Unit）と呼ばれ、そのサイズは最大６４×６４画素である。ＣＴＵはさらに予測や変換を行う単位となるサブブロックに分割される。 As a coding method for compressed recording of moving images, a HEVC (High Efficiency Video Coding) coding method is known. In HEVC, in order to improve coding efficiency, a basic block having a size larger than that of a conventional macroblock (16 × 16 pixels) has been adopted. This large size basic block is called a CTU (Coding Tree Unit), and its size is a maximum of 64 x 64 pixels. The CTU is further divided into sub-blocks that serve as units for prediction and conversion.
　また、ＨＥＶＣにおいては、量子化マトリクスと呼ばれる、直交変換を施した後の係数（以下、変換係数と記す）を、周波数成分に応じて重み付けをする処理が用いられている。人間の視覚には劣化が目立ちにくい高周波成分のデータをより削減することで、画質を維持しながら圧縮効率を高めることが可能となっている。特許文献１には、このような量子化マトリクスを符号化する技術が開示されている。 Further, in HEVC, a process called a quantization matrix, in which a coefficient after orthogonal conversion (hereinafter referred to as a conversion coefficient) is weighted according to a frequency component is used. By further reducing the data of high-frequency components whose deterioration is not noticeable to human vision, it is possible to improve the compression efficiency while maintaining the image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.
　近年、ＨＥＶＣの後継として、さらに高効率な符号化方式の国際標準化を行う活動が開始された。ＪＶＥＴ（Joint Video Experts Team）がＩＳＯ／ＩＥＣとＩＴＵ－Ｔの間で設立され、ＶＶＣ（Versatile Video Coding）符号化方式として標準化が進められている。ＶＣＣ符号化方式では、符号化効率向上のため、従来のイントラ予測、インター予測に加え、符号化対象ピクチャのブロック状の予測画素群を用いた新たな予測方法（以下、現ピクチャ参照予測と呼称する）が検討されている。 In recent years, as a successor to HEVC, activities to carry out international standardization of more efficient coding methods have started. JVET (Joint Video Experts Team) was established between ISO / IEC and ITU-T, and standardization is being promoted as a VVC (Versatile Video Coding) coding method. In the VCS coding method, in order to improve the coding efficiency, in addition to the conventional intra prediction and inter prediction, a new prediction method using a block-shaped prediction pixel group of the picture to be coded (hereinafter referred to as the current picture reference prediction). ) Is being considered.
特開２０１３－３８７５８号公報Japanese Unexamined Patent Publication No. 2013-38758
　ＶＶＣにおいても、ＨＥＶＣと同様に量子化マトリクスの導入が検討されている。しかしながら、ＨＥＶＣにおける量子化マトリクスは従来のイントラ予測やインター予測といった予測方法を前提としており、新しい予測方法である現ピクチャ参照予測には対応できていない。このため、現ピクチャ参照予測の誤差に対しては、周波数成分に応じた量子化制御を行うことができず、主観画質の向上は期待できない。 In VVC, the introduction of a quantization matrix is being considered in the same way as HEVC. However, the quantization matrix in HEVC is premised on the conventional prediction methods such as intra-prediction and inter-prediction, and cannot correspond to the current picture reference prediction which is a new prediction method. Therefore, the quantization control according to the frequency component cannot be performed for the error of the current picture reference prediction, and the improvement of the subjective image quality cannot be expected.
　本発明は上述した課題を解決するためになされたものであり、上記定義の現ピクチャ参照予測に対応した量子化マトリクスを用いた量子化処理を可能とすることで、主観画質を向上する技術を提供しようとするものである。 The present invention has been made to solve the above-mentioned problems, and a technique for improving subjective image quality by enabling a quantization process using a quantization matrix corresponding to the current picture reference prediction of the above definition. It is what we are trying to provide.
　この課題を解決するため、例えば本発明の画像符号化装置は以下の構成を備える。すなわち、
　画像を複数のブロックに分割し、ブロック単位で符号化を行う画像符号化装置であって、
　符号化対象の着目ブロックの予測画像を、当該着目ブロックが属するフレームの符号化済の領域におけるブロック状の画素群を用いて生成する予測手段と、
　前記着目ブロックの画素と前記予測画像との誤差を直交変換し、変換係数を生成する変換手段と、
　前記変換手段で生成された前記変換係数を、量子化パラメータと量子化マトリクスとを用いて量子化する量子化手段とを有する。 In order to solve this problem, for example, the image coding apparatus of the present invention has the following configuration. That is,
An image coding device that divides an image into multiple blocks and encodes them in block units.
A prediction means for generating a prediction image of a block of interest to be encoded by using a block-shaped pixel group in a coded region of a frame to which the block of interest belongs.
A conversion means that generates a conversion coefficient by orthogonally converting the error between the pixel of the block of interest and the predicted image.
It has a quantization means for quantizing the conversion coefficient generated by the conversion means by using a quantization parameter and a quantization matrix.
　本発明によれば、符号化済ピクチャのブロック状の予測画素群を用いた新しい予測方法に対する誤差に対しても量子化マトリクスを用いた量子化を行い、主観画質を向上させることができる。
　本発明のその他の特徴及び利点は、添付図面を参照とした以下の説明により明らかになるであろう。なお、添付図面においては、同じ若しくは同様の構成には、同じ参照番号を付す。 According to the present invention, it is possible to improve the subjective image quality by performing quantization using a quantization matrix even for an error with respect to a new prediction method using a block-shaped prediction pixel group of a coded picture.
Other features and advantages of the present invention will become apparent in the following description with reference to the accompanying drawings. In the attached drawings, the same or similar configurations are given the same reference numbers.
　添付図面は明細書に含まれ、その一部を構成し、本発明の実施の形態を示し、その記述と共に本発明の原理を説明するために用いられる。
第１の実施形態における画像符号化装置のブロック構成図。 第２の実施形態における画像復号装置のブロック構成図。 第１の実施形態の画像符号化装置の符号化処理を示すフローチャート。 第２の実施形態の画像復号装置の復号処理を示すフローチャート。 画像符号化装置、復号装置に適用可能なコンピュータのハードウェア構成例を示す図。 実施形態におけるビットストリームの構造例を示す図。 実施形態におけるビットストリームの構造例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられるサブブロック分割の例を示す図。 実施形態で用いられる量子化マトリクスの例を示す図。 実施形態で用いられる量子化マトリクスの例を示す図。 実施形態で用いられる量子化マトリクスの例を示す図。 実施形態で用いられる量子化マトリクスの要素の走査方法を示す図。 実施形態で生成される量子化マトリクスの差分値行列を示す図。 実施形態で生成される量子化マトリクスの差分値行列を示す図。 実施形態で生成される量子化マトリクスの差分値行列を示す図。 実施形態での符号化処理に用いられる符号化テーブルの例を示す図。 実施形態での符号化処理に用いられる符号化テーブルの例を示す図。 The accompanying drawings are included in the specification and are used to form a part thereof, show embodiments of the present invention, and explain the principles of the present invention together with the description thereof.
The block block diagram of the image coding apparatus in 1st Embodiment. The block block diagram of the image decoding apparatus in the 2nd Embodiment. The flowchart which shows the coding process of the image coding apparatus of 1st Embodiment. The flowchart which shows the decoding process of the image decoding apparatus of 2nd Embodiment. The figure which shows the hardware configuration example of the computer applicable to the image coding apparatus and the decoding apparatus. The figure which shows the structural example of the bit stream in embodiment. The figure which shows the structural example of the bit stream in embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the sub-block division used in an embodiment. The figure which shows the example of the quantization matrix used in embodiment. The figure which shows the example of the quantization matrix used in embodiment. The figure which shows the example of the quantization matrix used in embodiment. The figure which shows the scanning method of the element of the quantization matrix used in embodiment. The figure which shows the difference value matrix of the quantization matrix generated by embodiment. The figure which shows the difference value matrix of the quantization matrix generated by embodiment. The figure which shows the difference value matrix of the quantization matrix generated by embodiment. The figure which shows the example of the coding table used for the coding process in an embodiment. The figure which shows the example of the coding table used for the coding process in an embodiment.
　以下、添付図面を参照して実施形態を詳しく説明する。尚、以下の実施形態は特許請求の範囲に係る発明を限定するものでするものでない。実施形態には複数の特徴が記載されているが、これらの複数の特徴の全てが発明に必須のものとは限らず、また、複数の特徴は任意に組み合わせられてもよい。さらに、添付図面においては、同一若しくは同様の構成に同一の参照番号を付し、重複した説明は省略する。また、基本ブロックや、サブブロック、量子化マトリクスといった呼称は、各実施形態において便宜的に用いている呼称であり、その意味が変わらない範囲で、適宜、他の呼称を用いてもよい。例えば、基本ブロックやサブブロックは、基本ユニットやサブユニットと称されてもよいし、単にブロックやユニットと称されてもよい。 Hereinafter, embodiments will be described in detail with reference to the attached drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate description is omitted. Further, the names such as the basic block, the sub-block, and the quantization matrix are the names used for convenience in each embodiment, and other names may be appropriately used as long as the meanings do not change. For example, a basic block or subunit may be referred to as a basic unit or subunit, or may simply be referred to as a block or unit.
　［第１の実施形態］
　図１は本実施形態が適用する画像符号化装置を示すブロック図である。なお、実施形態における画像符号化装置は、符号化対象の画像データを発生する撮像素子を持つデジタルビデオカメラに実装されるものとして説明する。しかし、これはあくまで理解を容易にするためであり、符号化対象の画像発生源は特に問わない点に留意願いたい。 [First Embodiment]
FIG. 1 is a block diagram showing an image coding device to which the present embodiment is applied. The image coding device according to the embodiment will be described as being mounted on a digital video camera having an image sensor that generates image data to be encoded. However, it should be noted that this is just for ease of understanding, and the source of the image to be encoded does not matter.
　制御部１５０は、ＣＰＵ、ＣＰＵが実行するプログラムや各種パラメータを記憶しているＲＯＭ、及び、ＣＰＵのワークエリアとして利用されるＲＡＭで構成され、以下に説明する各構成要素の制御を行うことで画像符号化装置の全体の制御を司る。 The control unit 150 is composed of a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM that is used as a work area of the CPU, and controls each component described below. It controls the entire image encoding device.
　ブロック分割部１０２は、撮像部で撮像された画像データを、入力端子１０１を介して入力する。そして、ブロック分割部１０２は、その入力画像を複数の基本ブロックに分割し、基本ブロック単位の画像データを後段の予測部１０４に出力する。 The block division unit 102 inputs the image data captured by the imaging unit via the input terminal 101. Then, the block division unit 102 divides the input image into a plurality of basic blocks, and outputs the image data of the basic block unit to the prediction unit 104 in the subsequent stage.
　量子化マトリクス保持部１０３は、量子化マトリクスを生成、及び、保持する。量子化マトリクスの生成方法は特に問わないが、ユーザが量子化マトリクスを入力しても良いし、入力画像の特性から算出しても、初期値として予め指定されたものを使用しても良い。本実施形態の量子化マトリクス保持部１０３は、図８Ａ～８Ｃに示される３種類の８×８画素の直交変換に対応した二次元の量子化マトリクスを生成し、保持するものとする。ここで、量子化マトリクスとは、周波数成分に応じて、変換係数に対する量子化処理を重み付けするためのものである。各変換係数のための量子化ステップは、基準となる値に、量子化マトリクスにおける各要素の値を乗算することで重み付けされる。なお、量子化ステップの基準となる値は、量子化パラメータによって、定義される。 The quantization matrix holding unit 103 generates and holds a quantization matrix. The method of generating the quantization matrix is not particularly limited, but the user may input the quantization matrix, calculate from the characteristics of the input image, or use a predetermined value as an initial value. The quantization matrix holding unit 103 of the present embodiment generates and holds a two-dimensional quantization matrix corresponding to the orthogonal conversion of three types of 8 × 8 pixels shown in FIGS. 8A to 8C. Here, the quantization matrix is for weighting the quantization process with respect to the conversion coefficient according to the frequency component. The quantization step for each transformation factor is weighted by multiplying the reference value by the value of each element in the quantization matrix. The reference value of the quantization step is defined by the quantization parameter.
　予測部１０４は、ブロック分割部１０２から受信した基本ブロック単位の画像データに対し、サブブロック分割法を決定し、サブブロック単位でフレーム内予測であるイントラ予測やフレーム間予測であるインター予測などを行い、予測画像データを生成する。さらに、予測部１０４は、サブブロックの画像データと予測画像データとの差分である予測誤差を算出し、出力する。また、予測部１０４は、予測に必要な情報、例えばサブブロック分割法、予測モードや動きベクトル等の情報も予測誤差と併せて出力される。以下ではこの予測に必要な情報を予測情報と呼称する。 The prediction unit 104 determines the sub-block division method for the image data in the basic block unit received from the block division unit 102, and performs intra-frame prediction, which is intra-frame prediction, inter-frame prediction, which is inter-frame prediction, and the like in sub-block units. And generate the predicted image data. Further, the prediction unit 104 calculates and outputs a prediction error which is a difference between the image data of the subblock and the predicted image data. In addition, the prediction unit 104 also outputs information necessary for prediction, for example, information such as a sub-block division method, a prediction mode, and a motion vector, together with a prediction error. Hereinafter, the information necessary for this prediction is referred to as prediction information.
　変換・量子化部１０５は、予測部１０４からのサブブロック単位の予測誤差を直交変換することで変換係数を生成する。さらに変換・量子化部１０５は、量子化マトリクス保持部１０３に格納されている量子化マトリクスと、量子化パラメータとを用いて、生成した変換係数の量子化を行うことで量子化後の変換係数（以下、単に量子化係数）を得る。なお、直交変換を行う機能と、量子化を行う機能とは別々の構成にしてもよい。 The conversion / quantization unit 105 generates a conversion coefficient by orthogonally converting the prediction error in subblock units from the prediction unit 104. Further, the conversion / quantization unit 105 uses the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameter to quantize the generated conversion coefficient, so that the conversion coefficient after quantization is performed. (Hereinafter, simply the quantization coefficient) is obtained. The function of performing orthogonal conversion and the function of performing quantization may be configured separately.
　逆量子化・逆変換部１０６は、変換・量子化部１０５から出力された量子化係数を量子化マトリクス保持部１０３に格納されている量子化マトリクスと、量子化パラメータとを用いて逆量子化することで変換係数を再生する。さらに逆量子化・逆変換部１０６は、再生した変換係数に対して逆直交変換して予測誤差を再生する。なお、逆量子化を行う機能と、逆直交変換を行う機能とは別々の構成にしてもよい。 The inverse quantization / inverse conversion unit 106 dequantizes the quantization coefficient output from the conversion / quantization unit 105 using the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameters. By doing so, the conversion coefficient is reproduced. Further, the inverse quantization / inverse conversion unit 106 reproduces the prediction error by performing inverse orthogonal conversion with respect to the reproduced conversion coefficient. The function of performing inverse quantization and the function of performing inverse orthogonal transformation may be configured separately.
　画像再生部１０７は、予測部１０４から出力された予測情報に基づいて、フレームメモリ１０８を適宜参照して、符号化対象の着目ブロック用の予測画像データを生成する。そして、画像再生部１０７は、この予測画像データに、逆量子化・逆変換部１０６から入力された予測誤差を加算することで、再生画像データを生成し、フレームメモリ１０８に格納する。 The image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information output from the prediction unit 104, and generates the prediction image data for the block of interest to be encoded. Then, the image reproduction unit 107 generates the reproduction image data by adding the prediction error input from the inverse quantization / inverse conversion unit 106 to the predicted image data, and stores the reproduced image data in the frame memory 108.
　インループフィルタ部１０９は、フレームメモリ１０８に格納された再生画像に対し、デブロッキングフィルタやサンプルアダプティブオフセットなどのインループフィルタ処理を行い、フィルタ処理後の画像データを再びフレームメモリ１０８に格納する。 The in-loop filter unit 109 performs in-loop filter processing such as a deblocking filter and a sample adaptive offset on the reproduced image stored in the frame memory 108, and stores the filtered image data in the frame memory 108 again.
　符号化部１１０は、変換・量子化部１０５から出力された量子化係数および予測部１０４から出力された予測情報を符号化して、符号化データを生成し、出力する。なお、画像復号装置が量子化パラメータを導出するための情報も符号化部１１０によってビットストリームに符号化される。 The coding unit 110 encodes the quantization coefficient output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104, generates coded data, and outputs the coded data. The information for the image decoding device to derive the quantization parameter is also encoded in the bit stream by the coding unit 110.
　量子化マトリクス符号化部１１３は、量子化マトリクス保持部１０３に保持されている量子化マトリクスを符号化して、量子化マトリクスの符号化データを生成し、出力する。 The quantization matrix coding unit 113 encodes the quantization matrix held in the quantization matrix holding unit 103, generates the coding data of the quantization matrix, and outputs it.
　統合符号化部１１１は、量子化マトリクス符号化部１１３からの出力である量子化マトリクスの符号化データを用いてヘッダデータを生成する。更に、統合符号化部１１１は、ヘッダデータと、符号化部１１０から出力された符号化データと合わせて、ビットストリームを形成して、出力端子１１２を介して外部に出力する。出力対象は、実施形態の場合にメモリカード等の記録媒体となるが、出力対象はネットワーク等でも構わず、その種類は特に問わない。 The integrated coding unit 111 generates header data using the coding data of the quantization matrix which is the output from the quantization matrix coding unit 113. Further, the integrated coding unit 111 forms a bit stream together with the header data and the coded data output from the coding unit 110, and outputs the bit stream to the outside via the output terminal 112. In the case of the embodiment, the output target is a recording medium such as a memory card, but the output target may be a network or the like, and the type thereof is not particularly limited.
　実施形態における画像符号化装置の構成と動作は、概ね上記の通りであるが、以下、より詳しく説明する。本実施形態では符号化対象は動画像であり、フレーム単位に入力する構成とする。さらに本実施形態では説明を単純化するため、ブロック分割部１０２においては８×８画素の基本ブロックに分割するものとして説明するが、これはあくまで理解を容易にするためであると理解されたい。例えば、基本ブロックよりも大きいブロック（例えば、６４×６４画素のブロックや１２８×１２８画素のブロック）を基準として、それを様々な大きさの基本ブロックに分割するようにしてもよい。 The configuration and operation of the image coding device in the embodiment are generally as described above, but will be described in more detail below. In the present embodiment, the coding target is a moving image, and it is configured to be input in frame units. Further, in the present embodiment, in order to simplify the description, the block division unit 102 will be described as being divided into basic blocks of 8 × 8 pixels, but it should be understood that this is only for facilitating understanding. For example, a block larger than the basic block (for example, a block having 64 × 64 pixels or a block having 128 × 128 pixels) may be used as a reference and divided into basic blocks having various sizes.
　画像の符号化に先立ち、量子化マトリクスの生成および符号化が行われる。 Prior to image coding, the quantization matrix is generated and coded.
　最初に、制御部１５０は、量子化マトリクス保持部１０３を制御し、量子化マトリクスを生成させる。量子化マトリクス保持部１０３は、符号化を行うサブブロックのサイズや予測方法の種類に応じて、量子化マトリクスを生成する。本実施形態では、図７Ａに示される８×８画素の基本ブロックに対して、分割無しとした８×８画素のサイズのサブブロック用の量子化マトリクスを生成するものとする。ただし、生成される量子化マトリクスはこれに限定されず、４×８や８×４、４×４など、サブブロックの形状に対応した量子化マトリクスが生成されても良い。量子化マトリクスを構成する各要素の決定方法は特に問わない。例えば、所定の初期値を用いても良いし、個別に設定しても良い。また、画像の特性に応じて生成されても構わない。 First, the control unit 150 controls the quantization matrix holding unit 103 to generate a quantization matrix. The quantization matrix holding unit 103 generates a quantization matrix according to the size of the subblock to be encoded and the type of prediction method. In the present embodiment, it is assumed that a quantization matrix for a subblock having a size of 8 × 8 pixels without division is generated for the basic block of 8 × 8 pixels shown in FIG. 7A. However, the generated quantization matrix is not limited to this, and a quantization matrix corresponding to the shape of the subblock such as 4 × 8 or 8 × 4, 4 × 4 may be generated. The method for determining each element constituting the quantization matrix is not particularly limited. For example, a predetermined initial value may be used, or may be set individually. Further, it may be generated according to the characteristics of the image.
　量子化マトリクス保持部１０３は、このようにして生成した量子化マトリクスを保持する。量子化マトリクス保持部１０３が生成する量子化マトリクス例を図８Ａ～８Ｃに示す。図８Ａはイントラ予測用、図８Ｂはインター予測用、図８Ｃは現ピクチャ参照予測用に対応する量子化マトリクスの一例である。太枠は、量子化マトリクスを表している。説明を簡易にするため、それぞれ８×８の６４画素分の構成とし、太枠内の各正方形は量子化マトリクスを構成している各要素値を表しているものとする。本実施形態では、図８Ａ～８Ｃに示された三種の量子化マトリクス８００～８０２が二次元の形状で保持されているものとするが、量子化マトリクス８００～８０２内の各要素はもちろんこれに限定されない。また、サブブロックのサイズによって、あるいは符号化対象が輝度ブロックか色差ブロックかによって、同じ予測方法に対して複数の量子化マトリクスを保持することも可能である。一般的に、量子化マトリクスは人間の視覚特性に応じた量子化処理を実現するため、図８Ａ～８Ｃに示すように量子化マトリクス８００～８０２の左上部分に相当する低周波部分の要素値は小さく、右下部分に相当する高周波部分の要素値は大きくなっている。 The quantization matrix holding unit 103 holds the quantization matrix generated in this way. Examples of the quantization matrix generated by the quantization matrix holding unit 103 are shown in FIGS. 8A to 8C. FIG. 8A is an example of a quantization matrix corresponding to intra-prediction, FIG. 8B is for inter-prediction, and FIG. 8C is for current picture reference prediction. The thick frame represents the quantization matrix. For the sake of simplicity, it is assumed that each square is composed of 64 pixels of 8 × 8, and each square in the thick frame represents each element value constituting the quantization matrix. In the present embodiment, it is assumed that the three types of quantization matrices 800 to 802 shown in FIGS. 8A to 8C are held in a two-dimensional shape, but each element in the quantization matrix 800 to 802 is of course included in this. Not limited. It is also possible to hold a plurality of quantization matrices for the same prediction method depending on the size of the subblock or whether the coding target is a luminance block or a color difference block. In general, since the quantization matrix realizes the quantization process according to the human visual characteristics, as shown in FIGS. 8A to 8C, the element values of the low frequency portion corresponding to the upper left portion of the quantization matrices 800 to 802 are set. It is small and the element value of the high frequency part corresponding to the lower right part is large.
　量子化マトリクス符号化部１１３は、図８Ａ～８Ｃに示すような二次元形状で格納されている量子化マトリクス８００～８０２を量子化マトリクス保持部１０６から順に読み出し、各要素を走査（スキャン）して差分を計算し、一次元の行列に配置する。本実施形態の量子化マトリクス符号化部１１３は、図８Ａ～８Ｃに示された各量子化マトリクス８００～８０２を、図９に示された走査方法を用い、要素ごとに走査順に直前の要素との差分を計算する。例えば図８Ｃで示された８×８画素分の量子化マトリクス８０２は図９で示された走査方法によって走査されるが、左上隅に位置する最初の要素“８”の次はそのすぐ下に位置する要素“１１”が走査され、差分である＋３が計算される。また、量子化マトリクス８０２の最初の要素（本実施形態では“８”）の符号化には、所定の初期値（例えば“８”）との差分を計算するものとするが、もちろんこれに限定されず、任意の値との差分や、最初の要素の値そのものを用いても良い。 The quantization matrix coding unit 113 reads out the quantization matrices 800 to 802 stored in the two-dimensional shape as shown in FIGS. 8A to 8C in order from the quantization matrix holding unit 106, and scans (scans) each element. Calculate the difference and place it in a one-dimensional matrix. The quantization matrix coding unit 113 of the present embodiment uses each of the quantization matrices 800 to 802 shown in FIGS. 8A to 8C by the scanning method shown in FIG. 9, and sets each element as the immediately preceding element in the scanning order. Calculate the difference between. For example, the 8 × 8 pixel quantization matrix 802 shown in FIG. 8C is scanned by the scanning method shown in FIG. 9, but immediately below the first element “8” located in the upper left corner. The located element "11" is scanned and the difference +3 is calculated. Further, in the coding of the first element (“8” in this embodiment) of the quantization matrix 802, the difference from a predetermined initial value (for example, “8”) is calculated, but of course, it is limited to this. Instead, the difference from an arbitrary value or the value of the first element itself may be used.
　このようにして、図８Ａ～８Ｃの量子化マトリクス８００～８０２は、図９の走査方法を用い、図１０Ａ～１０Ｃに示される差分行列１０００～１００２が生成される。量子化マトリクス符号化部１１３は、さらに差分行列１０００～１００２を符号化して量子化マトリクスの符号化データを生成する。本実施形態では図１１Ａに示される符号化テーブルを用いて符号化するものとするが、符号化テーブルはこれに限定されず、例えば図１１Ｂに示される符号化テーブルを用いても良い。量子化マトリクス符号化部１１３は、このようにして生成された図８Ａ～８Ｃに示す量子化マトリクス８００～８０２の符号化データを後段の統合符号化部１１１に出力する。 In this way, the quantization matrices 800 to 802 of FIGS. 8A to 8C use the scanning method of FIG. 9, and the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C are generated. The quantization matrix coding unit 113 further encodes the difference matrices 1000 to 1002 to generate the coding data of the quantization matrix. In the present embodiment, coding is performed using the coding table shown in FIG. 11A, but the coding table is not limited to this, and for example, the coding table shown in FIG. 11B may be used. The quantization matrix coding unit 113 outputs the coded data of the quantization matrices 800 to 802 shown in FIGS. 8A to 8C thus generated to the integrated coding unit 111 in the subsequent stage.
　図１に戻り、統合符号化部１１１は画像データの符号化に必要なヘッダ情報を符号化し、量子化マトリクスの符号化データを統合する。 Returning to FIG. 1, the integrated coding unit 111 encodes the header information necessary for encoding the image data and integrates the encoded data of the quantization matrix.
　続いて、画像データの符号化が行われる。端子１０１から入力された１フレーム分の画像データはブロック分割部１０２に供給される。ブロック分割部１０２は、入力されたフレームの画像データを複数の基本ブロックに分割し、基本ブロック単位の画像データを予測部１０４に出力する。本実施形態での基本ブロックのサイズは８×８画素である。 Subsequently, the image data is encoded. The image data for one frame input from the terminal 101 is supplied to the block dividing unit 102. The block division unit 102 divides the image data of the input frame into a plurality of basic blocks, and outputs the image data of each basic block unit to the prediction unit 104. The size of the basic block in this embodiment is 8 × 8 pixels.
　予測部１０４は、ブロック分割部１０２から入力された基本ブロックの画像データに対し予測処理を実行する。具体的には、基本ブロックをさらに細かいサブブロックに分割するサブブロック分割法を決定し、さらにサブブロック単位でイントラ予測やインター予測、現ピクチャ参照予測などの予測モードを決定する。 The prediction unit 104 executes prediction processing on the image data of the basic block input from the block division unit 102. Specifically, a subblock division method for dividing a basic block into finer subblocks is determined, and a prediction mode such as intra prediction, inter prediction, or current picture reference prediction is determined for each subblock.
　図７Ａ～７Ｆにサブブロック分割方法の例を示す。太枠を表す参照符号７００～７０５はそれぞれ基本ブロックを表しており、説明を簡易にするため、８×８画素の構成とし、太枠内の各四角形はサブブロックを表している。図７Ａは基本ブロック７００＝サブブロックの例を示している。図７Ｂは従来の正方形サブブロック分割の一例を表しており、８×８画素の基本ブロック７０１が４つの４×４画素のサブブロックに分割されている。一方、図７Ｃ～７Ｆは、基本ブロック７０２乃至７０５が２つまたは３つの長方形サブブロックで構成されることを示している。図７Ｃは基本ブロック７０２が垂直方向に長手の２つの４×８画素へのサブブロック分割を示している。図７Ｄは基本ブロック７０３が水平方向に長手の２つの８×４画素へのサブブロック分割を示している。また、図７Ｅは、基本ブロック７０４が、垂直方向に長手で、幅が１：２：１の比の３つの長方形サブブロック分割を示している。そして、図７Ｆは、基本ブロック７０５が、水平方向に長手で、幅が１：２：１の比の３つの長方形サブブロック分割を示している。このように正方形だけではなく、長方形のサブブロックも用いて符号化処理を行っている。 Figures 7A to 7F show examples of subblock division methods. Reference numerals 700 to 705 representing the thick frame each represent a basic block, and for the sake of simplicity, the configuration is 8 × 8 pixels, and each quadrangle in the thick frame represents a subblock. FIG. 7A shows an example of a basic block 700 = a subblock. FIG. 7B shows an example of the conventional square subblock division, in which the 8 × 8 pixel basic block 701 is divided into four 4 × 4 pixel subblocks. On the other hand, FIGS. 7C to 7F show that the basic blocks 702 to 705 are composed of two or three rectangular subblocks. FIG. 7C shows a subblock division in which the basic block 702 is vertically elongated into two 4 × 8 pixels. FIG. 7D shows the subblock division of the basic block 703 into two horizontally long 8 × 4 pixels. Further, FIG. 7E shows three rectangular sub-block divisions in which the basic block 704 is vertically longitudinal and has a width of 1: 2: 1. And FIG. 7F shows three rectangular subblock divisions in which the basic block 705 is horizontally longitudinal and has a width of 1: 2: 1. In this way, not only squares but also rectangular sub-blocks are used for encoding processing.
　本実施形態では、８×８画素の基本ブロックを、分割しない図７Ａのみが用いられるものとするが、サブブロック分割方法はこれに限定されない。図７Ｂのような四分木分割や、図７Ｅ，７Ｆのような三分木分割または図７Ｃや図７Ｄのような二分木分割を用いても構わない。図７Ａ以外のサブブロック分割も用いられる場合には、量子化マトリクス保持部１０３にて使用されるサブブロックに対応する量子化マトリクスが生成される。また、生成された量子化マトリクスは量子化マトリクス符号化部１１３にて符号化されることとなる。 In the present embodiment, it is assumed that only FIG. 7A, which does not divide the basic block of 8 × 8 pixels, is used, but the sub-block division method is not limited to this. A quadtree division as shown in FIG. 7B, a ternary division as shown in FIGS. 7E and 7F, or a binary tree division as shown in FIGS. 7C and 7D may be used. When the subblock division other than FIG. 7A is also used, the quantization matrix corresponding to the subblock used in the quantization matrix holding unit 103 is generated. Further, the generated quantization matrix will be encoded by the quantization matrix coding unit 113.
　また、本実施形態で用いられる予測方法について、改めて説明する。本実施形態ではイントラ予測、インター予測、現ピクチャ参照予測の３種類の予測方法が用いられる。 In addition, the prediction method used in this embodiment will be described again. In this embodiment, three types of prediction methods are used: intra prediction, inter prediction, and current picture reference prediction.
　イントラ予測は、符号化対象ブロックの空間的に周辺に位置する符号化済画素を用いて符号化対象ブロックの予測画素を生成するものである。予測画素の生成方法は、水平予測や垂直予測などの３３種類の方向にかかるＡｎｇｕｌａｒ予測、隣接ブロックの復号画素値の平均値を予測値とするＤＣ予測、隣接ブロックの復号画素から平面近似により予測画素値を生成するＰｌａｎａｒ予測などが含まれ、いずれを用いたのかを示す情報も生成する。 Intra-prediction is to generate predicted pixels of a coded block by using coded pixels located spatially around the coded block. Prediction pixel generation methods include Angular prediction for 33 directions such as horizontal prediction and vertical prediction, DC prediction with the average value of the decoded pixel values of adjacent blocks as the predicted value, and plane approximation from the decoded pixels of the adjacent block. Planar prediction and the like for generating pixel values are included, and information indicating which one is used is also generated.
　インター予測は、符号化対象ブロックとは時間的に異なるフレームの符号化済画素を用いて符号化対象ブロックの予測画素を生成し、参照するフレームや動きベクトルなどを示す動き情報も生成する。 Inter-prediction generates predicted pixels of a coded target block using encoded pixels of a frame that is different in time from the coded target block, and also generates motion information indicating a frame to be referred to, a motion vector, and the like.
　そして、現ピクチャ参照予測は、符号化対象ブロックが属するフレームの符号化済領域内から、類似したブロック状の画素群を符号化対象ブロックの予測ブロックとして生成するものである。例えば符号化対象ブロックから左方向にｘ画素、上方向にｙ画素離れた位置に符号化対象ブロックと類似したブロック状の画素群が存在した場合、この「左にｘ画素、上にｙ画素」を変位情報として生成する。こうして現ピクチャ参照予測では、符号化対象ブロックの予測ブロックを生成し、予測画素の生成に用いられた変位情報も生成する。この現ピクチャ参照予測は、特に符号化・復号対象のフレーム内に同一の文字やテクスチャが繰り返し出現するコンピュータの画面のような人工の画像において、符号化効率を向上させるのに適した技術であると言える。 Then, in the current picture reference prediction, a similar block-shaped pixel group is generated as a prediction block of the coding target block from within the coded region of the frame to which the coding target block belongs. For example, if there is a block-shaped pixel group similar to the coded block at a position separated from the coded block by x pixels in the left direction and y pixels in the upward direction, this "x pixel on the left and y pixel on the top". Is generated as displacement information. In this way, in the current picture reference prediction, the prediction block of the coded block is generated, and the displacement information used for generating the prediction pixel is also generated. This current picture reference prediction is a technique suitable for improving the coding efficiency, especially in an artificial image such as a computer screen in which the same characters and textures repeatedly appear in a frame to be coded / decoded. It can be said that.
　予測部１０４は、上記いずれかの予測モードの中から誤差が最小となる予測モードを決定する。そして、予測部１０４は、決定した予測モードおよび符号化済の画素から予測画像データを生成し、さらに入力された画像データと予測画像データから予測誤差を生成し、予測誤差を変換・量子化部１０５に出力する。また、予測部１０４は、サブブロック分割や予測モードなどの情報を予測情報として、変換・量子化部１０５、符号化部１１０、画像再生部１０７、逆量子化・逆変換部１０６に出力する。 The prediction unit 104 determines the prediction mode that minimizes the error from any of the above prediction modes. Then, the prediction unit 104 generates prediction image data from the determined prediction mode and encoded pixels, further generates prediction error from the input image data and prediction image data, and converts and quantizes the prediction error. Output to 105. Further, the prediction unit 104 outputs information such as subblock division and prediction mode as prediction information to the conversion / quantization unit 105, the coding unit 110, the image reproduction unit 107, and the inverse quantization / inverse conversion unit 106.
　変換・量子化部１０５は、予測部１０４より入力した予測誤差に対して直交変換を行い、変換係数を得る。そして、変換・量子化部１０５は、予測モードに従った量子化マトリクスと、量子化パラメータとを用いて、変換係数に対して量子化を行い、量子化係数を生成する。具体的には、変換・量子化部１０５は、サブブロックのサイズに対応した直交変換処理を施し直交変換係数を生成し、次に変換係数を予測モードに応じて量子化マトリクス保持部１０３に格納されている量子化マトリクスと、量子化パラメータとを用いて量子化し、量子化係数を生成する。本実施形態の変換・量子化部１０５は、イントラ予測で予測が行われたサブブロックの変換係数に対しては図８Ａの量子化マトリクス８００を用いて量子化を行う。また、インター予測が行われたサブブロックの変換係数には図８Ｂの量子化マトリクス８０１を用いて量子化を行う。そして、現ピクチャ参照予測が行われたサブブロックの変換係数には図８Ｃの量子化マトリクス８０２を用いて量子化を行う。ただし、使用される量子化マトリクスはこれに限定されない。そして、変換・量子化部１０５は、量子化処理で得た量子化係数を符号化部１１０および逆量子化・逆変換部１０６に出力する。 The conversion / quantization unit 105 performs orthogonal conversion on the prediction error input from the prediction unit 104 to obtain a conversion coefficient. Then, the conversion / quantization unit 105 performs quantization with respect to the conversion coefficient using the quantization matrix according to the prediction mode and the quantization parameter, and generates the quantization coefficient. Specifically, the conversion / quantization unit 105 performs orthogonal conversion processing corresponding to the size of the subblock to generate an orthogonal conversion coefficient, and then stores the conversion coefficient in the quantization matrix holding unit 103 according to the prediction mode. Quantization is performed using the quantization matrix and the quantization parameters, and the quantization coefficient is generated. The conversion / quantization unit 105 of the present embodiment quantizes the conversion coefficient of the subblock predicted by the intra prediction using the quantization matrix 800 of FIG. 8A. Further, the conversion coefficient of the sub-block in which the inter-prediction is performed is quantized using the quantization matrix 801 of FIG. 8B. Then, the conversion coefficient of the sub-block for which the current picture reference prediction has been performed is quantized using the quantization matrix 802 of FIG. 8C. However, the quantization matrix used is not limited to this. Then, the conversion / quantization unit 105 outputs the quantization coefficient obtained in the quantization process to the coding unit 110 and the inverse quantization / inverse conversion unit 106.
　逆量子化・逆変換部１０６は、入力された量子化係数を量子化マトリクス保持部１０３に保持された量子化マトリクスと、量子化パラメータとを用いて逆量子化して変換係数を再生する。このように、量子化マトリクスと量子化パラメータとを用いて、変換係数を再生（導出）する処理を逆量子化と称するものとする。更に、逆量子化・逆変換部１０６は、再生された変換係数を逆直交変換して予測誤差を再生する。逆量子化・逆変換部１０６による逆量子化処理で用いる量子化マトリクスは、変換・量子化部１０５が符号化対象ブロックの変換係数に対して用いた量子化マトリクスと同じものが用いられる。そのため、逆量子化・逆変換部１０６は、予測部１０４からの予測情報に基づいて、利用する量子化マトリクスを決定することになる。逆量子化・逆変換部１０６は、再生した予測誤差を画像再生部１０７に出力する。 The inverse quantization / inverse conversion unit 106 inversely quantizes the input quantization coefficient using the quantization matrix held in the quantization matrix holding unit 103 and the quantization parameter, and reproduces the conversion coefficient. In this way, the process of reproducing (deriving) the conversion coefficient using the quantization matrix and the quantization parameter is referred to as inverse quantization. Further, the inverse quantization / inverse conversion unit 106 performs inverse orthogonal conversion of the reproduced conversion coefficient to reproduce the prediction error. As the quantization matrix used in the inverse quantization process by the inverse quantization / inverse conversion unit 106, the same one as the quantization matrix used by the conversion / quantization unit 105 for the conversion coefficient of the coded block is used. Therefore, the inverse quantization / inverse conversion unit 106 determines the quantization matrix to be used based on the prediction information from the prediction unit 104. The inverse quantization / inverse conversion unit 106 outputs the reproduced prediction error to the image reproduction unit 107.
　画像再生部１０７は、予測部１０４から入力した予測情報に基づいて、フレームメモリ１０８を適宜参照し、サブブロックの予測画像を再生する。そして再生された予測画像と、逆量子化・逆変換部１０６より入力したサブブロックの予測誤差から画像データを再生し、フレームメモリ１０８に格納する。 The image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104, and reproduces the predicted image of the subblock. Then, the image data is reproduced from the reproduced predicted image and the prediction error of the subblock input from the inverse quantization / inverse conversion unit 106, and stored in the frame memory 108.
　インループフィルタ部１０９は、フレームメモリ１０８から再生画像を読み出し、デブロッキングフィルタなどのインループフィルタ処理を行う。そして、インループフィルタ部１０９は、フィルタ処理後の画像データをフレームメモリ１０８に再格納する。 The in-loop filter unit 109 reads the reproduced image from the frame memory 108 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 109 re-stores the image data after the filter processing in the frame memory 108.
　符号化部１１０は、ブロック単位で、変換・量子化部１０５で生成された量子化係数、予測部１０４から入力された予測情報をエントロピー符号化し、符号化データを生成する。エントロピー符号化の方法は特に指定しないが、ゴロム符号化、算術符号化、ハフマン符号化などを用いることができる。生成された符号化データは統合符号化部１１１に出力される。 The coding unit 110 entropy-encodes the quantization coefficient generated by the conversion / quantization unit 105 and the prediction information input from the prediction unit 104 in block units to generate coded data. The method of entropy coding is not particularly specified, but Golomb coding, arithmetic coding, Huffman coding and the like can be used. The generated coded data is output to the integrated coding unit 111.
　統合符号化部１１１では、前述のヘッダの符号化データ（量子化マトリクスの符号化データが含まれる）と、符号化部１１０から入力した符号化データなどを多重化してビットストリームを形成する。そして、統合符号化部１１１は、形成したビットストリームは端子１１２から本装置外（例えば記録媒体）に出力する。 The integrated coding unit 111 forms a bit stream by multiplexing the above-mentioned header coding data (including the coding data of the quantization matrix) and the coding data input from the coding unit 110. Then, the integrated coding unit 111 outputs the formed bit stream from the terminal 112 to the outside of the apparatus (for example, a recording medium).
　図６Ａは第１の実施形態で出力されるビットストリームの一例である。シーケンスヘッダには量子化マトリクスの符号化データが含まれ、各要素の符号化結果で構成されている。ただし、符号化される位置はこれに限定されず、ピクチャヘッダ部やその他のヘッダ部に符号化される構成をとってももちろん構わない。また、１つのシーケンスの中で量子化マトリクスの変更を行う場合、量子化マトリクスを新たに符号化することで更新することも可能である。この際、全ての量子化マトリクスを書き換えても良いし、書き換える量子化マトリクスに対応する量子化マトリクスの予測モードを指定することでその一部を変更するようにすることも可能である。 FIG. 6A is an example of a bit stream output in the first embodiment. The sequence header contains the coded data of the quantization matrix, and is composed of the coded results of each element. However, the encoded position is not limited to this, and of course, a configuration in which the image header portion and other header portions are encoded may be used. Further, when the quantization matrix is changed in one sequence, it can be updated by newly encoding the quantization matrix. At this time, all the quantization matrices may be rewritten, or a part of them may be changed by specifying the prediction mode of the quantization matrix corresponding to the rewritten quantization matrix.
　図３は、第１の実施形態の画像符号化装置における制御部１５０の制御下での１フレーム分の符号化処理を示すフローチャートである。 FIG. 3 is a flowchart showing the coding process for one frame under the control of the control unit 150 in the image coding device of the first embodiment.
　まず、画像の符号化に先立ち、Ｓ３０１にて、量子化マトリクス保持部１０３は二次元の量子化マトリクスの生成、保持する。本実施形態の量子化マトリクス保持部１０３は、図８Ａ～８Ｃに示される、８×８画素のサブブロックに対応するイントラ予測用、インター予測用および現ピクチャ参照予測用の量子化マトリクス８００～８０２を生成し、保持することになる。 First, prior to image coding, the quantization matrix holding unit 103 generates and holds a two-dimensional quantization matrix in S301. The quantization matrix holding unit 103 of the present embodiment is a quantization matrix 800 to 802 for intra prediction, inter prediction, and current picture reference prediction corresponding to a subblock of 8 × 8 pixels shown in FIGS. 8A to 8C. Will be generated and retained.
　Ｓ３０２にて、量子化マトリクス符号化部１１３は、Ｓ３０１にて生成・保持された量子化マトリクス８００～８０２それぞれを走査して各要素の差分を算出し、差分行列１０００～１００２を生成する。すなわち、本実施形態では、図８Ａ～８Ｃに示された量子化マトリクス８００～８０２から、図９の走査方法で走査され、図１０Ａ～１０Ｃに示される差分行列１０００～１００２が生成されることになる。量子化マトリクス符号化部１１３は、このように生成された差分行列１０００～１００２を符号化し、量子化マトリクスの符号化データを生成する。 In S302, the quantization matrix coding unit 113 scans each of the quantization matrices 800 to 802 generated and held in S301, calculates the difference of each element, and generates the difference matrix 1000 to 1002. That is, in the present embodiment, the quantization matrices 800 to 802 shown in FIGS. 8A to 8C are scanned by the scanning method of FIG. 9, and the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C are generated. Become. The quantization matrix coding unit 113 encodes the difference matrices 1000 to 1002 thus generated, and generates the coded data of the quantization matrix.
　Ｓ３０３にて、統合符号化部１１１は、生成された量子化マトリクスの符号化データを含む、画像データの符号化に必要なヘッダ情報を符号化し、出力する。 In S303, the integrated coding unit 111 encodes and outputs the header information necessary for coding the image data, including the coded data of the generated quantization matrix.
　Ｓ３０４にて、ブロック分割部１０２はフレーム単位の入力画像を基本ブロック単位に分割する。 In S304, the block division unit 102 divides the input image in frame units into basic block units.
　Ｓ３０５にて、予測部１０４はＳ３０４にて生成された基本ブロック単位の画像データに対して、予測処理を実行し、サブブロック分割情報や予測モードなどの予測情報および予測画像データを生成する。本実施形態の予測部１０４は、イントラ予測、インター予測および現ピクチャ参照予測の３種類の予測方法を試し、予測誤差が最小となる一の予測方法を決定する。ただし、予測方法の決定方法はこれに限定されず、予測誤差の大きさと発生符号量から符号化コストを算出し、符号化コストが最小となる予測方法を決定する構成としても良い。そして、予測部１０４は、入力された画像データと予測画像データから予測誤差を算出し、予測情報と共に出力する。 In S305, the prediction unit 104 executes prediction processing on the image data in basic block units generated in S304, and generates prediction information such as sub-block division information and prediction mode and prediction image data. The prediction unit 104 of the present embodiment tries three types of prediction methods, intra prediction, inter prediction, and current picture reference prediction, and determines one prediction method that minimizes the prediction error. However, the method for determining the prediction method is not limited to this, and the coding cost may be calculated from the magnitude of the prediction error and the amount of code generated to determine the prediction method that minimizes the coding cost. Then, the prediction unit 104 calculates a prediction error from the input image data and the prediction image data, and outputs the prediction error together with the prediction information.
　Ｓ３０６にて、変換・量子化部１０５はＳ３０５で算出された予測誤差を直交変換して変換係数を生成する。さらに変換・量子化部１０５は、予測情報に基づき、量子化マトリクス保持部１０３に保持された該当する量子化マトリクス８００～８０２のいずれかと、量子化パラメータとを用いて、生成した変換係数の量子化を行い、量子化係数を生成する。本実施形態では、イントラ予測が用いられたサブブロックには図８Ａの量子化マトリクス８００、インター予測が用いられたサブブロックには図８Ｂの量子化マトリクス８０１、現ピクチャ参照予測が用いられたサブブロックには図８Ｃの量子化マトリクス８０２が用いられることになる。 In S306, the conversion / quantization unit 105 orthogonally converts the prediction error calculated in S305 to generate a conversion coefficient. Further, the conversion / quantization unit 105 uses any of the corresponding quantization matrices 800 to 802 held in the quantization matrix holding unit 103 and the quantization parameter based on the prediction information to generate a quantum of the conversion coefficient. Is performed to generate a quantization coefficient. In the present embodiment, the sub-block in which the intra-prediction is used is the quantization matrix 800 of FIG. 8A, the sub-block in which the inter-prediction is used is the quantization matrix 801 of FIG. The quantization matrix 802 of FIG. 8C will be used for the block.
　Ｓ３０７にて、逆量子化・逆変換部１０６はＳ３０６で生成された量子化係数を、量子化マトリクス保持部１０３に保持された該当する量子化マトリクスを用いて逆量子化を行い、変換係数を再生する。さらに、逆量子化・逆変換部１０６は、再生した変換係数に対して逆直交変換し、予測誤差を再生する。本ステップにおいて、逆量子化で用いる量子化マトリクスは、Ｓ３０６で用いられた量子化マトリクスと同じとなる。 In S307, the inverse quantization / inverse conversion unit 106 inversely quantizes the quantization coefficient generated in S306 using the corresponding quantization matrix held in the quantization matrix holding unit 103, and obtains the conversion coefficient. Reproduce. Further, the inverse quantization / inverse conversion unit 106 performs inverse orthogonal conversion with respect to the reproduced conversion coefficient and reproduces the prediction error. In this step, the quantization matrix used in the inverse quantization is the same as the quantization matrix used in S306.
　Ｓ３０８にて、画像再生部１０７はＳ３０５で生成された予測情報に基づいて予測画像を再生する。さらに画像再生部１０７は、再生された予測画像とＳ３０７で生成された予測誤差から画像データを再生する。 In S308, the image reproduction unit 107 reproduces the predicted image based on the prediction information generated in S305. Further, the image reproduction unit 107 reproduces the image data from the reproduced predicted image and the prediction error generated in S307.
　３０９にて、符号化部１１０は、Ｓ３０５で生成された予測情報およびＳ３０６で生成された量子化係数を符号化し、画像データに対する符号化データを生成する。また、符号化部１１０は、他の符号化データも含め、ビットストリームを生成する。 At 309, the coding unit 110 encodes the prediction information generated in S305 and the quantization coefficient generated in S306 to generate coded data for the image data. In addition, the coding unit 110 generates a bit stream including other coded data.
　Ｓ３１０にて、制御部１５０は、フレーム内の全ての基本ブロックの符号化が終了したか否かの判定を行い、終了したと判定した場合は処理をＳ３１１に、未符号化の基本ブロックが残っていると判定した場合は処理をＳ３０４に戻す。 In S310, the control unit 150 determines whether or not the coding of all the basic blocks in the frame is completed, and if it is determined that the coding is completed, the processing is performed in S311 and the uncoded basic blocks remain. If it is determined that the process is returned to S304.
　Ｓ３１１にて、インループフィルタ部１０９はＳ３０８で再生された画像データに対し、インループフィルタ処理を行い、フィルタ処理された画像データを生成し、処理を終了する。 In S311 the in-loop filter unit 109 performs in-loop filter processing on the image data reproduced in S308, generates the filtered image data, and ends the processing.
　以上の構成と動作により、特にＳ３０６において、現ピクチャ参照予測を用いたサブブロックに対して量子化マトリクスを用いた量子化処理をすることで、周波数成分ごとに量子化を制御し主観画質を向上させることができる。 With the above configuration and operation, especially in S306, by performing the quantization process using the quantization matrix on the subblock using the current picture reference prediction, the quantization is controlled for each frequency component and the subjective image quality is improved. Can be made to.
　なお、本実施形態では、イントラ予測、インター予測および現ピクチャ参照予測の３種類の予測方法に対して、個別に量子化マトリクスを定義し、３種類の量子化マトリクス全てを符号化する構成としたが、そのうちのいくつかを共通化しても構わない。 In this embodiment, the quantization matrix is individually defined for the three types of prediction methods of intra-prediction, inter-prediction, and current picture reference prediction, and all three types of quantization matrices are encoded. However, some of them may be shared.
　例えば、現ピクチャ予測を用いたサブブロックに対しても、イントラ予測画像用の量子化マトリクス８００（図８Ａ）を用いて量子化し、図８Ｃの量子化マトリクス８０２の符号化を省略することも可能である。これにより、図８Ｃの量子化マトリクス８０２分の符号量を削減しつつ、ブロック歪みなどの同一フレーム内の画素を用いた予測による誤差から生じる画質劣化を軽減させることができる。 For example, it is also possible to quantize the subblock using the current picture prediction by using the quantization matrix 800 (FIG. 8A) for the intra prediction image, and omit the coding of the quantization matrix 802 in FIG. 8C. Is. As a result, it is possible to reduce the code amount for the quantization matrix 802 in FIG. 8C, and reduce the image quality deterioration caused by the error due to the prediction using the pixels in the same frame such as block distortion.
　同様に、現ピクチャ参照予測を用いたサブブロックに対しても、インター予測画像用の量子化マトリクス８０１（図８Ｂ）を用いて量子化し、図８Ｃの量子化マトリクス８０２の符号化を省略することも可能である。これにより、図８Ｃの量子化マトリクス８０２分の符号量を削減しつつ、ガクガクした動きなどのブロック状の画素群を用いた予測による誤差から生じる画質劣化を低減させることもできる。 Similarly, the sub-block using the current picture reference prediction is also quantized using the quantization matrix 801 (FIG. 8B) for the inter-prediction image, and the coding of the quantization matrix 802 in FIG. 8C is omitted. Is also possible. As a result, it is possible to reduce the amount of code equivalent to 802 minutes of the quantization matrix of FIG. 8C, and reduce the image quality deterioration caused by an error due to prediction using a block-shaped pixel group such as a jerky motion.
　また、画像符号化装置は、不図示の操作部からのユーザによる指示に応じて、現ピクチャ参照予測のサブブロックで用いる量子化マトリクスをどれにするかを選択できるようにしても良い。この場合、いずれが選択されたのかを示す情報を、ストリームのヘッダに、量子化マトリクス設定情報として格納する。 Further, the image coding device may be able to select which quantization matrix is used in the subblock of the current picture reference prediction in response to an instruction from a user from an operation unit (not shown). In this case, the information indicating which is selected is stored in the header of the stream as the quantization matrix setting information.
　この場合の、ストリーム構成の例を図６Ｂに示す。量子化マトリクス設定情報が“０”である場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、イントラ予測用の量子化マトリクス８００（図８Ａ）であるものとする。また、量子化マトリクス設定情報が“１”の場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、インター予測用の量子化マトリクス８０１（図８Ｂ）であるものとする。そして、量子化マトリクス設定情報が“２”の場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、現ピクチャ参照予測のため専用の量子化マトリクス８０２（図８Ｃ）であるものとする。これにより、量子化マトリクス符号量削減と現ピクチャ参照予測を用いたサブブロックに対する独自の量子化制御とを選択的に実現することが可能となる。 An example of the stream configuration in this case is shown in FIG. 6B. When the quantization matrix setting information is "0", it is assumed that the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 800 (FIG. 8A) for intra prediction. .. When the quantization matrix setting information is "1", the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 801 (FIG. 8B) for inter-prediction. To do. When the quantization matrix setting information is "2", the quantization matrix used for the subblock using the current picture reference prediction is the dedicated quantization matrix 802 for the current picture reference prediction (FIG. 8C). It is assumed that. This makes it possible to selectively realize the reduction of the quantization matrix code amount and the original quantization control for the subblock using the current picture reference prediction.
　［第２の実施形態］
　本第２の実施形態は画像復号装置について説明する。本第２の実施形態に係る画像復号装置の構成を図２に示す。この画像復号装置は、第１の実施形態の画像符号化装置が生成した符号化ビットストリームを復号するものとして説明する。 [Second Embodiment]
The second embodiment describes an image decoding apparatus. FIG. 2 shows the configuration of the image decoding device according to the second embodiment. This image decoding device will be described as decoding the coded bit stream generated by the image coding device of the first embodiment.
　制御部２５０は、ＣＰＵ、ＣＰＵが実行するプログラムや各種パラメータを記憶しているＲＯＭ、及び、ＣＰＵのワークエリアとして利用されるＲＡＭで構成され、以下に説明する各構成要素の制御を行うことで画像復号装置の全体の制御を司る。 The control unit 250 is composed of a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM that is used as a work area of the CPU, and controls each component described below. It controls the entire image decoding device.
　分離復号部２０２は、入力端子２０１より入力した符号化ビットストリームから復号処理に関する情報や係数に関する符号化データに分離し、また符号化ビットストリームのヘッダ部に存在する符号化データを復号する。具体的には、分離復号部２０２は、符号化ビットストリームから量子化マトリクスの符号化データを分離し、分離した量子化マトリクスの符号化データを量子化マトリクス復号部２０９に出力する。また、分離復号部２０２は、符号化ビットストリームから画像の符号化データを分離し、その分離した画像の符号化データを復号部２０３に出力する。つまり、分離復号部２０２は図１の統合符号化部１１１と逆の処理を行う。 The separation / decoding unit 202 separates the coded bit stream input from the input terminal 201 into coded data related to information related to decoding processing and coefficients, and decodes the coded data existing in the header part of the coded bit stream. Specifically, the separation / decoding unit 202 separates the coded data of the quantization matrix from the coded bit stream, and outputs the coded data of the separated quantization matrix to the quantization matrix decoding unit 209. Further, the separation / decoding unit 202 separates the coded data of the image from the coded bit stream, and outputs the coded data of the separated image to the decoding unit 203. That is, the separation / decoding unit 202 performs the reverse processing of the integrated coding unit 111 of FIG.
　量子化マトリクス復号部２０９は、分離復号部２０２より入力した量子化マトリクスの符号化データを復号し、量子化マトリクス８００～８０２を保持する。 The quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and holds the quantization matrices 800 to 802.
　復号部２０３は、分離復号部２０２より入力した画像の符号化データを復号し、量子化係数および予測情報を再生する。そして、復号部２０３は、量子化係数を逆量子化・逆変換部２０４に供給すると共に、予測情報を逆量子化・逆変換部２０４、及び、画像再生部２０５に出力する。なお、量子化パラメータを導出するための情報も復号部２０３によってビットストリームから復号される。 The decoding unit 203 decodes the encoded data of the image input from the separation decoding unit 202, and reproduces the quantization coefficient and the prediction information. Then, the decoding unit 203 supplies the quantization coefficient to the inverse quantization / inverse conversion unit 204, and outputs the prediction information to the inverse quantization / inverse conversion unit 204 and the image reproduction unit 205. The information for deriving the quantization parameter is also decoded from the bit stream by the decoding unit 203.
　逆量子化・逆変換部２０４は復号部２０３から量子化係数を入力する。そして、逆量子化・逆変換部２０４は、図１の逆量子化・逆変換部１０６と同様に、予測情報で示される予測モードに応じた量子化マトリクスと、量子化パラメータとを用いて、量子化係数の逆量子化処理を行って変換係数を得る。さらに逆量子化・逆変換部２０４は、得られた変換係数に対して逆直交変換を行い、予測誤差を再生し、画像再生部２０５に出力する。 The inverse quantization / inverse conversion unit 204 inputs the quantization coefficient from the decoding unit 203. Then, the inverse quantization / inverse conversion unit 204 uses the quantization matrix corresponding to the prediction mode indicated by the prediction information and the quantization parameter, similarly to the inverse quantization / inverse conversion unit 106 of FIG. The conversion coefficient is obtained by performing the inverse quantization process of the quantization coefficient. Further, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion on the obtained conversion coefficient, reproduces the prediction error, and outputs the prediction error to the image reproduction unit 205.
　画像再生部２０５は、予測部２０３より入力した予測情報に基づいて、フレームメモリ２０６（復号済の画像データが格納されている）を適宜参照して予測画像データを生成する。そして、画像再生部２０５は、この予測画像データと、逆量子化・逆変換部２０４で再生された予測誤差から再生画像データを生成する。そして、画像再生部２０５は、生成した再生画像をフレームメモリ２０６に格納する。 The image reproduction unit 205 generates the predicted image data by appropriately referring to the frame memory 206 (which stores the decoded image data) based on the prediction information input from the prediction unit 203. Then, the image reproduction unit 205 generates the reproduction image data from the predicted image data and the prediction error reproduced by the inverse quantization / inverse conversion unit 204. Then, the image reproduction unit 205 stores the generated reproduced image in the frame memory 206.
　インループフィルタ部２０７は、図１のインループフィルタ部１０９と同様であり、フレームメモリ２０６に格納された再生画像に対し、デブロッキングフィルタなどのインループフィルタ処理を行い、フィルタ処理された画像をフレームメモリ２０６に再格納する。この結果、フレームメモリにはフィルタ処理後の表示可能な画像データが格納されるので、出力端子２０８を介して外部装置（表示装置がその代表例）に出力されることになる。 The in-loop filter unit 207 is the same as the in-loop filter unit 109 in FIG. 1, and performs an in-loop filter process such as a deblocking filter on the reproduced image stored in the frame memory 206 to obtain the filtered image. Restore in frame memory 206. As a result, since the image data that can be displayed after the filter processing is stored in the frame memory, it is output to an external device (a display device is a typical example thereof) via the output terminal 208.
　本第２の実施形態の画像復号装置における画像の復号動作を更に詳しく以下に説明する。本第２の実施形態では、先に説明した第１の実施形態の画像符号化装置で生成されたビットストリームをフレーム単位で入力する構成となっている。 The image decoding operation in the image decoding device of the second embodiment will be described in more detail below. In the second embodiment, the bit stream generated by the image coding apparatus of the first embodiment described above is input in frame units.
　分離復号部２０２は、入力端子２０１を介してビットストリームを入力し、復号処理に関する情報や係数に関する符号化データに分離し、ビットストリームのヘッダ部に存在する符号化データを復号する。より具体的には、分離復号部２０２は、ビットストリームのヘッダ部に格納されている、量子化マトリクスの符号化データを分離し、量子化マトリクス復号部２０９に出力する。本実施形態の分離復号部２０２は、図６Ａに示されるビットストリームのシーケンスヘッダから量子化マトリクスの符号化データを抽出し、量子化マトリクス復号部２０９に出力する。この結果、本実施形態では、図８Ａ～８Ｃに示される量子化マトリクス８０１～８０２の符号化データが量子化マトリクス復号部２０９に出力されることになる。また、分離復号部２０２は、ヘッダ部に後続する画像の符号化データをビットストリームから分離し、復号部２０３に出力する。 The separation / decoding unit 202 inputs a bitstream via the input terminal 201, separates it into information related to decoding processing and coded data related to coefficients, and decodes the coded data existing in the header part of the bitstream. More specifically, the separation / decoding unit 202 separates the coded data of the quantization matrix stored in the header unit of the bitstream and outputs it to the quantization matrix decoding unit 209. The separation / decoding unit 202 of the present embodiment extracts the coded data of the quantization matrix from the sequence header of the bitstream shown in FIG. 6A and outputs the coded data to the quantization matrix decoding unit 209. As a result, in the present embodiment, the coded data of the quantized matrices 801 to 802 shown in FIGS. 8A to 8C is output to the quantized matrix decoding unit 209. Further, the separation / decoding unit 202 separates the coded data of the image following the header unit from the bit stream and outputs it to the decoding unit 203.
　量子化マトリクス復号部２０９は、分離復号部２０２から入力した量子化マトリクスの符号化データを復号し、図１０Ａ～１０Ｃに示される一次元の差分行列１０００～１００２を再生する。本実施形態では、第１の実施形態と同様、図１１Ａに示される符号化テーブルを用いて復号するものとするが、符号化テーブルはこれに限定されず、符号化装置と同じものを用いる限りは他の符号化テーブルを用いても良い。さらに量子化マトリクス復号部２０９は、再生された一次元の差分行列を走査し、二次元の量子化マトリクスを再生する。ここでは第１の実施形態の量子化マトリクス符号化部１１３の動作とは逆の動作を行う。すなわち、量子化マトリクス復号部２０９は、図１０Ａ～１０Ｃに示される差分行列１０００～１００２から、図９に示される走査方法を用いて、それぞれ図８Ａ～８Ｃに示される３種の量子化マトリクス８００～８０２を再生し、保持する。 The quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and reproduces the one-dimensional difference matrices 1000 to 1002 shown in FIGS. 10A to 10C. In the present embodiment, as in the first embodiment, decoding is performed using the coding table shown in FIG. 11A, but the coding table is not limited to this, as long as the same one as the coding device is used. May use other coding tables. Further, the quantization matrix decoding unit 209 scans the reproduced one-dimensional difference matrix and reproduces the two-dimensional quantization matrix. Here, the operation opposite to the operation of the quantization matrix coding unit 113 of the first embodiment is performed. That is, the quantization matrix decoding unit 209 uses the scanning methods shown in FIGS. 9A to 9 from the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C, and the three types of quantization matrices 800 shown in FIGS. 8A to 8C, respectively. 802 is regenerated and retained.
　復号部２０３は、分離復号部２０２から入力した符号化データを復号し、量子化係数および予測情報を再生する。復号部２０３は、再生された量子化係数を逆量子化・逆変換部２０４に、再生された予測情報を逆量子化・逆変換部２０４及び画像再生部２０５に出力する。 The decoding unit 203 decodes the coded data input from the separation decoding unit 202 and reproduces the quantization coefficient and the prediction information. The decoding unit 203 outputs the reproduced quantization coefficient to the inverse quantization / inverse conversion unit 204, and outputs the reproduced prediction information to the inverse quantization / inverse conversion unit 204 and the image reproduction unit 205.
　逆量子化・逆変換部２０４は、入力された量子化係数に対して、量子化マトリクスと、量子化パラメータとを用いて逆量子化を行うことで、変換係数を生成する。逆量子化・逆変換部２０４は、得られた変換係数を逆直交変換して、予測誤差を再生する。そして、逆量子化・逆変換部２０４は、再生された予測誤差を画像再生部２０５に出力する。本実施形態の逆量子化・逆変換部２０４は、復号部２０３で再生された予測情報によって定まった復号対象ブロックの予測モードに応じて、逆量子化処理において使用される量子化マトリクスを決定する。すなわち、イントラ予測が用いられているサブブロックには図８Ａの量子化マトリクス８００、インター予測が用いられているサブブロックには図８Ｂの量子化マトリクス８０１、現ピクチャ参照予測が用いられているサブブロックには図８Ｃの量子化マトリクス８０２が用いられる。ただし、使用される量子化マトリクスはこれに限定されず、第１の実施形態の変換・量子化部１０５および逆量子化・逆変換部１０６で用いられた量子化マトリクスと同一のものであれば良い。 The inverse quantization / inverse conversion unit 204 generates a conversion coefficient by performing inverse quantization on the input quantization coefficient using the quantization matrix and the quantization parameter. The inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion of the obtained conversion coefficient to reproduce the prediction error. Then, the inverse quantization / inverse conversion unit 204 outputs the reproduced prediction error to the image reproduction unit 205. The inverse quantization / inverse conversion unit 204 of the present embodiment determines the quantization matrix used in the inverse quantization process according to the prediction mode of the decoding target block determined by the prediction information reproduced by the decoding unit 203. .. That is, the sub-block in which the intra-prediction is used is the quantization matrix 800 of FIG. 8A, the sub-block in which the inter-prediction is used is the quantization matrix 801 of FIG. 8B, and the sub-block in which the current picture reference prediction is used. The quantization matrix 802 of FIG. 8C is used for the block. However, the quantization matrix used is not limited to this, as long as it is the same as the quantization matrix used in the conversion / quantization unit 105 and the inverse quantization / inverse conversion unit 106 of the first embodiment. good.
　画像再生部２０５では、復号部２０３から入力された予測情報に基づいて、フレームメモリ２０６を適宜参照し、予測画像を再生する。本第２の実施形態の画像再生部２０５は、第１の実施形態における予測部１０４と同様、イントラ予測、インター予測および現ピクチャ参照予測の３種類の予測方法が用いられる。具体的な予測の処理については、第１の実施形態の予測部１０４と同様であるため、説明を省略する。画像再生部２０５は、この予測画像と逆量子化・逆変換部２０４から入力された予測誤差とを足し合わせて画像データを再生し、フレームメモリ２０６に格納する。格納された画像データは予測の際の参照に用いられる。 The image reproduction unit 205 reproduces the predicted image by appropriately referring to the frame memory 206 based on the prediction information input from the decoding unit 203. Similar to the prediction unit 104 in the first embodiment, the image reproduction unit 205 of the second embodiment uses three types of prediction methods: intra prediction, inter prediction, and current picture reference prediction. Since the specific prediction process is the same as that of the prediction unit 104 of the first embodiment, the description thereof will be omitted. The image reproduction unit 205 reproduces the image data by adding the predicted image and the prediction error input from the inverse quantization / inverse conversion unit 204, and stores the image data in the frame memory 206. The stored image data is used as a reference when making a prediction.
　インループフィルタ部２０７は、図１のインループフィルタ部１０９と同様、フレームメモリ２０６から再生画像を読み出し、デブロッキングフィルタなどのインループフィルタ処理を行う。そして、インループフィルタ部２０７は、フィルタ処理後の画像を再びフレームメモリ２０６に格納する。 Similar to the in-loop filter unit 109 in FIG. 1, the in-loop filter unit 207 reads the reproduced image from the frame memory 206 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 207 stores the filtered image in the frame memory 206 again.
　そして、このフレームメモリ２０６に格納された再生画像は、最終的には出力端子２０８から外部に出力される。 Then, the reproduced image stored in the frame memory 206 is finally output to the outside from the output terminal 208.
　図４は、第２の実施形態の画像復号装置における制御部２５０の制御下での１フレーム分の符号化処理を示すフローチャートである。 FIG. 4 is a flowchart showing the coding process for one frame under the control of the control unit 250 in the image decoding apparatus of the second embodiment.
　まず、Ｓ４０１にて、分離復号部２０２はビットストリームから復号処理に関する情報や係数に関する符号化データに分離して、ヘッダ部分の符号化データを復号する。この結果、量子化マトリクスの符号化データが得られる。分離復号部２０２は、得られた量子化マトリクスの符号化データを量子化マトリクス復号部２０９に、後続する画像データの符号化データを復号部２０３に出力する。 First, in S401, the separation / decoding unit 202 separates the bit stream into coded data related to information related to decoding processing and coefficients, and decodes the coded data of the header portion. As a result, the coded data of the quantization matrix is obtained. The separation decoding unit 202 outputs the coded data of the obtained quantization matrix to the quantization matrix decoding unit 209, and outputs the coded data of the subsequent image data to the decoding unit 203.
　Ｓ４０２にて、量子化マトリクス復号部２０９は、分離復号部２０２から入力した量子化マトリクスの符号化データを復号し、図１０Ａ～１０Ｃで示される一次元の差分行列１０００～１００２を再生する。さらに、量子化マトリクス復号部２０９は、再生された一次元の差分行列１０００～１００２それぞれを走査し、二次元の量子化マトリクス８００～８０２を再生する。すなわち、本実施形態では、量子化マトリクス復号部２０９は、図１０Ａ～１０Ｃに示される差分行列１０００～１００２を、図９に示される走査方法を用いて走査することで、図８Ａ～８Ｃに示される３種の量子化マトリクス８００～８０２を再生し、保持する。 In S402, the quantization matrix decoding unit 209 decodes the coded data of the quantization matrix input from the separation decoding unit 202, and reproduces the one-dimensional difference matrices 1000 to 1002 shown in FIGS. 10A to 10C. Further, the quantization matrix decoding unit 209 scans each of the reproduced one-dimensional difference matrices 1000 to 1002, and reproduces the two-dimensional quantization matrices 800 to 802. That is, in the present embodiment, the quantization matrix decoding unit 209 scans the difference matrices 1000 to 1002 shown in FIGS. 10A to 10C by using the scanning method shown in FIG. 9, and is shown in FIGS. 8A to 8C. The three types of quantization matrices 800 to 802 are regenerated and retained.
　Ｓ４０３にて、復号部２０３は、分離復号部２０２から入力した符号化データを復号し、量子化係数および予測情報を再生する。 In S403, the decoding unit 203 decodes the coded data input from the separation decoding unit 202 and reproduces the quantization coefficient and the prediction information.
　Ｓ４０４にて、逆量子化・逆変換部２０４は、復号部２０３より入力した量子化係数に対し、量子化マトリクス復号部２０９が保持している量子化マトリクス８００～８０２のいずれかと、量子化パラメータとを用いて逆量子化を行って変換係数を得る。具体的には、逆量子化・逆変換部２０４は、得られた変換係数を逆直交変換し、予測誤差を再生する。本実施形態の逆量子化・逆変換部２０４は、Ｓ４０３で再生された予測情報によって定まった予測モードに応じて、逆量子化処理において使用される量子化マトリクスを選択する。すなわち、逆量子化・逆変換部２０４は、イントラ予測が用いられているサブブロックには図８Ａの量子化マトリクス８００を、インター予測が用いられているサブブロックには図８Ｂの量子化マトリクス８０１を、現ピクチャ参照予測が用いられているサブブロックには図８Ｃの量子化マトリクス８０２を選択し、量子化マトリクス復号部２０９から選択した量子化マトリクスを取得する。ただし、使用される量子化マトリクスはこれに限定されず、第１の実施形態のＳ３０６およびＳ３０７で用いられた量子化マトリクスと同一のものであれば良い。 In S404, the inverse quantization / inverse conversion unit 204 has a quantization parameter with any of the quantization matrices 800 to 802 held by the quantization matrix decoding unit 209 with respect to the quantization coefficient input from the decoding unit 203. Inverse quantization is performed using and to obtain the conversion coefficient. Specifically, the inverse quantization / inverse conversion unit 204 performs inverse orthogonal conversion of the obtained conversion coefficient and reproduces the prediction error. The inverse quantization / inverse conversion unit 204 of the present embodiment selects the quantization matrix used in the inverse quantization process according to the prediction mode determined by the prediction information reproduced in S403. That is, the inverse quantization / inverse conversion unit 204 uses the quantization matrix 800 of FIG. 8A for the subblock in which the intra prediction is used, and the quantization matrix 801 of FIG. 8B for the subblock in which the inter prediction is used. The quantization matrix 802 of FIG. 8C is selected for the subblock in which the current picture reference prediction is used, and the quantization matrix selected from the quantization matrix decoding unit 209 is acquired. However, the quantization matrix used is not limited to this, and may be the same as the quantization matrix used in S306 and S307 of the first embodiment.
　Ｓ４０５にて、画像再生部２０５は、復号部２０３から入力した予測情報を用い、フレームメモリ２０７を参照して予測画像を再生する。本実施形態では、第１の実施形態のＳ３０５と同様、イントラ予測、インター予測および現ピクチャ参照予測の３種類の予測方法が用いられる。画像再生部２０５は、さらに再生された予測画像と、逆量子化・逆変換部２０４で生成された予測誤差とを足し合わせこととで、画像データを再生する。画像再生部２０５は、再生した画像データをフレームメモリ２０７に格納する。 In S405, the image reproduction unit 205 reproduces the predicted image by referring to the frame memory 207 using the prediction information input from the decoding unit 203. In this embodiment, as in S305 of the first embodiment, three types of prediction methods are used: intra prediction, inter prediction, and current picture reference prediction. The image reproduction unit 205 reproduces the image data by further adding the reproduced predicted image and the prediction error generated by the inverse quantization / inverse conversion unit 204. The image reproduction unit 205 stores the reproduced image data in the frame memory 207.
　Ｓ４０６にて、制御部２５０は、着目フレーム内の全ての基本ブロックの復号が終了したか否かの判定を行い、終了していれば処理をＳ４０７に進み、そうでなければ次の基本ブロックを復号するため、処理をＳ４０３に戻す。 In S406, the control unit 250 determines whether or not the decoding of all the basic blocks in the frame of interest has been completed, and if so, proceeds to S407, and if not, proceeds to the next basic block. The process returns to S403 for decoding.
　Ｓ４０７にて、インループフィルタ部２０７は、フレームメモリ２０７に格納されたた画像データに対し、インループフィルタ処理を行い、フィルタ処理された画像を生成し、フレームメモリ２０７に再格納し、本処理を終える。 In S407, the in-loop filter unit 207 performs in-loop filter processing on the image data stored in the frame memory 207, generates a filtered image, re-stores it in the frame memory 207, and performs this processing. To finish.
　以上の構成と動作により、第１の実施形態で生成された、現ピクチャ参照予測を用いたサブブロックに対しても、周波数成分ごとに量子化を制御し主観画質を向上したビットストリームを復号することができる。 With the above configuration and operation, even for the subblock using the current picture reference prediction generated in the first embodiment, the quantization is controlled for each frequency component to decode the bit stream with improved subjective image quality. be able to.
　なお、本第２の実施形態では、イントラ予測、インター予測および現ピクチャ参照予測の３種類の予測方法に対して、個別に量子化マトリクスを定義し、３種類の量子化マトリクス全てを復号する構成としたが、そのうちのいくつかを共通化しても構わない。例えば、現ピクチャ参照予測を用いたサブブロックに対しても、イントラ予測を用いたサブブロック同様に図８Ａの量子化マトリクス８００を用いて逆量子化し、図８Ｃの量子化マトリクス８０２の復号を省略することも可能である。これにより、図８Ｃの量子化マトリクス８０２分の符号量を削減しつつ、ブロック歪みなど同一フレーム内の画素を用いた予測による誤差から生じる画質劣化を軽減させたビットストリームを復号することができる。 In the second embodiment, the quantization matrix is individually defined for the three types of prediction methods of intra prediction, inter prediction, and current picture reference prediction, and all three types of quantization matrices are decoded. However, some of them may be shared. For example, the sub-block using the current picture reference prediction is also dequantized using the quantization matrix 800 of FIG. 8A in the same manner as the sub-block using the intra prediction, and the decoding of the quantization matrix 802 of FIG. 8C is omitted. It is also possible to do. As a result, it is possible to decode a bit stream in which the amount of code equivalent to 802 minutes of the quantization matrix of FIG. 8C is reduced, and the image quality deterioration caused by an error due to prediction using pixels in the same frame such as block distortion is reduced.
　同様に、現ピクチャ参照予測を用いたサブブロックに対しても、インター予測を用いたサブブロック同様に図８Ｂの量子化マトリクス８０１を用いて逆量子化し、図８Ｃの量子化マトリクス８０２の復号を省略することも可能である。これにより、図８Ｃの量子化マトリクス８０２分の符号量を削減しつつ、ガクガクした動きなどのブロック状の画素群を用いた予測による誤差から生じる画質劣化を低減させたビットストリームを復号することもできる。 Similarly, the sub-block using the current picture reference prediction is also dequantized using the quantization matrix 801 of FIG. 8B in the same manner as the sub-block using the inter-prediction, and the quantization matrix 802 of FIG. 8C is decoded. It can be omitted. As a result, while reducing the code amount of 802 minutes of the quantization matrix of FIG. 8C, it is also possible to decode a bit stream in which image quality deterioration caused by an error due to prediction using a block-shaped pixel group such as jerky motion is reduced. it can.
　さらには、本実施形態では、現ピクチャ参照予測を用いたサブブロックに対する量子化マトリクスが一意に決まる構成としたが、識別子を導入することで選択可能とする構成としても構わない。 Furthermore, in the present embodiment, the quantization matrix for the sub-block using the current picture reference prediction is uniquely determined, but it may be configured to be selectable by introducing an identifier.
　復号対象のビットストリームとして、図６Ｂに示すようにヘッダ部に、量子化マトリクス設定情報を設ける。量子化マトリクス設定情報が“０”である場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、イントラ予測用の量子化マトリクス８００（図８Ａ）であるものとする。また、量子化マトリクス設定情報が“１”の場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、インター予測用の量子化マトリクス８０１（図８Ｂ）であるものとする。そして、量子化マトリクス設定情報が“２”の場合は、現ピクチャ参照予測を用いたサブブロックに対して用いた量子化マトリクスは、現ピクチャ参照予測のため専用の量子化マトリクス８０２（図８Ｃ）であるものとする。これにより、量子化マトリクス符号量削減と現ピクチャ参照予測を用いたサブブロックに対する独自の量子化制御とを選択的に実現したビットストリームを復号することが可能となる。 As a bit stream to be decoded, quantization matrix setting information is provided in the header portion as shown in FIG. 6B. When the quantization matrix setting information is "0", it is assumed that the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 800 (FIG. 8A) for intra prediction. .. When the quantization matrix setting information is "1", the quantization matrix used for the subblock using the current picture reference prediction is the quantization matrix 801 (FIG. 8B) for inter-prediction. To do. When the quantization matrix setting information is "2", the quantization matrix used for the subblock using the current picture reference prediction is the dedicated quantization matrix 802 for the current picture reference prediction (FIG. 8C). It is assumed that. This makes it possible to decode a bitstream that selectively realizes the reduction of the quantization matrix code amount and the original quantization control for the subblock using the current picture reference prediction.
　［第３の実施形態］
　上記第１、第２の実施形態では、図１、図２に示した各処理部はハードウェアでもって構成しているものとして説明した。しかし、これらの図に示した各処理部で行う処理をコンピュータプログラムでもって構成しても良い。 [Third Embodiment]
In the first and second embodiments described above, each of the processing units shown in FIGS. 1 and 2 has been described as being configured by hardware. However, the processing performed by each processing unit shown in these figures may be configured by a computer program.
　図５は、上記各実施形態に係る画像符号化装置や復号装置に適用可能なコンピュータのハードウェアの構成例を示すブロック図である。 FIG. 5 is a block diagram showing a configuration example of computer hardware applicable to the image coding device and the decoding device according to each of the above embodiments.
　ＣＰＵ５０１は、ＲＡＭ５０２やＲＯＭ５０３に格納されているコンピュータプログラムやデータを用いてコンピュータ全体の制御を行うと共に、上記各実施形態に係る画像処理装置が行うものとして上述した各処理を実行する。即ち、ＣＰＵ５０１は、図１、図２に示した各処理部として機能することになる。 The CPU 501 controls the entire computer by using the computer programs and data stored in the RAM 502 and the ROM 503, and also executes each of the above-described processes as performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 501 functions as each processing unit shown in FIGS. 1 and 2.
　ＲＡＭ５０２は、外部記憶装置５０６からロードされたコンピュータプログラムやデータ、Ｉ／Ｆ（インターフェース）５０７を介して外部から取得したデータなどを一時的に記憶するためのエリアを有する。更に、ＲＡＭ５０２は、ＣＰＵ５０１が各種の処理を実行する際に用いるワークエリアを有する。即ち、ＲＡＭ５０２は、例えば、フレームメモリとして割り当てたり、その他の各種のエリアを適宜提供したりすることができる。 The RAM 502 has an area for temporarily storing computer programs and data loaded from the external storage device 506, data acquired from the outside via the I / F (interface) 507, and the like. Further, the RAM 502 has a work area used by the CPU 501 when executing various processes. That is, the RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as appropriate.
　ＲＯＭ５０３には、本コンピュータの設定データや、ブートプログラムなどが格納されている。操作部５０４は、キーボードやマウスなどにより構成されており、本コンピュータのユーザが操作することで、各種の指示をＣＰＵ５０１に対して入力することができる。出力部５０５は、ＣＰＵ５０１による処理結果を出力する。また出力部５０５は例えば液晶ディスプレイで構成される。 The ROM 503 stores the setting data of this computer, the boot program, and the like. The operation unit 504 is composed of a keyboard, a mouse, and the like, and can be operated by a user of the computer to input various instructions to the CPU 501. The output unit 505 outputs the processing result by the CPU 501. The output unit 505 is composed of, for example, a liquid crystal display.
　外部記憶装置５０６は、ハードディスクドライブ装置に代表される、大容量情報記憶装置である。外部記憶装置５０６には、ＯＳ（オペレーティングシステム）や、図１、図２に示した各部の機能をＣＰＵ５０１に実現させるためのコンピュータプログラムが保存されている。更には、外部記憶装置５０６には、処理対象としての各画像データが保存されていても良い。 The external storage device 506 is a large-capacity information storage device represented by a hard disk drive device. The external storage device 506 stores an OS (operating system) and a computer program for realizing the functions of the respective parts shown in FIGS. 1 and 2 in the CPU 501. Further, each image data as a processing target may be stored in the external storage device 506.
　外部記憶装置５０６に保存されているコンピュータプログラムやデータは、ＣＰＵ５０１による制御に従って適宜、ＲＡＭ５０２にロードされ、ＣＰＵ５０１による処理対象となる。Ｉ／Ｆ５０７には、ＬＡＮやインターネット等のネットワーク、投影装置や表示装置などの他の機器を接続することができ、本コンピュータはこのＩ／Ｆ５０７を介して様々な情報を取得したり、送出したりすることができる。参照符号５０８は上述の各部を繋ぐバスである。 The computer programs and data stored in the external storage device 506 are appropriately loaded into the RAM 502 according to the control by the CPU 501, and are processed by the CPU 501. A network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 507, and the computer acquires and sends various information via the I / F 507. Can be done. Reference numeral 508 is a bus connecting the above-mentioned parts.
　上記構成において、本装置の電源がＯＮになると、ＣＰＵ５０１はＲＯＭ５０３のブートプログラムを実行して、外部記憶装置５０６に格納されたＯＳをＲＡＭ５０２にロードし、ＯＳを起動する。この結果、本装置が、インターフェース５０７を介した通信が可能となり、情報処理装置として機能する。そして、ＯＳの制御下で、ＣＰＵ５０１は画像符号化に係るアプリケーション（図３に相当する）を外部記憶装置５０６からＲＡＭ５０２にロードして実行することで、ＣＰＵ５０１が図１に示す各種処理部として機能することになり、本装置が画像符号化装置として機能することになる。一方、ＣＰＵ５０１は画像復号に係るアプリケーション（図４に相当する）を外部記憶装置５０６からＲＡＭ５０２にロードして実行した場合、ＣＰＵ５０１は図２に示す各種処理部として機能することになり、本装置が画像復号装置として機能することになる。 In the above configuration, when the power of this device is turned on, the CPU 501 executes the boot program of the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and starts the OS. As a result, the present device enables communication via the interface 507 and functions as an information processing device. Then, under the control of the OS, the CPU 501 loads an application related to image coding (corresponding to FIG. 3) from the external storage device 506 into the RAM 502 and executes it, so that the CPU 501 functions as various processing units shown in FIG. This device will function as an image coding device. On the other hand, when the CPU 501 loads an application related to image decoding (corresponding to FIG. 4) from the external storage device 506 into the RAM 502 and executes the CPU 501, the CPU 501 functions as various processing units shown in FIG. It will function as an image decoding device.
　（その他の実施例）
　本発明は、上述の実施形態の１以上の機能を実現するプログラムを、ネットワーク又は記憶媒体を介してシステム又は装置に供給し、そのシステム又は装置のコンピュータにおける１つ以上のプロセッサーがプログラムを読出し実行する処理でも実現可能である。また、１以上の機能を実現する回路（例えば、ＡＳＩＣ）によっても実現可能である。 (Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
　発明は上記実施形態に制限されるものではなく、発明の精神及び範囲から離脱することなく、様々な変更及び変形が可能である。従って、発明の範囲を公にするために請求項を添付する。 The invention is not limited to the above embodiment, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.
　本願は、２０１９年３月１１日提出の日本国特許出願特願２０１９－０４４１９５を基礎として優先権を主張するものであり、その記載内容の全てを、ここに援用する。 This application claims priority based on Japanese Patent Application No. 2019-044195 submitted on March 11, 2019, and all the contents thereof are incorporated herein by reference.
　本発明は静止画・動画の符号化・復号を行う符号化装置・復号装置に用いられる。特に、量子化マトリクスを使用する符号化方式および復号方式に適用が可能である。 The present invention is used in a coding device / decoding device that encodes / decodes still images / moving images. In particular, it can be applied to a coding method and a decoding method using a quantization matrix.

Claims (11)

  1. 　画像を複数のブロックに分割し、ブロック単位で符号化を行う画像符号化装置であって、
    　符号化対象の着目ブロックの予測画像を、当該着目ブロックが属するフレームの符号化済の領域におけるブロック状の画素群を用いて生成する予測手段と、
    　前記着目ブロックの画素と前記予測画像との誤差を直交変換し、変換係数を生成する変換手段と、
    　前記変換手段で生成された前記変換係数を、量子化パラメータと量子化マトリクスとを用いて量子化する量子化手段と
    　を有することを特徴とする画像符号化装置。     An image coding device that divides an image into multiple blocks and encodes them in block units.
    A prediction means for generating a prediction image of a block of interest to be encoded by using a block-shaped pixel group in a coded region of a frame to which the block of interest belongs.
    A conversion means that generates a conversion coefficient by orthogonally converting the error between the pixel of the block of interest and the predicted image.
    An image coding apparatus comprising: a quantization means for quantizing the conversion coefficient generated by the conversion means by using a quantization parameter and a quantization matrix.
  2. 　前記着目ブロックが属するフレームの符号化済の領域における前記ブロック状の画素群を用いて生成される予測画像が用いられる場合における量子化マトリクスは、
    　前記着目ブロックの予測画像が当該着目ブロックに空間的に隣接する符号化済の画素群のみから生成される予測画像が用いられる場合における量子化マトリクスと同じである
    　ことを特徴とする請求項１に記載の画像符号化装置。     The quantization matrix in the case where the predicted image generated by using the block-shaped pixel group in the coded region of the frame to which the block of interest belongs is used is
    Claim 1 is characterized in that the predicted image of the block of interest is the same as the quantization matrix in the case where a predicted image generated only from a group of encoded pixels spatially adjacent to the block of interest is used. The image encoding device described.
  3. 　前記着目ブロックが属するフレームの符号化済の領域における前記ブロック状の画素群を用いて生成される予測画像が用いられる場合における量子化マトリクスは、
    　前記着目ブロックの予測画像が当該着目ブロックが属するフレームとは異なるフレームの画素群から生成される予測画像が用いられる場合における量子化マトリクスと同じである
    　ことを特徴とする請求項１に記載の画像符号化装置。     The quantization matrix in the case where the predicted image generated by using the block-shaped pixel group in the coded region of the frame to which the block of interest belongs is used is
    The image according to claim 1, wherein the predicted image of the block of interest is the same as the quantization matrix in the case where a predicted image generated from a pixel group of a frame different from the frame to which the block of interest belongs is used. Encoding device.
  4. 　画像を複数のブロックに分割し、ブロック単位で符号化を行う画像符号化方法であって、
    　符号化対象の着目ブロックの予測画像を、当該着目ブロックが属するフレームの符号化済の領域におけるブロック状の画素群を用いて生成する予測工程と、
    　前記着目ブロックの画素と前記予測画像との誤差を直交変換し、変換係数を生成する変換工程と、
    　前記変換工程で生成された前記変換係数を、量子化パラメータと量子化マトリクスとを用いて量子化する量子化工程と
    　を有することを特徴とする画像符号化方法。     An image coding method in which an image is divided into a plurality of blocks and coded in block units.
    A prediction step of generating a prediction image of the block of interest to be encoded by using a block-shaped pixel group in the coded region of the frame to which the block of interest belongs.
    A conversion step of orthogonally converting the error between the pixel of the block of interest and the predicted image to generate a conversion coefficient, and
    An image coding method characterized by having a quantization step of quantizing the conversion coefficient generated in the conversion step using a quantization parameter and a quantization matrix.
  5. 　ビットストリームからブロック単位に画像を復号する画像復号装置であって、
    　前記ビットストリームから、量子化マトリクスの符号化データを復号する第１の復号手段と、
    　前記ビットストリームから、ブロック単位の量子化係数の符号化データと予測情報を復号する第２の復号手段と、
    　前記予測情報に基づいて特定される量子化マトリクスと、量子化パラメータとを用いて、前記量子化係数の符号化データに基づく着目ブロックの量子化係数から当該着目ブロックの変換係数を導出する逆量子化を行う逆量子化手段と、
    　前記逆量子化手段で導出した前記変換係数に逆直交変換を行って、前記着目ブロックの予測誤差を導出する逆直交変換手段と
    　前記予測情報に従って前記着目ブロックの予測画像を生成し、当該予測画像と、前記逆直交変換手段で導出した前記予測誤差とから前記着目ブロックの画像を再生する再生手段と
    　を有し、
    　前記再生手段による前記予測画像を生成する方法は、
    　前記着目ブロックが属するフレームの復号済の領域におけるブロック状の画素群を用いる方法を含む
    　ことを特徴とする画像復号装置。     An image decoding device that decodes an image from a bitstream in block units.
    A first decoding means for decoding the coded data of the quantization matrix from the bit stream,
    A second decoding means for decoding the coded data and the prediction information of the quantization coefficient in block units from the bit stream,
    Inverse quantum that derives the conversion coefficient of the block of interest from the quantization coefficient of the block of interest based on the coded data of the quantization coefficient using the quantization matrix specified based on the prediction information and the quantization parameter. Inverse quantization means for conversion and
    The conversion coefficient derived by the inverse quantization means is subjected to inverse orthogonal transformation to generate a prediction image of the attention block according to the inverse orthogonal conversion means for deriving the prediction error of the attention block and the prediction information, and the prediction image is generated. And the reproduction means for reproducing the image of the block of interest from the prediction error derived by the inverse orthogonal conversion means.
    The method of generating the predicted image by the reproduction means is
    An image decoding apparatus comprising a method of using a block-shaped pixel group in a decoded region of a frame to which the block of interest belongs.
  6. 　前記再生手段は、
    　前記予測情報に従って、前記着目ブロックが属するフレームとは異なるフレームの画素群を用いる方法、前記着目ブロックが属するフレームにおける当該着目ブロックに空間的に隣接する復号済の画素群のみを用いる方法、前記着目ブロックが属するフレームの復号済の領域における前記ブロック状の画素群を用いる方法のいずれかを用いて前記予測画像を生成する
    　ことを特徴とする請求項５に記載の画像復号装置。     The regeneration means
    According to the prediction information, a method of using a pixel group of a frame different from the frame to which the focus block belongs, a method of using only a decoded pixel group spatially adjacent to the focus block in the frame to which the focus block belongs, the focus. The image decoding apparatus according to claim 5, wherein the predicted image is generated by using any of the methods using the block-shaped pixel group in the decoded region of the frame to which the block belongs.
  7. 　前記逆量子化手段は、
    前記着目ブロックが属するフレームの復号済の領域における前記ブロック状の画素群を用いて、前記着目ブロックの予測画像が生成される場合、
    　前記着目ブロックに対し、当該着目ブロックに空間的に隣接する復号済の画素群のみから生成される予測画像が用いられる場合における量子化マトリクスと同じ量子化マトリクスを用いて逆量子化を行う
    　ことを特徴とする請求項５に記載の画像復号装置。     The inverse quantization means
    When the predicted image of the block of interest is generated by using the block-shaped pixel group in the decoded region of the frame to which the block of interest belongs.
    Inverse quantization is performed on the block of interest using the same quantization matrix as the quantization matrix when a predicted image generated only from decoded pixel groups spatially adjacent to the block of interest is used. The image decoding apparatus according to claim 5.
  8. 　前記逆量子化手段は、
    前記着目ブロックが属するフレームの復号済の領域における前記ブロック状の画素群を用いて、前記着目ブロックの予測画像が生成される場合、
    　前記着目ブロックに対し、当該着目ブロックが属するフレームとは異なるフレームの画素群から生成される予測画像が用いられる場合における量子化マトリクスと同じ量子化マトリクスを用いて逆量子化を行う
    　ことを特徴とする請求項５に記載の画像復号装置。     The inverse quantization means
    When the predicted image of the block of interest is generated by using the block-shaped pixel group in the decoded region of the frame to which the block of interest belongs.
    The feature is that the block of interest is inversely quantized using the same quantization matrix as the quantization matrix when a predicted image generated from a pixel group of a frame different from the frame to which the block of interest belongs is used. The image decoding apparatus according to claim 5.
  9. 　ビットストリームからブロック単位に画像を復号する画像復号方法であって、
    　前記ビットストリームから、量子化マトリクスの符号化データを復号する第１の復号工程と、
    　前記ビットストリームから、ブロック単位の量子化係数の符号化データと予測情報を復号する第２の復号工程と、
    　前記予測情報に基づいて特定される量子化マトリクスと、量子化パラメータとを用いて、前記量子化係数の符号化データに基づく着目ブロックの量子化係数から当該着目ブロックの変換係数を導出する逆量子化を行う逆量子化工程と、
    　前記逆量子化工程で導出した前記変換係数に逆直交変換を行って、前記着目ブロックの予測誤差を導出する逆直交変換工程と
    　前記予測情報に従って前記着目ブロックの予測画像を生成し、当該予測画像と、前記逆直交変換工程で導出した前記予測誤差とから前記着目ブロックの画像を再生する再生工程と
    　を有し、
    　前記再生工程による前記予測画像を生成する方法は、
    　前記着目ブロックが属するフレームの復号済の領域におけるブロック状の画素群を用いる方法を含む
    　ことを特徴とする画像復号方法。     An image decoding method that decodes an image from a bitstream in block units.
    The first decoding step of decoding the coded data of the quantization matrix from the bit stream, and
    A second decoding step of decoding the coded data and the prediction information of the quantization coefficient in block units from the bit stream,
    Inverse quantum that derives the conversion coefficient of the block of interest from the quantization coefficient of the block of interest based on the coded data of the quantization coefficient using the quantization matrix specified based on the prediction information and the quantization parameter. Inverse quantization process to perform
    The conversion coefficient derived in the inverse quantization step is subjected to inverse orthogonal transformation to derive the prediction error of the block of interest, and a predicted image of the block of interest is generated according to the prediction information and the predicted image. And a reproduction step of reproducing the image of the block of interest from the prediction error derived in the inverse orthogonal conversion step.
    The method for generating the predicted image by the reproduction step is
    An image decoding method comprising a method of using a block-shaped pixel group in a decoded region of a frame to which the block of interest belongs.
  10. 　コンピュータが読み込み実行することで、前記コンピュータに、請求項１乃至３のいずれか１項に記載の画像符号化装置の各手段として機能させるためのプログラム。 A program for causing the computer to function as each means of the image coding device according to any one of claims 1 to 3 by being read and executed by the computer.
  11. 　コンピュータが読み込み実行することで、前記コンピュータに、請求項５乃至８のいずれか１項に記載の画像復号装置の各手段として機能させるためのプログラム。 A program for causing the computer to function as each means of the image decoding device according to any one of claims 5 to 8 by reading and executing the computer.
PCT/JP2019/050359 2019-03-11 2019-12-23 Image coding device, image decoding device, image coding method, image decoding method, and program WO2020183859A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-044195 2019-03-11
JP2019044195A JP2020150334A (en) 2019-03-11 2019-03-11 Image encoder, image decoder, image encoding method, image decoding method, and program

Publications (1)

Publication Number Publication Date
WO2020183859A1 true WO2020183859A1 (en) 2020-09-17

Family

ID=72427233

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/050359 WO2020183859A1 (en) 2019-03-11 2019-12-23 Image coding device, image decoding device, image coding method, image decoding method, and program

Country Status (3)

Country Link
JP (1) JP2020150334A (en)
TW (1) TW202034699A (en)
WO (1) WO2020183859A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150103910A1 (en) * 2013-10-14 2015-04-16 Texas Instruments Incorporated Intra Block Copy (IntraBC) Cost Estimation
JP2016046641A (en) * 2014-08-21 2016-04-04 三菱電機株式会社 Image coding device, image decoding device, image coding method and image decoding method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150103910A1 (en) * 2013-10-14 2015-04-16 Texas Instruments Incorporated Intra Block Copy (IntraBC) Cost Estimation
JP2016046641A (en) * 2014-08-21 2016-04-04 三菱電機株式会社 Image coding device, image decoding device, image coding method and image decoding method

Also Published As

Publication number Publication date
TW202034699A (en) 2020-09-16
JP2020150334A (en) 2020-09-17

Similar Documents

Publication Publication Date Title
US11985352B2 (en) Image coding apparatus, image coding method, and storage media
JP2023181542A (en) Image encoding device, image decoding device, control method thereof, and program
JP2024015184A (en) Image decoding device and method, and program
JP7497486B2 (en) Image decoding device, image decoding method, and program
JP2023113858A (en) Image decoding device, image decoding method, and program
WO2020003740A1 (en) Image coding device, image decoding device, control methods therefor, and program
WO2020183859A1 (en) Image coding device, image decoding device, image coding method, image decoding method, and program
KR20220051022A (en) Image encoding apparatus, image encoding method, image decoding apparatus, image decoding method, and non-transitory computer-readable storage medium
WO2021054012A1 (en) Image encoding device, image decoding device, control method therefor, and program
WO2021187023A1 (en) Image encoding device, image encoding method and program, image decoding device, and image decoding method and program
WO2020129489A1 (en) Image encoding device, image decoding device, and method and program for controlling same
JP2021150723A (en) Image encoding device, image encoding method and program, and image decoding device, image decoding method and program
WO2020129498A1 (en) Image encoding device, image encoding method, image decoding device, and image decoding method
KR20220055495A (en) Image encoding apparatus, image encoding method, image decoding apparatus, image decoding method, and computer program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19918715

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19918715

Country of ref document: EP

Kind code of ref document: A1