US20200195969A1 - Method for coding a digital image, decoding method, devices, terminal equipment and related computer programs - Google Patents
Method for coding a digital image, decoding method, devices, terminal equipment and related computer programs Download PDFInfo
- Publication number
- US20200195969A1 US20200195969A1 US16/338,878 US201716338878A US2020195969A1 US 20200195969 A1 US20200195969 A1 US 20200195969A1 US 201716338878 A US201716338878 A US 201716338878A US 2020195969 A1 US2020195969 A1 US 2020195969A1
- Authority
- US
- United States
- Prior art keywords
- vector
- assumption
- signs
- sign
- coefficients
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000004590 computer program Methods 0.000 title description 14
- 239000013598 vector Substances 0.000 claims abstract description 274
- 238000012545 processing Methods 0.000 claims abstract description 27
- 230000006870 function Effects 0.000 claims description 47
- 230000001131 transforming effect Effects 0.000 claims description 27
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 9
- 230000009466 transformation Effects 0.000 description 9
- 101000597862 Homo sapiens Transmembrane protein 199 Proteins 0.000 description 7
- 102100035335 Transmembrane protein 199 Human genes 0.000 description 7
- 230000008901 benefit Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 7
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 101000805729 Homo sapiens V-type proton ATPase 116 kDa subunit a 1 Proteins 0.000 description 6
- 101000854879 Homo sapiens V-type proton ATPase 116 kDa subunit a 2 Proteins 0.000 description 6
- 101000854873 Homo sapiens V-type proton ATPase 116 kDa subunit a 4 Proteins 0.000 description 6
- 102100020737 V-type proton ATPase 116 kDa subunit a 4 Human genes 0.000 description 6
- 238000007792 addition Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 238000000844 transformation Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 101000957313 Acanthamoeba polyphaga mimivirus Mitochondrial carrier-like protein L276 Proteins 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- TWRXJAOTZQYOKJ-UHFFFAOYSA-L magnesium chloride Substances [Mg+2].[Cl-].[Cl-] TWRXJAOTZQYOKJ-UHFFFAOYSA-L 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 239000001119 stannous chloride Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 229940000425 combination drug Drugs 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003116 impacting effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/48—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/521—Processing of motion vectors for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Definitions
- the field of the invention is that of the encoding and decoding of images or sequences of images, and in particular of video streams.
- the invention relates to the compression of images or sequences of images using a block representation of the images.
- the invention may especially apply to the image or video encoding implemented in the current encoders (JPEG, MPEG, H.264, HEVC, etc. and their amendments) or to future ones, and to the corresponding decoding process.
- a current block to be coded is predicted from a previously coded decoded block.
- a residual block is obtained by subtracting the original values from the predicted values. It is then transformed using a DCT type transformation (for “Discrete Cosine Transform”) or wavelets.
- the transformed coefficients are quantized and their amplitudes are subjected to entropic coding of the Huffmann or arithmetic type.
- Such coding provides efficient performances because, due to the transformation, the values of the amplitudes to be coded are mostly equal to zero.
- the signs of the coefficients are coded by a bit 0 or 1 .
- One advantage is that it ensures that for a large majority of predicted signs, the prediction indicator will indicate a correct prediction and thus provide a favourable context for entropy coding. In this way, compression performance is increased.
- a disadvantage of this technique is that it requires, for M signs to be predicted, to evaluate the prediction obtained for each of the 2 M combinations of values of these signs and, to do this, to reconstruct the current block from this prediction.
- Such a reconstruction notably requires the transformation of the prediction residue of the current block, which introduces a significant complexity on the coder side as the decoder side.
- An embodiment of the invention relates to a method for coding a digital image, said image being divided into a plurality of blocks of pixels processed in a defined order, said method comprising the following steps, implemented for a current block:
- the pixel vector associated with the current assumption is obtained by simple linear combination of a pixel vector associated with an assumption already processed and at least one pattern vector of pixels previously calculated, without needing to resort to the application of a transform.
- the pixel pattern vector is obtained by transforming a coefficient vector, comprising a single non-zero coefficient, which makes the operation much less complex.
- the coefficient pattern vectors comprise a single non-zero coefficient
- the application of the inverse transform generates a very moderate calculation cost. Indeed, for a current block of size N ⁇ N pixels of which all the coefficients are likely to be non-zero, for a normal inverse transform N ⁇ N projections on basic vectors of dimension N ⁇ N are necessary. We need N 4 multiplications and N 4 additions.
- N 2 multiplications and N 2 additions are necessary.
- the pixel vector of the current assumption is constructed by linear combination of the pixel vector associated with an already processed assumption and as many vectors as there are sign differences between the current assumption and this assumption.
- the invention is based on an entirely new and inventive approach to prediction, which consists in exploiting the result of the processing of at least one previous assumption in order to more effectively address the current assumption.
- the invention also relates to a method for decoding a digital image from a bitstream, said image being divided into a plurality of blocks processed in a defined order, the bitstream comprising, for a block, coded data representative of a vector of transformed coefficients, said method comprising the following steps, implemented for a block, so-called current block:
- the decoder evaluates the different sign prediction assumptions according to the principle of the invention and in a manner similar to the encoder.
- the step of obtaining at least one pattern vector is implemented M times and the step of obtaining a pixel vector by transforming at least one vector of coefficients is carried out for M+1 first assumptions.
- the pixel vectors of the other assumptions can then all be obtained by linear combination of these M pattern vectors and M+1 pixel vectors.
- the state of the art implements 2 M transformations, each of N 4 complexity, i.e.: 2 M ⁇ N 4 multiplications and 2 M ⁇ N 4 additions, while the invention implements M+1 transformations and 2 M -M ⁇ 1 linear combinations, representing (M+1) ⁇ N 4 multiplications and (M+1) ⁇ N 4 +(2 M -M ⁇ 1)*N 2 additions.
- the sign of the non-zero coefficient is chosen equal to +.
- said at least one pattern vector is calculated in a step prior to the processing of the assumptions and stored in memory and obtaining said at least one pattern vector comprises reading said memory.
- the obtaining of the pattern vectors is not dependent on the pixel vectors of previous assumptions, it can be done at the initialisation of the prediction step for the current block.
- M vector patterns are calculated beforehand and the step of obtaining a pixel vector is performed for a single first assumption.
- the cost function is calculated for a subset of the pixels of a pixel vector, said subset comprising at least the pixels located at a boundary of the current block with a neighbouring block already processed and the transforming step transforms the coefficient vector into a sub-vector of pixels corresponding to said subset.
- a particular transform is used which reconstructs only the pixels needed to evaluate the cost function.
- One advantage is a reduction in the complexity of the evaluation of the cost function.
- the invention also relates to a coding device adapted to implement the coding method according to any one of the particular embodiments defined above.
- This coding device may of course include the various characteristics relating to the coding method according to the invention. Thus, the characteristics and advantages of this coding device are the same as those of the decoding method, and are not detailed further.
- the invention also relates to a decoding device adapted to implement the decoding method according to any one of the particular embodiments defined above.
- This decoding device may of course include the various characteristics relating to the decoding method according to the invention. Thus, the characteristics and advantages of this decoding device are the same as those of the decoding method, and are not detailed further.
- the invention also relates to a terminal equipment comprising a transmission module and a data receiving module to and from a telecommunications network, a coding device and a decoding device according to the invention.
- the invention also relates to a computer program comprising instructions for implementing the steps of a coding method as described above, when this program is executed by a processor.
- the invention also relates to a computer program comprising instructions for implementing the steps of a decoding method of a digital image as described above, when this program is executed by a processor.
- These programs can use any programming language. They can be downloaded from a communication network and/or recorded on a computer-readable medium.
- the invention finally relates to recording media, which are readable by a processor, whether or not integrated into the coding device of a digital image and to the device for decoding a digital image according to the invention, which might be removable, respectively storing a computer program implementing a coding method and a computer program implementing a decoding method, as described above.
- FIG. 1 schematically shows a sequence of digital images cut into blocks
- FIG. 2 schematically shows the steps of a method for coding images according to the invention
- FIG. 3 details the processing step of a current block
- FIGS. 4A to 4C schematically illustrate the pixels of the current block and of the neighbouring blocks already processed implemented for the evaluation of a cost function
- FIG. 5 describes in detail the step of predicting signs of the current block according to a first embodiment of the invention
- FIG. 6 describes in detail the step of predicting a sequence of signs of the current block according to a second embodiment of the invention.
- FIG. 7 schematically shows the steps of a method for decoding an image according to the invention
- FIG. 8 details an example of a method for selecting the best combination of signs when decoding according to the invention.
- FIG. 9 schematically shows the hardware structure of a device for coding an image according to the invention, when it is integrated with a terminal equipment.
- FIG. 10 schematically shows the hardware structure of a device for decoding an image according to the invention, when it is integrated with a terminal equipment.
- the general principle of the invention relates to the prediction of signs of the current block and rests on a reconstruction of the decoded block for a combination of sign values to be predicted, based on reusing the results obtained for combinations of values already evaluated. More specifically, the invention consists in calculating a decoded pixel vector of the current block for a current assumption by combining at least one decoded pixel vector already calculated for at least one preceding assumption and at least one pixel pattern vector previously calculated.
- an original video consisting of a sequence of M images I 1 , I 2 , . . . Im, with M a non-zero integer.
- the images are encoded and decoded in a predefined order known from the encoder and the decoder, for example in the time order I 1 then I 2 , . . . , then IM (this order may differ according to the embodiment).
- a first block is selected.
- the first block to be processed is selected as the current block C.
- it is the first block (in lexicographic order). It has N ⁇ N pixels, with N a non-zero integer, but of course, the invention would apply similarly to a rectangular block.
- the decoded current picture (which, in a video encoder, is built into the encoder so as to allow a prediction of subsequent blocks of the video) will be noted ID.
- a prediction Pr of the block C is determined. It is a prediction block constructed by known means, typically by motion compensation (block from a previously decoded reference image), or by intra prediction (block constructed from decoded pixels immediately close to the current block in the image ID).
- the prediction information related to P is encoded in the compressed file. It is assumed here that there are K possible prediction modes m 1 , m 2 , . . . , mK, and that the prediction mode chosen for block C is the mode mkj.
- the residue R is transformed by a DCT transform or into a wavelet, known per se (REF JPEG standard DCT, JPEG2000 standard for wavelet transform), into a block called RT.
- the transformed residue RT is quantized by conventional quantization means, for example scalar or vector, into a quantized residual block RQ.
- This quantized block RQ contains N ⁇ N coefficients. As known in the state of the art, these coefficients are scanned in a predetermined order so as to constitute a one-dimensional vector RQ[i] where the index i varies from 0 to N 2 ⁇ 1. The index i is called the frequency of the coefficient RQ [i].
- the amplitude information of the RQ coefficients is encoded by entropy coding (Huffman coding or arithmetic coding).
- amplitude here is meant the absolute value of the coefficient.
- Amplitude coding means are for example described in the HEVC standard (explained in Sole J. et al., «Transform Coefficient Coding in HEVC», IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 12, December 2012).
- a piece of information representative of the fact that the coefficient is non-zero can be encoded for each coefficient. Then, for each non-zero coefficient, one or more pieces of information relating to the amplitude are encoded.
- the invention is valid for any other descriptive information of the compressed image (syntax element).
- the invention can be applied to the significance information (a syntax element representative of the zero or non-zero value of a transformed coefficient amplitude), an amplitude information, a mode information of coding (inter or intra for example), etc.
- a step E 2 selects the signs to be predicted, in the current block, according to known prior probabilities, as described in document FR3023112 already cited.
- the encoder may use a set of probabilities for correct detection of sign, also called scores. In the preferred embodiment, these probabilities are dependent on the size of the current block C, the amplitude of the coefficient associated with the sign, its frequency, and the prediction mode used among the K possible modes.
- the signs to be predicted are selected while keeping those whose probability is greater than a predetermined threshold.
- the signs to be predicted SP are scheduled.
- This order can be predefined: for example, in the scanning order of the signs as defined in the HEVC standard.
- the signs are ordered by decreasing probability of correct detection.
- the purpose is to select the best assumption of combination of the values of M signs of the sequence SP among 2 M possible assumptions.
- a cost associated with each assumption is evaluated using a cost function FC and the assumption that minimizes the cost function is chosen.
- the evaluation function must make sure to generate a minimal cost when the sign assumption is the most likely.
- the cost function measures a distortion along the left and top border of the current block, as shown in FIG. 4A .
- the decoded pixels y 1 to y 8 coming from the blocks previously processed are situated on one side of a border of the current block C, and on the other side, the pixels x 1 to x 7 of the current block decoded with the assumption of signs of which the cost is to be measured.
- the cost function FC is obtained from the calculation of a difference between the values of x 1 pixels x 7 located in the blocks previously processed, and y 1 to y 7 pixels located on the first line and the first column of the current block.
- a cost measurement could be based not only on the pixels located at the borders, but also on the interior pixel rows, neighbouring the previous ones, as shown by FIGS. 4B to 4C .
- the difference is calculated between the values of the pixels of previously processed blocks and the pixels y 1 to y 12 located on the first line, the first column as well as the second line and the second column of the current block. For example, we evaluate the difference between y 1 and x 1 and y 8 and x 1 and, similarly, between y 2 and x 2 and y 9 and x 2 , and so on.
- the difference is calculated between the values of the pixels located in the previously processed blocks, and all the pixels of the current block.
- the invention proposes a simplifying solution, two particular embodiments of which will now be described with reference to FIGS. 5 and 6 .
- the assumptions are browsed according to a predetermined order.
- the assumptions are browsed in an order such that the sign of a single coefficient changes between two consecutive assumptions. This order is known to a person having ordinary skills in the art as “Gray code” or “reflected binary code”.
- E 512 the pixel vectors of VPH 0 and VPH 1 associated with each of these assumptions are calculated by inverse transform of the coefficient vectors obtained from the vector of transformed coefficients VC of the current block by assigning thereto the signs of the assumption considered.
- the vectors VPH 0 and VPH 1 obtained are stored in memory.
- a first pattern vector VM 2 is calculated by difference between the pixel vectors VPH 0 and VPH 1 associated with the assumptions H 0 and H 1 . It is understood that according to this first embodiment of the invention, this pattern vector is associated with a value change of the sign s 2 and represents in some way an impact of the change of the sign s 2 between the combination of signs of H 0 and that of H 1 in the spatial domain.
- E 511 a third first assumption H 2 , according to the predetermined order. Its combination of signs is ⁇ +, ⁇ , + ⁇ and differs from H 0 by the value of the sign s 1 .
- the vector of corresponding pixels VPH 2 is then calculated in E 512 in a conventional manner, by applying an inverse transform and stored in memory.
- a second pattern vector VM 1 is calculated by difference between the pixel vectors VPH 0 and VPH 2 associated with the assumptions H 0 and H 2 . This pattern vector is associated with the sign change of the sign s 1 between the assumptions H 0 and H 2 . It is stored in memory.
- a fourth assumption H 3 is considered, whose combi-nation is ⁇ +, ⁇ , ⁇ . It differs from H 2 by the value of the sign s 2 .
- a pixel vector VPH 3 associated with this assumption H 3 is constructed in E 54 by linear combination of the vector of pixels VPH 0 and the vector pattern VM 2 as follows:
- VPH 3 VPH 0+ VM 2
- the vector VPH 0 and the pattern vector VM 2 to be combined are obtained in a predefined table and stored in memory, so as to avoid the previous step of identification.
- Its pixel vector VPH 4 is calculated by applying an inverse transform and stored in memory.
- a pattern vector VM 0 is calculated by vector difference between VPH 4 and VPH 0 . It is associated with a sign change of the sign s 0 . It is stored in memory for later use.
- this embodiment implements 4 transforms for 3 signs to be predicted. More generally, according to this approach, if M signs are to be predicted, the prior art requires the calculation of 2 M inverse transforms, whereas the proposed method requires only M+1 transformed. The invention therefore allows a substantial reduction in complexity. Furthermore, when a vector of coefficients is decoded by adding pattern vectors, the value is likely to be slightly different from that obtained classically by inverse transform. However, the error is in general negligible and has no impact on the compression efficiency of the sign prediction method.
- the sign of the non-zero coefficient it can be chosen arbitrarily. For simplicity it is allocated the value +.
- the coefficient of the pattern vector corresponding to the position of the sign sj takes the absolute value of the co-located coefficient in the vector VC of the current block.
- the vectors of coefficients VC are assumed to be ⁇ +4, 0, ⁇ 2, ⁇ 3, 0, +1, 0, 0, ⁇ 1, 0, 0, . . . , 0 ⁇ and the three signs to be predicted are associated with the first, third and fourth coefficients. It is therefore ⁇ +, ⁇ , ⁇ .
- a first assumption to be processed is selected, according to a predetermined scanning order of the 2 M assumptions, for example the order of the Gray code.
- H 1 is associated with the combination ⁇ +, +, ⁇ .
- E 5 ′ 32 which previously calculated assumption differs only from a sign with the current assumption is identified and its sign is also identified.
- the previous assumption identified is H 0 and the sign that differs between the current combination and that of assumption H 0 is the sign s 2 .
- the pixel vector VPH 0 associated with the assumption H 0 and the pattern vector VM′ 2 are obtained by reading from a predefined table of a record associated with the current assumption H 1 .
- the pixel vector VPH 1 associated with the assumption H 0 is constructed by linear combination of the vector VP 0 and the vector pattern VM′ 2 as follows:
- VPH 3 is therefore calculated for example by linear combination of VPH 2 and VM′ 2 as follows:
- VPH 3 VPH 2 ⁇ 2 ⁇ VM′ 2
- VPH 3 is stored in memory.
- VPH 4 For the assumption H 4 associated with the combination ⁇ , +, + ⁇ , VPH 4 is calculated for example by linear combination of VPH 0 and VM′ 0 in the following way:
- VPH 4 VPH 0 ⁇ 2 ⁇ VM′ 0
- VPH 4 is stored in memory.
- VPH 5 VPH 4 ⁇ 2 ⁇ VM′ 2
- VPH 6 is calculated for example by linear combination of VPH 2 and VM′ 0 in the following way:
- VPH 7 VPH 3 ⁇ 2 ⁇ VM′ 0
- VPH 7 is stored in memory.
- this embodiment also implements 4 transforms for 3 signs to be predicted.
- the transforms implemented to calculate the M pattern vectors VM′j are not very complex, since the coefficient pattern vectors VMCj only include a non-zero coefficient. This second embodiment is therefore even more economical in operations than the previous one.
- the costs associated with each of the assumptions are calculated from the vectors obtained and from a cost function FC.
- step E 57 the purpose is to decide on the predicted sequence from the costs that have just been calculated. There are different approaches to exploit the costs associated with each of the assumptions. An exemplary method will now be detailed.
- This comparison makes it possible to define the value of a prediction indicator IP intended to be coded and then transmitted in the bitstream.
- this indicator indicates whether the predicted sign s 0 is equal to or different from the real sign (for example, respectively 0 or 1). If we take the previous example ⁇ +4, 0, ⁇ 2, ⁇ 3, 0, +1, 0, 0, ⁇ 1, 0, 0, . . . , 0 ⁇ , it is a +. There is then coding of an indicator 0. This value IPs 0 is stored in memory.
- E 6 the values of indicators IP decided during the preceding step of sign prediction are coded. Since only the signs that are associated with a representative score of a sufficient reliability level are predicted, the prediction indicator more often takes the value 1 than the value 0. This is exploited by the entropy coding for reducing the size of the compressed signal.
- the entropy coding takes into account the score S j associated with the predicted sign for coding the indicator IP.
- the score has a value between 0 (low reliability of the prediction) and 10 (high reliability of the prediction)
- the entropic coding of the indicators is parametered taking into account the score, so as to exploit the more or less uniform distribution of the indicators.
- CABAC-type entropic coding known from the HEVC standard, is used by initializing the probabilities used in CABAC based on the predetermined scores.
- the decoded block is reconstructed by applying to the quantized residual RQ the steps of dequantisation and inverse transform (known per se).
- a decoded residual block RD is obtained.
- RD is added the predictor block P to obtain the decoded block BD.
- the decoded block BD is also added to the reconstructed image ID. This makes it possible to have in the encoder a decoded version of the current image. This decoded version is used in particular during the step of constructing a prediction of the signs selected to be predicted.
- E 8 the values encoded in the bitstream TB are inserted.
- step E 9 whether the current block is the last block to be processed will be tested, given the browsing order defined previously. If yes, the coding unit has finished processing. If no, the next step is the step of selecting the next block.
- the next block to be processed by the encoding unit is selected, according to the sequence defined previously. This block becomes the current block to be processed, and the next step is the step of testing E 1 the current block.
- the steps of a method of decoding a coded digital image according to an embodiment of the invention are described. It is assumed that the bitstream TB has been received by a decoding device implementing the decoding method according to the invention. In a variation, the decoding device obtains a compressed file FC.
- the first block to be processed is initially selected as the current block C′.
- this is the first block (in lexicographic order).
- This block comprises for example N ⁇ N pixels.
- the invention would also apply to a rectangular block.
- the current block C′ is processed by implementing the decoding scheme corresponding to the coding scheme used by the encoder, for example as specified in the HEVC standard. In particular, during this step, it identifies a set of elements ED of data description to be decoded for the current block C′.
- a prediction P′ of the block to be decoded C′ is produced.
- the prediction information related to P′ is read in the bitstream or in the compressed file and then decoded.
- the prediction mode information mj is therefore decoded.
- the amplitude information of the residue to be decoded RQ′ is also decoded into the bitstream or the compressed file and then decoded. We now know the amplitudes of RQ′[i], but not yet the signs.
- the decoding method according to the invention implements the step of selecting the signs to be predicted from the identified description elements. This step has already been described in detail for the coding method in relation to FIG. 2 .
- the signs to be predicted are advantageously selected according to predetermined scores. A subset SP is obtained.
- the method reads in the bitstream TB the coded data relating to the signs of the coefficients of the unpredicted current block and decodes them with means adapted to those used during coding (typically binary decoding).
- entropy decoding uses the probability pi obtained previously.
- the elements of the set SP obtained in an initial sequence SSP are ordered, for example by decreasing scores, as already described for the coding method according to the invention.
- D 5 the sign values of the SSP sequence are predicted. To do so, the best combination in the sense of a cost function FC is selected among the combinations of possible values of the current sequence SP.
- FC cost function
- D 7 the decoded signs are reconstructed from the indicator values IP decoded and the predicted values.
- the next step D 8 is for reconstructing the current block C′ from the description elements decoded, EP predicted (D 5 ) and ENP not predicted (D 3 ), the amplitude information of the residual block coefficients RQ′ and the prediction P′ obtained in D 1 .
- the dequantized residue is then subjected to a reverse transform from that used in the coding.
- the decoded residue is then obtained.
- the decoded block BD′ is reconstructed by adding the decoded residue to the prediction P′.
- This block is integrated with the image being decoded.
- step D 9 whether the current block is the last block to be processed is tested, given the scanning order of the blocs, as defined previously. If yes, the decoding is complete. If not, the next step is the step D 0 for selecting the next block and the steps of the decoding method are repeated.
- the prediction step D 5 and the step D 6 of decoding the signs of the sequence SSP are combined in a single step D′ 5 :
- D′ 5 1 a current sequence Seq c that is initialised to the initial sequence SP is considered.
- D′ 5 2 the best combination within the meaning of a cost function FC is selected, the same as that used by the coding method which produced the bitstream to be decoded, from the combinations of possible values for the current sequence SP.
- the first sign of the sequence Seq c is decoded.
- the costs corresponding to each assumption or possible combination for the set of signs to be predicted are generated, using the solution of the invention, which has just been described in connection with FIG. 7 .
- D′ 5 3 the value of the first sign s 0 is predicted by its value in the combination corresponding to the assumption H 2 . This is +.
- the prediction indicator IP is decoded for that first sign s 0 , from coded data extracted from the bitstream or compressed file. This indicator indicates whether the predicted sign has been correctly predicted or not. For example, let us assume that the decoded value is 0 and is associated with a correct prediction.
- D′ 5 6 we test whether the sign s 0 is the last of the current sequence. If this is the case, the processing ends. Otherwise, the current sequence is updated in D 5 7 by deleting the element s 0 which has just been decoded.
- the second element s 1 which has then become the first one in the current sequence, is processed.
- C 1 is identified as the minimum cost.
- s 1 is predicted by its value in the combination of assumption H 1 , that is to say +.
- the indicator IP corresponding to s 1 is decoded from the coded data extracted from the bitstream or compressed file.
- An indicator is decoded, which indicates whether the predicted sign is equal to or different from the actual sign. In our example, let us assume that the decoded value of IP is 0, which means that the prediction of this sign is correct.
- D′ 5 6 it is tested whether s is the last element. Since this is not the case, the current sequence in D′ 5 7 is updated, by removing s 1 . The new sequence Seq c is reduced to the element s 2 .
- the last sign s 2 is decoded.
- D′ 5 2 the combinations of values of the original sequence Seq i are considered, for which the elements of description already processed s 0 and s 1 take on their decoded value.
- C 3 is identified as the minimum cost.
- the last sign s 2 in the predefined order is predicted, in D 5 3 , by its value in the combination of assumption H 3 : it is ⁇ .
- an indicator IP associated with the sign s 2 is decoded at D 5 4 . It indicates whether the sign s 2 has been correctly predicted or not. In our example, let us assume that the decoded value of IP is 0, which corresponds to a correct prediction.
- the invention is not limited to the embodiment that has just been presented. Other implementation choices can be made, such as the one to keep the sequence at its original length, to update the index of the current element, initialised to the first description element ED 0 and to terminate the iterations once the last element ED M-1 has been processed.
- step D 2 since all the elements of the initial sequence selected according to the predetermined scores are predicted, it is therefore known as of the outcome of step D 2 how many prediction indicators IP are to be extracted from the bitstream or compressed file.
- this makes it possible to implement the invention in such a way that it decorrelates the reading and parsing operations of the coded data contained in the bitstream or compressed file of the current block processing operations in accordance with the coding/decoding scheme implemented.
- the decoding could be organised by using a specific component for the parsing/reading of the coded data in the bitstream and another for the reconstruction operations of the decoded blocks.
- the encoding and decoding methods described above can be integrated into standard video encoders/decoders such as HEVC/H.265, AVC/H.264 or any type of proprietary video encoders/decoders.
- the coding and decoding methods according to the invention also apply to all types of coders/decoders of still images and more generally of signals using a predictive coding process.
- the coding and decoding methods have been previously described in the case of the prediction of the signs of the coefficients of a residue block of the image to be encoded/decoded. These methods are easily applicable to the case of the prediction of another block description element for example.
- FIG. 9 shows the simplified structure of a coding device 100 adapted to implement the coding method according to any of the particular embodiments of the invention.
- the coding device 100 is adapted to encode at least one image in the form of a coded data stream, said image being divided into blocks.
- the coding device 100 is notably configured for:
- the steps of the coding method are implemented by computer program instructions.
- the coding device 100 has the conventional architecture of a computer and notably comprises a memory MEM 1 , a processing unit UT 1 , equipped for example with a microprocessor ⁇ 1, and driven by the computer program Pg 1 stored in the memory MEM 1 .
- the computer program Pg 1 includes instructions for implementing the steps of the coding method as described above, when the program is executed by the processor PROC.
- the code instructions of the computer program Pg 1 are for example loaded into a RAM before being executed by the processor PROC.
- the processor ⁇ 1 of the processing unit UT 1 implements in particular the steps of the coding method described above, according to the instructions of the computer program Pg 1 .
- the coding method is implemented by functional modules or units.
- the coding device 100 furthermore comprises the following modules:
- the device 100 further comprises a unit M 1 for storing the coding contexts of the coefficients, predetermined scores associated with each of these contexts, the pixel vectors associated with the 2 M assumptions for the sequence of signs to predict, their related costs, as well as the pattern vectors from which the pixel vectors are derived.
- These units are driven by the processor ⁇ 1 of the processing unit 110 .
- such a device 100 can be integrated in a user terminal equipment TU, such as an encoder, a personal computer, a tablet, a digital camera, a smart phone, etc.
- the device 100 is then arranged to cooperate at least with the next module of the terminal TU:
- Such a functional module can be in hardware and/or software form.
- a functional module may include a processor, a memory and program code instructions for implementing the function corresponding to the module when the code instructions are executed by a processor.
- a functional module can be implemented by any type of suitable coding circuits, such as for example and without limitation microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), a logic unit wiring.
- DSPs Digital Signal Processors
- ASICs Application Specific Integrated Circuits
- FPGAs Field Programmable Gate Arrays
- FIG. 10 shows the simplified structure of a decoding device 200 adapted to implement the decoding method according to any of the particular embodiments of the invention which have just been described in connection to FIGS. 7, 8, 5 and 6 .
- the decoding device 200 is adapted to decode at least one image in the form of a coded data stream representative of at least one image, said image being divided into blocks.
- the decoding device 200 is notably configured for:
- the decoding device 200 has the conventional architecture of a computer and notably comprises a memory MEM 2 , a processing unit UT 2 , equipped for example with a microprocessor ⁇ 2, and driven by the computer program Pg 2 stored in the memory MEM 2 .
- the computer program Pg 1 includes instructions for implementing the steps of the decoding method as described above, when the program is executed by the processor ⁇ 2.
- the code instructions of the computer program Pg 2 are for example loaded into a RAM before being executed by the processor ⁇ 2.
- the processor ⁇ 2 of the processing unit UT 0 implements in particular the steps of the decoding method described above, according to the instructions of the computer program Pg 2 .
- the decoding method is implemented by functional modules.
- the decoding device 200 furthermore comprises the following modules:
- the device 200 furthermore comprises a unit M 2 for storing the predetermined scores or correct detection probability used to select the signs to be predicted, pixel vectors corresponding to the 2 M sign combination assumptions to be predicted, their related costs as well as the pattern vectors from which the pixel vectors are derived.
- These units are driven by the processor ⁇ 2 of the processing unit UT 2 .
- Such a functional module can be in hardware and/or software form.
- a functional module may include a processor, a memory and program code instructions for implementing the function corresponding to the module when the code instructions are executed by a processor.
- a functional module can be implemented by any type of suitable coding circuits, such as for example and without limitation microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), a logic unit wiring.
- DSPs Digital Signal Processors
- ASICs Application Specific Integrated Circuits
- FPGAs Field Programmable Gate Arrays
- such a device 200 can be integrated into a user terminal TU, for example a decoder, a Set-Top-Box, a digital television, a computer, a tablet, a smartphone, etc.
- the device 200 is then arranged to cooperate at least with the next module of the terminal UT:
- the invention which has just been presented can find many applications, in particular in the context of video signal compression, audio (speech, sound), still images, images acquired by an medical imaging module. It applies for example to two-dimensional (2D), three-dimensional (3D) contents including a depth map, or multispectral images (whose colour intensities are different from the three red green blue bands) or finally to full images.
- An exemplary embodiment of the invention proposes a solution that considerably simplifies the process of predicting signs without impacting compression performance.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This application is a Section 371 National Stage Application of International Application No. PCT/FR2017/052422, filed Sep. 12, 2017, the content of which is incorporated herein by reference in its entirety, and published as WO 2018/065687 on Apr. 12, 2018, not in English.
- The field of the invention is that of the encoding and decoding of images or sequences of images, and in particular of video streams.
- More specifically, the invention relates to the compression of images or sequences of images using a block representation of the images.
- The invention may especially apply to the image or video encoding implemented in the current encoders (JPEG, MPEG, H.264, HEVC, etc. and their amendments) or to future ones, and to the corresponding decoding process.
- We consider a conventional compression scheme of a digital image, according to which the image is divided into blocks of pixels. A current block to be coded is predicted from a previously coded decoded block. A residual block is obtained by subtracting the original values from the predicted values. It is then transformed using a DCT type transformation (for “Discrete Cosine Transform”) or wavelets. The transformed coefficients are quantized and their amplitudes are subjected to entropic coding of the Huffmann or arithmetic type. Such coding provides efficient performances because, due to the transformation, the values of the amplitudes to be coded are mostly equal to zero.
- On the other hand, it does not apply to the values of the signs of the coefficients, whose + and − values are generally associated with equivalent appearance probabilities. Thus, the signs of the coefficients are coded by a
bit 0 or 1. - We know of the French patent application published under the number FR3023112 selecting the signs of the coefficients to be predicted in the current block according to a score associated with a coding context of the coefficient, the score being representative of a reliability level of a prediction of the sign.
- One advantage is that it ensures that for a large majority of predicted signs, the prediction indicator will indicate a correct prediction and thus provide a favourable context for entropy coding. In this way, compression performance is increased.
- A disadvantage of this technique is that it requires, for M signs to be predicted, to evaluate the prediction obtained for each of the 2M combinations of values of these signs and, to do this, to reconstruct the current block from this prediction. Such a reconstruction notably requires the transformation of the prediction residue of the current block, which introduces a significant complexity on the coder side as the decoder side.
- An embodiment of the invention relates to a method for coding a digital image, said image being divided into a plurality of blocks of pixels processed in a defined order, said method comprising the following steps, implemented for a current block:
-
- Processing the current block for providing a vector of transformed coefficients;
- Selecting a sequence of M signs of coefficients to be predicted in the coefficient vector, with M being an integer greater than or equal to 2;
- Predicting values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- For a sign of the sequence, coding an indicator representative of a difference between its actual value and its predicted value;
According to the invention, the prediction step comprises the following substeps: - for at least one second assumption, so-called current assumption:
- Obtaining a pixel vector associated with the current assumption, by linear combination of a pixel vector associated with at least one first assumption and obtained by transforming a vector of coefficients comprising the combination of signs of the first assumption previously processed and different from the current assumption by the value of at least one sign (sj), with j an integer between 0 and M, and at least one pixel pattern vector obtained by transforming a coefficient vector (VMj) associated with said at least one sign (sj) of the sequence, of the same dimension as the vector of transformed coefficients, and comprising an amplitude coefficient equal to that of said vector at the position of the coefficient corresponding to the sign sj in said vector, all the other coefficients being set to zero;
- Calculating the cost function for the current assumption from the pixel vector obtained.
- With the invention, the pixel vector associated with the current assumption is obtained by simple linear combination of a pixel vector associated with an assumption already processed and at least one pattern vector of pixels previously calculated, without needing to resort to the application of a transform.
- The pixel pattern vector is obtained by transforming a coefficient vector, comprising a single non-zero coefficient, which makes the operation much less complex.
- Since the coefficient pattern vectors comprise a single non-zero coefficient, the application of the inverse transform generates a very moderate calculation cost. Indeed, for a current block of size N×N pixels of which all the coefficients are likely to be non-zero, for a normal inverse transform N×N projections on basic vectors of dimension N×N are necessary. We need N4 multiplications and N4 additions. On the other hand, in the case of a coefficient vector of coefficients, because a priori only a single coefficient is non-zero, there remains only one projection to realise, i.e. N2 multiplications and N2 additions.
- The pixel vector of the current assumption is constructed by linear combination of the pixel vector associated with an already processed assumption and as many vectors as there are sign differences between the current assumption and this assumption.
- The invention is based on an entirely new and inventive approach to prediction, which consists in exploiting the result of the processing of at least one previous assumption in order to more effectively address the current assumption.
- The evaluation of the costs of the different assumptions of sign prediction combinations is therefore simplified, without prejudice to the quality of this evaluation and the resulting prediction decision.
- Correspondingly, the invention also relates to a method for decoding a digital image from a bitstream, said image being divided into a plurality of blocks processed in a defined order, the bitstream comprising, for a block, coded data representative of a vector of transformed coefficients, said method comprising the following steps, implemented for a block, so-called current block:
-
- Decoding the amplitudes of the coefficients of the vector of transformed coefficients, extracted from the bitstream;
- Selecting a sequence of M signs of coefficients to be predicted in the coefficient vector, with M being an integer greater than or equal to 2;
- Predicting values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- Decoding values of a prediction indicator of the signs selected from coded data extracted from the bitstream, the prediction indicator of a sign being indicative of a difference between its actual value and its predicted value;
- Calculating the decoded values of the selected signs from the decoded prediction indicator values and the predicted values;
- Reconstructing coefficients of the block from decoded amplitudes and decoded signs;
According to the invention, the prediction step comprises the following substeps: - for at least one second assumption, so-called current assumption:
- Obtaining a pixel vector associated with the current assumption, by linear combination of a pixel vector associated with at least a first assumption previously processed and different from the current assumption by the value of at least one sign (sj), with j an integer between 0 and M, and obtained by transforming a coefficient vector comprising the combination of signs of the first assumption and at least one pixel pattern vector obtained by transforming a coefficient vector (VMj) associated with said at least one sign (sj) of the sequence, of the same dimension as the vector of transformed coefficients, and comprising an amplitude coefficient equal to that of said vector at the position of the coefficient corresponding to the sign sj in said vector, all the other coefficients being set to zero;
- Calculating the cost function for the current assumption from the pixel vector obtained.
- Thus, the same mechanism applies to the decoder as well as to the encoder. To find the prediction used by the encoder and to determine the value of the prediction indicators of the signs to be predicted from the current block, the decoder evaluates the different sign prediction assumptions according to the principle of the invention and in a manner similar to the encoder.
- Advantageously, the step of obtaining at least one pattern vector is implemented M times and the step of obtaining a pixel vector by transforming at least one vector of coefficients is carried out for M+1 first assumptions.
- In this way, the number of transformations to be applied is decreased, down to the number M+1 pixel vectors to be calculated.
- The pixel vectors of the other assumptions can then all be obtained by linear combination of these M pattern vectors and M+1 pixel vectors.
- The state of the art implements 2M transformations, each of N4 complexity, i.e.: 2M×N4 multiplications and 2M×N4 additions, while the invention implements M+1 transformations and 2M-M−1 linear combinations, representing (M+1)×N4 multiplications and (M+1)×N4+(2M-M−1)*N2 additions.
- If we approximate in a real case (N is large and M small), the complexity of the state of the art is in O(2M×N4) and that of the invention in O(M×N4) which is much lower: for the variable M, we go from exponential complexity to linear complexity.
- Advantageously, to obtain the pixel vector, the sign of the non-zero coefficient is chosen equal to +.
- An advantage of this solution is its simplicity. Of course, we could choose other sign values.
- Advantageously, said at least one pattern vector is calculated in a step prior to the processing of the assumptions and stored in memory and obtaining said at least one pattern vector comprises reading said memory.
- According to this embodiment, since the obtaining of the pattern vectors is not dependent on the pixel vectors of previous assumptions, it can be done at the initialisation of the prediction step for the current block.
- Advantageously, M vector patterns are calculated beforehand and the step of obtaining a pixel vector is performed for a single first assumption.
- In this way, the number of transformations to be applied is minimised. Among the M+1 transformations implemented, M are applied to vectors comprising only one non-zero coefficient. The operation is therefore overall very little complex.
- According to another aspect of the invention, the cost function is calculated for a subset of the pixels of a pixel vector, said subset comprising at least the pixels located at a boundary of the current block with a neighbouring block already processed and the transforming step transforms the coefficient vector into a sub-vector of pixels corresponding to said subset.
- A particular transform is used which reconstructs only the pixels needed to evaluate the cost function. One advantage is a reduction in the complexity of the evaluation of the cost function.
- The various embodiments or features mentioned below may be added independently or in combination with each other, to the characteristics of the decoding method and/or coding method defined above.
- The invention also relates to a coding device adapted to implement the coding method according to any one of the particular embodiments defined above. This coding device may of course include the various characteristics relating to the coding method according to the invention. Thus, the characteristics and advantages of this coding device are the same as those of the decoding method, and are not detailed further.
- The invention also relates to a decoding device adapted to implement the decoding method according to any one of the particular embodiments defined above. This decoding device may of course include the various characteristics relating to the decoding method according to the invention. Thus, the characteristics and advantages of this decoding device are the same as those of the decoding method, and are not detailed further.
- Correlatively, the invention also relates to a terminal equipment comprising a transmission module and a data receiving module to and from a telecommunications network, a coding device and a decoding device according to the invention.
- The invention also relates to a computer program comprising instructions for implementing the steps of a coding method as described above, when this program is executed by a processor.
- The invention also relates to a computer program comprising instructions for implementing the steps of a decoding method of a digital image as described above, when this program is executed by a processor.
- These programs can use any programming language. They can be downloaded from a communication network and/or recorded on a computer-readable medium.
- The invention finally relates to recording media, which are readable by a processor, whether or not integrated into the coding device of a digital image and to the device for decoding a digital image according to the invention, which might be removable, respectively storing a computer program implementing a coding method and a computer program implementing a decoding method, as described above.
- Other features and advantages of the invention will appear more clearly on reading the following description of an embodiment of the invention, given as a simple illustrative and non-limiting example, and the appended drawings among which:
-
FIG. 1 schematically shows a sequence of digital images cut into blocks; -
FIG. 2 schematically shows the steps of a method for coding images according to the invention; -
FIG. 3 details the processing step of a current block; -
FIGS. 4A to 4C schematically illustrate the pixels of the current block and of the neighbouring blocks already processed implemented for the evaluation of a cost function; -
FIG. 5 describes in detail the step of predicting signs of the current block according to a first embodiment of the invention; -
FIG. 6 describes in detail the step of predicting a sequence of signs of the current block according to a second embodiment of the invention; -
FIG. 7 schematically shows the steps of a method for decoding an image according to the invention; -
FIG. 8 details an example of a method for selecting the best combination of signs when decoding according to the invention; -
FIG. 9 schematically shows the hardware structure of a device for coding an image according to the invention, when it is integrated with a terminal equipment; and -
FIG. 10 schematically shows the hardware structure of a device for decoding an image according to the invention, when it is integrated with a terminal equipment. - The general principle of the invention relates to the prediction of signs of the current block and rests on a reconstruction of the decoded block for a combination of sign values to be predicted, based on reusing the results obtained for combinations of values already evaluated. More specifically, the invention consists in calculating a decoded pixel vector of the current block for a current assumption by combining at least one decoded pixel vector already calculated for at least one preceding assumption and at least one pixel pattern vector previously calculated.
- In relation to
FIG. 1 , an original video consisting of a sequence of M images I1, I2, . . . Im, with M a non-zero integer, is considered. The images are encoded and decoded in a predefined order known from the encoder and the decoder, for example in the time order I1 then I2, . . . , then IM (this order may differ according to the embodiment). - When encoding an image, it is subdivided into blocks C of maximum size which can in turn be subdivided into smaller blocks.
- In relation to
FIG. 2 , the steps of a method of coding an image Im according to the invention are described. During a step E0, a first block is selected. - In E1, the current block C undergoes a sequence of operations E1 (applying prediction steps, calculating a residue, quantising, entropy coding) which will now be detailed in relation with
FIG. 3 . - For example, the first block to be processed is selected as the current block C. In the preferred embodiment, it is the first block (in lexicographic order). It has N×N pixels, with N a non-zero integer, but of course, the invention would apply similarly to a rectangular block.
- It is assumed that there are L possible cuts into block numbered from 1 to L, and that the cut used on block C is the
cut number 1. For example, there may be 4 possible cuts, in blocks of size 4×4, 8×8, 16×16, and 32×32. - On the other hand, the decoded current picture (which, in a video encoder, is built into the encoder so as to allow a prediction of subsequent blocks of the video) will be noted ID.
- During a step E11, a prediction Pr of the block C is determined. It is a prediction block constructed by known means, typically by motion compensation (block from a previously decoded reference image), or by intra prediction (block constructed from decoded pixels immediately close to the current block in the image ID). The prediction information related to P is encoded in the compressed file. It is assumed here that there are K possible prediction modes m1, m2, . . . , mK, and that the prediction mode chosen for block C is the mode mkj.
- In E12, an original residue block R is formed by difference between the current block and the predicted block R=C−P.
- In E13, the residue R is transformed by a DCT transform or into a wavelet, known per se (REF JPEG standard DCT, JPEG2000 standard for wavelet transform), into a block called RT.
- In E14, the transformed residue RT is quantized by conventional quantization means, for example scalar or vector, into a quantized residual block RQ. This quantized block RQ contains N×N coefficients. As known in the state of the art, these coefficients are scanned in a predetermined order so as to constitute a one-dimensional vector RQ[i] where the index i varies from 0 to N2−1. The index i is called the frequency of the coefficient RQ [i]. Classically, these coefficients are scanned in increasing order of frequency, for example according to a diagonal scan as described in the document by Sole et al, entitled «Transform Coefficient Coding in HEVC», published in the journal «IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 12, in December 2012.
- In E15, the amplitude information of the RQ coefficients is encoded by entropy coding (Huffman coding or arithmetic coding). By amplitude, here is meant the absolute value of the coefficient. Amplitude coding means are for example described in the HEVC standard (explained in Sole J. et al., «Transform Coefficient Coding in HEVC», IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 12, December 2012).
- Conventionally, a piece of information representative of the fact that the coefficient is non-zero, can be encoded for each coefficient. Then, for each non-zero coefficient, one or more pieces of information relating to the amplitude are encoded.
- All the steps which will now be detailed apply to the sign information. However, the invention is valid for any other descriptive information of the compressed image (syntax element). For example, the invention can be applied to the significance information (a syntax element representative of the zero or non-zero value of a transformed coefficient amplitude), an amplitude information, a mode information of coding (inter or intra for example), etc.
- In relation with
FIG. 2 , a step E2 selects the signs to be predicted, in the current block, according to known prior probabilities, as described in document FR3023112 already cited. The encoder may use a set of probabilities for correct detection of sign, also called scores. In the preferred embodiment, these probabilities are dependent on the size of the current block C, the amplitude of the coefficient associated with the sign, its frequency, and the prediction mode used among the K possible modes. - These probabilities were constructed before encoding and decoding, either by statistical accumulation on a set of signals representative of the signals to be coded, or by mathematical calculation from assumptions on the distribution of the signs of the coefficients.
- For each coefficient RQ[i], a correct prediction probability of the sign of the coefficient RQ[i] can be obtained.
- Advantageously, the signs to be predicted are selected while keeping those whose probability is greater than a predetermined threshold.
- During a step E3, the set of SNP signs of RQ which are not predicted are encoded conventionally. Thus, it is known in the HEVC standard to transmit each sign in the form of a
binary element 0 or 1, with a convention associating one with the plus sign and the other with the minus sign. - During a step E4, the signs to be predicted SP are scheduled. This order can be predefined: for example, in the scanning order of the signs as defined in the HEVC standard. In the preferred embodiment, the signs are ordered by decreasing probability of correct detection. Thus, let us assume that we have at this stage a sequence of M ordered signs to be predicted SP={s0, s1, . . . , sM−1}.
- During a step E5, the M signs of the sequence SP are predicted.
- The purpose is to select the best assumption of combination of the values of M signs of the sequence SP among 2M possible assumptions.
- By way of example, a cost associated with each assumption is evaluated using a cost function FC and the assumption that minimizes the cost function is chosen.
- Thus, for a given assumption, this function FC produces a cost, for example FC ({s1=+,s2=−,s3=−, . . . , sM−1=+})=4240.
- Here, the assumption is {+, −, −, . . . , +} and the resulting cost is 4240. Of course, the evaluation function must make sure to generate a minimal cost when the sign assumption is the most likely.
- Cost functions are described in the state of the art for example in the document by Ponomarenko et al., entitled «Prediction of signs of DCT coefficients in block-based lossy image compression», published in Image Processing: Algorithms and Systems, Proceedings of the SPIE, Volume 6497 in February 2007, whose principle will be described below.
- For example, the cost function measures a distortion along the left and top border of the current block, as shown in
FIG. 4A . The decoded pixels y1 to y8 coming from the blocks previously processed are situated on one side of a border of the current block C, and on the other side, the pixels x1 to x7 of the current block decoded with the assumption of signs of which the cost is to be measured. - To generate a decoded current block corresponding to an assumption of signs, the process is as follows:
-
- each coefficient of the residual block is allocated its real sign (if there is a coefficient whose sign is not predicted), or the sign assumption (if it is a coefficient whose sign is to be predicted).
- using conventional means for dequantising and inverse transform, a block of decoded residual is obtained, in particular the residual samples {r1, r2, . . . , r7} neighbouring the boundary of the block with previously decoded blocks;
- the predicted block P is then added to them to produce a decoded block DV.
- It is not necessary to reconstruct the complete block and it may be enough to reconstruct the pixels x1 to x7 which are situated along the border. In connection with
FIG. 4A , the cost function FC is obtained from the calculation of a difference between the values of x1 pixels x7 located in the blocks previously processed, and y1 to y7 pixels located on the first line and the first column of the current block. - Of course, other cost measurements can be considered. For example, a cost measurement could be based not only on the pixels located at the borders, but also on the interior pixel rows, neighbouring the previous ones, as shown by
FIGS. 4B to 4C . Thus, inFIG. 4B , the difference is calculated between the values of the pixels of previously processed blocks and the pixels y1 to y12 located on the first line, the first column as well as the second line and the second column of the current block. For example, we evaluate the difference between y1 and x1 and y8 and x1 and, similarly, between y2 and x2 and y9 and x2, and so on. - In the example shown in
FIG. 4C , the difference is calculated between the values of the pixels located in the previously processed blocks, and all the pixels of the current block. - Subsequently, the vector consisting of the decoded pixels of the current block used to calculate the cost function for the kth assumption on the signs will be designed VDk.
- The vector VDk can be written VDk=P+VPHk, where P is the prediction vector of the current block, or advantageously a sub-vector restricted for example to the pixels y1 to y7 or still to the pixels y1 to y12 and VPHk is the decoded residual vector corresponding to the kth assumption Hk on the signs or, a sub-vector restricted for example to the pixels y1 to y7 or to the pixels y1 to y12.
- To predict the sequence of M signs of the current block, the costs associated with each assumption of combinations of the sign values are generated using a cost function FC. For these costs, it is necessary to calculate the decoded residual blocks associated with each assumption, hence the vectors of pixels VPH0 to
VPH 2M. - In the state of the art it was necessary to completely recalculate these vectors. However, this process is computationally intensive, in particular because the calculation of the residual vector VPHk requires the application of an inverse transform each time, which requires significant computational resources.
- In order to avoid having to calculate all these inverse transforms, the invention proposes a simplifying solution, two particular embodiments of which will now be described with reference to
FIGS. 5 and 6 . - In connection with
FIG. 5 , where the assumptions are browsed according to a predetermined order. Advantageously, the assumptions are browsed in an order such that the sign of a single coefficient changes between two consecutive assumptions. This order is known to a person having ordinary skills in the art as “Gray code” or “reflected binary code”. - In the particular case where the sequence of signs to be predicted comprises M=3 signs {s0, s1, s2}, the assumptions will be examined in the following order:
- H0={+,+,+}
- H1={+,+,−}
- H2={+,−,+}
- H3={+,−,−}
- H4={−,+,+}
- H5={−,+,−}
- H6={−,−,+}
- H7={−,−,−}
- During a step E511, we first consider two first assumptions H0, H1 such that their combinations differ only in the value of one and the same sign, for example the first one. The result is then H0={+,+,+} and H1={+,+,−}. In E512 the pixel vectors of VPH0 and VPH1 associated with each of these assumptions are calculated by inverse transform of the coefficient vectors obtained from the vector of transformed coefficients VC of the current block by assigning thereto the signs of the assumption considered. The vectors VPH0 and VPH1 obtained are stored in memory.
- During a step E52, a first pattern vector VM2 is calculated by difference between the pixel vectors VPH0 and VPH1 associated with the assumptions H0 and H1. It is understood that according to this first embodiment of the invention, this pattern vector is associated with a value change of the sign s2 and represents in some way an impact of the change of the sign s2 between the combination of signs of H0 and that of H1 in the spatial domain.
- We then consider in E511 a third first assumption H2, according to the predetermined order. Its combination of signs is {+, −, +} and differs from H0 by the value of the sign s1. The vector of corresponding pixels VPH2 is then calculated in E512 in a conventional manner, by applying an inverse transform and stored in memory. During a step E52, a second pattern vector VM1 is calculated by difference between the pixel vectors VPH0 and VPH2 associated with the assumptions H0 and H2. This pattern vector is associated with the sign change of the sign s1 between the assumptions H0 and H2. It is stored in memory.
- During a step E53, a fourth assumption H3 is considered, whose combi-nation is {+, −, −}. It differs from H2 by the value of the sign s2.
- Advantageously, which previously calculated assumption differs only from a sign with the current H3 assumption is identified and its sign is also identified. In the present case, H2 is identified and the sign which differs between the current combination and that of assumption H2 is the sign s2. The memory will be scanned for finding the vector VPH0 and the vector pattern VM2.
- According to the invention, a pixel vector VPH3 associated with this assumption H3 is constructed in E54 by linear combination of the vector of pixels VPH0 and the vector pattern VM2 as follows:
-
VPH3=VPH0+VM2 - The application of an inverse transform is thus avoided.
- According to a variant, the vector VPH0 and the pattern vector VM2 to be combined are obtained in a predefined table and stored in memory, so as to avoid the previous step of identification.
- In E55, any assumption to be processed is investigated. If yes, for the following assumptions, the process is as follows:
- Let us consider the assumption H4 corresponding to the combination {−, +, +}.
- Its pixel vector VPH4 is calculated by applying an inverse transform and stored in memory.
- A pattern vector VM0 is calculated by vector difference between VPH4 and VPH0. It is associated with a sign change of the sign s0. It is stored in memory for later use.
- For the assumption H5 corresponding to the combination {−, +, −}, its pixel vector VPH5 is calculated by linear combination of VPH4 and of the pattern vector VM2.
- The assumption H6 corresponding to the combination {−, −, +}. Its pixel vector VPH6 is obtained for example by linear combination of VPH4 and the pattern vector VM1. VPH2 and VM0 could also be combined.
- The assumption H7 corresponding to the combination {−, −, −}. Its pixel vector VPH7 is obtained for example by linear combination of VPH6 and the pattern vector VM2.
- Note that this embodiment implements 4 transforms for 3 signs to be predicted. More generally, according to this approach, if M signs are to be predicted, the prior art requires the calculation of 2M inverse transforms, whereas the proposed method requires only M+1 transformed. The invention therefore allows a substantial reduction in complexity. Furthermore, when a vector of coefficients is decoded by adding pattern vectors, the value is likely to be slightly different from that obtained classically by inverse transform. However, the error is in general negligible and has no impact on the compression efficiency of the sign prediction method.
- In relation to
FIG. 6 , a second embodiment of the invention will now be described. - A sequence of M=3 signs to be predicted SP={s0, s1, S2} is again considered.
- During a previous step E5′11, the vector of transformed coefficients VC of the current residual block is considered and M=3 pattern vectors of coefficients from the vector of transformed coefficients VC of the current block are constructed, as follows:
-
- a pattern vector of coefficients VMCj associated with the sign sj, with j an integer between 0 and 2, has the same dimension as the vector of coefficients VC and it is constructed while keeping the amplitude of the coefficient of the vector VC corresponding to the position of the sign sj, all the other coefficients being set to zero.
- Regarding the sign of the non-zero coefficient, it can be chosen arbitrarily. For simplicity it is allocated the value +. In this case, the coefficient of the pattern vector corresponding to the position of the sign sj takes the absolute value of the co-located coefficient in the vector VC of the current block.
- For example, the vectors of coefficients VC are assumed to be {+4, 0, −2, −3, 0, +1, 0, 0, −1, 0, 0, . . . , 0} and the three signs to be predicted are associated with the first, third and fourth coefficients. It is therefore {+, −, −}.
- The three following coefficients pattern vectors VMC0, VMC1, VMC2 are obtained in such a case:
-
- VMC0={+4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, . . . , 0}
- VMC1={0, 0, +2, 0, 0, 0, 0, 0, 0, 0, 0, . . . , 0}
- VMC2={0, 0, 0, +3, 0, 0, 0, 0, 0, 0, 0, . . . , 0}
- In E5′11, these patterns vectors of coefficients are transformed into in pixel pattern vectors VM′0, VM′1, VM′2 conventionally, by applying a conventional inverse transform. They are stored in memory.
- Since the coefficient pattern vectors comprise a single non-zero coefficient, the application of the inverse transform generates a very moderate calculation cost. Indeed, for a current block of size N×N pixels of which all the coefficients are likely to be non-zero, for a normal inverse transform N×N projections on basic vectors of dimension N×N are necessary. We need N4 multiplications and N4 additions. However, the fact that a priori only one pixel is not zero, there remains more than one projection to be carried out, i.e. or N2 multiplications and N2 additions.
- In E5′21, a first assumption to be processed is selected, according to a predetermined scanning order of the 2M assumptions, for example the order of the Gray code.
- It is in this case the assumption H0 equal to {+, +, +}. In E5′22, the vector of associated pixels VP0 is constructed by inverse transformation of the coefficient vector obtained from the vector of transformed coefficients VC of the current block by assigning thereto the signs of the assumption H0. The vector VPH0 obtained is stored in memory.
- The following assumptions are now processed by iterating the succession of following steps, starting from i=1:
- In E5′31, an assumption Hi is considered, for example H1.
- H1 is associated with the combination {+, +, −}.
- In E5′32, which previously calculated assumption differs only from a sign with the current assumption is identified and its sign is also identified. In this case, the previous assumption identified is H0 and the sign that differs between the current combination and that of assumption H0 is the sign s2.
- In E5′33, the vector of pixels VPH0 associated with the assumption H0 is read in memory and in E5′34 the vector pattern VM′2 associated with the coefficient of the vector VC bearing the sign s2, is read in memory.
- It will be noted that, according to a variation, the pixel vector VPH0 associated with the assumption H0 and the pattern vector VM′2 are obtained by reading from a predefined table of a record associated with the current assumption H1.
- In E5′4, the pixel vector VPH1 associated with the assumption H0 is constructed by linear combination of the vector VP0 and the vector pattern VM′2 as follows:
-
VPH1=VPH0−2×VM′2 - In E5′5, any assumptions to be processed are investigated. If yes, i is incremented and the method resumes at step E5′31.
- For assumption H2 associated with the combination {+,−, +}, which previously calculated assumption differs only from a sign with the current assumption is identified and its sign is also identified. In the present case, the assumption is H0 and the sign that differs between the current combination and that of the assumption H0 is the sign s1. VP2 is calculated by linear combination of the vector VPH0 and of the pattern vector VM′1 as follows:
-
VPH2=VPH0−2×VM′1 - VPH2 is stored in memory.
- For the assumption H3 associated with the combination {+,−,−}, which previously calculated assumption differs only from a sign with the current assumption is identified and its sign is also identified. In this case, the assumption is H2 and the sign that differs between the current combination and that of assumption H2 is the sign s2.
- VPH3 is therefore calculated for example by linear combination of VPH2 and VM′2 as follows:
-
VPH3=VPH2−2×VM′2 - VPH3 is stored in memory.
- We note that we could also combine VPH1 and VM′1.
- For the assumption H4 associated with the combination {−, +, +}, VPH4 is calculated for example by linear combination of VPH0 and VM′0 in the following way:
-
VPH4=VPH0−2×VM′0 - VPH4 is stored in memory.
- For the assumption H5 associated with the combination {−, +, −}, VPH5 is calculated for example by linear combination of VPH4 and VM′2 in the following way:
-
VPH5=VPH4−2×VM′2 - VPH5 is stored in memory.
- For the assumption H6 associated with the combination {−, −, +}, VPH6 is calculated for example by linear combination of VPH2 and VM′0 in the following way:
-
VPH6=VPH2−2×VM′0 - VPH6 is stored in memory.
- For the assumption H7 associated with the combination {−, −, −}, VPH7 is calculated for example by linear combination of VPH3 and VM′0 in the following way:
-
VPH7=VPH3−2×VM′0 - VPH7 is stored in memory.
- For simplicity, it has been chosen in this example to favour the combinations of a vector pixel with a vector pattern because they generate fewer operations, but it should be noted that a vector with two pattern vectors could also be combined when the identified assumption presents two signs distinct from that of the current assumption.
- Note that this embodiment also implements 4 transforms for 3 signs to be predicted. In addition, the transforms implemented to calculate the M pattern vectors VM′j are not very complex, since the coefficient pattern vectors VMCj only include a non-zero coefficient. This second embodiment is therefore even more economical in operations than the previous one.
- The vectors or sub-vectors of pixels VPHi of the current block are now available for each of the assumptions of sign combinations.
- During a step E56, the costs associated with each of the assumptions are calculated from the vectors obtained and from a cost function FC.
- During a next step E57, the purpose is to decide on the predicted sequence from the costs that have just been calculated. There are different approaches to exploit the costs associated with each of the assumptions. An exemplary method will now be detailed.
- The following substeps are iterated:
- First Iteration (Coding of the First Sign)
- Using the cost function FC, the costs corresponding to each possible assumption are generated for all the signs to be predicted. Thus, let us assume that the number of signs to be predicted is M=3, and that the sequence of signs to be predicted is SP={s0, s1, s2}. We will generate the following costs:
- C0=FC({+,+,+})
- C1=FC({+,+,−})
- C2=FC({+,−,+})
- C3=FC({+,−,−})
- C4=FC({−,+,+})
- C4=FC({−,+,−})
- C6=FC({−,−,+})
- C7=FC({−,−,−})
- We begin by identifying the assumption that obtained the minimal cost. Let us assume that it is H3. Then, the first sign in the predefined order s0 is predicted by the value it has in the combination of H3: it is a +. Then, the predicted value is compared to the actual value of s0 in the actual pixel vector of the current block.
- This comparison makes it possible to define the value of a prediction indicator IP intended to be coded and then transmitted in the bitstream.
- As described in document FR3023112, this indicator indicates whether the predicted sign s0 is equal to or different from the real sign (for example, respectively 0 or 1). If we take the previous example {+4, 0, −2, −3, 0, +1, 0, 0, −1, 0, 0, . . . , 0}, it is a +. There is then coding of an indicator 0. This value IPs0 is stored in memory.
- During a second iteration (coding of the second sign s1), the costs for which s1 does not have its real value are eliminated. Here, the real value of s0 is +, so we keep the following costs:
- C1=FC({+,+,+})
- C2=FC({+,+,−})
- C3=FC({+,−,+})
- C4=FC({+,−,−})
- We thus have the 4 costs corresponding to the 4 possible assumptions on the signs that remain to be predicted. We will identify the minimum cost: let us assume that it is C3. Then, the second sign in the predefined order s1 is predicted by the value it has in the combination of H3: it is −. Then, the predicted value is compared to the actual value of s1. In our example, the actual value is −. There is then coding of an indicator IP equal to 0. This value IPs1 is stored in memory.
- During a third iteration (coding of the last sign s2), the costs for which s2 does not have its real value are eliminated. Here, the real value of s2 is −, so we keep the following costs:
- C3=FC({+,−,+})
- C4=FC({±,−,−})
- We thus have the 2 costs corresponding to the 2 possible assumptions on the signs that remain to be predicted. We will identify the minimum cost: let us assume that it is C4. Then, the last sign in the predefined order s2 is predicted by the value it has in the combination of H4: it is −. Then, the predicted value is compared to the actual value of s3. In our example, it is is −. The indicator IP is therefore set to the value IPs3=0 for the sign s3.
- We have now completed the prediction step E5 and we have a prediction of the signs of the current residual block.
- In relation with
FIG. 2 , the coding of the current block is continued. - In E6, the values of indicators IP decided during the preceding step of sign prediction are coded. Since only the signs that are associated with a representative score of a sufficient reliability level are predicted, the prediction indicator more often takes the
value 1 than the value 0. This is exploited by the entropy coding for reducing the size of the compressed signal. - Advantageously, the entropy coding takes into account the score Sj associated with the predicted sign for coding the indicator IP. For example, in the embodiment of the invention in which the score has a value between 0 (low reliability of the prediction) and 10 (high reliability of the prediction), the entropic coding of the indicators is parametered taking into account the score, so as to exploit the more or less uniform distribution of the indicators. For example, CABAC-type entropic coding, known from the HEVC standard, is used by initializing the probabilities used in CABAC based on the predetermined scores.
- In E7, the decoded block is reconstructed by applying to the quantized residual RQ the steps of dequantisation and inverse transform (known per se). A decoded residual block RD is obtained. RD is added the predictor block P to obtain the decoded block BD.
- During this step, the decoded block BD is also added to the reconstructed image ID. This makes it possible to have in the encoder a decoded version of the current image. This decoded version is used in particular during the step of constructing a prediction of the signs selected to be predicted.
- In E8, the values encoded in the bitstream TB are inserted.
- During this step E9, whether the current block is the last block to be processed will be tested, given the browsing order defined previously. If yes, the coding unit has finished processing. If no, the next step is the step of selecting the next block.
- During this step E0, the next block to be processed by the encoding unit is selected, according to the sequence defined previously. This block becomes the current block to be processed, and the next step is the step of testing E1 the current block.
- In relation to
FIG. 7 , the steps of a method of decoding a coded digital image according to an embodiment of the invention are described. It is assumed that the bitstream TB has been received by a decoding device implementing the decoding method according to the invention. In a variation, the decoding device obtains a compressed file FC. - In D0, the first block to be processed is initially selected as the current block C′. For example, this is the first block (in lexicographic order). This block comprises for example N×N pixels. As already mentioned for coding, the invention would also apply to a rectangular block.
- In D1, the current block C′ is processed by implementing the decoding scheme corresponding to the coding scheme used by the encoder, for example as specified in the HEVC standard. In particular, during this step, it identifies a set of elements ED of data description to be decoded for the current block C′.
- In addition, a prediction P′ of the block to be decoded C′ is produced. The prediction information related to P′ is read in the bitstream or in the compressed file and then decoded. Thus, the prediction mode information mj is therefore decoded.
- The amplitude information of the residue to be decoded RQ′ is also decoded into the bitstream or the compressed file and then decoded. We now know the amplitudes of RQ′[i], but not yet the signs.
- In D2, the decoding method according to the invention implements the step of selecting the signs to be predicted from the identified description elements. This step has already been described in detail for the coding method in relation to
FIG. 2 . The signs to be predicted are advantageously selected according to predetermined scores. A subset SP is obtained. - During a step D3, the method reads in the bitstream TB the coded data relating to the signs of the coefficients of the unpredicted current block and decodes them with means adapted to those used during coding (typically binary decoding). In particular, entropy decoding uses the probability pi obtained previously.
- In D4, the elements of the set SP obtained in an initial sequence SSP are ordered, for example by decreasing scores, as already described for the coding method according to the invention.
- In D5, the sign values of the SSP sequence are predicted. To do so, the best combination in the sense of a cost function FC is selected among the combinations of possible values of the current sequence SP. As already mentioned for the coding method, there are several cost functions. In the following, the same as that of the coding method is used, for example a measurement of distortion along a boundary with previously processed blocks, and the combination of values which minimises this measurement is selected.
- To obtain the costs associated with 2M combinations of signs in each case, it is necessary to calculate pixel vectors VPHi corresponding to the values of the values of the current residual block for the combination of signs of the assumption Hi.
- In the state of the art it was necessary to completely recalculate these vectors. This process is computationally intensive, in particular because the calculation of the residual vector VPHi requires the application of an inverse transform each time, which requires significant computational resources.
- To avoid having to calculate all of these inverse transform, the invention described in connection with the
FIGS. 4, 5 is applied again, for example by implementing one of the embodiments already described in relation withFIGS. 5 and 6 . - In D6, the values of the prediction indicators IP read in the bitstream or the compressed file for the signs of the sequence SP are decoded.
- In D7, the decoded signs are reconstructed from the indicator values IP decoded and the predicted values.
- The next step D8 is for reconstructing the current block C′ from the description elements decoded, EP predicted (D5) and ENP not predicted (D3), the amplitude information of the residual block coefficients RQ′ and the prediction P′ obtained in D1.
- To do this, we first dequantize the RQ′ block to obtain a dequantized block. This is done by means adapted to the quantization used during the coding (scalar dequantization, vector dequantization . . . ), known to those skilled in the art.
- The dequantized residue is then subjected to a reverse transform from that used in the coding. The decoded residue is then obtained.
- Lastly, the decoded block BD′ is reconstructed by adding the decoded residue to the prediction P′.
- This block is integrated with the image being decoded.
- During a step D9, whether the current block is the last block to be processed is tested, given the scanning order of the blocs, as defined previously. If yes, the decoding is complete. If not, the next step is the step D0 for selecting the next block and the steps of the decoding method are repeated.
- According to one embodiment of the invention which will now be detailed with reference to
FIG. 8 , the prediction step D5 and the step D6 of decoding the signs of the sequence SSP are combined in a single step D′5: - In D′5 1, a current sequence Seqc that is initialised to the initial sequence SP is considered.
- The following steps (D′5 2 to D5′6) form an iterative loop that will be repeated several times, depending on the M number of signs to be predicted, selected from Seqc.
- In D′5 2, the best combination within the meaning of a cost function FC is selected, the same as that used by the coding method which produced the bitstream to be decoded, from the combinations of possible values for the current sequence SP.
- During a first iteration, the first sign of the sequence Seqc is decoded. With the aid of the cost function, the costs corresponding to each assumption or possible combination for the set of signs to be predicted are generated, using the solution of the invention, which has just been described in connection with
FIG. 7 . - Thus, let us assume that the number of signs to be predicted is equal to M=3, and Seqc={s0, s1, s2}. We will generate the following costs:
- C0=FC({+,+,+})
- C1=FC({+,+,−})
- C2=FC({+,−,+})
- C3=FC({+,−,−})
- C4=FC({−,+,+})
- C5=FC({−,+,−})
- C6=FC({−,−,+})
- C7=FC({−,−,−})
- We thus have the 8 costs corresponding to the 8 possible assumptions on the signs that remain to be predicted. The minimum cost is assumed to be C2.
- In D′5 3, the value of the first sign s0 is predicted by its value in the combination corresponding to the assumption H2. This is +.
- In D′5 4, the prediction indicator IP is decoded for that first sign s0, from coded data extracted from the bitstream or compressed file. This indicator indicates whether the predicted sign has been correctly predicted or not. For example, let us assume that the decoded value is 0 and is associated with a correct prediction.
- In D′5 5, it is deduced that the decoded value of the sign s0 is +.
- In D′5 6, we test whether the sign s0 is the last of the current sequence. If this is the case, the processing ends. Otherwise, the current sequence is updated in D5 7 by deleting the element s0 which has just been decoded.
- During a second iteration, the second element s1, which has then become the first one in the current sequence, is processed.
- In D′5 2, the combinations Cbk, where s0 is the decoded value, the others being discarded, are considered.
- In the example considered, the actual value of s0 is −, so the following costs are compared:
- C0=FC({+,+,+})
- C1=FC({+,+,−})
- C2=FC({+,−,+})
- C3=FC({+,−,−})
- These 4 costs correspond to the 4 possible assumptions on the signs that remain to be predicted, knowing the decoded value of the one that has already been processed.
- C1 is identified as the minimum cost.
- In D′5 3, s1 is predicted by its value in the combination of assumption H1, that is to say +.
- In D′5 4, the indicator IP corresponding to s1 is decoded from the coded data extracted from the bitstream or compressed file. An indicator is decoded, which indicates whether the predicted sign is equal to or different from the actual sign. In our example, let us assume that the decoded value of IP is 0, which means that the prediction of this sign is correct.
- In D′5 5, it follows that s1 is −.
- In D′5 6, it is tested whether s is the last element. Since this is not the case, the current sequence in D′5 7 is updated, by removing s1. The new sequence Seqc is reduced to the element s2.
- During a third and last iteration, the last sign s2 is decoded.
- In D′5 2, the combinations of values of the original sequence Seqi are considered, for which the elements of description already processed s0 and s1 take on their decoded value.
- The following costs are therefore compared:
- C2=FC({+,−,+})
- C3=FC({+,−,−})
- C3 is identified as the minimum cost.
- Then, the last sign s2 in the predefined order is predicted, in D5 3, by its value in the combination of assumption H3: it is −.
- Then, an indicator IP associated with the sign s2 is decoded at D5 4. It indicates whether the sign s2 has been correctly predicted or not. In our example, let us assume that the decoded value of IP is 0, which corresponds to a correct prediction.
- It is deduced in D′55 that the decoded value of s2 is the sign +.
- In D′5 6, we find that s2 is the last element to be processed.
- As for the encoding, it will be noted that the invention is not limited to the embodiment that has just been presented. Other implementation choices can be made, such as the one to keep the sequence at its original length, to update the index of the current element, initialised to the first description element ED0 and to terminate the iterations once the last element EDM-1 has been processed.
- According to this embodiment of the invention, since all the elements of the initial sequence selected according to the predetermined scores are predicted, it is therefore known as of the outcome of step D2 how many prediction indicators IP are to be extracted from the bitstream or compressed file. It will be understood that this makes it possible to implement the invention in such a way that it decorrelates the reading and parsing operations of the coded data contained in the bitstream or compressed file of the current block processing operations in accordance with the coding/decoding scheme implemented. For example, the decoding could be organised by using a specific component for the parsing/reading of the coded data in the bitstream and another for the reconstruction operations of the decoded blocks. An advantage of this parsing independence is to allow parallelisation of the decoding operations.
- The encoding and decoding methods described above can be integrated into standard video encoders/decoders such as HEVC/H.265, AVC/H.264 or any type of proprietary video encoders/decoders. The coding and decoding methods according to the invention also apply to all types of coders/decoders of still images and more generally of signals using a predictive coding process.
- The coding and decoding methods have been previously described in the case of the prediction of the signs of the coefficients of a residue block of the image to be encoded/decoded. These methods are easily applicable to the case of the prediction of another block description element for example.
-
FIG. 9 shows the simplified structure of acoding device 100 adapted to implement the coding method according to any of the particular embodiments of the invention. Thecoding device 100 is adapted to encode at least one image in the form of a coded data stream, said image being divided into blocks. - The
coding device 100 is notably configured for: -
- Processing the current block for providing a vector of transformed coefficients;
- Selecting a sequence of N signs of coefficients to be predicted in the coefficient vector, with N being an integer greater than or equal to 2;
- Predicting values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- For a sign of the sequence, coding an indicator representative of a difference between its actual value and its predicted value;
According to the invention the prediction comprises: - obtaining at least one pattern vector of dimension equal to that of a vector of coefficients, from the vector of transformed coefficients;
- for at least one first assumption, obtaining a virtual pixel vector by transforming a coefficient vector associated with the first assumption; and
- for at least one second assumption, so-called current assumption:
- obtaining a pixel vector associated with the current assumption, distinct from the first assumption, by linear combination of at least one pixel vector associated with the first assumption and said at least one pattern vector;
- calculating the cost function for the current assumption from the pixel vector obtained.
Thedevice 100 is further configured to implement the various embodiments of the invention which have just been described in relation toFIGS. 2, 5 and 6 .
- According to a particular embodiment of the invention, the steps of the coding method are implemented by computer program instructions. To do so, the
coding device 100 has the conventional architecture of a computer and notably comprises a memory MEM1, a processing unit UT1, equipped for example with a microprocessor μ1, and driven by the computer program Pg1 stored in the memory MEM1. The computer program Pg1 includes instructions for implementing the steps of the coding method as described above, when the program is executed by the processor PROC. - At initialisation, the code instructions of the computer program Pg1 are for example loaded into a RAM before being executed by the processor PROC. The processor μ1 of the processing unit UT1 implements in particular the steps of the coding method described above, according to the instructions of the computer program Pg1.
According to another particular embodiment of the invention, the coding method is implemented by functional modules or units. For this purpose, thecoding device 100 furthermore comprises the following modules: -
- Processing (PROC) the current block for providing a vector of transformed coefficients;
- Selecting (SEL SP) a sequence of N signs of coefficients to be predicted in the coefficient vector, with N being an integer greater than or equal to 2;
- Predicting (PRED) values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- For a sign of the sequence, coding (COD IP) an indicator representative of a difference between its actual value and its predicted value.
According to the invention, the prediction module comprises the following units: - for at least one second assumption, so-called current assumption:
- Obtaining a pixel vector associated with the current assumption, by linear combination of at least one pixel vector associated with at least one first assumption and obtained by transforming a vector of coefficients comprising the signs of the combination of the first assumption and at least one pattern vector obtained from the vector of transformed coefficients; and
- Calculating the cost function for the current assumption from the pixel vector obtained.
The processing unit UT1 cooperates with the various functional modules described above and the memory MEM1 in order to implement the steps of the coding method.
- According to one embodiment of the invention, the
device 100 further comprises a unit M1 for storing the coding contexts of the coefficients, predetermined scores associated with each of these contexts, the pixel vectors associated with the 2M assumptions for the sequence of signs to predict, their related costs, as well as the pattern vectors from which the pixel vectors are derived. - These units are driven by the processor μ1 of the processing unit 110.
- Advantageously, such a
device 100 can be integrated in a user terminal equipment TU, such as an encoder, a personal computer, a tablet, a digital camera, a smart phone, etc. Thedevice 100 is then arranged to cooperate at least with the next module of the terminal TU: -
- a data transmission/reception module E/R, through which the bitstream TB or the compressed file FC is transmitted over a telecommunications network, for example a wired network or a radio network.
- The various functional modules described above can be in hardware and/or software form. Under a software form, such a functional module may include a processor, a memory and program code instructions for implementing the function corresponding to the module when the code instructions are executed by a processor. In a material form, such a functional module can be implemented by any type of suitable coding circuits, such as for example and without limitation microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), a logic unit wiring.
-
FIG. 10 shows the simplified structure of adecoding device 200 adapted to implement the decoding method according to any of the particular embodiments of the invention which have just been described in connection toFIGS. 7, 8, 5 and 6 . Thedecoding device 200 is adapted to decode at least one image in the form of a coded data stream representative of at least one image, said image being divided into blocks. Thedecoding device 200 is notably configured for: -
- Decoding (DEC RQ) the amplitudes of the coefficients of the vector of transformed coefficients, extracted from the bitstream (TB);
- Selecting (SEL′ SP) a sequence of N signs of coefficients to be predicted in the coefficient vector, with N being an integer greater than or equal to 2;
- Predicting (PRED′) values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- Decoding (DEC IP) values of a prediction indicator of the signs selected from coded data extracted from the bitstream, the prediction indicator of a sign being indicative of a difference between its actual value and its predicted value;
- Calculating (CALC) the decoded values of the selected signs from the decoded prediction indicator values;
- Reconstructing (RECONST) coefficients of the block from decoded amplitudes and decoded signs;
According to the invention, the prediction moreover includes: - for at least one second assumption, so-called current assumption:
- Obtaining a pixel vector associated with the current assumption, by linear combination of at least one pixel vector associated with at least one first assumption and obtained by transforming a vector of coefficients comprising the signs of the combination of the first assumption and at least one pattern vector obtained from the vector of transformed coefficients; and
- Calculating the cost function for the current assumption from the pixel vector obtained.
- According to a particular embodiment of the invention, the
decoding device 200 has the conventional architecture of a computer and notably comprises a memory MEM2, a processing unit UT2, equipped for example with a microprocessor μ2, and driven by the computer program Pg2 stored in the memory MEM2. The computer program Pg1 includes instructions for implementing the steps of the decoding method as described above, when the program is executed by the processor μ2. - At initialisation, the code instructions of the computer program Pg2 are for example loaded into a RAM before being executed by the processor μ2. The processor μ2 of the processing unit UT0 implements in particular the steps of the decoding method described above, according to the instructions of the computer program Pg2.
According to another particular embodiment of the invention, the decoding method is implemented by functional modules. For this purpose, thedecoding device 200 furthermore comprises the following modules: -
- Decoding (DEC C) the amplitudes of the coefficients of the vector of transformed coefficients of a current block, extracted from the bitstream (TB);
- Selecting (SEL′ SP) a sequence SP of N signs of coefficients to be predicted in the coefficient vector, with N being an integer greater than or equal to 2;
- Predicting (PRED′) values of the signs of the sequence from a plurality of assumptions of combinations and a cost function;
- Decoding (DEC IP) values of a prediction indicator of the signs selected from coded data extracted from the bitstream, the prediction indicator of a sign being indicative of a difference between its actual value and its predicted value;
- Calculating (CALC) the decoded values of the selected signs from the decoded prediction indicator values;
- Reconstructing (RECONST) the coefficients of the block from decoded amplitudes and decoded signs;
According to the invention, the prediction module moreover includes:
for at least one second assumption, so-called current assumption: - Obtaining a pixel vector associated with the current assumption, by linear combination of at least one pixel vector associated with at least one first assumption and obtained by transforming a vector of coefficients comprising the signs of the combination of the first assumption and at least one pattern vector obtained from the vector of transformed coefficients; and
- Calculating the cost function for the current assumption from the pixel vector obtained.
The processing unit UT2 cooperates with the various functional modules described above and the memory MEM2 in order to implement the steps of the decoding method.
- The
device 200 furthermore comprises a unit M2 for storing the predetermined scores or correct detection probability used to select the signs to be predicted, pixel vectors corresponding to the 2M sign combination assumptions to be predicted, their related costs as well as the pattern vectors from which the pixel vectors are derived. - These units are driven by the processor μ2 of the processing unit UT2.
- The various functional modules described above can be in hardware and/or software form. Under a software form, such a functional module may include a processor, a memory and program code instructions for implementing the function corresponding to the module when the code instructions are executed by a processor. In a material form, such a functional module can be implemented by any type of suitable coding circuits, such as for example and without limitation microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), a logic unit wiring.
- Advantageously, such a
device 200 can be integrated into a user terminal TU, for example a decoder, a Set-Top-Box, a digital television, a computer, a tablet, a smartphone, etc. Thedevice 200 is then arranged to cooperate at least with the next module of the terminal UT: -
- a data transmission/reception module E/R, through which the bitstream TB or the compressed file FC is transmitted in a telecommunications network.
- a module DISP for displaying decoded digital images.
- The invention which has just been presented can find many applications, in particular in the context of video signal compression, audio (speech, sound), still images, images acquired by an medical imaging module. It applies for example to two-dimensional (2D), three-dimensional (3D) contents including a depth map, or multispectral images (whose colour intensities are different from the three red green blue bands) or finally to full images.
- An exemplary embodiment of invention particularly aims to overcome the disadvantages of the prior art.
- An exemplary embodiment of the invention proposes a solution that considerably simplifies the process of predicting signs without impacting compression performance.
- It goes without saying that the embodiments which have been described above have been given for purely indicative and non-limiting reasons, and that many modifications can easily be made by those skilled in the art without departing from the scope of the invention.
Claims (18)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1659493A FR3057130B1 (en) | 2016-10-03 | 2016-10-03 | METHOD OF ENCODING AN IMAGE, DECODING METHOD, DEVICES, TERMINAL EQUIPMENT AND COMPUTER PROGRAMS |
FR1659493 | 2016-10-03 | ||
PCT/FR2017/052422 WO2018065687A1 (en) | 2016-10-03 | 2017-09-12 | Method for encoding an image and associated decoding method, devices, terminal equipment and computer programs |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200195969A1 true US20200195969A1 (en) | 2020-06-18 |
US10805638B2 US10805638B2 (en) | 2020-10-13 |
Family
ID=58455101
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/338,878 Active US10805638B2 (en) | 2016-10-03 | 2017-09-12 | Method for coding a digital image, decoding method, devices, terminal equipment and related computer programs |
Country Status (4)
Country | Link |
---|---|
US (1) | US10805638B2 (en) |
EP (1) | EP3520416B1 (en) |
FR (1) | FR3057130B1 (en) |
WO (1) | WO2018065687A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210014509A1 (en) * | 2018-03-07 | 2021-01-14 | Huawei Technologies Co., Ltd. | Signaling residual signs predicted in transform domain |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6940557B2 (en) * | 2001-02-08 | 2005-09-06 | Micronas Semiconductors, Inc. | Adaptive interlace-to-progressive scan conversion algorithm |
WO2007078800A1 (en) * | 2005-12-15 | 2007-07-12 | Analog Devices, Inc. | Motion estimation using prediction guided decimated search |
KR20070069615A (en) * | 2005-12-28 | 2007-07-03 | 삼성전자주식회사 | Motion estimator and motion estimating method |
JP5533886B2 (en) * | 2010-02-12 | 2014-06-25 | 富士通株式会社 | Image encoding apparatus and image decoding apparatus |
FR3023112A1 (en) | 2014-06-27 | 2016-01-01 | Bcom | METHOD FOR ENCODING A DIGITAL IMAGE, DECODING METHOD, DEVICES AND COMPUTER PROGRAMS |
-
2016
- 2016-10-03 FR FR1659493A patent/FR3057130B1/en not_active Expired - Fee Related
-
2017
- 2017-09-12 EP EP17783910.7A patent/EP3520416B1/en active Active
- 2017-09-12 US US16/338,878 patent/US10805638B2/en active Active
- 2017-09-12 WO PCT/FR2017/052422 patent/WO2018065687A1/en active Application Filing
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210014509A1 (en) * | 2018-03-07 | 2021-01-14 | Huawei Technologies Co., Ltd. | Signaling residual signs predicted in transform domain |
US11856216B2 (en) * | 2018-03-07 | 2023-12-26 | Huawei Technologies Co., Ltd. | Signaling residual signs predicted in transform domain |
Also Published As
Publication number | Publication date |
---|---|
EP3520416A1 (en) | 2019-08-07 |
FR3057130A1 (en) | 2018-04-06 |
EP3520416B1 (en) | 2020-06-24 |
US10805638B2 (en) | 2020-10-13 |
FR3057130B1 (en) | 2019-08-02 |
WO2018065687A1 (en) | 2018-04-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
RU2608682C2 (en) | Image encoding and decoding method, device for encoding and decoding and corresponding software | |
US11350106B2 (en) | Method for encoding and decoding images, device for encoding and decoding images and corresponding computer programs | |
US10575017B2 (en) | Data encoding and decoding based on transform-skip mode detection | |
US11825129B2 (en) | Segmentation method and method for signaling segmentation of a coding tree unit | |
CN112335241A (en) | Method, apparatus and system for encoding and decoding a transform block of video samples | |
US10965938B2 (en) | Method and apparatus for encoding a video | |
JP2022043029A (en) | Method for encoding digital image, decoding method, associated device, user terminal and computer program | |
KR20160034914A (en) | Scanning orders for non-transform coding | |
US11323715B2 (en) | Method for decoding an image, encoding method, devices, terminal equipment and associated computer programs | |
US10805638B2 (en) | Method for coding a digital image, decoding method, devices, terminal equipment and related computer programs | |
US20170353722A1 (en) | A method for encoding a digital image, decoding method, devices, and associated computer programs | |
KR20190068588A (en) | A method for coding and decoding image parameters, an apparatus for coding and decoding image parameters, and an equivalent computer program | |
US8532179B2 (en) | Method and device for optimizing the compression of a video stream | |
CN108432254B (en) | Image encoding and decoding method, apparatus and computer storage medium | |
US20240244266A1 (en) | Methods and devices for encoding and decoding a data stream representing at least one image | |
US11863751B2 (en) | Methods and devices for coding and decoding a data stream representative of at least one image | |
CN112292858B (en) | Method and device for encoding and decoding a data stream representing at least one image | |
KR20170120634A (en) | Encoding of images by vector quantization | |
CN112740690B (en) | Method and device for encoding and decoding a data stream representing at least one image | |
RU2812248C2 (en) | Image encoding method based on transformation and device for its implementation | |
CN112740691A (en) | Method and device for encoding and decoding a data stream representing at least one image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: B COM, FRANCE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HENRY, FELIX;CLARE, GORDON;SIGNING DATES FROM 20190903 TO 20190919;REEL/FRAME:051980/0968 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: FONDATION B-COM, FRANCE Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME SHOULD BE: FONDATION B-COM. PREVIOUSLY RECORDED AT REEL: 051980 FRAME: 0968. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNORS:HENRY, FELIX;CLARE, GORDON;SIGNING DATES FROM 20220317 TO 20220324;REEL/FRAME:061463/0711 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |