WO2019114294A1 - 图像编解码方法、装置、***及存储介质 - Google Patents
图像编解码方法、装置、***及存储介质 Download PDFInfo
- Publication number
- WO2019114294A1 WO2019114294A1 PCT/CN2018/099255 CN2018099255W WO2019114294A1 WO 2019114294 A1 WO2019114294 A1 WO 2019114294A1 CN 2018099255 W CN2018099255 W CN 2018099255W WO 2019114294 A1 WO2019114294 A1 WO 2019114294A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image block
- transform
- core group
- encoded
- determining
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
- H04N19/64—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission
- H04N19/645—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission by grouping of coefficients into blocks after the transform
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N19/45—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder performing compensation of the inverse transform mismatch, e.g. Inverse Discrete Cosine Transform [IDCT] mismatch
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/649—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding the transform being applied to non rectangular image segments
Definitions
- the present application relates to image processing technologies, and in particular, to an image encoding and decoding method, apparatus, system, and storage medium.
- Digital video is a video recorded in digital form.
- the digital video consists of a multi-frame digital image.
- the original video contains a large amount of data, which is not suitable for storage and transmission, and requires efficient video compression coding technology to compress the original video. This is also the encoding purpose of the encoding end.
- the encoding process of the encoding end includes: after receiving the original video by the encoding end, dividing the image into a plurality of current image blocks to be encoded for each frame image constituting the original video.
- the reference image block is used to provide the reference pixel required for the current image block to be encoded
- the reference pixel is used to predict the current image block to be encoded
- the block performs intra prediction or inter prediction to obtain a prediction signal of the current image block to be encoded; and subtracts the prediction signal from the original signal of the current image block to be encoded to obtain a residual signal.
- the magnitude of the residual signal is much smaller than the original signal.
- the residual signal is transformed and quantized. After transform quantization, the transform quantized coefficients are obtained, and the quantized coefficients and other indication information in the encoding are encoded by the entropy coding technique to obtain a code stream.
- the transformation is an important part of the video coding process. It converts the information sampled value according to some form of transformation function from the current domain to the transform domain, and then compresses and encodes according to the distribution characteristics of the information in the transform domain.
- the reason for the transformation is that image or video data tends to have large data correlation in the spatial domain, resulting in the existence of a large amount of redundant information, and direct encoding requires a large amount of bits.
- the data correlation is greatly reduced, so that the redundant information of the encoding is reduced, and the amount of data required for encoding is also greatly reduced, so that a higher compression ratio can be obtained.
- a multi-core transform is generally composed into a plurality of transform core groups.
- the current image block to be encoded is transformed by each transform core group, and the encoding end is subjected to a rate distortion cost (Rate Distortion). Cost, RDCost)
- RDO Rate Distortion Optimization
- the application provides an image encoding and decoding method, device, system and storage medium, thereby reducing coding complexity.
- the present application provides an image encoding method, including: determining location information of a target reconstructed image block of an image block to be currently encoded, wherein the target reconstructed image block is used to determine motion information of a current image block to be encoded. And the first transform kernel group is determined according to the location information of the target reconstructed image block; and the residual signal of the current image block to be encoded is transformed according to the first transform kernel group to obtain a transform coefficient.
- the beneficial effects of the present application include: the image encoding method provided by the present application, so that the encoding end does not need to try multiple transform core groups, thereby reducing coding complexity and improving coding efficiency.
- the method before determining the first transform core group according to the location information of the target reconstructed image block, the method further includes: acquiring a correspondence between the location information of the at least one reconstructed image block and the at least one transform core group; and correspondingly, reconstructing the image according to the target Determining the first transform core group, the location information of the block includes: determining the first transform core group according to the correspondence relationship and the location information of the target reconstructed image block.
- the beneficial effects of the present application include: by using the image coding method provided by the present application, the first transform core group can be effectively determined, so that the coding end does not need to try multiple transform core groups, thereby reducing coding complexity.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group includes: a correspondence between a left reconstructed image block of the current image block to be encoded and a transform core group (DST7, DCT2), and a current Corresponding relationship between the reconstructed image block and the transform kernel group (DCT2, DST7) of the image block to be encoded, the corresponding relationship between the upper right reconstructed image block of the current image block to be encoded and the transform kernel group (DCT8, DST7), and the current image to be encoded Corresponding relationship between the reconstructed image block and the transform kernel group (DST7, DCT8) at the lower left of the block and the corresponding relationship between the upper left reconstructed image block and the transform kernel group (DST7, DST7) of the current image block to be encoded.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group further includes: a correspondence between the other target reconstructed image blocks and the transform kernel group (DCT2, DCT2).
- determining the first transform core group according to the correspondence information and the location information of the target reconstructed image block including: determining, according to the correspondence relationship and the location information of the target reconstructed image block, at least two second transform core groups; Each second transform core group of the two second transform core groups encodes the current image block to be encoded, and determines a rate distortion cost corresponding to the encoded current image block to be encoded; and the corresponding minimum rate distortion cost
- the second transform kernel group is used as the first transform kernel group.
- the beneficial effects of the present application include: by using the image coding method provided by the present application, the first transform core group can be effectively determined, so that the coding end does not need to try multiple transform core groups, thereby reducing coding complexity.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group includes: a left reconstructed image block of the current image block to be encoded and two second transform kernel groups (DST7, DST7) Correspondence relationship between (DST7, DCT8), the corresponding relationship between the upper reconstructed image block of the current image block to be encoded and the two second transform kernel groups (DST7, DST7) (DCT8, DST7), and the upper right reconstruction of the current image block to be encoded
- DCT8 DST7 corresponds the image block and the two second transform kernel groups (DST7, DST7)
- DCT8 DST7 the lower left reconstructed image block of the current image block to be encoded
- DCT8 DST7 DCT2 Correspondence relationship between DCT8 and the correspondence between the upper left reconstructed image block of the current image block to be encoded and the two second transform kernel groups (DST7, DST7) (DCT8, DST7).
- the residual signal of the current image block to be encoded is transformed according to the first transform core group, and after the transform coefficient is obtained, the method further includes: performing quantization on the transform coefficient to obtain a quantized transform coefficient of the current image block to be encoded. And entropy encoding the quantized transform coefficient and the auxiliary coding information of the current image block to be encoded to obtain a code stream; wherein the auxiliary coding information includes: an identifier of the inter-frame transform mode and indication information, where the indication information is used to indicate the first transform The index of the core group.
- the beneficial effects of the present application include: for the encoding end, only one bit of indication information needs to be transmitted, so that the decoding end can determine the first transform core group, and in the prior art, the encoding end needs to transmit the corresponding one of each transform core group.
- Identification taking the H.266 standard as an example, indicating that there are 5 transform core groups. In this case, the encoding end needs to transmit 3 bits of identification information, so that the decoding end determines the first transform core group, so the method can be reduced by the method.
- the overhead of the encoding side is an example.
- the present application provides an image decoding method, including: performing entropy decoding on a code stream to obtain a quantized transform coefficient of a current image block to be reconstructed, and performing inverse quantization on the quantized transform coefficient to obtain a current Transforming a transform coefficient of the image block; determining location information of the target reconstructed image block of the current image block to be reconstructed, wherein the target reconstructed image block is a reconstructed image block for determining motion information of the current image block to be reconstructed; reconstructing the image block according to the target The position information determines a first transform kernel group; and inverse transforms the transform coefficients according to the first transform kernel group to reconstruct the current image block to be reconstructed.
- the beneficial effects of the present application include: the image decoding method provided by the present application, so that the decoding end does not need to try multiple transform core groups, thereby reducing decoding complexity and improving decoding efficiency.
- the method before determining the first transform core group according to the location information of the target reconstructed image block, the method further includes: acquiring a correspondence between the location information of the at least one reconstructed image block and the at least one transform core group; and correspondingly, reconstructing the image according to the target Determining the first transform core group, the location information of the block includes: determining the first transform core group according to the correspondence relationship and the location information of the target reconstructed image block.
- the beneficial effects of the present application include that the first transform core group can be effectively determined by the image decoding method provided by the present application, so that the decoding end does not need to try multiple transform core groups, thereby reducing decoding complexity.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group includes: a correspondence between a left reconstructed image block of the current image block to be reconstructed and a transform kernel group (DST7, DCT2), and a current Corresponding relationship between the reconstructed image block and the transform kernel group (DCT2, DST7) of the image block to be reconstructed, the corresponding relationship between the upper right reconstructed image block and the transform kernel group (DCT8, DST7) of the current image block to be reconstructed, and the current image to be reconstructed Corresponding relationship between the reconstructed image block and the transform kernel group (DST7, DCT8) and the corresponding relationship between the upper left reconstructed image block and the transform kernel group (DST7, DST7) of the current image block to be reconstructed.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group further includes: a correspondence between the other target reconstructed image blocks and the transform kernel group (DCT2, DCT2).
- the code stream further carries indication information, where the indication information is used to indicate an index of the first transform core group; and correspondingly, determining the first transform core group according to the correspondence relationship and the location information of the target reconstructed image block, including: Corresponding relationship and location information of the target reconstructed image block, determining at least two second transform core groups; determining, in the at least two second transform core groups, the first transform core group according to an index of the first transform core group.
- the beneficial effects of the present application include that the first transform core group can be effectively determined by the image decoding method provided by the present application, so that the decoding end does not need to try multiple transform core groups, thereby reducing decoding complexity. Further, for the decoding end, the first transform core group is determined without calculating the rate distortion cost corresponding to the two second transform core groups, and the first transform core group is determined according to the indication information. Thereby reducing the decoding complexity, thereby improving the decoding efficiency.
- the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group includes: a left reconstructed image block of the current image block to be reconstructed and two second transform kernel groups (DST7, DST7) Correspondence relationship between (DST7, DCT8), the corresponding relationship between the upper reconstructed image block of the current image block to be reconstructed and the two second transform kernel groups (DST7, DST7) (DCT8, DST7), and the upper right reconstruction of the current image block to be reconstructed
- DCT8 DST7 corresponds the image block and the two second transform kernel groups
- DST7, DCT8 DST7 corresponds the lower left reconstructed image block of the current image block to be reconstructed
- DCT8 DCT2 corresponding relationship between DCT8 and the correspondence between the upper left reconstructed image block of the current image block to be reconstructed and the two second transform kernel groups (DST7, DST7) (DCT8, DST
- the implementation principle and technical effects can refer to the method related to the first aspect and the principle and technical effects of the optional mode of the first aspect, and details are not described herein again.
- an image encoding apparatus including:
- a first determining module configured to determine location information of a target reconstructed image block of the current image block to be encoded, where the target reconstructed image block is a reconstructed image block for determining motion information of the current image block to be encoded.
- the second determining module is configured to determine the first transform core group according to the location information of the target reconstructed image block.
- a transform module configured to transform, according to the first transform core group, a residual signal of the current image block to be encoded, to obtain a transform coefficient.
- the device further includes: an obtaining module, configured to acquire a correspondence between the location information of the at least one reconstructed image block and the at least one transform core group; and correspondingly, the second determining module is specifically configured to reconstruct according to the corresponding relationship and the target The position information of the image block determines the first transform core group.
- the second determining module is specifically configured to: determine, according to the correspondence relationship and the location information of the target reconstructed image block, the at least two second transform core groups; respectively, according to each of the at least two second transform core groups Transforming the kernel group, encoding the current image block to be encoded, and determining a rate distortion cost corresponding to the encoded current image block to be encoded; and using the second transform kernel group corresponding to the minimum rate distortion cost as the first transform core group .
- the device further includes:
- a quantization module configured to quantize the transform coefficients to obtain quantized transform coefficients of the current image block to be encoded.
- An entropy coding module is configured to entropy encode the quantized transform coefficients and the auxiliary coding information of the current image block to be encoded to obtain a code stream.
- the auxiliary coding information includes: an identifier of the inter-frame transformation mode and indication information, where the indication information is used to indicate an index of the first transform core group.
- the image decoding device is described below.
- the implementation principle and technical effects can refer to the method related to the second aspect and the principle and technical effects of the optional mode of the second aspect, and details are not described herein again.
- the application provides an image decoding apparatus, including:
- An entropy decoding module is configured to perform entropy decoding on the code stream to obtain quantized transform coefficients of the current image block to be reconstructed.
- An inverse quantization module is configured to inverse quantize the quantized transform coefficients to obtain transform coefficients of the current image block to be reconstructed.
- a first determining module configured to determine location information of the target reconstructed image block of the current image block to be reconstructed, where the target reconstructed image block is a reconstructed image block used to determine motion information of the current image block to be reconstructed.
- the second determining module is configured to determine the first transform core group according to the location information of the target reconstructed image block.
- an inverse transform module configured to inverse transform the transform coefficients according to the first transform core group to reconstruct the current image block to be reconstructed.
- the device further includes: an obtaining module, configured to acquire a correspondence between the location information of the at least one reconstructed image block and the at least one transform core group; and correspondingly, the second determining module is specifically configured to reconstruct according to the corresponding relationship and the target The position information of the image block determines the first transform core group.
- the code stream further carries indication information, where the indication information is used to indicate an index of the first transform core group.
- the second determining module is specifically configured to: determine at least two according to the corresponding relationship and the location information of the target reconstructed image block. a second transform core group; determining, in the at least two second transform core groups, the first transform core group according to an index of the first transform core group.
- the present application provides an image coding and decoding system, comprising: the image coding apparatus according to any one of the third aspect and the optional aspect of the third aspect, and the optional manners of the fourth aspect and the fourth aspect An image decoding device according to any one of the preceding claims.
- the present application provides a storage medium comprising instructions for causing an image encoding device to perform the method of any of the first aspect and the first aspect, when the instruction is run on the image encoding device.
- the present application provides a storage medium comprising instructions that, when executed on an image decoding device, cause the image decoding device to perform the method of any of the second aspect and the second aspect.
- the present application provides a computer program product, when the computer program product is run on an image encoding device, causing the image encoding device to perform the image encoding method according to any of the first aspect and the optional aspect of the first aspect .
- the present application provides a computer program product, which, when run on an image decoding device, causes the image decoding device to perform the image decoding method according to any of the second aspect and the second aspect .
- the present application provides an image encoding and decoding method, apparatus, system, and storage medium, the method comprising: determining location information of a target reconstructed image block of an image block to be currently encoded, wherein the target reconstructed image block is used to determine an image to be encoded currently. Reconstructing the image block of the motion information of the block; determining the first transform core group according to the position information of the target reconstructed image block; and transforming the residual signal of the current image block to be encoded according to the first transform kernel group to obtain a transform coefficient.
- the encoding end does not need to try multiple transform core groups, thereby reducing coding complexity and improving coding efficiency.
- FIG. 1 is a flowchart of an image encoding method according to an embodiment of the present application
- FIG. 2 is a schematic diagram of a location of a target reconstructed image block according to an embodiment of the present disclosure
- FIG. 3A to FIG. 3E are schematic diagrams showing residual distribution of a current image block to be encoded according to an embodiment of the present disclosure
- FIG. 4 is a flowchart of a method for determining a first transform core group according to another embodiment of the present disclosure
- FIG. 5 is a schematic diagram of residual distribution of a current image block to be encoded according to an embodiment of the present disclosure
- FIG. 6 is an interaction flowchart of an image encoding and decoding method according to an embodiment of the present application.
- FIG. 7 is a schematic structural diagram of an image coding apparatus 70 according to an embodiment of the present disclosure.
- FIG. 8 is a schematic structural diagram of an image decoding apparatus 80 according to an embodiment of the present disclosure.
- FIG. 9 is a schematic block diagram of a video codec device or electronic device 50.
- FIG. 10 is a schematic diagram of an apparatus for video encoding according to an embodiment of the present application.
- FIG. 11 is a schematic block diagram of a video codec system 10 in accordance with an embodiment of the present application.
- Digital video is a video recorded in digital form.
- Digital video consists of multi-frame digital images.
- the original video contains a large amount of data, which is not suitable for storage and transmission, and requires efficient video compression coding technology to compress the original video.
- the RDO method is an optimized method to improve video compression performance.
- the principle is to optimize the video's lossy (picture quality) and bit rate (the amount of data required for encoding) to achieve an optimal balance point.
- this algorithm was originally used in video compression encoders, it can also be used for various multimedia encodings including video, video, audio, etc., as long as the encoding takes into account both quality and file size.
- Video compression technology achieves compression by eliminating video redundancy.
- Video redundancy mainly includes the following items: spatial redundancy, temporal redundancy, visual redundancy, and information entropy redundancy.
- Spatial redundancy It is the most important data redundancy for static images. It means that the amplitudes of adjacent pixels are similar in an image. This spatial coherence is called spatial correlation or spatial redundancy. Spatial redundancy is mainly eliminated by the intra prediction method.
- the intra prediction method refers to using the correlation of the video spatial domain to predict the pixels of the currently reconstructed image block by using the pixels of the reference image block to achieve the purpose of removing the video space redundancy. .
- Temporal redundancy It is the redundancy that is often included in video sequences. Since adjacent images of video often contain the same or similar background and moving objects, only the spatial position of the moving object is slightly different. The high correlation of data is called time redundancy. Temporal redundancy is mainly eliminated by inter-frame prediction techniques, which refer to the use of pixels of temporally adjacent images to predict the current pixel.
- Visual redundancy The human visual system is insensitive to changes in image detail, and even if these subtle changes are lost, the human eye does not feel it.
- the visual redundancy is mainly eliminated by transforming and transforming technology, which refers to transforming the image signal into the frequency domain for processing, and performing data expression and bit redistribution according to the contribution of the different frequency signals to the visual quality, so that the space can be corrected. Unreasonable expression of uniform sampling over a domain.
- the need to remove visual redundancy is taken into consideration, and the quantization operation is omitted, and the excessively high-frequency component expression is omitted to achieve effective compression.
- Information entropy redundancy It can be known from information theory that as a pixel representing image data, it is only necessary to allocate the corresponding number of bits according to the size of its information entropy, and for each pixel of image data, it is difficult to obtain its information at the time of image acquisition. Entropy, therefore, is generally expressed by the same number of bits per pixel, so there must be redundancy. Information entropy redundancy is mainly solved by entropy coding technology. Entropy coding technology distributes different bit numbers for data with different information entropy through the information entropy distribution of statistical coefficients.
- the current mainstream video compression coding architecture is a hybrid coding architecture.
- different technologies are adopted to eliminate redundancy, and these technologies are combined to form a hybrid architecture of video coding.
- the image is divided into the current image block to be encoded for each frame image constituting the original video.
- the current image block to be encoded is first predicted by the reference image block to obtain a prediction signal of the current image block to be encoded; and the prediction signal is subtracted from the original signal of the current image block to be encoded to obtain a residual signal.
- the magnitude of the residual signal is much smaller than the original signal.
- the residual signal is transformed and quantized. After transform quantization, the transform quantized coefficients are obtained, and the quantized coefficients and other indication information in the encoding are encoded by the entropy coding technique to obtain a code stream.
- the decoding end After obtaining the code stream, the decoding end first performs entropy decoding on the code stream to obtain a transform quantized coefficient of the current image block to be reconstructed, and then inverse quantizes and inverse transforms the transformed quantized coefficient to obtain a reconstructed residual of the current image block to be reconstructed. signal. Predicting the current image block to be reconstructed by using the reference image block, obtaining a prediction signal of the current image block to be reconstructed, and then adding the prediction signal and the reconstructed residual signal to obtain a reconstruction signal of the current image block to be reconstructed, and then according to the The reconstructed signal obtains a reconstructed image block corresponding to the current image block to be reconstructed.
- the encoding end may use an intra prediction technique or an inter prediction technique to generate a prediction signal of the current image block to be encoded.
- the inter prediction process is: the encoding end determines the target reconstructed image block for each inter-frame transform mode, and the motion information of the target reconstructed image block is used as the motion information of the reconstructed image block to be encoded.
- the motion information includes: an index of a reference image frame in which the target reconstructed image block is located, and a spatial displacement between the target reconstructed image block and the reference image block of the target reconstructed image block, that is, a motion vector; the encoding end is current according to the motion information
- the image block at the corresponding position in a certain image frame before the image block to be encoded is motion-compensated to obtain a prediction signal at the encoding end.
- the encoding end reconstructs the image block by searching in each reference image frame to determine a reconstructed image block having the lowest rate distortion coding cost as the above target.
- the target reconstructed image block is determined according to a certain criterion.
- a multi-core transform is generally composed into a plurality of transform core groups, such as a plurality of transform core groups including DST1, DST2, DCT5, DST7, and DCT8 in the H.266 video coding standard, where each transform core group The horizontal transform kernel and the vertical transform kernel are included.
- the current image block to be encoded is transformed by each transform kernel group, and the encoding end selects the optimal transform kernel group according to the minimum rate penalty cost criterion. That is, RDO needs to be performed on the current image block to be encoded. Because the prior art requires multiple RDOs on the current image block to be encoded, the coding complexity is high.
- FIG. 1 is a flowchart of an image encoding method according to an embodiment of the present application. As shown in FIG. 1 , the method includes the following steps:
- Step S101 determining location information of a target reconstructed image block of the current image block to be encoded
- Step S102 Determine a first transform core group according to location information of the target reconstructed image block.
- Step S103 Perform a transform on the residual signal of the current image block to be encoded according to the first transform core group to obtain a transform coefficient.
- the inter-frame transform mode adopted by the encoding end may be a Merge mode, a template matching (TM) mode, or an advanced motion vector prediction (AMVP) mode.
- TM template matching
- AMVP advanced motion vector prediction
- the target reconstructed image block is a reconstructed image block for determining motion information of the current image block to be encoded.
- the motion information of the target reconstructed image block may be used as motion information of the current image block to be encoded, and the motion information includes: An index of a reference image frame in which the target reconstructed image block is located, and a spatial bit between the target reconstructed image block and the reference image block of the target reconstructed image block, that is, a motion vector; wherein the encoding end can be based on the motion information
- the image block at the corresponding position in a certain image frame before the image block to be encoded is motion-compensated to obtain a prediction signal at the encoding end. Further, the encoding end may determine location information of the target reconstructed image block.
- FIG. 2 is a schematic diagram of the position of the target reconstructed image block according to an embodiment of the present application.
- the position of the target reconstructed image block of the current image block to be encoded may be currently in the current state.
- the upper left, upper, upper right, left or lower left of the coded image block, etc., the present application does not limit the position of the target reconstructed image block.
- FIG. 3A to FIG. 3E are the current image block to be encoded according to an embodiment of the present application. Schematic diagram of the residual distribution, as shown in FIG.
- the encoding end is more accurate in predicting the upper left pixel included in the current image block to be encoded, that is, the residual is smaller. Based on this, the residual of the current image block to be encoded changes from small to large with the direction of the arrow pointing.
- the target reconstructed image block is above the current image block to be encoded, the upper pixel included in the current image block to be encoded has motion consistency with the target reconstructed image block, and therefore, the encoding end pairs the current image to be encoded.
- the prediction of the upper pixel included in the block is more accurate, that is, the residual is smaller. Based on this, the residual of the current image block to be encoded changes from small to large with the direction of the arrow pointing.
- the upper right pixel included in the current image block to be encoded has motion consistency with the target reconstructed image block, and therefore, the encoding end is currently waiting.
- the prediction of the upper right pixel included in the coded image block is more accurate, that is, the residual is smaller. Based on this, the residual of the current image block to be coded changes from small to large with the direction of the arrow pointing. As shown in FIG.
- the left pixel included in the current image block to be encoded has motion consistency with the target reconstructed image block, and therefore, the encoding end is currently waiting.
- the prediction of the left pixel included in the coded image block is more accurate, that is, the residual is small, and based on this, the residual of the current image block to be encoded changes from small to large as the direction of the arrow points. As shown in FIG.
- the lower left pixel included in the current image block to be encoded has motion consistency with the target reconstructed image block, and therefore, the encoding end is currently waiting.
- the prediction of the lower left pixel included in the coded image block is more accurate, that is, the residual is smaller. Based on this, the residual of the current image block to be coded changes from small to large with the direction of the arrow pointing.
- step S102 is described. Before the step S102, the encoding end may acquire the correspondence between the location information of the at least one reconstructed image block and the at least one transform core group. Based on this, step S102 specifically includes: determining, according to the correspondence relationship and the location information of the target reconstructed image block, the first transform core group.
- the encoding end may directly determine the first transform core group according to the correspondence information and the position information of the target reconstructed image block.
- the correspondence relationship includes: the correspondence between the location information of the target reconstructed image block and the first transform core group.
- the corresponding relationship shown in Table 1 is determined according to the residual distribution of the image block to be encoded and the characteristics of the transform kernel group. For example, DCT2 is more suitable for image blocks with uniform residual parts, and DST7 is more suitable for horizontal direction. On top, the image block whose residual distribution gradually increases from left to right. It should be noted that the correspondence between the plurality of location information and the plurality of transform core groups described in the present application is not limited to the content shown in Table 1.
- step S102 can be implemented by means of table lookup. For example, when the target reconstructed image block is on the left side of the current image block to be encoded, then the determined first transform kernel group is (DST7, DCT2); when the target reconstructed image block is above the current image block to be encoded, then it is determined.
- the first transform kernel group is (DCT2, DST7); when the target reconstructed image block is at the upper right of the current image block to be encoded, then the determined first transform kernel group is (DCT8, DST7); when the target reconstructed image block is at When the lower left corner of the image block to be encoded is currently, the determined first transform kernel group is (DST7, DCT8); when the target reconstructed image block is at the upper left of the current image block to be encoded, the determined first transform kernel group is (DST7, DST7).
- FIG. 4 is a flowchart of a method for determining a first transform core group according to another embodiment of the present application. As shown in FIG. 4, the method includes the following process:
- Step S401 The encoding end determines at least two second transform core groups according to the correspondence relationship and the location information of the target reconstructed image block.
- Step S402 Encode the current image block to be encoded according to each second transform core group in the at least two second transform core groups, and determine a rate distortion cost corresponding to the encoded current image block to be encoded.
- Step S403 The second transform kernel group that is the smallest of the corresponding rate distortion costs is used as the first transform core group.
- the correspondence between the at least one location information of the reconstructed image block and the at least one transform core group is shown in Table 2.
- the correspondence relationship includes: location information of the target reconstructed image block and two Corresponding relationship of the second transform kernel groups.
- the correspondence relationship shown in Table 2 is determined according to the residual distribution of the current image block to be encoded and the characteristics of the transform kernel group. For example, DCT2 is more suitable for image blocks with uniform residual parts, and DST7 is more suitable for horizontal direction. On top, the image block whose residual distribution gradually increases from left to right. It should be noted that the correspondence between the plurality of location information and the plurality of transform core groups described in the present application is not limited to the content shown in Table 2.
- step S102 can be implemented by means of table lookup.
- the determined two second transform kernel groups are (DST7, DST7), (DST7, DCT8); when the target reconstructed image block is currently to be encoded
- the two determined second transform kernel groups are (DST7, DST7), (DCT8, DST7); when the target reconstructed image block is at the upper right of the current image block to be encoded, the two determined The second transform kernel group is (DST7, DST7), (DCT8, DST7); when the target reconstructed image block is at the lower left of the current image block to be encoded, then the determined two second transform kernel groups are (DST7, DCT8) (DCT2, DCT8); when the target reconstructed image block is at the upper left of the current image block to be encoded, then the determined two second transform kernel groups are (DST7, DST7), (DST7, DST7), (DCT2, DCT8); when the target reconstructed image block is at the upper left of the current image
- the encoding end calculates a rate distortion cost corresponding to the current image block to be encoded by using a rate distortion method, and selects a transform kernel group with the lowest rate distortion cost as the first transform core group. .
- FIG. 5 is a schematic diagram of residual distribution of a current image block to be encoded according to an embodiment of the present application.
- the image block is reconstructed above the current image block to be encoded (can be understood as target reconstruction).
- the image block) and the left reconstructed image block (which can be understood as the target reconstructed image block) constitute an inverted "L" type template, and the encoding end can determine the motion information of the current image block to be encoded through the "L" type template, and further,
- the encoding end may obtain a reference image block of the current image block to be encoded according to the motion information, where the reference image block is used to generate a prediction signal of the image block to be encoded.
- the current image block to be encoded can correspond to two second transform kernel groups, namely (DCT2, DCT2) and (DST7, DST7).
- the transform core group with the lowest rate distortion cost and the lowest rate distortion cost of the current image block to be encoded under each transform kernel group can be calculated as the first transform core group.
- the present application does not limit the content and number of the second transform core group corresponding to the current image block to be encoded.
- step S103 for the case where there is only one type of inter-frame transform mode, the encoding end directly uses the first transform core group as an optimal transform core group, and encodes the current image block to be encoded by using the first transform core group. .
- the encoding end since the encoding end has determined the corresponding first transform kernel group for each inter-frame transform mode, finally selecting the transform with the lowest rate distortion cost among the first transform kernel groups.
- the kernel group is used as an optimal transform kernel group, and the current image block to be encoded is encoded by the optimal transform kernel group.
- the first transform core group may be determined according to the position information of the target reconstructed image block, and the first transform kernel is determined for the position information that cannot reconstruct the image block according to the target.
- the prior art method may be adopted, that is, each transform core group is tried, and the transform kernel group with the smallest corresponding rate distortion cost is determined as the first transform core group.
- at least one transform core group such as (DCT2, DCT2), etc., may be used by default.
- the current image block to be encoded may be transformed by using an existing transformation manner. This application will not go into details here.
- the present application provides an image encoding method, including: determining, in an inter-frame transform mode, location information of a target reconstructed image block of a current image block to be encoded; and determining a first transform kernel according to location information of the target reconstructed image block. And transforming the residual signal of the current image block to be encoded according to the first transform core group.
- the encoding end does not need to try multiple transform core groups, thereby reducing coding complexity and improving coding efficiency.
- FIG. 6 is an interaction flowchart of an image encoding and decoding method according to an embodiment of the present application. As shown in FIG. 6, the method includes the following steps:
- Step S601 The encoding end determines location information of the target reconstructed image block of the current image block to be encoded in the inter-frame transform mode;
- Step S602 The encoding end determines the first transform core group according to the location information of the target reconstructed image block.
- Step S603 The encoding end transforms the residual signal of the current image block to be encoded according to the first transform core group to obtain a transform coefficient.
- the steps S601 to S603 are the same as the steps S101 to S103, and the explanations thereof can be referred to the explanations of the steps S101 to S103.
- Step S604 The encoding end quantizes the transform coefficients to obtain the quantized transform coefficients of the current image block to be encoded.
- Step S605 The encoding end entropy encodes the quantized transform coefficient and the auxiliary coding information of the current image block to be encoded to obtain a code stream.
- Step S606 The decoding end acquires a code stream.
- Step S607 Entropy decoding the code stream to obtain a quantized transform coefficient of the current image block to be encoded, and inversely quantizing the quantized transform coefficient to obtain a transform coefficient of the current image block to be reconstructed;
- Step S608 determining location information of a target reconstructed image block of the current image block to be reconstructed, wherein the target reconstructed image block is a reconstructed image block for determining motion information of the current image block to be reconstructed;
- Step S609 The decoding end determines the first transform core group according to the location information of the target reconstructed image block.
- Step S610 The decoding end inversely transforms the transform coefficients according to the first transform core group to reconstruct the current image block to be reconstructed.
- Step S604 is explained: the process of quantization is a process of forcibly planning a large dynamic input value into fewer output values. Because the range of quantized input values is large, more bit numbers are needed, and the range of output values after "forced planning" is small, so that only a small number of bits can be represented.
- Each quantized input is normalized to a quantized output, that is, quantized into an order of magnitude, often referred to as a quantization level (usually specified by the encoding side).
- the quantification process mentioned in this application can adopt the existing quantification technology, which is not described in this application.
- the auxiliary coding information of the current to-be-encoded image block described in the present application refers to all the information except the quantized transform coefficient of the current image block to be coded involved in the entropy coding process.
- the method includes: an identifier of an inter-frame transform mode, and indication information (optionally), where the indication information is used to indicate an index of the first transform core group, and the auxiliary coding information may further include: data of the current image block to be encoded and associated data.
- the associated data may include a Sequence Paramater Set (SPS), a Picture Paramater Set (PPS), and other syntax structures.
- SPS Sequence Paramater Set
- PPS Picture Paramater Set
- An SPS can contain parameters that are applied to zero or more sequences.
- the PPS can contain parameters that are applied to zero or more pictures.
- a grammatical structure refers to a collection of zero or more syntax elements arranged in a specified order in a code stream.
- the auxiliary coding information may further include: an encoding mode of the current image block to be encoded, block division information of the current image block to be encoded, and other indication information of the current image block to be encoded, and the like.
- the existing entropy coding technology is not described in this application.
- the inter-frame transform mode is used to determine the mode of the first transform core group by reconstructing the position information of the image block, and the corresponding relationship between the inter-frame transform modes is as follows. As shown in Table 1, that is, one location corresponds to one transform core group, and the auxiliary coding information carries identification information of the inter-frame transform mode finally determined by the encoding end.
- the decoding end only needs to know the identification information of the inter-frame transformation mode, and according to the identification information, the corresponding inter-frame transformation mode and the correspondence relationship as shown in Table 1 can be determined.
- the decoding end may further reconstruct the current image block to be reconstructed according to the code stream, where the current image block to be reconstructed corresponds to the current image block to be reconstructed, that is, after the image block to be reconstructed is reconstructed, the original image of the current image block to be encoded is obtained.
- the decoding end can also use the same rules as the encoding end to know the target reconstructed image block of the current image block to be reconstructed, and the motion information of the target reconstructed image block can be used as the motion information of the current image block to be reconstructed.
- the reference image block of the current image block to be reconstructed is obtained by using the motion information, and the reference image block is used to generate a prediction signal of the current image block to be reconstructed.
- the so-called "decoding end can also use the same rule as the encoding end to know the target reconstructed image block of the current image block to be reconstructed” means that if the target reconstructed image block determined by the encoding end is the upper left corner reconstructed image block of the current image block to be encoded, Then, the target reconstructed image block determined by the decoding end is also the upper left corner reconstructed image block of the current image block to be reconstructed. Based on this, after the decoding end acquires the target reconstructed image block, the decoding end may determine the location information of the target reconstructed image block.
- the first transform kernel group is determined by looking up Table 1.
- the beneficial effect of the method is that, for the encoding end, the decoding end can determine the first transform core group without transmitting any indication information, and in the prior art, the encoding end needs to transmit the identifier corresponding to each transform core group, For example, the H.266 standard assumes that there are five transform core groups. In this case, the encoding end needs to transmit 3-bit identification information, so that the decoding end determines the first transform core group. Therefore, the overhead of the encoding end can be reduced by this method.
- the inter-frame transform mode is used to determine the mode of the first transform core group by using the location information of the target reconstructed image block, and the corresponding relationship between the inter-frame transform modes is as follows. As shown in Table 2, that is, one location corresponds to two transform core groups, and the help coding information carries identification information and indication information of the inter-frame transform mode finally determined by the encoding end.
- the indication information is used to indicate an index of the first transform core group; for example, the index is 1 or 2.
- the index When the index is 1, it indicates that when the decoding end determines two second transform core groups, the decoding end finally selects the first one of the two second transform core groups as the first transform core group, and when the index is 2, it indicates When the decoding end determines two second transform core groups, the decoding end finally selects the second one of the two second transform core groups as the first transform core group.
- the decoding end needs to know the identification information and the indication information of the inter-frame transformation mode, and according to the identification information, the corresponding inter-frame transformation mode and the correspondence relationship as shown in Table 2 can be determined.
- the decoding end may further reconstruct the current image block to be reconstructed according to the code stream, and the decoding end may also use the same rule as the encoding end to learn the target reconstructed image block of the current image block to be reconstructed, and the motion information of the target reconstructed image block may be used as The motion information of the current image block to be reconstructed, by which the reference image block of the current image block to be reconstructed can be obtained, and the reference image block is used to generate a prediction signal of the current image block to be reconstructed.
- the so-called "decoding end can also use the same rule as the encoding end, and the target reconstructed image block of the image block to be reconstructed" means that if the target reconstructed image block determined by the encoding end is the reconstructed image block of the upper left corner of the current image block to be encoded, Then, the target reconstructed image block determined by the decoding end is also the upper left corner reconstructed image block of the current image block to be reconstructed. Based on this, after the decoding end acquires the target reconstructed image block, the decoding end may determine the location information of the target reconstructed image block. By checking the manner of Table 2, two second transform core groups are determined. In this case, the decoding end does not need to calculate the rate distortion cost corresponding to the two second transform core groups respectively, to determine the first transform core group, but The first transform core group is determined according to the indication information.
- the advantage of this is that, on the one hand, for the coding end, only one bit of indication information needs to be transmitted, so that the decoding end can determine the first transform core group, and in the prior art, the coding end needs to transmit each transform core.
- the corresponding identifier of the group taking the H.266 standard as an example, indicates that there are five transform core groups.
- the encoding end needs to transmit the 3-bit identification information, so that the decoding end determines the first transform core group, so The method can reduce the overhead of the encoding end.
- the first transform core group is determined without calculating the rate distortion cost corresponding to the two second transform core groups, and the first transform core group is determined according to the indication information. Thereby reducing the decoding complexity, thereby improving the decoding efficiency.
- the decoding end directly uses the first transform core group as an optimal transform core group, and performs the current image block to be reconstructed by using the first transform core group. Inverse transform to reconstruct the current image block to be reconstructed.
- the decoding end since the decoding end has determined the corresponding first transform core group for each inter-frame transform mode, finally selecting the transform with the lowest rate distortion cost among the first transform kernel groups.
- the kernel group is used as an optimal transform kernel group, and the current image block to be reconstructed is inversely transformed by the optimal transform kernel group to reconstruct the current image block to be reconstructed.
- the current inverse transform mode may be used to inverse transform the current image block to be reconstructed. This application will not go into details here.
- the method provided by the present application can reduce the overhead of the coding end on the one hand, and reduce the decoding complexity on the other hand, thereby improving the decoding efficiency.
- Prior Art 1 has been adopted by standards and is currently integrated in the Joint Exploration Model (JEM). Based on the transformation of the original H.265 standard, the H.266 video coding standard introduces new The transform cores DST7 and DCT8 are combined and combined to form a multi-core transform technique, which is specifically as follows:
- Step 1 Calculate the rate distortion cost of the default transform kernel group
- This step uses the transform kernel set of the original H.265 standard to transform, quantize, and entropy the current image block to be encoded, and calculate the rate distortion cost.
- Step 2 Calculation of rate distortion cost of multi-core transform kernel group
- the H.266 standard uses (DCT, DCT2) to transform the current image block to be encoded, and then quantizes and entropy codes to calculate the rate distortion cost (RDCost1). If the calculated rate distortion cost is greater than 1.1 times the minimum rate distortion cost of the tried mode calculated in step 1, the rate distortion cost calculation process of the multi-core transform kernel group is skipped, and the original H.265 standard transform kernel pair is selected.
- the image block to be encoded is encoded.
- the four groups of transform cores (DCT8, DCT8), (DST7, DST7), (DCT8, DST7), (DST7, DCT8) are respectively used to transform the current image block to be encoded, and then Quantization, entropy coding, and four sets of rate distortion costs are calculated, which are RDCost2, RDCost3, RDCost4, and RDCost5.
- Step 3 Transform the selection of the core group
- Step 3 is to determine the optimal transform kernel group for the current image block to be coded according to the criterion of the lowest rate distortion cost.
- Step 4 Entropy coding
- the binary coding uses context-based arithmetic coding, the initial model number of the context and the binary bit or the current The depth of the image block to be encoded is related.
- the optimal transform kernel group of the current image block to be encoded is (DCT2, DCT2)
- the flag is 1, otherwise the flag is 0, indicating that the multi-core transform technique is used.
- Selecting (DCT8, DCT8) in the multi-core transform the current block is marked as 0, the selected (DCT8, DST7) is marked as 1, the selected (DST7, DCT8) is marked as 2, the selected (DST7, DST7) is marked as 3, and Binarization is shown in Table 3 below, and the binarized bits are encoded using the mainstream context-based arithmetic coding.
- the coding uses two context models.
- the context-based arithmetic coding is also used for the flag bits of the technique using multi-core transform. 5 context models.
- the decoding end can determine which set of transform kernel groups to use, and does not need to be determined by the RDO process.
- the specific process is the inverse process of the encoding end entropy encoding process, and the context model is the same as the encoding end.
- Example 1 For the Merge mode, it is assumed that there is a correspondence relationship as shown in Table 1. For other inter-frame transform modes, the coding end uses the transform core group (DCT2, DCT2) by default.
- DCT2, DCT2 transform core group
- Example 2 For the Merge mode, it is assumed that there is a correspondence relationship as shown in Table 2; for the TM mode, there are two transform core groups (DCT2, DCT2) and (DST7, DST7); for other inter-frame transform modules, There is a technical method of attempting each transform kernel group to determine that the transform kernel group having the smallest rate distortion cost is the first transform core group.
- Example 3 For the Merge mode, it is assumed that there is a correspondence relationship as shown in Table 2; for the TM mode, there are two transform kernel groups (DCT2, DCT2) and (DST7, DST7); other inter-frame transform modules also have two Transform kernel groups (DCT2, DCT2) and (DST7, DST7).
- Example 1 and Example 2 are implemented in JEM and tested in full sequence.
- the test sequence is the general test sequence of H.266 standard.
- the test conditions are the general conditions specified by the Joint Video Exploration Team (JVET). Under the access (RandomAccess, RA), LDP, LDB structure, BD-RATE is used as the evaluation index of the coding performance (the coding performance refers to the bit savings under the same code reconstruction video quality), and the negative value indicates that the coding performance is improved. A positive value indicates a loss of coding performance.
- JVET Joint Video Exploration Team
- the test sequence is 420 YUV format, which corresponds to three sets of data, namely luminance (Y) and chrominance (U, V) components, and the complexity is measured by the time of the codec end, where the coding end
- the time is called Encoding Time (EncT)
- the time of the decoding end is called Decoding Time (DecT)
- the following Class represents the test sequence set, specifically the video
- Overall(Ref) represents the average value.
- JEM4.0 code sequence length is 1 second
- Frame Rate Up-Conversion (FRUC) and inter-frame EMT are turned on, and other tools are turned off.
- Example 1 The coding performance of Example 1 is shown in Table 7 to Table 9.
- JEM4.0 code sequence length is 1 second, FRUC and interframe EMT are turned on, and other tools are turned off.
- JEM 5.0.1 code sequence length is 1 second, FRUC and interframe EMT are turned on, and other tools are turned off.
- columns 2-6 are the performance of the second example relative to the EMT OFF in the JEM
- columns 7-11 are the performance of the second example relative to the JEM.
- Test conditions performance of LDB JEM5.0.1 general test conditions
- Test conditions performance of LDP JEM 5.0.1 general test conditions
- JEM 5.0.1 code sequence length is 1 second, FRUC and interframe EMT are turned on, and other tools are turned off.
- the second to sixth columns are the performance data of the third example in the JEM
- the seventh to the seventh are the performance data of the prior art in the JEM.
- the position-dependent inter-frame transformation is adopted only for the merge mode and the TM mode. From the performance data of the second example, it can be concluded that the coding end reduces the coding complexity under the premise that there is almost no coding loss under the LDP and LDB configurations. 3%-6%), the RA configuration also reduces the complexity of the coding, achieving a better balance of performance and complexity.
- the coding under LDP and LDB coding configurations can be improved under the general conditions specified by JVET. Performance, with an average performance increase of 0.11% and 0.10% for the luminance component.
- Example 3 not only adopts position-dependent inter-frame transformation for merge mode and TM mode, but also uses two transformations for other inter-frame modes. Comparing the performance data of example three can be obtained, the encoding end will be JEM in the performance with little loss. The complexity of the multi-core transform technology is reduced by about 50%, achieving a good balance between coding performance and coding complexity.
- FIG. 7 is a schematic structural diagram of an image encoding apparatus 70 according to an embodiment of the present disclosure.
- the apparatus 70 may be an encoder, a terminal device including an encoder, or a chip having a similar function to an encoder, as shown in FIG. 7.
- the device 70 includes:
- the first determining module 71 is configured to determine location information of a target reconstructed image block of the current image block to be encoded, where the target reconstructed image block is a reconstructed image block for determining motion information of the current image block to be encoded.
- the second determining module 72 is configured to determine the first transform core group according to the location information of the target reconstructed image block determined by the first determining module 71.
- the transforming module 73 is configured to transform the residual signal of the current image block to be encoded according to the first transform core group determined by the second determining module 72 to obtain a transform coefficient.
- the image encoding device further includes: an obtaining module 74, configured to acquire a correspondence between location information of the at least one reconstructed image block and the at least one transform core group.
- the second determining module 72 is specifically configured to determine the first transform core group according to the correspondence relationship and the location information of the target reconstructed image block.
- the second determining module 72 is configured to: determine, according to the correspondence relationship and location information of the target reconstructed image block, at least two second transform core groups; respectively, according to the at least two second transform cores Each second transform core group in the group encodes the current image block to be encoded, and determines a rate distortion cost corresponding to the encoded current image block to be encoded; and a second transform corresponding to the minimum rate distortion cost a core group as the first transform core group.
- the image encoding device 70 further includes:
- the quantization module 75 is configured to quantize the transform coefficients to obtain quantized transform coefficients of the current image block to be encoded.
- the entropy coding module 76 is configured to entropy encode the quantized transform coefficient and the auxiliary coding information of the current image block to be encoded to obtain a code stream.
- the auxiliary coding information includes: an identifier of the inter-frame transform mode and indication information, where the indication information is used to indicate an index of the first transform core group.
- the image encoding device provided by the present application can perform the image encoding method described above, and the content and effect of the image encoding method can be referred to in the corresponding content and effect, which will not be further described herein.
- FIG. 8 is a schematic structural diagram of an image decoding apparatus 80 according to an embodiment of the present disclosure.
- the apparatus 80 may be a decoder, a terminal device including a decoder, or a chip having a similar function to a decoder, as shown in FIG. 8.
- the device 80 includes:
- the entropy decoding module 81 is configured to perform entropy decoding on the code stream to obtain a quantized transform coefficient of the current image block to be reconstructed.
- the inverse quantization module 82 is configured to perform inverse quantization on the quantized transform coefficients to obtain transform coefficients of the current image block to be reconstructed.
- the first determining module 83 is configured to determine location information of a target reconstructed image block of the current image block to be reconstructed, where the target reconstructed image block is a reconstructed image block used to determine motion information of the current image block to be reconstructed.
- the second determining module 84 is configured to determine the first transform core group according to the location information of the target reconstructed image block.
- the inverse transform module 85 is configured to inverse transform the transform coefficients according to the first transform core group to reconstruct the current image block to be reconstructed.
- the device 80 further includes:
- the obtaining module 86 is configured to acquire a correspondence between location information of the at least one reconstructed image block and the at least one transform core group.
- the second determining module 84 is specifically configured to determine the first transform core group according to the correspondence relationship and location information of the target reconstructed image block.
- the code stream further carries indication information, where the indication information is used to indicate an index of the first transform core group;
- the second determining module 84 is specifically configured to: determine, according to the correspondence relationship and the location information of the target reconstructed image block, at least two second transform core groups; according to the index of the first transform core group In the at least two second transform core groups, the first transform core group is determined.
- the image decoding device provided by the present application can perform the image decoding method described above, and the content and effect of the image decoding method can be referred to in the corresponding content and effect, which will not be further described herein.
- An embodiment of the present application provides an image coding and decoding system, also referred to as a video codec system, comprising: an image coding device as shown in FIG. 7, and an image decoding device shown in FIG.
- the system can be disposed in the same video codec device or electronic device, in which the image encoding device and the image decoding device can be integrated, and can be collectively referred to as a codec.
- the system is provided in a different device, and the two cases are separately described below with reference to the accompanying drawings:
- FIG. 9 is a schematic block diagram of a video codec device or electronic device 50 that may incorporate the codec described above.
- FIG. 10 is a schematic diagram of an apparatus for video encoding in accordance with an embodiment of the present application. The units in Figs. 9 and 10 will be explained below.
- the electronic device 50 can be, for example, a mobile terminal or user equipment of a wireless communication system. It should be understood that embodiments of the present application may be implemented in any electronic device or device that may require encoding and decoding, or encoding, or decoding of a video image.
- the electronic device 50 can include a housing 30 for incorporating and protecting the device.
- Electronic device 50 may also include display 32 in the form of a liquid crystal display.
- the display may be any suitable display technology suitable for displaying images or video.
- the electronic device 50 can also include a keypad 34.
- any suitable data or user interface mechanism can be utilized.
- the user interface can be implemented as a virtual keyboard or data entry system as part of a touch sensitive display.
- Electronic device 50 may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input.
- the electronic device 50 may also include an audio output device, which in the embodiment of the present application may be any of the following: an earphone 38, a speaker, or an analog audio or digital audio output connection.
- Electronic device 50 may also include battery 40, and in other embodiments of the present application, the device may be powered by any suitable mobile energy device, such as a solar cell, fuel cell, or clock mechanism generator.
- the electronic device 50 may also include an infrared port 42 for short-range line of sight communication with other devices.
- electronic device 50 may also include any suitable short range communication solution, such as a Bluetooth wireless connection or a USB/Firewire wired connection.
- Electronic device 50 may include a controller 56 or processor for controlling electronic device 50.
- the controller 56 can be coupled to a memory 58, which in the embodiments of the present application can store data in the form of data and audio in the form of images, and/or can also store instructions for execution on the controller 56.
- Controller 56 may also be coupled to codec circuitry 54 suitable for implementing encoding and decoding of audio and/or video data or assisted encoding and decoding by controller 56.
- the electronic device 50 may also include a card reader 48 and a smart card 46, such as a UICC and UICC reader, for providing user information and for providing authentication information for authenticating and authorizing users on the network.
- a card reader 48 and a smart card 46 such as a UICC and UICC reader, for providing user information and for providing authentication information for authenticating and authorizing users on the network.
- the electronic device 50 may also include a radio interface circuit 52 coupled to the controller and adapted to generate, for example, a wireless communication signal for communicating with a cellular communication network, a wireless communication system, or a wireless local area network.
- the electronic device 50 may also include an antenna 44 coupled to the radio interface circuit 52 for transmitting radio frequency signals generated at the radio interface circuit 52 to other device(s) and for receiving radio frequency signals from other device(s).
- electronic device 50 includes a camera capable of recording or detecting a single frame, and codec 54 or controller receives these single frames and processes them.
- the device may receive video image data to be processed from another device prior to transmission and/or storage.
- electronic device 50 may receive images for encoding/decoding via a wireless or wired connection.
- FIG. 10 is only a schematic diagram of the electronic device 50 and the various software and hardware modules it includes, and has various implementations, such as the keypad 34 may be a touch screen, and the touch screen may be part of the display 32. Not limited.
- FIG. 11 is a schematic block diagram of a video codec system 10 in accordance with an embodiment of the present application.
- the video codec system 10 includes a source device 12 and a destination device 14.
- Source device 12 produces encoded video data.
- source device 12 may be referred to as a video encoding device or a video encoding device or an image encoding device as described herein.
- Destination device 14 may decode the encoded video data produced by source device 12.
- destination device 14 may be referred to as a video decoding device or video decoding device or an image decoding device as described herein.
- Source device 12 and destination device 14 may be examples of video codec devices or video codec devices.
- Source device 12 and destination device 14 may include a wide range of devices including desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set top boxes, smart phones, etc., televisions, cameras, display devices , digital media player, video game console, on-board computer, or the like.
- Channel 16 may include one or more media and/or devices capable of moving encoded video data from source device 12 to destination device 14.
- channel 16 may include one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real time.
- source device 12 may modulate the encoded video data in accordance with a communication standard (eg, a wireless communication protocol) and may transmit the modulated video data to destination device 14.
- the one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines.
- RF radio frequency
- the one or more communication media may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network (e.g., the Internet)).
- the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from source device 12 to destination device 14.
- channel 16 can include a storage medium that stores encoded video data generated by source device 12.
- destination device 14 can access the storage medium via disk access or card access.
- the storage medium may include a variety of locally accessible data storage media, such as Blu-ray Disc, DVD, CD-ROM, flash memory, or other suitable digital storage medium for storing encoded video data.
- channel 16 can include a file server or another intermediate storage device that stores encoded video data generated by source device 12.
- destination device 14 may access the encoded video data stored at a file server or other intermediate storage device via streaming or download.
- the file server may be a server type capable of storing encoded video data and transmitting the encoded video data to the destination device 14.
- the instance file server includes a web server (eg, for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive.
- FTP file transfer protocol
- NAS network attached storage
- Destination device 14 can access the encoded video data via a standard data connection (e.g., an internet connection).
- a standard data connection e.g., an internet connection.
- An instance type of a data connection includes a wireless channel (eg, a Wi-Fi connection), a wired connection (eg, DSL, cable modem, etc.), or both, suitable for accessing encoded video data stored on a file server. combination.
- the transmission of the encoded video data from the file server may be streaming, downloading, or a combination of both.
- the technology of the present application is not limited to a wireless application scenario.
- the technology can be applied to video codecs supporting various multimedia applications such as aerial television broadcasting, cable television transmission, satellite television transmission, and streaming video. Transmission (eg, via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other application.
- video codec system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.
- source device 12 includes video source 18, video encoder 20, and output interface 22.
- output interface 22 can include a modulator/demodulator (modem) and/or a transmitter.
- Video source 18 may include a video capture device (eg, a video camera), a video archive containing previously captured video data, a video input interface to receive video data from a video content provider, and/or a computer for generating video data.
- Video encoder 20 may encode video data from video source 18.
- source device 12 transmits the encoded video data directly to destination device 14 via output interface 22.
- the encoded video data may also be stored on a storage medium or file server for later access by the destination device 14 for decoding and/or playback.
- destination device 14 includes an input interface 28, a video decoder 30, and a display device 32.
- input interface 28 includes a receiver and/or a modem.
- Input interface 28 can receive the encoded video data via channel 16.
- Display device 32 may be integral with destination device 14 or may be external to destination device 14. In general, display device 32 displays the decoded video data.
- Display device 32 may include a variety of display devices such as liquid crystal displays (LCDs), plasma displays, organic light emitting diode (OLED) displays, or other types of display devices.
- LCDs liquid crystal displays
- OLED organic light emitting diode
- Video encoder 20 and video decoder 30 may operate in accordance with a video compression standard (eg, the High Efficiency Video Codec H.265 standard) and may conform to the HEVC Test Model (HM).
- a video compression standard eg, the High Efficiency Video Codec H.265 standard
- HM HEVC Test Model
- a textual description of the H.265 standard is published on April 29, 2015, ITU-T.265(V3) (04/2015), available for download from http://handle.itu.int/11.1002/1000/12455 The entire contents of the document are incorporated herein by reference.
- the present application provides a storage medium including instructions that, when executed on an image encoding device, cause the image encoding device to perform an image encoding method as described above.
- the present application provides a storage medium including instructions that, when executed on an image decoding device, cause the image decoding device to perform an image decoding method as described above.
- the present application provides a computer program product that, when executed on an image encoding device, causes the image encoding device to perform the image encoding method as described above.
- the application provides a computer program product that, when executed on an image decoding device, causes the image decoding device to perform an image decoding method as described above.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
本申请提供一种图像编解码方法、装置、***及存储介质,该方法包括:确定当前待编码图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块;根据目标重建图像块的位置信息,确定第一变换核组;根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。从而降低编码复杂度,提高编码效率。
Description
本申请要求于2017年12月13日提交中国专利局、申请号为201711327990.X、申请名称为“图像编解码方法、装置、***及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及图像处理技术,尤其涉及一种图像编解码方法、装置、***及存储介质。
数字视频是以数字形式记录的视频。其中数字视频由多帧数字图像组成。通常原始视频包括的数据量较大,不适合存储和传输,需要利用高效的视频压缩编码技术来压缩原始视频。这也是编码端的编码目的。
具体地,编码端的编码流程包括:编码端接收到原始视频之后,对于构成原始视频的每帧图像,将该图像划分成多个当前待编码图像块。对于当前待编码图像块,首先通过参考图像块(该参考图像块用于提供当前待编码图像块所需的参考像素,该参考像素用于对当前待编码图像块进行预测)对当前待编码图像块进行帧内预测或帧间预测,得到当前待编码图像块的预测信号;用当前待编码图像块的原始信号减去预测信号,得到残差信号。经过预测后,残差信号的幅值远小于原始信号。将残差信号进行变换和量化操作。经过变换量化后,得到变换量化系数,再通过熵编码技术编码量化系数以及编码中的其他指示信息,得到码流。
其中,变换是视频编码过程中的一个重要环节,它是将信息采样值按照某种形式的变换函数,从当前域转换到变换域上,再根据信息在变换域的分布特性进行压缩编码。变换的原因是:图像或视频数据往往在空间域的数据相关性大,导致大量冗余信息的存在,直接编码需要很大的比特量。而在变换域中数据相关性则大大减少,使得编码的冗余信息减少,编码所需的数据量也随之大大减少,这样就能够得到较高的压缩比。
目前在帧间变换模式下,通常采用多核变换(Explicit Multiple Core Transform,EMT),多核变换可以弥补帧间预测不准确,导致图像残差较大的不足。现有技术中,通常将多核变换组成多个变换核组,针对每种帧间变换模式,通过每个变换核组对当前待编码图像块进行变换,编码端都要根据率失真代价(Rate Distortion Cost,RDCost)最小准则,选择最优的变换核组,即需要对当前待编码图像块进行多次率失真优化(Rate Distortion Optimization,RDO)。由于现有技术需要对当前待编码图像块进行多次RDO,从而造成编码复杂度较高的问题。
发明内容
本申请提供一种图像编解码方法、装置、***及存储介质,从而降低编码复杂度。
第一方面,本申请提供一种图像编码方法,包括:确定当前待编码图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块;根据目标重建图像块的位置信息,确定第一变换核组;根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。
本申请的有益效果包括:本申请提供的图像编码方法,可以使得编码端无需尝试多个变换核组,从而降低编码复杂度,提高编码效率。
可选地,根据目标重建图像块的位置信息,确定第一变换核组之前,还包括:获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,根据目标重建图像块的位置信息,确定第一变换核组,包括:根据对应关系和目标重建图像块的位置信息,确定第一变换核组。
本申请的有益效果包括:通过本申请提供的图像编码方法,可以有效的确定第一变换核组,从而使得编码端无需尝试多个变换核组,进而降低编码复杂度。
一种可选方式,至少一个重建图像块的位置信息与至少一个变换核组的对应关系包括:当前待编码图像块的左侧重建图像块与变换核组(DST7,DCT2)的对应关系、当前待编码图像块的上方重建图像块与变换核组(DCT2,DST7)的对应关系、当前待编码图像块的右上方重建图像块与变换核组(DCT8,DST7)的对应关系、当前待编码图像块的左下方重建图像块与变换核组(DST7,DCT8)的对应关系和当前待编码图像块的左上方重建图像块与变换核组(DST7,DST7)的对应关系。
可选地,至少一个重建图像块的位置信息与至少一个变换核组的对应关系还包括:其他目标重建图像块与变换核组(DCT2,DCT2)的对应关系。
可选地,根据对应关系和目标重建图像块的位置信息,确定第一变换核组,包括:根据对应关系和目标重建图像块的位置信息,确定至少两个第二变换核组;分别根据至少两个第二变换核组中的每个第二变换核组,对当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;将对应最小的率失真代价的第二变换核组,作为第一变换核组。
本申请的有益效果包括:通过本申请提供的图像编码方法,可以有效的确定第一变换核组,从而使得编码端无需尝试多个变换核组,进而降低编码复杂度。
另一种可选方式,至少一个重建图像块的位置信息与至少一个变换核组的对应关系包括:当前待编码图像块的左侧重建图像块与两个第二变换核组(DST7,DST7)(DST7,DCT8)的对应关系、当前待编码图像块的上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系、当前待编码图像块的右上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系、当前待编码图像块的左下方重建图像块与两个第二变换核组(DST7,DCT8)(DCT2,DCT8)的对应关系和当前待编码图像块的左上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系。
可选地,根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数之后,还包括:对变换系数进行量化,得到当前待编码图像块的量化后的变换 系数;对量化后的变换系数和当前待编码图像块的辅助编码信息进行熵编码,得到码流;其中,辅助编码信息包括:帧间变换模式的标识和指示信息,指示信息用于指示第一变换核组的索引。
本申请的有益效果包括:对于编码端来讲,仅需传输1比特的指示信息,使得解码端可以确定第一变换核组,而现有技术中,编码端需要传输每个变换核组对应的标识,以H.266标准为例,指示存在5个变换核组,在此情况下,编码端需要传输3比特的标识信息,以使解码端确定第一变换核组,因此通过该方法可以降低编码端的开销。
第二方面,本申请提供一种图像解码方法,包括:对码流进行熵解码,以得到当前待重建图像块的量化后的变换系数,并对量化后的变换系数进行反量化,得到当前待重建图像块的变换系数;确定当前待重建图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待重建图像块的运动信息的重建图像块;根据目标重建图像块的位置信息,确定第一变换核组;根据第一变换核组,对变换系数进行反变换,以重建当前待重建图像块。
本申请的有益效果包括:本申请提供的图像解码方法,可以使得解码端无需尝试多个变换核组,从而降低解码复杂度,提高解码效率。
可选地,根据目标重建图像块的位置信息,确定第一变换核组之前,还包括:获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,根据目标重建图像块的位置信息,确定第一变换核组,包括:根据对应关系和目标重建图像块的位置信息,确定第一变换核组。
本申请的有益效果包括:通过本申请提供的图像解码方法,可以有效的确定第一变换核组,从而使得解码端无需尝试多个变换核组,进而降低解码复杂度。
一种可选方式,至少一个重建图像块的位置信息与至少一个变换核组的对应关系包括:当前待重建图像块的左侧重建图像块与变换核组(DST7,DCT2)的对应关系、当前待重建图像块的上方重建图像块与变换核组(DCT2,DST7)的对应关系、当前待重建图像块的右上方重建图像块与变换核组(DCT8,DST7)的对应关系、当前待重建图像块的左下方重建图像块与变换核组(DST7,DCT8)的对应关系和当前待重建图像块的左上方重建图像块与变换核组(DST7,DST7)的对应关系。
可选地,至少一个重建图像块的位置信息与至少一个变换核组的对应关系还包括:其他目标重建图像块与变换核组(DCT2,DCT2)的对应关系。
可选地,码流还携带指示信息,该指示信息用于指示第一变换核组的索引;相应的,根据对应关系和目标重建图像块的位置信息,确定第一变换核组,包括:根据对应关系和目标重建图像块的位置信息,确定至少两个第二变换核组;根据第一变换核组的索引在至少两个第二变换核组中,确定第一变换核组。
本申请的有益效果包括:通过本申请提供的图像解码方法,可以有效的确定第一变换核组,从而使得解码端无需尝试多个变换核组,进而降低解码复杂度。进一步地,对于解码端来讲,无需计算两个第二变换核组分别对应的率失真代价,来确定第一变换核组,而是根据指示信息确定第一变换核组。从而降低了解码复杂度,进而提高了解码效率。
另一种可选方式,至少一个重建图像块的位置信息与至少一个变换核组的对应关系包括:当前待重建图像块的左侧重建图像块与两个第二变换核组(DST7,DST7)(DST7,DCT8)的对应关系、当前待重建图像块的上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系、当前待重建图像块的右上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系、当前待重建图像块的左下方重建图像块与两个第二变换核组(DST7,DCT8)(DCT2,DCT8)的对应关系和当前待重建图像块的左上方重建图像块与两个第二变换核组(DST7,DST7)(DCT8,DST7)的对应关系。
下面介绍图像编码装置,其实现原理和技术效果可参照第一方面涉及的方法以及第一方面的可选方式的原理和技术效果,此处不再赘述。
第三方面,本申请提供一种图像编码装置,包括:
第一确定模块,用于确定当前待编码图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块。
第二确定模块,用于根据目标重建图像块的位置信息,确定第一变换核组。
变换模块,用于根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。
可选地,该装置还包括:获取模块,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,第二确定模块,具体用于根据对应关系和目标重建图像块的位置信息,确定第一变换核组。
可选地,第二确定模块具体用于:根据对应关系和目标重建图像块的位置信息,确定至少两个第二变换核组;分别根据至少两个第二变换核组中的每个第二变换核组,对当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;将对应最小的率失真代价的第二变换核组,作为第一变换核组。
可选地,该装置还包括:
量化模块,用于对变换系数进行量化,得到当前待编码图像块的量化后的变换系数。
熵编码模块,用于对量化后的变换系数和当前待编码图像块的辅助编码信息进行熵编码,得到码流。其中,辅助编码信息包括:帧间变换模式的标识和指示信息,指示信息用于指示第一变换核组的索引。
下面介绍图像解码装置,其实现原理和技术效果可参照第二方面涉及的方法以及第二方面的可选方式的原理和技术效果,此处不再赘述。
第四方面,本申请提供一种图像解码装置,包括:
熵解码模块,用于对码流进行熵解码,以得到当前待重建图像块的量化后的变换系数。
反量化模块,用于对量化后的变换系数进行反量化,得到当前待重建图像块的变换系数。
第一确定模块,用于确定当前待重建图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待重建图像块的运动信息的重建图像块。
第二确定模块,用于根据目标重建图像块的位置信息,确定第一变换核组。
反变换模块,用于根据第一变换核组,对变换系数进行反变换,以重建当前待重建图像块。
可选地,该装置还包括:获取模块,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,第二确定模块,具体用于根据对应关系和目标重建图像块的位置信息,确定第一变换核组。
可选地,码流还携带指示信息,指示信息用于指示第一变换核组的索引;相应的,第二确定模块具体用于:根据对应关系和目标重建图像块的位置信息,确定至少两个第二变换核组;根据第一变换核组的索引在至少两个第二变换核组中,确定第一变换核组。
第五方面,本申请提供一种图像编解码***,包括:如第三方面和第三方面的可选方式中任一项的图像编码装置,以及如第四方面和第四方面的可选方式中任一项的图像解码装置。
第六方面,本申请提供一种存储介质,包括指令,当指令在图像编码装置上运行时,使得图像编码装置执行如第一方面和第一方面的可选方式中任一项的方法。
第七方面,本申请提供一种存储介质,包括指令,当指令在图像解码装置上运行时,使得图像解码装置执行如第二方面和第二方面的可选方式中任一项的方法。
第八方面,本申请提供一种计算机程序产品,当计算机程序产品在图像编码装置上运行时,使得图像编码装置执行如第一方面和第一方面的可选方式中任一项的图像编码方法。
第九方面,本申请提供一种计算机程序产品,当计算机程序产品在图像解码装置上运行时,使得图像解码装置执行如第二方面和第二方面的可选方式中任一项的图像解码方法。
本申请提供一种图像编解码方法、装置、***及存储介质,该方法包括:确定当前待编码图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块;根据目标重建图像块的位置信息,确定第一变换核组;根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。使得编码端无需尝试多个变换核组,从而降低编码复杂度,提高编码效率。
图1为本申请一实施例提供的图像编码方法的流程图;
图2为本申请一实施例提供的目标重建图像块的位置示意图;
图3A至图3E为本申请一实施例提供的当前待编码图像块的残差分布示意图;
图4为本申请另一实施例提供的确定第一变换核组的方法流程图;
图5为本申请一实施例提供的当前待编码图像块的残差分布示意图;
图6为本申请一实施例提供的图像编解码方法的交互流程图;
图7为本申请一实施例提供的一种图像编码装置70的结构示意图;
图8为本申请一实施例提供的一种图像解码装置80的结构示意图;
图9是视频编解码装置或电子设备50的示意性框图;
图10是根据本申请实施例的用于视频编码的示意性装置图;
图11是根据本申请实施例的视频编解码***10的示意性框图。
以下,对本申请中的部分专业用语进行解释说明,以便于本领域技术人员理解。
数字视频是以数字形式记录的视频。数字视频由多帧数字图像组成。通常原始视频包括的数据量较大,不适合存储和传输,需要利用高效的视频压缩编码技术来压缩原始视频。
RDO方法是一种提升视频压缩性能的最优化方法。其原理是对视频的有损(画面质量)与比特率(编码所需的数据量)同时进行最优化,以求达到一个最佳的平衡点。虽然此算法一开始是在视频压缩的编码器中被使用,但也可以用于各种多媒体编码包含视频、视频、音频等等,只要编码时会同时考虑到质量及文件大小皆可使用。
视频压缩技术是通过消除视频冗余来达到压缩的目的。视频冗余主要包括如下几项:空间冗余,时间冗余,视觉冗余和信息熵冗余。
空间冗余:它是静态图像存在的最主要的数据冗余。它是指在一幅图像中,相邻的像素的幅值都比较相近,这种空间连贯性被称为空间相关或空间冗余。空间冗余主要是通过帧内预测方法来消除,帧内预测方法是指利用视频空间域的相关性,使用参考图像块的像素预测当前重建图像块的像素,以达到去除视频空间冗余的目的。
时间冗余:它是视频序列中经常包含的冗余,由于视频的相邻图像往往包含相同或类似的背景和运动物体,只是运动物体所在的空间位置略有不同,这种相邻图像间的数据的高度相关性就称为时间冗余。时间冗余主要是通过帧间预测技术来消除,帧间预测技术是指利用时间上相邻图像的像素来预测当前像素。
视觉冗余:人眼视觉***对图像细节的变化不敏感,这些细微变化信息即使丢失,人眼也感受不到。而在记录原始视频数据时,通常假定视觉***是对各种内容的敏感度是一致的,这样就产生了比理想编码更多的数据,称为视觉冗余。视觉冗余主要是通过变换,量化技术来消除,该变换技术是指将图像信号变换到频率域进行处理,根据不同频率信号对视觉质量的贡献大小进行数据表达和比特再分配,这样可以纠正空间域上均匀采样的不合理表达。同时在比特再分配过程中融合考虑去除视觉冗余的需要,通过量化操作,省略过分精细的高频分量表达,实现有效压缩。
信息熵冗余:由信息论可知,为表示图像数据的一个像素,只要按其信息熵的大小分配相应比特数即可,而对于图像数据的每个像素,在图像获取时很难得到它的信息熵,因此一般是对每个像素采用相同的比特数来表示,这样必然存在冗余。信息熵冗余主要是通过熵编码技术来消除熵编码技术是通过统计系数的信息熵分布,为具有不同信息熵的数据分配不同的比特数。
当前主流的视频压缩编码架构是混合编码架构,针对上述冗余,采取不同的技术来消除冗余,并将这些技术结合在一起,形成了视频编码的混合架构。编码端接收到原始视频之后,对于构成原始视频的每帧图像,将该图像划分成当前待编码图像块。对于当前待编码图像块,首先通过参考图像块对当前待编码图像块进行预测,得到当前待编码图像块的预测信号;用当前待编码图像块的原始信号减去预测信号,得到残差信号。经过预测后,残差信号的幅值远小于原始信号。将残差信号进行变换和量化 操作。经过变换量化后,得到变换量化系数,再通过熵编码技术编码量化系数以及编码中的其他指示信息,得到码流。
解码端获取到码流之后,首先对码流进行熵解码,得到当前待重建图像块的变换量化系数,然后对变换量化系数进行反量化和反变换,得到当前待重建图像块的重建的残差信号。通过参考图像块对当前待重建图像块进行预测,得到当前待重建图像块的预测信号,然后将预测信号和上述重建的残差信号相加,得到当前待重建图像块的重建信号,然后根据该重建信号得到当前待重建图像块对应的重建图像块。
其中,编码端可以采用帧内预测技术或帧间预测技术,生成当前待编码图像块的预测信号。在现有技术中,帧间预测过程是:编码端针对每种帧间变换模式,编码端确定目标重建图像块,将该目标重建图像块的运动信息作为待编码重建图像块的运动信息,该运动信息包括:目标重建图像块所在的参考图像帧的索引,以及,目标重建图像块与该目标重建图像块的参考图像块之间的空间位移,即运动矢量;编码端根据运动信息,对当前待编码图像块之前的某一图像帧中对应位置上的图像块进行运动补偿,得到编码端的预测信号。对于部分帧间预测模式,编码端通过在各个参考图像帧中搜索,以确定率失真编码代价最小的重建图像块作为上述目标重建图像块。对于另一部分帧间预测模式,如合并(Merge)模式,无需编码端在各个参考图像帧中不断搜索,而是在当前待编码图像块的左上重建图像块、上方重建图像块、右上重建图像块、左边重建图像块和左下重建图像块中,根据某种准则确定上述目标重建图像块。
如上所述,得到当前待编码图像块的预测信号之后,根据当前待编码图像块的预测信号和原始信号,得到残差信号。接着对残差信号进行变换、量化和熵编码,得到码流。现有技术中,通常将多核变换组成多个变换核组,如在H.266视频编码标准中包括DST1、DST2、DCT5、DST7和DCT8组成的多个变换核组,其中,每个变换核组包括水平变换核和垂直变换核,针对每种帧间变换模式,通过每个变换核组对当前待编码图像块进行变换,编码端都要根据率失真代价最小准则,选择最优的变换核组,即需要对当前待编码图像块进行RDO。由于现有技术需要对当前待编码图像块进行多次RDO,从而造成编码复杂度较高的问题。
为了解决上述技术问题,本申请提供一种图像编解码方法、装置、***及存储介质。具体地,图1为本申请一实施例提供的图像编码方法的流程图,如图1所示,该方法包括如下步骤:
步骤S101:确定当前待编码图像块的目标重建图像块的位置信息;
步骤S102:根据目标重建图像块的位置信息,确定第一变换核组;
步骤S103:根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。
针对步骤S101进行说明:编码端采用的帧间变换模式可以是Merge模式、模板匹配(Template Match,TM)模式或高级运动向量预测(advanced motion vector prediction,AMVP)模式等。
该目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块,可选地,该目标重建图像块的运动信息可以作为当前待编码图像块的运动信息,该运动信息包括:目标重建图像块所在的参考图像帧的索引,以及,目标重建图像块与该目标 重建图像块的参考图像块之间的空间位,即运动矢量;其中,编码端可以根据该运动信息,对当前待编码图像块之前的某一图像帧中对应位置上的图像块进行运动补偿,得到编码端的预测信号。进一步地,编码端可确定该目标重建图像块的位置信息。
其中目标重建图像块的位置如下,图2为本申请一实施例提供的目标重建图像块的位置示意图,如图2所示,当前待编码图像块的目标重建图像块的位置可以是在当前待编码图像块的左上方、上方、右上方、左侧或左下方等,本申请对目标重建图像块的位置不做限制。进一步地,当目标重建图像块的位置不同时,编码端根据该目标重建图像块确定的参考图像块也不尽相同,由于该参考图像块用于生成当前待编码图像块的预测信号,基于此,编码端生成的当前待编码图像块的残差信号也不同,即当前待编码图像块的残差分布不同,具体地,图3A至图3E为本申请一实施例提供的当前待编码图像块的残差分布示意图,如图3A所示,若目标重建图像块在当前待编码图像块的左上方,则当前待编码图像块包括的左上角像素与该目标重建图像块具有运动一致性,因此,编码端对当前待编码图像块包括的左上角像素的预测更为准确,即残差较小,基于此,当前待编码图像块的残差随箭头指向方向,由小至大发生变化。如图3B所示,若目标重建图像块在当前待编码图像块的上方,则当前待编码图像块包括的上方像素与该目标重建图像块具有运动一致性,因此,编码端对当前待编码图像块包括的上方像素的预测更为准确,即残差较小,基于此,当前待编码图像块的残差随箭头指向方向,由小至大发生变化。如图3C所示,若目标重建图像块在当前待编码图像块的右上方,则当前待编码图像块包括的右上角像素与该目标重建图像块具有运动一致性,因此,编码端对当前待编码图像块包括的右上角像素的预测更为准确,即残差较小,基于此,当前待编码图像块的残差随箭头指向方向,由小至大发生变化。如图3D所示,若目标重建图像块在当前待编码图像块的左侧,则当前待编码图像块包括的左侧像素与该目标重建图像块具有运动一致性,因此,编码端对当前待编码图像块包括的左侧像素的预测更为准确,即残差较小,基于此,当前待编码图像块的残差随箭头指向方向,由小至大发生变化。如图3E所示,若目标重建图像块在当前待编码图像块的左下方,则当前待编码图像块包括的左下方像素与该目标重建图像块具有运动一致性,因此,编码端对当前待编码图像块包括的左下方像素的预测更为准确,即残差较小,基于此,当前待编码图像块的残差随箭头指向方向,由小至大发生变化。
针对步骤S102进行说明:在步骤S102之前,编码端可获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系。基于此,步骤S102具体包括:根据对应关系和目标重建图像块的位置信息,确定第一变换核组。
一种情况,编码端可以直接根据对应关系和目标重建图像块的位置信息,确定第一变换核组。以Merge模式为例,至少一个重建图像块的位置信息与至少一个变换核组的对应关系见表1,该对应关系包括:目标重建图像块的位置信息与第一变换核组的对应关系。其中,该表1所示对应关系是根据当前待编码图像块的残差分布与变换核组的特性确定的,例如:DCT2更适用于残差部分均匀的图像块,DST7更适用于在水平方向上,残差分布从左到右逐渐增大的图像块。需要说明的是,本申请所述的多个位置信息与多个变换核组的对应关系不限于表1所示的内容。
表1
基于此,步骤S102可通过查表的方式实现。例如:当目标重建图像块在当前待编码图像块的左侧时,则确定的第一变换核组是(DST7,DCT2);当目标重建图像块在当前待编码图像块的上方时,则确定的第一变换核组是(DCT2,DST7);当目标重建图像块在当前待编码图像块的右上方时,则确定的第一变换核组是(DCT8,DST7);当目标重建图像块在当前待编码图像块的左下方时,则确定的第一变换核组是(DST7,DCT8);当目标重建图像块在当前待编码图像块的左上方时,则确定的第一变换核组是(DST7,DST7)。
另一种情况,编码端可以间接根据对应关系和目标重建图像块的位置信息,确定第一变换核组。具体地,图4为本申请另一实施例提供的确定第一变换核组的方法流程图,如图4所示,该方法包括如下流程:
步骤S401:编码端根据对应关系和目标重建图像块的位置信息,确定至少两个第二变换核组;
步骤S402:分别根据至少两个第二变换核组中的每个第二变换核组,对当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;
步骤S403:将对应率失真代价中最小的第二变换核组,作为第一变换核组。
具体地,以Merge模式为例,如表2所示,重建图像块的至少一个位置信息与至少一个变换核组的对应关系见表2,该对应关系包括:目标重建图像块的位置信息与两个第二变换核组的对应关系。其中,表2所示的对应关系是根据当前待编码图像块的残差分布与变换核组的特性确定的,例如:DCT2更适用于残差部分均匀的图像块,DST7更适用于在水平方向上,残差分布从左到右逐渐增大的图像块。需要说明的是,本申请所述的多个位置信息与多个变换核组的对应关系不限于表2所示的内容。
表2
基于此,步骤S102可通过查表的方式实现。例如:当目标重建图像块在当前待编码图像块的左侧时,则确定的两个第二变换核组是(DST7,DST7),(DST7,DCT8);当目标重建图像块在当前待编码图像块的上方时,则确定的两个第二变换核组是(DST7,DST7),(DCT8,DST7);当目标重建图像块在当前待编码图像块的右上方时,则确定的两个第二变换核组是(DST7,DST7),(DCT8,DST7);当目标重建图像块在当前待编码图像块的左下方时,则确定的两个第二变换核组是(DST7,DCT8),(DCT2,DCT8);当目标重建图像块在当前待编码图像块的左上方时,则确定的两个第二变换核组是(DST7,DST7),(DCT8,DST7)。其中,每个变换核组中的第一项表示水平方向上的变换核,第二项表示垂直方向上的变换核。
进一步地,当编码端确定了两个变换核组之后,编码端通过率失真方法,计算当前待编码图像块对应的率失真代价,并选择率失真代价最小的变换核组作为第一变换核组。
再以TM模式为例,图5为本申请一实施例提供的当前待编码图像块的残差分布示意图,如图5所示,当前待编码图像块的上方重建图像块(可以理解为目标重建图像块)和左侧重建图像块(可以理解为目标重建图像块)构成一个倒置“L”型模板,编码端通过该“L”型模板可以确定当前待编码图像块的运动信息,进一步地,编码端可根据该运动信息得到当前待编码图像块的参考图像块,该参考图像块用于生成待编码图像块的预测信号。基于此,当前待编码图像块中越靠近该“L”型模板的像素,残 差越小,相反,越远离“L”型模板的像素,则残差越大。基于该特征,在TM模式下,当前待编码图像块可对应两个第二变换核组,分别是(DCT2,DCT2)和(DST7,DST7)。这种情况下,可以计算当前待编码图像块在每个变换核组下,对应的率失真代价,最后率失真代价较小的变换核组作为第一变换核组。本申请对当前待编码图像块对应的第二变换核组的内容和个数不做限制。
在步骤S103中:对于只存在一种帧间变换模式的情况,编码端直接将该第一变换核组作为最优变换核组,并通过该第一变换核组对当前待编码图像块进行编码。对于存在多种帧间变换模式的情况,由于编码端针对每种帧间变换模式,已经确定了对应的第一变换核组,最终在这些第一变换核组中选择对应率失真代价最小的变换核组作为最优变换核组,通过该最优变换核组对当前待编码图像块进行编码。
需要说明的是,并不是在所有帧间变换模式下,都可以根据目标重建图像块的位置信息,确定第一变换核组,对于那些不能根据目标重建图像块的位置信息,确定第一变换核组的帧间变换模式,可以采用现有技术的方法,即尝试每个变换核组,确定对应率失真代价最小的变换核组为第一变换核组。或者,可以默认使用至少一个变换核组,如(DCT2,DCT2)等。
进一步地,可采用现有变换方式对当前待编码图像块进行变换。本申请对此不再赘述。
综上,本申请提供一种图像编码方法,包括:在帧间变换模式下,确定当前待编码图像块的目标重建图像块的位置信息;根据目标重建图像块的位置信息,确定第一变换核组;根据第一变换核组,对当前待编码图像块的残差信号进行变换。使得编码端无需尝试多个变换核组,从而降低编码复杂度,提高编码效率。
进一步地,对当前待编码图像块的残差信号变换完之后,编码端还可以对当前待编码图像块的变换系数进行量化,得到量化后的变换系数,该系数也被称为变换量化系数或量化系数,进一步地,编码端对该量化后的变换系数和辅助编码信息进行熵编码,得到对应码流。具体地,图6为本申请一实施例提供的图像编解码方法的交互流程图,如图6所示,该方法包括如下步骤:
步骤S601:编码端在帧间变换模式下,确定当前待编码图像块的目标重建图像块的位置信息;
步骤S602:编码端根据目标重建图像块的位置信息,确定第一变换核组;
步骤S603:编码端根据第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数;
其中,步骤S601至步骤S603,与,步骤S101至步骤S103相同,其解释说明可参考步骤S101至步骤S103的解释说明,本申请对此不再赘述。
步骤S604:编码端对变换系数进行量化,得到当前待编码图像块的量化后的变换系数;
步骤S605:编码端对量化后的变换系数和当前待编码图像块的辅助编码信息进行熵编码,得到码流;
步骤S606:解码端获取码流;
步骤S607:对所述码流进行熵解码,以得到当前待编码图像块的量化后的变换系 数,并对量化后的变换系数进行反量化,得到当前待重建图像块的变换系数;
步骤S608:确定当前待重建图像块的目标重建图像块的位置信息,其中,该目标重建图像块是用于确定当前待重建图像块的运动信息的重建图像块;
步骤S609:解码端根据目标重建图像块的位置信息,确定第一变换核组;
步骤S610:解码端根据第一变换核组,对变换系数进行反变换,以重建当前待重建图像块。
针对步骤S604进行说明:量化的过程就是将动态范围较大的输入值强行规划成较少的输出值的过程。因为量化输入值范围较大,需要较多的比特数表示,而“强行规划”后的输出值范围较小,从而只需要少量的比特数即可表示。每个量化输入被归一化到一个量化输出,即量化到某个数量级中,这些数量级通常被称之为量化级(通常由编码端规定)。本申请提及的量化过程可采用现有的量化技术,本申请对此不再赘述。
针对步骤S605进行说明:本申请所述的当前待编码图像块的辅助编码信息是指在熵编码过程中所涉及的,除当前待编编码图像块的量化后的变换系数之外的其他所有信息,例如包括:帧间变换模式的标识、指示信息(可选地),该指示信息用于指示第一变换核组的索引,辅助编码信息还可以包括:当前待编码图像块的数据及相关联数据。相关联数据可包含序列参数集(Sequence Paramater Set,SPS)、图片参数集(Picture Paramater Set,PPS)及其它语法结构。SPS可含有应用于零个或多个序列的参数。PPS可含有应用于零个或多个图片的参数。语法结构是指码流中以指定次序排列的零个或多个语法元素的集合。进一步地,辅助编码信息还可以包括:当前待编码图像块的编码模式、当前待编码图像块的块划分信息以及当前待编码图像块的其他指示信息等,本申请提及的熵编码过程可采用现有的熵编码技术,本申请对此不再赘述。
需要说明的是,在本申请中,编码端和解码端中都可以获取如表1或表2所示的对应关系,由于表1和表2所示的对应关系所包括的变换核组个数不同,基于此,辅助编码信息携带的内容也不尽相同。具体如下:
可选方式一,若编码端最终进行编码时,采用的帧间变换模式为可以通过目标重建图像块的位置信息,确定第一变换核组的模式,且该帧间变换模式涉及的对应关系如表1所示,即一个位置对应一个变换核组,则辅助编码信息携带编码端最终确定的帧间变换模式的标识信息。
基于此,结合步骤S607至步骤S609进行说明:解码端的熵解码和反量化过程可以采用现有技术的方法,本申请对此不再赘述。解码端只需获知帧间变换模式的标识信息,根据该标识信息可以确定对应的帧间变换模式和如表1所示的对应关系。此外,解码端还可以根据码流重建当前待重建图像块,该当前待重建图像块与当前待编码图像块对应,即当前待重建图像块重建之后,得到的就是当前待编码图像块的原始图像块(通常存在残差);解码端也可以采用与编码端同样的规则,获知当前待重建图像块的目标重建图像块,该目标重建图像块的运动信息可作为当前待重建图像块的运动信息,通过该运动信息可得到当前待重建图像块的参考图像块,参考图像块用于生成当前待重建图像块的预测信号。所谓“解码端也可以采用与编码端同样的规则,获知当前待重建图像块的目标重建图像块”是指,若编码端确定的目标重建图像块为当前 待编码图像块的左上角重建图像块,则解码端确定的目标重建图像块也是当前待重建图像块的左上角重建图像块。基于此,解码端获取到目标重建图像块之后,解码端可以确定该目标重建图像块的位置信息。通过查表1的方式,确定第一变换核组。
这样做的有益效果是:对于编码端来讲,无需传输任何指示信息,解码端即可确定第一变换核组,而现有技术中,编码端需要传输每个变换核组对应的标识,以H.266标准为例,假设存在5个变换核组,在此情况下,编码端需要传输3比特的标识信息,以使解码端确定第一变换核组。因此通过该方法可以降低编码端的开销。
可选方式二,若编码端最终进行编码时,采用的帧间变换模式为可以通过目标重建图像块的位置信息,确定第一变换核组的模式,且该帧间变换模式涉及的对应关系如表2所示,即一个位置对应两个变换核组,则助编码信息携带编码端最终确定的帧间变换模式的标识信息以及指示信息。该指示信息用于指示第一变换核组的索引;例如:索引为1或者为2。当索引为1时,表示当解码端确定两个第二变换核组时,解码端最终选择两个第二变换核组中的第一个作为第一变换核组,当索引为2时,表示当解码端确定两个第二变换核组时,解码端最终选择两个第二变换核组中的第二个作为第一变换核组。
基于此,结合步骤S607至步骤S609进行说明:解码端的熵解码和反量化过程可以采用现有技术的方法,本申请对此不再赘述。解码端需获知帧间变换模式的标识信息和指示信息,根据该标识信息可以确定对应的帧间变换模式和如表2所示的对应关系。此外,解码端还可以根据码流重建当前待重建图像块,解码端也可以采用与编码端同样的规则,获知当前待重建图像块的目标重建图像块,该目标重建图像块的运动信息可作为当前待重建图像块的运动信息,通过该运动信息可以得到当前待重建图像块的参考图像块,参考图像块用于生成当前待重建图像块的预测信号。所谓“解码端也可以采用与编码端同样的规则,获知待重建图像块的目标重建图像块”是指,若编码端确定的目标重建图像块为当前待编码图像块的左上角重建图像块,则解码端确定的目标重建图像块也是当前待重建图像块的左上角重建图像块。基于此,解码端获取到目标重建图像块之后,解码端可以确定该目标重建图像块的位置信息。通过查表2的方式,确定两个第二变换核组,这种情况下,解码端无需再计算两个第二变换核组分别对应的率失真代价,来确定第一变换核组,而是根据指示信息确定第一变换核组。
这样做的有益效果是:一方面,对于编码端来讲,仅需传输1比特的指示信息,使得解码端可以确定第一变换核组,而现有技术中,编码端需要传输每个变换核组对应的标识,以H.266标准为例,指示存在5个变换核组,在此情况下,编码端需要传输3比特的标识信息,以使解码端确定第一变换核组,因此通过该方法可以降低编码端的开销。另一方面,对于解码端来讲,无需计算两个第二变换核组分别对应的率失真代价,来确定第一变换核组,而是根据指示信息确定第一变换核组。从而降低了解码复杂度,进而提高了解码效率。
针对步骤S610进行说明:对于只存在一种帧间变换模式的情况,解码端直接将该第一变换核组作为最优变换核组,并通过该第一变换核组对当前待重建图像块进行反变换,以重建当前待重建图像块。对于存在多种帧间变换模式的情况,由于解码端针对每种帧间变换模式,已经确定了对应的第一变换核组,最终在这些第一变换核组中 选择对应率失真代价最小的变换核组作为最优变换核组,通过该最优变换核组对当前待重建图像块进行反变换,以重建当前待重建图像块。
进一步地,可采用现有反变换方式对当前待重建图像块进行反变换。本申请对此不再赘述。
综上,通过本申请提供的方法,一方面可以降低编码端的开销,另一方面可以降低解码复杂度,进而提高解码效率。
为了更好的说明本申请的技术效果,下面提供本申请技术方案对应的性能数据和现有技术对应的性能数据。
现有技术:现有技术1已被标准采纳,目前集成在联合探索模型(Joint Exploration Model,JEM)中,该技术在原有H.265标准的变换基础上,H.266视频编码标准引入了新的变换核DST7和DCT8,并进行组合,形成了多核变换的技术,该技术具体如下:
步骤1:默认变换核组的率失真代价计算
这一步使用原有H.265标准的变换核组,对当前待编码图像块进行变换,量化,熵编码,计算出率失真代价。
步骤2:多核变换核组的率失真代价计算
例如:H.266标准中使用(DCT,DCT2)对当前待编码图像块进行变换,然后经过量化、熵编码,计算出率失真代价(RDCost1)。如果计算的率失真代价大于步骤1计算的已尝试模式的最小率失真代价的1.1倍,则跳过多核变换核组的率失真代价计算过程,选择原有H.265标准的变换核组对当前待编码图像块进行编码,否则,用(DCT8,DCT8),(DST7,DST7),(DCT8,DST7),(DST7,DCT8)四组变换核组分别对当前待编码图像块进行变换,然后经过量化、熵编码,计算出四组率失真代价,分别为RDCost2、RDCost3、RDCost4、RDCost5。
步骤3:变换核组的选择
结合步骤1和步骤2,得到了若干个率失真代价,步骤3是根据率失真代价最小的准则,为当前待编码图像块确定最优的变换核组。
例如:在H.266中,在上述步骤中,得到5个率失真代价,这一步,比较5个率失真代价的大小,选择最小的率失真代价所对应的变换核组作为当前块的最优变换核组,并标记。
如果RDCost1<min{RDCOst2,RDCos3,RDCost4,RDCost5},则选择(DCT2,DCT2)
如果RDCost2<min{RDCOst1,RDCos3,RDCost4,RDCost5},则选择(DCT8,DCT8)
如果RDCost3<min{RDCOst2,RDCos1,RDCost4,RDCost5},则选择(DCT8,DST7)
如果RDCost4<min{RDCOst2,RDCos3,RDCost1,RDCost5},则选择(DST7,DCT8)
如果RDCost5<min{RDCOst2,RDCos3,RDCost4,RDCost1},则选择(DST7,DST7)
注释:min{A,B}=A,表示选择A和B的最小者。
步骤4:熵编码
这一步根据需要将上两步选择哪组变换核组的标记信息传输到解码端,因此需要熵编码的过程,二进制的编码采用基于上下文的算术编码,上下文的初始模型个数与二进制位或者当前待编码图像块的深度有关。
例如:在H.266标准中,如果当前待编码图像块的最优变换核组为(DCT2,DCT2),则标记为1,否则标记为0,表示使用多核变换技术。在多核变换中选择(DCT8,DCT8),则当前块标记为0,选择(DCT8,DST7)标记为1,选择(DST7,DCT8)标记为2,选择(DST7,DST7),标记为3,并且二进制化如下表3,然后采用主流的基于上下文的算术编码对二进制化的比特进行编码,编码采用2个上下文模型,对于是否使用多核变换的技术的标记位也采用基于上下文的算术编码,编码采用5个上下文模型。
表3
变换核组 | 标记 | 二进制化 |
(DCT8,DCT8) | 0 | ‘00’ |
(DCT8,DST7) | 1 | ‘10’ |
(DST7,DCT8) | 2 | ‘01’ |
(DST7,DST7) | 3 | ‘11’ |
对于解码端根据解析出的语法信息即能判断使用哪组变换核组,无需经过RDO过程确定,具体过程为编码端熵编码过程的逆过程,上下文模型与编码端相同。
实例一:针对Merge模式,假设存在如表1所示的对应关系,针对其他帧间变换模式,编码端默认使用变换核组(DCT2,DCT2)。
实例二:针对Merge模式,假设存在如表2所示的对应关系;针对TM模式,存在两个变换核组(DCT2,DCT2)和(DST7,DST7);对于其他帧间变换模块,可以采用现有技术的方法,即尝试每个变换核组,确定对应率失真代价最小的变换核组为第一变换核组。
实例三:针对Merge模式,假设存在如表2所示的对应关系;针对TM模式,存在两个变换核组(DCT2,DCT2)和(DST7,DST7);其他帧间变换模块,也存在两个变换核组(DCT2,DCT2)和(DST7,DST7)。
实例一和实例二在JEM中实现,并进行了全序测试,测试序列为H.266标准的通测序列,测试条件为联合视频探索小组(Joint VideoExploration Team,JVET)指定的通测条件,随机接入(RandomAccess,RA)、LDP、LDB结构下,采用BD-RATE作为编码性能(编码性能指在相同的编码重建视频质量下的比特节省)的评价指标,负值表示编码性能有所提高,正值表示编码性能有所损失,测试序列为420的YUV格式,对应有三组数据,分别是亮度(Y)、色度(U、V)分量,复杂度用编解码端的时间衡量,其中编码端的时间称为编码时间(Encodeing Time,EncT),解码端的时间称为解码时间(Decoding Time,DecT),下面的Class表示测试序列集,具体是视频,Overall(Ref)表示平均值。
1、现有技术的性能见表4至表6
表4
表5
表6
测试条件:JEM4.0编码序列长度1秒,帧率上转换(Frame Rate Up-Conversion,FRUC)和帧间EMT打开,其它工具关闭。
2、实例一的编码性能见表7至表9
表7
表8
表9
测试条件:JEM4.0编码序列长度1秒,FRUC和帧间EMT打开,其它工具关闭。
3、实例二的编码性能见表10至表12
表10
表11
表12
测试条件:JEM5.0.1编码序列长度1秒,FRUC和帧间EMT打开,其它工具关闭。
其中,第2-6列为实例二相对于JEM中的EMT OFF的性能,第7-11列为实例二相对于JEM的性能。
表13
Y | U | V | |
Class A1 | |||
Class A2 | |||
Class B | 0.08% | -0.07% | -0.41% |
Class C | -0.05% | -0.24% | -0.37% |
Class D | -0.24% | -0.12% | -0.16% |
Class E | -0.32% | 0.32% | -0.05% |
Overall(Ref) | -0.11% | -0.05% | -0.27% |
测试条件:LDB JEM5.0.1通测条件的性能
表14
Y | U | V | |
Class A1 | |||
Class A2 | |||
Class B | 0.03% | 0.23% | 0.46% |
Class C | -0.07% | -0.25% | -0.63% |
Class D | -0.18% | -0.25% | 0.43% |
Class E | -0.23% | 0.01% | -0.12% |
Overall(Ref) | -0.10% | -0.05% | 0.07% |
测试条件:LDP JEM5.0.1通测条件的性能
4、实例三的编码性能见表15至表17
表15
表16
表17
测试条件:JEM5.0.1编码序列长度1秒,FRUC和帧间EMT打开,其它工具关闭。
其中,第2-6列为实例三在JEM的性能数据,第7-11列为现有技术在JEM的性能数据。
从现有技术和实例一的性能数据可以得知,本申请相对于现有技术,在损失较小的性能的前提下,大幅度降低了编码的复杂度,很好的实现了编码复杂度和编码性能 的平衡。
实例二仅对merge模式和TM模式的采用位置依赖帧间变换,从实例二的性能数据可以得出,在LDP和LDB配置下几乎没有编码损失的前提下,编码端降低了编码复杂度(约3%-6%),RA配置下同样降低了编码的复杂度,实现了性能和复杂度更好的平衡,特别地,在JVET指定的通测条件下可以提高LDP和LDB编码配置下的编码性能,对于亮度分量平均有0.11%和0.10%的性能提升。
实例三不仅对merge模式和TM模式采用位置依赖帧间变换,对其他帧间模式均采用两次变换,对比实例三的性能数据可以得出,编码端在损失很小的性能下,将JEM中多核变换技术的复杂度降低了50%左右,实现了编码性能和编码复杂度很好的平衡。
图7为本申请一实施例提供的一种图像编码装置70的结构示意图,该装置70可以是编码器、包括编码器的终端设备或与编码器具有类似功能的芯片等,如图7所示,该装置70包括:
第一确定模块71,用于确定当前待编码图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待编码图像块的运动信息的重建图像块。
第二确定模块72,用于根据第一确定模块71确定的目标重建图像块的位置信息,确定第一变换核组。
变换模块73,用于根据第二确定模块72确定的第一变换核组,对当前待编码图像块的残差信号进行变换,得到变换系数。
可选地,该图像编码装置还包括:获取模块74,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系。
相应的,所述第二确定模块72,具体用于根据对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
可选地,第二确定模块72具体用于:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;分别根据所述至少两个第二变换核组中的每个第二变换核组,对所述当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;将对应最小的率失真代价的第二变换核组,作为所述第一变换核组。
可选地,如图7所示,图像编码装置70还包括:
量化模块75,用于对所述变换系数进行量化,得到所述当前待编码图像块的量化后的变换系数。
熵编码模块76,用于对所述量化后的变换系数和所述当前待编码图像块的辅助编码信息进行熵编码,得到码流。其中,所述辅助编码信息包括:所述帧间变换模式的标识和指示信息,所述指示信息用于指示所述第一变换核组的索引。
本申请提供的图像编码装置,可执行上述的图像编码方法,对应内容和效果可参考图像编码方法的内容和效果,本申请对此不再赘述。
图8为本申请一实施例提供的一种图像解码装置80的结构示意图,该装置80可以是解码器、包括解码器的终端设备或与解码器具有类似功能的芯片等,如图8所示,该装置80包括:
熵解码模块81,用于对码流进行熵解码,以得到当前待重建图像块的量化后的变 换系数。
反量化模块82,用于对量化后的变换系数进行反量化,得到所述当前待重建图像块的变换系数。
第一确定模块83,用于确定当前待重建图像块的目标重建图像块的位置信息,其中,目标重建图像块是用于确定当前待重建图像块的运动信息的重建图像块。
第二确定模块84,用于根据目标重建图像块的位置信息,确定第一变换核组。
反变换模块85,用于根据第一变换核组,对变换系数进行反变换,以重建当前待重建图像块。
可选地,装置80还包括:
获取模块86,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系。
相应的,所述第二确定模块84,具体用于根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
可选地,所述码流还携带指示信息,所述指示信息用于指示所述第一变换核组的索引;
相应的,所述第二确定模块84具体用于:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;根据所述第一变换核组的索引在所述至少两个第二变换核组中,确定所述第一变换核组。
本申请提供的图像解码装置,可执行上述的图像解码方法,对应内容和效果可参考图像解码方法的内容和效果,本申请对此不再赘述。
本申请一实施例提供一种图像编解码***,也被称为视频编解码***,该***包括:如图7所示的图像编码装置,以及图8所示的图像解码装置。其中,该***可设置在同一视频编解码装置或电子设备中,在该电子设备中图像编码装置和图像解码装置可以集成在一起,可统称为编码解码器。或者,该***设置在不同的装置中,下面结合附图对这两种情况分别进行说明:
图9是视频编解码装置或电子设备50的示意性框图,该装置或者电子设备可以并入上述的编码解码器。图10是根据本申请实施例的用于视频编码的示意性装置图。下面将说明图9和图10中的单元。
电子设备50可以例如是无线通信***的移动终端或者用户设备。应理解,可以在可能需要对视频图像进行编码和解码,或者编码,或者解码的任何电子设备或者装置内实施本申请的实施例。
电子设备50可以包括用于并入和保护设备的壳30。电子设备50还可以包括形式为液晶显示器的显示器32。在本申请的其它实施例中,显示器可以是适合于显示图像或者视频的任何适当的显示器技术。电子设备50还可以包括小键盘34。在本申请的其它实施例中,可以运用任何适当的数据或者用户接口机制。例如,可以实施用户接口为虚拟键盘或者数据录入***作为触敏显示器的一部分。电子设备50可以包括麦克风36或者任何适当的音频输入,该音频输入可以是数字或者模拟信号输入。电子设备50还可以包括如下音频输出设备,该音频输出设备在本申请的实施例中可以是以下各项中的任何一项:耳机38、扬声器或者模拟音频或者数字音频输出连接。电子设备50 也可以包括电池40,在本申请的其它实施例中,设备可以由任何适当的移动能量设备,比如太阳能电池、燃料电池或者时钟机构生成器供电。电子设备50还可以包括用于与其它设备的近程视线通信的红外线端口42。在其它实施例中,电子设备50还可以包括任何适当的近程通信解决方案,比如蓝牙无线连接或者USB/火线有线连接。
电子设备50可以包括用于控制电子设备50的控制器56或者处理器。控制器56可以连接到存储器58,该存储器在本申请的实施例中可以存储形式为图像的数据和音频的数据,和/或也可以存储用于在控制器56上实施的指令。控制器56还可以连接到适合于实现音频和/或视频数据的编码和解码或者由控制器56实现的辅助编码和解码的编码解码器电路54。
电子设备50还可以包括用于提供用户信息并且适合于提供用于在网络认证和授权用户的认证信息的读卡器48和智能卡46,例如UICC和UICC读取器。
电子设备50还可以包括无线电接口电路52,该无线电接口电路连接到控制器并且适合于生成例如用于与蜂窝通信网络、无线通信***或者无线局域网通信的无线通信信号。电子设备50还可以包括天线44,该天线连接到无线电接口电路52用于向其它(多个)装置发送在无线电接口电路52生成的射频信号并且用于从其它(多个)装置接收射频信号。
在本申请的一些实施例中,电子设备50包括能够记录或者检测单帧的相机,编码解码器54或者控制器接收到这些单帧并对它们进行处理。在本申请的一些实施例中,装置可以在传输和/或存储之前从另一设备接收待处理的视频图像数据。在本申请的一些实施例中,电子设备50可以通过无线或者有线连接接收图像用于编码/解码。
应理解,图10仅为电子设备50及其包含的各个软、硬件模块的示意图,具有多种不同的实现方式,比如,其中小键盘34可以是触摸屏,并且该触摸屏可以是显示器32的一部分,不做限定。
图11是根据本申请实施例的视频编解码***10的示意性框图。如图11所示,视频编解码***10包含源装置12及目的地装置14。源装置12产生经编码视频数据。因此,源装置12可被称作视频编码装置或视频编码设备或如本申请所述的图像编码装置。目的地装置14可解码由源装置12产生的经编码视频数据。因此,目的地装置14可被称作视频解码装置或视频解码设备或如本申请所述的图像解码装置。源装置12及目的地装置14可为视频编解码装置或视频编解码设备的实例。源装置12及目的地装置14可包括广泛范围的装置,包含台式计算机、移动计算装置、笔记本(例如,膝上型)计算机、平板计算机、机顶盒、智能电话等手持机、电视、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机,或其类似者。
目的地装置14可经由信道16接收来自源装置12的编码后的视频数据。信道16可包括能够将经编码视频数据从源装置12移动到目的地装置14的一个或多个媒体及/或装置。在一个实例中,信道16可包括使源装置12能够实时地将编码后的视频数据直接发射到目的地装置14的一个或多个通信媒体。在此实例中,源装置12可根据通信标准(例如,无线通信协议)来调制编码后的视频数据,且可将调制后的视频数据发射到目的地装置14。所述一个或多个通信媒体可包含无线及/或有线通信媒体,例如射频(RF)频谱或一根或多根物理传输线。所述一个或多个通信媒体可形成基于包的网络 (例如,局域网、广域网或全球网络(例如,因特网))的部分。所述一个或多个通信媒体可包含路由器、交换器、基站,或促进从源装置12到目的地装置14的通信的其它设备。
在另一实例中,信道16可包含存储由源装置12产生的编码后的视频数据的存储媒体。在此实例中,目的地装置14可经由磁盘存取或卡存取来存取存储媒体。存储媒体可包含多种本地存取式数据存储媒体,例如蓝光光盘、DVD、CD-ROM、快闪存储器,或用于存储经编码视频数据的其它合适数字存储媒体。
在另一实例中,信道16可包含文件服务器或存储由源装置12产生的编码后的视频数据的另一中间存储装置。在此实例中,目的地装置14可经由流式传输或下载来存取存储于文件服务器或其它中间存储装置处的编码后的视频数据。文件服务器可以是能够存储编码后的视频数据且将所述编码后的视频数据发射到目的地装置14的服务器类型。实例文件服务器包含web服务器(例如,用于网站)、文件传送协议(FTP)服务器、网络附加存储(NAS)装置,及本地磁盘驱动器。
目的地装置14可经由标准数据连接(例如,因特网连接)来存取编码后的视频数据。数据连接的实例类型包含适合于存取存储于文件服务器上的编码后的视频数据的无线信道(例如,Wi-Fi连接)、有线连接(例如,DSL、缆线调制解调器等),或两者的组合。编码后的视频数据从文件服务器的发射可为流式传输、下载传输或两者的组合。
本申请的技术不限于无线应用场景,示例性的,可将所述技术应用于支持以下应用等多种多媒体应用的视频编解码:空中电视广播、有线电视发射、***发射、流式传输视频发射(例如,经由因特网)、存储于数据存储媒体上的视频数据的编码、存储于数据存储媒体上的视频数据的解码,或其它应用。在一些实例中,视频编解码***10可经配置以支持单向或双向视频发射,以支持例如视频流式传输、视频播放、视频广播及/或视频电话等应用。
在图11的实例中,源装置12包含视频源18、视频编码器20及输出接口22。在一些实例中,输出接口22可包含调制器/解调器(调制解调器)及/或发射器。视频源18可包含视频俘获装置(例如,视频相机)、含有先前俘获的视频数据的视频存档、用以从视频内容提供者接收视频数据的视频输入接口,及/或用于产生视频数据的计算机图形***,或上述视频数据源的组合。
视频编码器20可编码来自视频源18的视频数据。在一些实例中,源装置12经由输出接口22将编码后的视频数据直接发射到目的地装置14。编码后的视频数据还可存储于存储媒体或文件服务器上以供目的地装置14稍后存取以用于解码及/或播放。
在图11的实例中,目的地装置14包含输入接口28、视频解码器30及显示装置32。在一些实例中,输入接口28包含接收器及/或调制解调器。输入接口28可经由信道16接收编码后的视频数据。显示装置32可与目的地装置14整合或可在目的地装置14外部。一般来说,显示装置32显示解码后的视频数据。显示装置32可包括多种显示装置,例如液晶显示器(LCD)、等离子体显示器、有机发光二极管(OLED)显示器或其它类型的显示装置。
视频编码器20及视频解码器30可根据视频压缩标准(例如,高效率视频编解码H.265标准)而操作,且可遵照HEVC测试模型(HM)。H.265标准的文本描述 ITU-TH.265(V3)(04/2015)于2015年4月29号发布,可从http://handle.itu.int/11.1002/1000/12455下载,所述文件的全部内容以引用的方式并入本文中。
本申请提供一种存储介质,包括指令,当所述指令在图像编码装置上运行时,使得所述图像编码装置执行如上所述的图像编码方法。
本申请提供一种存储介质,包括指令,当所述指令在图像解码装置上运行时,使得所述图像解码装置执行如上所述的图像解码方法。
本申请提供一种计算机程序产品,当所述计算机程序产品在图像编码装置上运行时,使得所述图像编码装置执行如上所述的图像编码方法。
本申请提供一种计算机程序产品,当所述计算机程序产品在图像解码装置上运行时,使得所述图像解码装置执行如上所述的图像解码方法。
Claims (17)
- 一种图像编码方法,其特征在于,包括:确定当前待编码图像块的目标重建图像块的位置信息,其中,所述目标重建图像块是用于确定所述当前待编码图像块的运动信息的重建图像块;根据所述目标重建图像块的位置信息,确定第一变换核组;根据所述第一变换核组,对所述当前待编码图像块的残差信号进行变换,得到变换系数。
- 根据权利要求1所述的方法,其特征在于,所述根据所述目标重建图像块的位置信息,确定第一变换核组之前,还包括:获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,所述根据所述目标重建图像块的位置信息,确定第一变换核组,包括:根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
- 根据权利要求2所述的方法,其特征在于,所述根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组,包括:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;分别根据所述至少两个第二变换核组中的每个第二变换核组,对所述当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;将对应最小的率失真代价的第二变换核组,作为所述第一变换核组。
- 根据权利要求3所述的方法,其特征在于,所述根据所述第一变换核组,对所述当前待编码图像块的残差信号进行变换,得到变换系数之后,还包括:对所述变换系数进行量化,得到所述当前待编码图像块的量化后的变换系数;对所述量化后的变换系数和所述当前待编码图像块的辅助编码信息进行熵编码,得到码流;其中,所述辅助编码信息包括:帧间变换模式的标识和指示信息,所述指示信息用于指示所述第一变换核组的索引。
- 一种图像解码方法,其特征在于,包括:对码流进行熵解码,以得到当前待重建图像块的量化后的变换系数,并对所述量化后的变换系数进行反量化,得到所述当前待重建图像块的变换系数;确定所述当前待重建图像块的目标重建图像块的位置信息,其中,所述目标重建图像块是用于确定所述当前待重建图像块的运动信息的重建图像块;根据所述目标重建图像块的位置信息,确定第一变换核组;根据所述第一变换核组,对所述变换系数进行反变换,以重建所述当前待重建图像块。
- 根据权利要求5所述的方法,其特征在于,所述根据所述目标重建图像块的位置信息,确定第一变换核组之前,还包括:获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,所述根据所述目标重建图像块的位置信息,确定第一变换核组,包括:根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
- 根据权利要求6所述的方法,其特征在于,所述码流还携带指示信息,所述指示信息用于指示所述第一变换核组的索引;相应的,所述根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组,包括:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;根据所述第一变换核组的索引在所述至少两个第二变换核组中,确定所述第一变换核组。
- 一种图像编码装置,其特征在于,包括:第一确定模块,用于确定当前待编码图像块的目标重建图像块的位置信息,其中,所述目标重建图像块是用于确定所述当前待编码图像块的运动信息的重建图像块;第二确定模块,用于根据所述目标重建图像块的位置信息,确定第一变换核组;所述变换模块,用于根据所述第一变换核组,对所述当前待编码图像块的残差信号进行变换,得到变换系数。
- 根据权利要求8所述的装置,其特征在于,还包括:获取模块,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,所述第二确定模块,具体用于根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
- 根据权利要求9所述的装置,其特征在于,所述第二确定模块具体用于:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;分别根据所述至少两个第二变换核组中的每个第二变换核组,对所述当前待编码图像块进行编码,并确定经过编码后的当前待编码图像块对应的率失真代价;将对应最小的率失真代价的第二变换核组,作为所述第一变换核组。
- 根据权利要求10所述的装置,其特征在于,还包括:量化模块,用于对所述变换系数进行量化,得到所述当前待编码图像块的量化后的变换系数;熵编码模块,用于对所述量化后的变换系数和所述当前待编码图像块的辅助编码信息进行熵编码,得到码流;其中,所述辅助编码信息包括:帧间变换模式的标识和指示信息,所述指示信息用于指示所述第一变换核组的索引。
- 一种图像解码装置,其特征在于,包括:熵解码模块,用于对码流进行熵解码,以得到当前待重建图像块的量化后的变换系数;反量化模块,用于对所述量化后的变换系数进行反量化,得到所述当前待重建图像块的变换系数;第一确定模块,用于确定所述当前待重建图像块的目标重建图像块的位置信息, 其中,所述目标重建图像块是用于确定所述当前待重建图像块的运动信息的重建图像块;第二确定模块,用于根据所述目标重建图像块的位置信息,确定第一变换核组;反变换模块,用于根据所述第一变换核组,对所述变换系数进行反变换,以重建所述当前待重建图像块。
- 根据权利要求12所述的装置,其特征在于,还包括:获取模块,用于获取至少一个重建图像块的位置信息与至少一个变换核组的对应关系;相应的,所述第二确定模块,具体用于根据所述对应关系和所述目标重建图像块的位置信息,确定所述第一变换核组。
- 根据权利要求13所述的装置,其特征在于,所述码流还携带指示信息,所述指示信息用于指示所述第一变换核组的索引;相应的,所述第二确定模块具体用于:根据所述对应关系和所述目标重建图像块的位置信息,确定至少两个第二变换核组;根据所述第一变换核组的索引在所述至少两个第二变换核组中,确定所述第一变换核组。
- 一种图像编解码***,其特征在于,包括:如权利要求8至11任一项所述的图像编码装置,以及如权利要求12至14任一项所述的图像解码装置。
- 一种存储介质,包括指令,其特征在于,当所述指令在图像编码装置上运行时,使得所述图像编码装置执行如权利要求1-4任一项所述的方法。
- 一种存储介质,包括指令,其特征在于,当所述指令在图像解码装置上运行时,使得所述图像解码装置执行如权利要求5-7任一项所述的方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/890,779 US11528507B2 (en) | 2017-12-13 | 2020-06-02 | Image encoding and decoding method, apparatus, and system, and storage medium to determine a transform core pair to effectively reduce encoding complexity |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711327990.X | 2017-12-13 | ||
CN201711327990.XA CN109922340B (zh) | 2017-12-13 | 2017-12-13 | 图像编解码方法、装置、***及存储介质 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/890,779 Continuation US11528507B2 (en) | 2017-12-13 | 2020-06-02 | Image encoding and decoding method, apparatus, and system, and storage medium to determine a transform core pair to effectively reduce encoding complexity |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019114294A1 true WO2019114294A1 (zh) | 2019-06-20 |
Family
ID=66820706
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/099255 WO2019114294A1 (zh) | 2017-12-13 | 2018-08-07 | 图像编解码方法、装置、***及存储介质 |
Country Status (3)
Country | Link |
---|---|
US (1) | US11528507B2 (zh) |
CN (1) | CN109922340B (zh) |
WO (1) | WO2019114294A1 (zh) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10931954B2 (en) * | 2018-11-20 | 2021-02-23 | Sony Corporation | Image coding modes selection for an embedded codec circuitry |
CN114071140B (zh) * | 2019-12-03 | 2022-12-23 | 杭州海康威视数字技术股份有限公司 | 解码方法、编码方法、装置、设备及机器可读存储介质 |
US11343512B1 (en) | 2021-01-08 | 2022-05-24 | Samsung Display Co., Ltd. | Systems and methods for compression with constraint on maximum absolute error |
WO2023044919A1 (zh) * | 2021-09-27 | 2023-03-30 | Oppo广东移动通信有限公司 | 视频编解码方法、设备、***、及存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103299642A (zh) * | 2011-01-07 | 2013-09-11 | Lg电子株式会社 | 编码和解码图像信息的方法和使用该方法的装置 |
CN105531999A (zh) * | 2013-07-09 | 2016-04-27 | 诺基亚技术有限公司 | 涉及用于信号传输运动信息的语法的视频编码方法及装置 |
CN106412601A (zh) * | 2010-09-02 | 2017-02-15 | Lg电子株式会社 | 编码和解码视频的方法和使用该方法的装置 |
CN106464878A (zh) * | 2014-05-14 | 2017-02-22 | 联发科技(新加坡)私人有限公司 | 数据压缩的备选转换的方法 |
CN107211144A (zh) * | 2015-01-26 | 2017-09-26 | 高通股份有限公司 | 用于预测残差的增强型多重变换 |
Family Cites Families (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101098473B (zh) * | 2006-06-30 | 2012-05-09 | 联想(北京)有限公司 | 一种图像编码方法及装置 |
CN101127903B (zh) * | 2007-09-29 | 2010-12-08 | 华南理工大学 | 基于dct的分辨率可伸缩图像编解码方法 |
KR101834825B1 (ko) * | 2009-01-27 | 2018-03-06 | 톰슨 라이센싱 | 비디오 인코딩 및 디코딩에서 변환 선택을 위한 방법 및 장치 |
KR102036118B1 (ko) * | 2010-01-12 | 2019-10-24 | 엘지전자 주식회사 | 비디오 신호의 처리 방법 및 장치 |
CN102202217A (zh) * | 2010-03-22 | 2011-09-28 | 北京中星微电子有限公司 | 一种jpeg编码方法和设备 |
US9661338B2 (en) * | 2010-07-09 | 2017-05-23 | Qualcomm Incorporated | Coding syntax elements for adaptive scans of transform coefficients for video coding |
CN101895756B (zh) * | 2010-07-15 | 2012-10-31 | 北京大学 | 视频图像块的编码、解码、重构方法及*** |
US8885701B2 (en) * | 2010-09-08 | 2014-11-11 | Samsung Electronics Co., Ltd. | Low complexity transform coding using adaptive DCT/DST for intra-prediction |
CN102065298B (zh) * | 2011-01-05 | 2012-10-24 | 西安电子科技大学 | 高性能宏块编码实现方法 |
CN102595112B (zh) * | 2011-01-12 | 2014-02-19 | 北京大学 | 视频编码中编码和重建图像块的方法 |
US9894386B2 (en) * | 2012-04-12 | 2018-02-13 | Goldpeak Innovations Inc. | Transform method based on block information, and apparatus using said method |
CN103581688B (zh) * | 2012-08-02 | 2017-09-12 | 北京大学 | 一种视频图像的编码和解码方法及装置 |
US9674530B1 (en) * | 2013-04-30 | 2017-06-06 | Google Inc. | Hybrid transforms in video coding |
CN103347188B (zh) * | 2013-07-24 | 2016-06-15 | 南京邮电大学 | 一种分布式视频编码非关键帧压缩感知编解码方法 |
CN104519368B (zh) * | 2013-09-30 | 2017-12-01 | 华为技术有限公司 | 图像编码、解码重构处理方法及装置 |
CN104702962B (zh) * | 2015-03-03 | 2019-04-16 | 华为技术有限公司 | 帧内编解码方法、编码器和解码器 |
US20170064298A1 (en) * | 2015-09-02 | 2017-03-02 | Blackberry Limited | Video coding with delayed reconstruction |
US10277905B2 (en) * | 2015-09-14 | 2019-04-30 | Google Llc | Transform selection for non-baseband signal coding |
CN110326294A (zh) * | 2017-01-03 | 2019-10-11 | Lg电子株式会社 | 使用二次变换编码/解码视频信号的方法和设备 |
US11405645B2 (en) * | 2017-02-28 | 2022-08-02 | Google Llc | Transform kernel selection and entropy coding |
JP2019017066A (ja) * | 2017-07-03 | 2019-01-31 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 符号化装置、復号装置、符号化方法及び復号方法 |
-
2017
- 2017-12-13 CN CN201711327990.XA patent/CN109922340B/zh active Active
-
2018
- 2018-08-07 WO PCT/CN2018/099255 patent/WO2019114294A1/zh active Application Filing
-
2020
- 2020-06-02 US US16/890,779 patent/US11528507B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106412601A (zh) * | 2010-09-02 | 2017-02-15 | Lg电子株式会社 | 编码和解码视频的方法和使用该方法的装置 |
CN103299642A (zh) * | 2011-01-07 | 2013-09-11 | Lg电子株式会社 | 编码和解码图像信息的方法和使用该方法的装置 |
CN105531999A (zh) * | 2013-07-09 | 2016-04-27 | 诺基亚技术有限公司 | 涉及用于信号传输运动信息的语法的视频编码方法及装置 |
CN106464878A (zh) * | 2014-05-14 | 2017-02-22 | 联发科技(新加坡)私人有限公司 | 数据压缩的备选转换的方法 |
CN107211144A (zh) * | 2015-01-26 | 2017-09-26 | 高通股份有限公司 | 用于预测残差的增强型多重变换 |
Also Published As
Publication number | Publication date |
---|---|
CN109922340A (zh) | 2019-06-21 |
CN109922340B (zh) | 2021-10-15 |
US11528507B2 (en) | 2022-12-13 |
US20200296422A1 (en) | 2020-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11558623B2 (en) | Bidirectional inter prediction method and terminal based on motion vector difference reduction | |
US20210297682A1 (en) | Chroma prediction method and device | |
RU2668723C2 (ru) | Способ и оборудование для кодирования и декодирования видеосигналов | |
TWI622288B (zh) | 視訊解碼方法 | |
WO2018058622A1 (zh) | 一种帧间预测的图像编解码方法和装置 | |
US11528507B2 (en) | Image encoding and decoding method, apparatus, and system, and storage medium to determine a transform core pair to effectively reduce encoding complexity | |
WO2017129023A1 (zh) | 解码方法、编码方法、解码设备和编码设备 | |
WO2019001024A1 (zh) | 一种确定运动矢量预测值的方法以及相关设备 | |
US12034963B2 (en) | Compound prediction for video coding | |
WO2019109955A1 (zh) | 帧间预测方法、装置及终端设备 | |
WO2019128716A1 (zh) | 图像的预测方法、装置及编解码器 | |
US9781447B1 (en) | Correlation based inter-plane prediction encoding and decoding | |
WO2019091302A1 (en) | Image prediction method and apparatus | |
WO2018120420A1 (zh) | 一种基于模板匹配的预测方法及装置 | |
WO2018119609A1 (zh) | 一种基于模板匹配的编解码方法及装置 | |
WO2019233423A1 (zh) | 获取运动矢量的方法和装置 | |
US11706444B2 (en) | Inter prediction method and apparatus | |
JP2022084683A (ja) | 映像符号化方法、映像復号方法、および端末 | |
WO2018120230A1 (zh) | 图像滤波方法、装置以及设备 | |
WO2018120290A1 (zh) | 一种基于模板匹配的预测方法及装置 | |
US20240251005A1 (en) | Method, apparatus, and medium for video processing | |
WO2023081820A1 (en) | Method, apparatus, and medium for media processing | |
WO2023137284A2 (en) | Method, apparatus, and medium for video processing | |
WO2023159143A2 (en) | Method, apparatus, and medium for video processing | |
CN118138770A (zh) | 视频处理方法、装置、电子设备和存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18889217 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18889217 Country of ref document: EP Kind code of ref document: A1 |