CN115988201A - Method, apparatus, electronic device and storage medium for encoding film grain - Google Patents
Method, apparatus, electronic device and storage medium for encoding film grain Download PDFInfo
- Publication number
- CN115988201A CN115988201A CN202310239786.1A CN202310239786A CN115988201A CN 115988201 A CN115988201 A CN 115988201A CN 202310239786 A CN202310239786 A CN 202310239786A CN 115988201 A CN115988201 A CN 115988201A
- Authority
- CN
- China
- Prior art keywords
- coding unit
- current
- distortion
- unit
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000009466 transformation Effects 0.000 claims description 46
- 238000004364 calculation method Methods 0.000 claims description 30
- 238000012937 correction Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 230000001131 transforming effect Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 8
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000013139 quantization Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a method, a device, an electronic device and a storage medium for encoding film grain, wherein the method comprises the following steps: calculating coding cost through a reconstruction coding unit according to pixel domain distortion and subjective distortion of an original coding unit and the reconstruction coding unit after the reconstruction coding unit; the predictive coding mode is then selected according to the coding cost. The invention realizes coding under the condition of keeping film grains, and simultaneously avoids the problems of nonuniform film grains, inconsistent definition of different frames and the like caused by reconstructing the film grains by using the traditional method.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, an electronic device, and a storage medium for encoding film grain.
Background
Conventional film grain is formed during the development of motion picture images. Film grain, while a random noise, can be present to make the video content more realistic and natural. There is an artistic need, particularly in the film industry, for controlling grain texture, which is considered part of the film authoring process and intent, moderate grain texture can help improve subjective perception, and can also be used to mask coding artifacts resulting from compression. However, with the widespread use of digital cameras, digital image video does not have film grain due to the different imaging principles. A viewer, when viewing image video without film grain, creates a perception that, although the rendered picture is sharp, it appears to lack realism. Therefore, in order to provide the presented content with a texture, the content producer may wish to preserve the original film grain in the image video, and may even actively add film grain to the image video content to create unique cinema features.
However, film grain is inherently high frequency noise and has temporal randomness, different from other types of noise, both in distribution and size. This randomness makes it difficult for conventional coding techniques to efficiently compress it, and at the same time, prediction becomes very difficult and the accuracy of motion estimation is reduced. Film grain is not well preserved in current video coding standards. In the encoding process, the original image video is subjected to filtering and lossy compression processing, and the encoder suppresses high-frequency signals such as film grains as noise and cannot reconstruct the high-frequency signals in the decoding process. If the film grain is reserved by performing lossless compression on the image video, huge code stream is consumed, and the practicability is lacked in practical application.
The current solution to film grain coding is to remove the film grain first, then encode the video with the film grain removed, model the film grain at the same time, and finally re-synthesize the film grain by modeling parameters during the decoding process. The new generation of coding standard AV1 first writes film grain processing tools into the standard, which also works by removing film grain by a de-noising process and estimating film grain parameters that will be sent to the decoder along with the compressed video stream. After decoding, the film grain is then synthesized and added to the reconstructed video frame. However, such a process has the problem that the reconstructed film grain is not uniform enough and the sharpness of different frames is not uniform, and particularly when processing already compressed video, it is difficult to reconstruct the film grain again. The above problems can be largely avoided if the film grain can be directly subjected to reserve compression. Direct-preserving compression of film grain presents challenges to image video compression algorithms, again due to the randomness of the film grain.
Disclosure of Invention
In view of the above problems with the prior art methods, embodiments of the present invention provide a method, apparatus, electronic device and storage medium for encoding film grain.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides a method for encoding film grain, comprising:
and traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
And calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
And calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
And selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
In a second aspect, embodiments of the present invention provide an apparatus for encoding film grain, comprising:
and the coding module is used for traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
And the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
And the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
And the mode selection module is used for selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
In a third aspect, embodiments of the present invention also provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method for encoding film grain as described in the first aspect when executing the computer program.
In a fourth aspect, embodiments of the present invention also provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of encoding film grain as described in the first aspect.
According to the technical scheme, the method and the device realize the encoding under the condition of keeping the film grains, and simultaneously avoid the problems of non-uniform film grains, inconsistent definition of different frames and the like which can occur when the film grains are reconstructed by using a traditional method.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is an exemplary diagram of a CTU partitioning a CU in the HEVC video coding standard.
Fig. 2 is an exemplary diagram of a CU divided into PUs and TUs in the HEVC video coding standard.
Fig. 3 is a flow chart of a method for encoding film grain according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of h.265/HEVC prediction mode.
Fig. 5 is a schematic structural diagram of a film grain encoding apparatus according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following further describes embodiments of the present invention with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
For a better understanding of the following examples, the basic concept is explained first.
Video is composed of a sequence of images. A video frame is an image sequence in a video, and each image in the video can be regarded as one frame. The current video frame is the video frame that needs to be encoded currently. Currently, a hybrid Coding framework is generally adopted in mainstream video Coding standards, including h.264/AVC and h.265/HEVC, in a video Coding process, a current video frame is block-coded, taking h.265/HEVC as an example, as shown in fig. 1, a block division mode in h.265/HEVC is a quad-Tree division mode, a video frame in a video is divided into Coding Tree Units (CTUs), the size of a Coding Tree is generally set to 64 × 64, each CTU can be further uniformly divided into 4 Coding Units (CUs), and one CU can be recursively divided into 4 small CUs according to a quad-Tree structure. As shown in fig. 2, each CU may include one or more Prediction Units (PUs) with different sizes, and each PU may further include several Transform Units (TUs), and when a TU is quantized, the TU is generally divided into multiple Coefficient Groups (CGs), and the quantization process is performed for each CG, i.e., quantization Unit.
The transformation of the image is to transform the spatial domain signal into the frequency domain signal, effectively removing the correlation of the signal and concentrating most of the energy into the low frequency region. Commonly used transform methods such as DCT transform, DST transform, hadamard transform, etc. The transform compresses the energy of the matrix into a first element, called the Direct Current (DC) coefficient. The remaining coefficients are referred to as Alternating Current (AC) coefficients. After transformation, the energy of the transformation coefficient is mainly concentrated in the upper left region. The coefficients in this region are very significant and tend to concentrate image information to which the human eye is sensitive. After the transform coefficients are quantized, the quantized coefficients in the upper left region are significant, and most of the quantized coefficients in the lower right region are zero. Quantization is one of the sources of distortion in video/image compression, and the parts with zero quantization coefficients are often compressed high frequency signals, which include film grain.
Fig. 3 is a flow chart illustrating a method of encoding film grain in accordance with an embodiment of the present invention, and as shown in fig. 3, the method of encoding film grain in accordance with an embodiment of the present invention includes the following:
In this step, it can be understood that the current mainstream coding standards are all based on the hybrid video coding framework. The hybrid framework mainly includes prediction (prediction), transformation (transform), quantization (quantization), entropy coding (entropy coding), and the like. The prediction unit generates a prediction pixel corresponding to an original pixel of a current coding unit by using a reconstructed pixel of the coded area. The coding unit where the original pixel is located is a current original coding unit, and the coding unit where the predicted pixel is located is a reconstruction coding unit. The prediction modes include intra prediction (intra prediction) and inter prediction (inter prediction). The specific coding prediction modes vary from one video standard to another. As shown in fig. 4, taking h.265/HEVC as an example, 35 intra prediction modes are specified in HEVC, including Planar mode, DC mode, and 33 angular modes. One or more coding prediction modes may be set in practical applications.
And 2, calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
The pixel domain distortion of the current original coding unit and the current reconstructed coding unit may be calculated by using an algorithm such as Sum of Absolute Differences (SAD), sum of hadamard Transformed Absolute differences (SATD), sum of Squared Differences (SSD), mean Absolute Differences (MAD), and Mean Squared Differences (MSD).
The calculating the subjective distortion of the current original coding unit and the current reconstruction coding unit includes:
in one embodiment, the subjective distortion of the current original coding unit and the current reconstructed coding unit can be represented by an energy difference value of the current original coding unit and the current reconstructed coding unit, and includes:
and carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit.
And transforming the original transformation unit to obtain the AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain the AC coefficient of the reconstruction transformation unit.
Wherein the transformation includes, but is not limited to: DCT transform, hadamard transform, DST transform.
Calculating the energy difference value of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion and abs is an absolute value function.
Further, in order to more accurately measure the subjective distortion between the current original coding unit and the current reconstructed coding unit, the subjective distortion calculation method may be further modified as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficient of the reconstruction transformation unit, distortion is subjective distortion, abs is an absolute value function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon The number of non-zero transform coefficients in the coding unit is currently reconstructed.
In another embodiment, the subjective distortion of the current original coding unit and the current reconstructed coding unit can be expressed by a pixel variance difference between the current original coding unit and the current reconstructed coding unit, and the calculation formula is as follows:
wherein, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current re-coding unit, N is the pixel number in the current coding unit, distortion is subjective distortion, and abs is an absolute value function.
And 3, calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
The calculation formula of the coding cost is as follows:
wherein D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
Further, the higher the subjective distortion between the current original coding unit and the current reconstructed coding unit, the greater the probability that film grain is removed. To avoid removing film grain, the subjective distortion needs to be weighted up, and the coding cost is corrected as follows:
wherein k is a correction factor, k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
Further, film grain tends to be weaker for flat areas, and the human eye is relatively less sensitive to film grain in such areas. The calculation of the coding cost can also be modified based on the level of film grain in order to more accurately preserve the film grain.
The correcting the calculation of the coding cost according to the intensity degree of the film grain comprises the following steps:
and judging the strength of the film grain of the current original coding unit.
And according to the judgment result, correcting the calculation of the coding cost:
if the current original coding unit is judged to be the area with weak film grain degree, the calculation formula of the coding cost is as follows:
wherein m is a correction factor, m is less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
If the current original coding unit is judged to be the area with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor, k is larger than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrange multiplier, and R is the current coding rate.
The determining the level of the film grain of the current original coding unit includes:
calculating the pixel variance of the current original coding unit:
wherein, X is the pixel value in the current original coding unit, X' is the pixel average value in the current original coding unit, and N is the pixel number in the coding unit.
And presetting a pixel variance threshold T of the current original coding unit.
And if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is an area with weaker film grain degree, and if not, judging that the current original coding unit is an area with stronger film grain degree.
And 4, selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
Fig. 5 is a schematic diagram of an apparatus for encoding film grain according to an embodiment of the present invention, as shown in fig. 5:
and the coding module is used for traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
And the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
The pixel domain distortion of the current original coding unit and the current reconstruction coding unit may be calculated by using algorithms such as Sum of Absolute Differences (SAD), sum of hadamard Transformed Absolute differences (SATD), sum of Squared Differences (SSD), mean Absolute Differences (MAD), and Mean Squared Differences (MSD).
The calculating the subjective distortion of the current original coding unit and the current reconstructed coding unit includes:
in one embodiment, the subjective distortion of the current original coding unit and the current reconstructed coding unit can be represented by an energy difference value of the current original coding unit and the current reconstructed coding unit, and includes:
and carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit.
And transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit.
Wherein the transformation includes, but is not limited to: DCT transform, hadamard transform, DST transform.
Calculating the energy difference value of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion and abs is an absolute value function.
Further, in order to more accurately measure the subjective distortion between the current original coding unit and the current reconstructed coding unit, the subjective distortion calculation method may be further modified as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion, abs is an absolute value function, N scr For the number of non-zero transform coefficients, N, of the current original coding unit recon The number of non-zero transform coefficients in the current reconstructed coding unit.
In another embodiment, the subjective distortion of the current original coding unit and the current reconstructed coding unit can be expressed by a pixel variance difference between the current original coding unit and the current reconstructed coding unit, and the calculation formula is as follows:
wherein, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current re-coding unit, N is the pixel number in the current coding unit, distortion is subjective distortion, and abs is an absolute value function.
And the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
The calculation formula of the coding cost is as follows:
wherein D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrange multiplier, and R is current coding rate.
Further, the higher the subjective distortion between the current original coding unit and the current reconstructed coding unit, the greater the probability that film grain is removed. To avoid removing film grain, the subjective distortion needs to be weighted up, and the coding cost is corrected as follows:
wherein k is a correction factor, k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
Further, film grain tends to be weaker for flat areas, and the human eye is relatively less sensitive to film grain in such areas. The calculation of the coding cost can also be modified based on the level of film grain in order to more accurately preserve the film grain.
The correcting the calculation of the coding cost according to the intensity degree of the film grain comprises the following steps:
and judging the intensity degree of the film grain of the current original coding unit.
And according to the judgment result, correcting the calculation of the coding cost:
if the current original coding unit is judged to be the area with weak film grain degree, the calculation formula of the coding cost is as follows:
wherein m is a correction factor, m is less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
If the current original coding unit is judged to be the area with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor, k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
The determining the level of the film grain of the current original coding unit includes:
calculating the pixel variance of the current original coding unit:
wherein, X is the pixel value in the current original coding unit, X' is the pixel average value in the current original coding unit, and N is the number of pixels in the coding unit.
And presetting a pixel variance threshold T of the current original coding unit.
If the pixel variance of the current original coding unit is smaller than the variance threshold T, the current original coding unit is judged to be an area with a weak film grain degree, otherwise, the current original coding unit is judged to be an area with a strong film grain degree.
And the mode selection module is used for selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
Based on the same inventive concept, another embodiment of the present invention provides an electronic device, as shown in fig. 6, which specifically includes the following contents: a processor 601, a memory 602, a communication interface 603, and a communication bus 604.
The processor 601, the memory 602 and the communication interface 603 complete mutual communication through the communication bus 604; the communication interface 603 is used for implementing information transmission between devices.
The processor 601 is operative to invoke a computer program in the memory 602 which, when executed, performs all of the steps of the above-described method of encoding film grain.
Based upon the same inventive concept, yet another embodiment of the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, performs all of the steps of the above-described method of encoding film grain.
The logic instructions in the memory may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention. One of ordinary skill in the art can understand and implement it without inventive effort.
Moreover, in embodiments of the present invention, relational terms such as "first" and "second", and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Reference throughout this specification to "one embodiment," "another embodiment," or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments. Furthermore, the various embodiments and features of the various embodiments described in this specification can be combined and combined by one skilled in the art without contradiction.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (20)
1. A method of encoding film grain comprising:
traversing a coding prediction mode for a current original coding unit to obtain a current reconstruction coding unit;
calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit;
calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion;
and selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
2. The method of claim 1, wherein said calculating pixel domain distortion for said current original coding unit and said current reconstructed coding unit uses an algorithm comprising:
sum of absolute errors, sum of hadamard transformed absolute errors, sum of squared differences, mean absolute errors, mean squared errors.
3. The method of claim 1, wherein said calculating subjective distortions of said current original coding unit and said current reconstructed coding unit comprises:
carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit;
transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit;
calculating the subjective distortion of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion and abs is an absolute value function.
4. The method of claim 3, wherein the subjective distortion of the current original coding unit and the current reconstructed coding unit is calculated based on the AC coefficients of the original transform unit and the AC coefficients of the reconstructed transform unit, and wherein the formula is further:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficient of the reconstruction transformation unit, distortion is subjective distortion, abs is an absolute value function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon The number of non-zero transform coefficients in the coding unit is currently reconstructed.
5. The method of claim 3, wherein said transforming comprises: DCT transform, hadamard transform, DST transform.
6. The method of claim 1, wherein said calculating subjective distortions for said current original coding unit and said current reconstructed coding unit further comprises:
wherein, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current re-coding unit, N is the pixel number in the current coding unit, distortion is subjective distortion, and abs is an absolute value function.
7. The method of encoding film grain as in claim 1 wherein said calculating an encoding cost for a current coding unit based on said pixel domain distortion and said subjective distortion is by the formula:
wherein k is a correction factor, k is not less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
8. The method of claim 1 wherein said calculating a coding cost for a current coding unit based on said pixel domain distortion and said subjective distortion, wherein said modifying the calculation of the coding cost based on the level of film grain comprises:
judging the strength of the film grain of the current original coding unit;
and according to the judgment result, correcting the calculation of the coding cost:
if the current original coding unit is judged to be the area with weak film grain degree, the calculation formula of the coding cost is as follows:
wherein m is a correction factor, m is less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate;
if the current original coding unit is judged to be the area with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor, k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
9. The method of claim 8, wherein said determining the level of film grain strength of said current original coding unit comprises:
calculating the pixel variance of the current original coding unit:
wherein, X is the pixel value in the current original coding unit, X' is the pixel average value in the current original coding unit, and N is the pixel number in the coding unit;
presetting a pixel variance threshold T of the current original coding unit;
and if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is an area with weaker film grain degree, and if not, judging that the current original coding unit is an area with stronger film grain degree.
10. An apparatus for encoding film grain, comprising:
the encoding module is used for traversing the encoding prediction mode of the current original encoding unit to obtain a current reconstruction encoding unit;
the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit;
the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion;
and the mode selection module is used for selecting the corresponding coding prediction mode with the minimum coding cost for coding according to the coding cost.
11. The apparatus of claim 10, wherein said computing pixel-domain distortion for said current original coding unit and said current reconstructed coding unit uses an algorithm comprising:
sum of absolute errors, sum of absolute errors of hadamard transform, sum of squared differences, mean absolute errors, mean squared errors.
12. The apparatus for encoding film grain as recited in claim 10, wherein said calculating subjective distortions for said current original coding unit and said current reconstructed coding unit comprises:
carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit;
transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit;
calculating the subjective distortion of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion and abs is an absolute value function.
13. The apparatus according to claim 12, wherein said subjective distortion of said current original coding unit and said current reconstructed coding unit is calculated based on AC coefficients of said original transform unit and AC coefficients of said reconstructed transform unit, and wherein the formula is further:
wherein ,Esrc For the energy value of the current original coding unit, AC src Is the AC coefficient of the original transform unit; e recon For the energy value of the current reconstructed coding unit, AC recon For the AC coefficients of the reconstruction transformation unit, distortion is subjective distortion, abs is an absolute value function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon The number of non-zero transform coefficients in the coding unit is currently reconstructed.
14. The apparatus for encoding film grain according to claim 12, wherein said transforming comprises: DCT transform, hadamard transform, DST transform.
15. The apparatus for encoding film grain as recited in claim 10, wherein said calculating subjective distortions for said current original coding unit and said current reconstructed coding unit further comprises:
wherein, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current re-coding unit, N is the pixel number in the current coding unit, distortion is subjective distortion, and abs is an absolute value function.
16. The apparatus for encoding film grain of claim 10 wherein said calculating the coding cost for the current coding unit based on said pixel domain distortion and said subjective distortion is by the formula:
wherein k is a correction factor, k is not less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrange multiplier, and R is the current coding rate.
17. The apparatus for encoding film grain as defined in claim 10 wherein the correction of the calculation of the coding cost based on the degree of film grain in said calculating the coding cost of the current coding unit based on said pixel domain distortion and said subjective distortion comprises:
judging the strength of the film grain of the current original coding unit;
and correcting the calculation of the coding cost according to a judgment result:
if the current original coding unit is judged to be the area with weak film grain degree, the calculation formula of the coding cost is as follows:
wherein m is a correction factor, m is less than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate;
if the current original coding unit is judged to be the area with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor, k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is a Lagrange multiplier, and R is the current coding rate.
18. The apparatus for encoding film grain as in claim 17 wherein said determining the level of film grain of said current original coding unit comprises:
calculating the pixel variance of the current original coding unit:
wherein, X is the pixel value in the current original coding unit, X' is the pixel average value in the current original coding unit, and N is the number of pixels in the coding unit;
presetting a pixel variance threshold T of the current original coding unit;
if the pixel variance of the current original coding unit is smaller than the variance threshold T, the current original coding unit is judged to be an area with a weak film grain degree, otherwise, the current original coding unit is judged to be an area with a strong film grain degree.
19. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements the method of encoding film grain as recited in any one of claims 1 through 9.
20. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method of encoding film grain according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310239786.1A CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310239786.1A CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115988201A true CN115988201A (en) | 2023-04-18 |
| CN115988201B CN115988201B (en) | 2023-05-30 |
Family
ID=85961138
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310239786.1A Active CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115988201B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101883280A (en) * | 2009-05-07 | 2010-11-10 | 香港科技大学 | A kind of video coding-decoding method and system of recovering noise |
| CN103634600A (en) * | 2012-08-22 | 2014-03-12 | 中国科学院声学研究所 | Video coding mode selection method and system based on SSIM evaluation |
| US20140119454A1 (en) * | 2012-10-25 | 2014-05-01 | Magnum Semiconductor, Inc. | Rate-distortion optimizers and optimization techniques including joint optimization of multiple color components |
| US20160142711A1 (en) * | 2014-11-17 | 2016-05-19 | Vixs Systems, Inc. | Video coder with adaptive bac engine and methods for use therewith |
| CN107548556A (en) * | 2015-04-21 | 2018-01-05 | Vid拓展公司 | Video coding based on artistic intent |
| CN109982082A (en) * | 2019-05-05 | 2019-07-05 | 山东大学深圳研究院 | A kind of more distortion criterion Rate-distortion optimization methods of HEVC based on local grain characteristic |
| CN112381744A (en) * | 2020-10-27 | 2021-02-19 | 杭州微帧信息科技有限公司 | Adaptive preprocessing method for AV1 synthetic film grains |
| CN115174898A (en) * | 2022-06-27 | 2022-10-11 | 福州大学 | Rate distortion optimization method based on visual perception |
-
2023
- 2023-03-14 CN CN202310239786.1A patent/CN115988201B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101883280A (en) * | 2009-05-07 | 2010-11-10 | 香港科技大学 | A kind of video coding-decoding method and system of recovering noise |
| CN103634600A (en) * | 2012-08-22 | 2014-03-12 | 中国科学院声学研究所 | Video coding mode selection method and system based on SSIM evaluation |
| US20140119454A1 (en) * | 2012-10-25 | 2014-05-01 | Magnum Semiconductor, Inc. | Rate-distortion optimizers and optimization techniques including joint optimization of multiple color components |
| US20160142711A1 (en) * | 2014-11-17 | 2016-05-19 | Vixs Systems, Inc. | Video coder with adaptive bac engine and methods for use therewith |
| CN107548556A (en) * | 2015-04-21 | 2018-01-05 | Vid拓展公司 | Video coding based on artistic intent |
| CN109982082A (en) * | 2019-05-05 | 2019-07-05 | 山东大学深圳研究院 | A kind of more distortion criterion Rate-distortion optimization methods of HEVC based on local grain characteristic |
| CN112381744A (en) * | 2020-10-27 | 2021-02-19 | 杭州微帧信息科技有限公司 | Adaptive preprocessing method for AV1 synthetic film grains |
| CN115174898A (en) * | 2022-06-27 | 2022-10-11 | 福州大学 | Rate distortion optimization method based on visual perception |
Non-Patent Citations (2)
| Title |
|---|
| SEONGWAN KIM: "SSIM-based distortion metric for film grain noise in HEVC", SIGNAL, IMAGE AND VIDEO PROCESSING * |
| 杨春玲;肖冬琴;: "基于SSE和SSIM的H.264帧内预测模式选择改进算法", 电子与信息学报, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115988201B (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11589041B2 (en) | Method and apparatus of neural network based processing in video coding | |
| CN111819854B (en) | Method and apparatus for coordinating multi-sign bit concealment and residual sign prediction | |
| JP2002531971A (en) | Image processing circuit and method for reducing differences between pixel values across image boundaries | |
| WO2002096118A2 (en) | Decoding compressed image data | |
| US8594189B1 (en) | Apparatus and method for coding video using consistent regions and resolution scaling | |
| CN115606179A (en) | CNN filter for learning-based downsampling for image and video coding using learned downsampling features | |
| EP4322523A1 (en) | Residual coding method and device, video coding method and device, and storage medium | |
| CN107113426B (en) | Method and apparatus for performing graph-based transformations using generalized graph parameters | |
| CN110944179B (en) | Video data decoding method and device | |
| CN115552905A (en) | Global skip connection based CNN filter for image and video coding | |
| JP2009153226A (en) | Decoding method | |
| CN114430904A (en) | Video compression using intra-loop image-level controllable noise generation | |
| CN112001854A (en) | Method for repairing coded image and related system and device | |
| CN115988201B (en) | Method, apparatus, electronic device and storage medium for encoding film grain | |
| CN111901595B (en) | Video coding method, device and medium based on deep neural network | |
| WO2021263251A1 (en) | State transition for dependent quantization in video coding | |
| CN116248895B (en) | Video cloud transcoding method and system for virtual reality panorama roaming | |
| US20230044603A1 (en) | Apparatus and method for applying artificial intelligence-based filtering to image | |
| WO2023132765A1 (en) | Filtering for image encoding and decoding | |
| US20200413105A1 (en) | Metric driven adaptive processing for video encoders | |
| WO2020248099A1 (en) | Perceptual adaptive quantization and rounding offset with piece-wise mapping function | |
| Yang et al. | Content adaptive spatial-temporal rescaling for video coding optimization | |
| WO2023102290A1 (en) | Method and apparatus for chart based mesh compression | |
| WO2020000487A1 (en) | Transformation method, inverse transformation method and apparatus | |
| US20200329232A1 (en) | Method and device for encoding or decoding video signal by using correlation of respective frequency components in original block and prediction block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |