CN115665413A - Method for estimating optimal quantization parameter of image compression - Google Patents
Method for estimating optimal quantization parameter of image compression Download PDFInfo
- Publication number
- CN115665413A CN115665413A CN202211184629.7A CN202211184629A CN115665413A CN 115665413 A CN115665413 A CN 115665413A CN 202211184629 A CN202211184629 A CN 202211184629A CN 115665413 A CN115665413 A CN 115665413A
- Authority
- CN
- China
- Prior art keywords
- image
- quantization parameter
- neural network
- multilayer neural
- characteristic vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses an estimation method of an optimal quantization parameter of image compression, which comprises the following steps: step 1, selecting an image set, and carrying out image processing on each image I in the image set i Setting a target code rate B 0 Calculating an optimal quantization parameter QP of each image; step 2, calculating the complexity characteristic vector C of each image in the image set i (ii) a Step 3, constructing a multilayer neural network NN, and regarding the multilayer neural network NN, an input layer is an image complexity characteristic vector C i The output layer is a regression type quantization parameter QP i Training a multilayer neural network NN by adopting an image set; step by stepAnd 4, step 4: and for the image I ', calculating an image complexity characteristic vector C ' of the image I ', and inputting the image complexity characteristic vector C ' into the trained multilayer neural network NN so as to output the optimal quantization parameter estimation QP '. The invention can estimate the optimal image compression quantization parameter under the limited bandwidth, fully utilizes the bandwidth and improves the image transmission quality.
Description
Technical Field
The invention belongs to the technical field of image compression, and particularly relates to an estimation method of an optimal quantization parameter of image compression.
Background
High-throughput image transmission is realized on a channel with limited bandwidth, and original image data needs to be subjected to lossy compression. On one hand, the lossy compression result generated by the encoder must ensure that the size is within the range of bandwidth limitation, so that information loss is avoided; on the other hand, it should occupy as much bandwidth as possible to ensure better transmission image quality. In order to meet the above two requirements, various encoders are required to have a reasonable code rate control algorithm.
The existing image compression algorithm JPEG or JPEG2000 can be summarized into input data through three steps of time-frequency transformation, quantization and encoding: (1) Preprocessing, which generally refers to performing multi-component conversion on a color map or performing time-frequency conversion on data in image compression; (2) Quantization, namely, accuracy down-sampling is carried out on the original image data by utilizing the limitation of human eyes on resolution accuracy, so that the data volume is reduced, and the compression of subsequent data is facilitated; (3) And (4) coding, namely performing compression coding on data according to different image coding standards, and finally generating a compressed code stream. The operation of down-sampling the precision of the quantized partial data is a main cause of the image quality damage. The setting of the Quantization Parameter QP directly affects the accuracy of the quantized data, and thus further affects the size and image quality of the compressed data stream. Therefore, the reasonable setting of the quantization parameter is crucial to realizing the code rate control.
At present, under limited bandwidth, a code rate control algorithm for quantization parameter QP optimization in high-throughput image coding can ensure that the use of the bandwidth does not exceed a limit range, data is not lost, and image reconstruction is not influenced; but also can use enough bandwidth as far as possible to achieve better image quality, and present better visual perception and lower distortion rate. According to the rate distortion theory, under the limited bandwidth, the higher the complexity of the image content is, the larger the quantization parameter QP is, so that the specified code rate is not exceeded, otherwise, when the complexity of the image content is low, the smaller the quantization parameter QP can be, the bandwidth is fully utilized, and the image restoration quality is improved.
The method for calculating the complexity of the image content includes Entropy (Entropy) of the image, sum of absolute difference of transformation (SATD), and Gradient (Gradient) of the image. These characteristics have representativeness of a certain aspect, and due to the image type, the illumination condition and other reasons, the quantitative parameter estimation accuracy has a further improved space.
Entropy (Entropy) of an image is a concept in statistics, and can reflect the amount of information of a mean value of an image in image processing. The information entropy of the image is a measure of uncertainty of image content, and can reflect the complexity [1] of the image content to a certain extent, and the calculation formula is shown as formula (1). Where p represents the frequency with which a certain pixel value k appears in the entire image, see equation (2). When the probability p of each gray value is equal, the information entropy of the image takes the maximum value; when only one gray value appears in the image, i.e. p is 1, the information entropy of the image takes the minimum value of 0. In the actual image, the information entropy of the image is reflected to be larger when the gray value of one image fluctuates on the whole value interval, otherwise, the information entropy is smaller when the gray value is single. The information entropy of an image is therefore often considered to reflect the texture complexity of the image [1]. In many image or video coding environments, information entropy plays an important role in content coding [2].
Entropy＝-∑plog 2 p； (1)
The Sum of Absolute Transformed Differences (SATD) can reflect the size of residual signals of images and video frames, is commonly used as a measure of coding complexity in video coding, and has proved to have a strong linear relation with the code rate occupied by compressed video, so that the quantization parameter QP decision [3] [4] can be participated in video coding. The SATD transforms the residual signal to the frequency domain and then sums the absolute values as shown in equation (3), where H represents the hadamard matrix, M is the size of the square matrix, and X is the residual signal square matrix:
SATD＝∑ M ∑ M |HXH|； (3)
the Gradient (Gradient) of the image can reflect the position relation of pixel values in the image [1]Compared with an image, the image contains more spatially distributed details, which is equivalent to a two-dimensional characteristic parameter. The gradient is intended to mean that the rate of change of a function is greatest at a point and changes fastest. The introduction of the gradient parameter into the image processing becomes a parameter reflecting the bumps and edges in the image [5 ]]. The formula for calculating the image gradient is shown in formula (4), wherein I i,j For the pixel value of the image I at the position (I, j), the calculation result is determined by the difference between the horizontal pixels and the difference between the vertical pixels, and can reflect the structure information of the two-dimensional image:
due to the diversity of image contents, the optimal quantization parameter prediction under the condition of limited bandwidth still has a failure condition by depending on a single characteristic parameter of image complexity or a linear weighting method, and therefore, an optimal quantization parameter estimation method with higher prediction accuracy needs to be developed.
Reference documents:
[1] rafael c. Gonzalez, richard e.woods, steven l.eddins.
[2]Tu C,Tran T D.Context-based entropy coding of block transform coefficients for image compression[J].IEEE Transactions on Image Processing,2002,11(11):1271-1283.
[3]W.Gao,S.Kwong,Q.Jiang,C.-K.Fong,P.H.Wong,W.Y.Yuen,Data-driven rate ontrol for rate-distortion optimization in HEVC based on simplified effective nitial QP learning,IEEE Trans.Broadcast.65(1)(2018)94–108.
[4]M.Karczewicz and X.Wang,“Intra Frame Rate Control Based on SATD,Document:JCTVC-M0257,13th Meeting,Incheon,KR,Apr.2013.
[5]Chen G H,Yang C L,Xie S L.Gradient-based structural similarity for image quality assessment[C]//2006International Conference on Image Processing.IEEE,2006:2929-2932.
Disclosure of Invention
The invention aims to provide an estimation method of an optimal quantization parameter for image compression aiming at the defects of the prior art, the method can estimate the optimal quantization parameter for image compression under the limited bandwidth, fully utilizes the bandwidth and improves the quality of image transmission.
In order to solve the technical problems, the invention adopts the following technical scheme:
a method for estimating an optimal quantization parameter for image compression comprises the following steps:
step 2, calculating a complexity characteristic vector C of each image in the image set i ；
Step 3, constructing a multilayer neural network NN, and regarding the multilayer neural network NN, an input layer is an image complexity characteristic vector C i The output layer is a regression type quantization parameter QP i Training a multilayer neural network NN by adopting an image set, verifying and predicting the trained multilayer neural network NN, and selecting the multilayer neural network NN with the highest test precision;
and 4, step 4: and (3) calculating an image complexity characteristic vector C 'of the image I', and inputting the image complexity characteristic vector C 'into the multilayer neural network NN obtained in the step 3, so as to obtain an output optimal quantization parameter estimation QP'.
Further, the specific method in step 1 is as follows:
for each image I i Adjusting a quantization parameter QP of an image encoder; when image compression outputs file size S>B 0 While, the quantization parameter QP is continuously reduced until S does not exceed B 0 And is closest to B 0 At this time, the quantization parameter QP is the picture I i Corresponding target code rate B 0 Optimal quantization parameter QP of i 。
Further, in step 2, the feature vector C of image complexity i Is shown as a drawingThe Entropy of the image, encopy, the sum of absolute difference of transformation, SATD, and the gradient of the image.
Further, the multi-layer neural network NN includes an input layer, two or more hidden layers, and an output layer, the activation function is ReLu, and the regression function is Linear.
Compared with the prior art, the invention has the following beneficial effects: the invention establishes image data with different content complexity, obtains the optimal quantization parameter of each picture through multiple encoding, calculates the image characteristic vector, forms a data set with marked characteristic vector and optimal quantization parameter, further utilizes supervised learning training multi-layer perception algorithm, establishes the optimal quantization parameter prediction method under the limited bandwidth, can be suitable for the diversity of image content, provides the quantization parameter suitable for the image complexity, fully utilizes the bandwidth, and improves the quality of image transmission.
Drawings
Fig. 1 is a schematic diagram illustrating filtering processing performed on pictures in an image set according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an NN optimal quantization parameter model structure of a multi-layer neural network according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the following embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive efforts based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The present invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
The invention provides an estimation method of an optimal quantization parameter of image compression, which comprises the following steps:
an image set is selected, and in the present embodiment, the image set includes an Inria aerial picture dataset and a group picture dataset, and as an embodiment, a specific quantization parameter estimation method is described.
In order to increase richness of image complexity, the data set is expanded. In this embodiment, the original pictures of the data set are subjected to mean filtering with windows of 3 × 3,5 × 5,9 × 9, and 11 × 11, as shown in fig. 1, and all the filtered images and the original images together form an image set. The formula for the mean filtering is:
wherein, corresponding to windows with different sizes of 3 × 3,5 × 5,9 × 9 and 11 × 11, N is respectively 3,5,9, 11, k is respectively 1,2,4,5.
After the image set is established, each image I in the image set i Setting a target code rate B 0 Calculating an optimal quantization parameter QP of each image;
in this embodiment, assume that the target code rate is B 0 =16KB, the optimal quantization parameter QP for each picture is calculated. For each image I in the set of images created above i The quantization parameter QP of the image encoder is adjusted. When image compression outputs file size S>16KB, the quantization parameter QP is reduced by the magnitude set each time until S is no more than 16KB and is closest to 16KB, at which time the quantization parameter QP is the picture I i Optimal quantization parameter QP corresponding to target code rate 16KB i 。
Step 2, calculating the complexity characteristic vector C of each image in the image set i ；
In this step, the image complexity feature vector C i One of Entropy (Encopy), sum of absolute difference of transformation (SATD), and Gradient (Gradient) of the image may be selectedThe calculation method of one or more parameters is shown in formulas (1) to (4) in the background art.
Step 3, constructing a multilayer neural network NN, and regarding the multilayer neural network NN, an input layer is an image complexity characteristic vector C i The output layer is a regression-type quantization parameter QP i (ii) a The NN structure of the multilayer neural network established in this embodiment is shown in table 1, a schematic structural diagram, as shown in fig. 2;
TABLE 1 Multi-layer neural network NN architecture
The input layer of the multi-layer neural network NN is an image I i Image complexity feature vector C of i The output layer is a regression type quantization parameter QP i The activation function is ReLu and the regression function is Linear. The image collection is divided into a training set, a testing set and a verification set, the multi-layer neural network NN is trained through the training set to obtain optimized parameters of the multi-layer neural network NN, the trained parameters of the multi-layer neural network NN are verified through the verification set, testing is carried out through the testing set, and when testing is carried out, two network structures of the multi-layer neural network NN are tested: the structure comprises a structure (a) and a structure (b), wherein the structure (a) comprises 2 hidden layers, and the number of nodes is 5 and 10 respectively; the structure (b) comprises 3 hidden layers, the number of the nodes is respectively 5, 10 and 5, and finally the optimal estimation QP is output.
To illustrate the beneficial effects of the present invention, the output structure of the multi-layer neural network NN was verified. In this embodiment, the rate accuracy BRA is defined as a reasonable degree for evaluating the selection of the quantization parameter QP, and the calculation formula is shown in formula (1). Wherein, TBR represents the target code rate and the actual code rate obtained after coding. In the formula, when ABR and TBR are closer, BRA is closer to 1, which means that the quantization parameter QP is more accurately set, and the NN effect of the multilayer neural network is better. Thus, the closer the BRA is to 1, the better the multi-layer neural network NN will perform when testing the multi-layer neural network NN. The BRA test results of the multi-layer neural network NN model with 7 different inputs are shown in table 2 as shown in table 1.
TABLE 2 BRA test results for the NN model of the multilayer neural network
(a) The number of nodes of each layer of the network is (n, 5, 10, 1)
(b) The number of nodes of each layer of the network is (n, 5, 10,5, 1)
(Note: bold data in Table for optimal results)
As can be seen from the table, using 3 features as inputs simultaneously, the code rate accuracy reaches 85.79%. And selecting the multi-layer neural network NN with the highest test precision as the multi-layer neural network NN finally used for prediction.
And 4, step 4: and (4) for the image I ', calculating an image complexity feature vector C', inputting the image complexity feature vector C 'into the multilayer neural network NN obtained in the step (3), and outputting the optimal quantization parameter estimation QP'.
And calculating an image complexity characteristic vector C 'of the image I', and respectively inputting the image complexity characteristic vector C 'into the two structures of the tested multilayer neural network NN to obtain an output quantization parameter optimal estimation QP'.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the invention.
Claims (4)
1. A method for estimating an optimal quantization parameter for image compression is characterized by comprising the following steps:
step 1, selecting an image set, and carrying out image processing on each image I in the image set i Setting a target code rate B 0 Calculating an optimal quantization parameter QP of each image;
step 2, calculatingComplexity feature vector C of each image in image set i ；
Step 3, constructing a multilayer neural network NN, and regarding the multilayer neural network NN, an input layer is an image complexity characteristic vector C i The output layer is a regression type quantization parameter QP i Training a multilayer neural network NN by adopting an image set, verifying and predicting the trained multilayer neural network NN, and selecting the multilayer neural network NN with the highest test precision;
and 4, step 4: and (3) calculating an image complexity characteristic vector C 'of the image I', and inputting the image complexity characteristic vector C 'into the multilayer neural network NN obtained in the step 3, so as to obtain an output optimal quantization parameter estimation QP'.
2. The method for estimating the optimal quantization parameter for image compression according to claim 1, wherein the specific method in step 1 is as follows:
for each image I i Adjusting a quantization parameter QP of an image encoder; when the size of the image compression output file is S > B 0 While, the quantization parameter QP is continuously reduced until S does not exceed B 0 And is closest to B 0 At this time, the quantization parameter QP is the picture I i Corresponding target code rate B 0 Optimal quantization parameter QP of i 。
3. The method according to claim 1, wherein in step 2, the image complexity eigenvector C is selected from the group consisting of i Is one or more of Entropy of information of the image, entropy, sum of absolute difference of transformation, SATD, and gradient of the image.
4. The method as claimed in claim 1, wherein the multi-layer neural network NN comprises an input layer, two or more hidden layers, and an output layer, the activation function is ReLu, and the regression function is Linear.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184629.7A CN115665413A (en) | 2022-09-27 | 2022-09-27 | Method for estimating optimal quantization parameter of image compression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211184629.7A CN115665413A (en) | 2022-09-27 | 2022-09-27 | Method for estimating optimal quantization parameter of image compression |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115665413A true CN115665413A (en) | 2023-01-31 |
Family
ID=84985520
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211184629.7A Pending CN115665413A (en) | 2022-09-27 | 2022-09-27 | Method for estimating optimal quantization parameter of image compression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115665413A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116911360A (en) * | 2023-09-15 | 2023-10-20 | 香港中文大学（深圳） | Method for solving minimum compression rate of semantic information by using neural network |
-
2022
- 2022-09-27 CN CN202211184629.7A patent/CN115665413A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116911360A (en) * | 2023-09-15 | 2023-10-20 | 香港中文大学（深圳） | Method for solving minimum compression rate of semantic information by using neural network |
| CN116911360B (en) * | 2023-09-15 | 2023-12-12 | 香港中文大学（深圳） | Method for solving minimum compression rate of semantic information by using neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7110370B2 (en) | Using Nonlinear Functions Applied to Quantization Parameters of Machine Learning Models for Video Coding | |
| CN111787323B (en) | Variable bit rate generation type compression method based on counterstudy | |
| CN110059796B (en) | Method and device for generating convolutional neural network | |
| US9282330B1 (en) | Method and apparatus for data compression using content-based features | |
| Cai et al. | Efficient variable rate image compression with multi-scale decomposition network | |
| CN110971901B (en) | Processing method, device and equipment of convolutional neural network and storage medium | |
| Valenzise et al. | Quality assessment of deep-learning-based image compression | |
| EP3743855A1 (en) | Receptive-field-conforming convolution models for video coding | |
| Hassan et al. | Color image compression based on DCT, differential pulse coding modulation, and adaptive shift coding | |
| Zhu et al. | Dictionary based surveillance image compression | |
| CN115665413A (en) | Method for estimating optimal quantization parameter of image compression | |
| Joshua et al. | Comparison of DCT and DWT image compression | |
| Dardouri et al. | Optimized lifting scheme based on a dynamical fully connected network for image coding | |
| Niu et al. | Edge-based perceptual image coding | |
| CN113822801B (en) | Compressed video super-resolution reconstruction method based on multi-branch convolutional neural network | |
| Bakhtiarnia et al. | Analysis of the effect of low-overhead lossy image compression on the performance of visual crowd counting for smart city applications | |
| Yin et al. | A co-prediction-based compression scheme for correlated images | |
| Schimpf et al. | Compressing of Medium-to Low-Rate Transform Residuals With Semi-Extreme Sparse Coding as an Alternate Transform in Video Coding | |
| CN117459737B (en) | Training method of image preprocessing network and image preprocessing method | |
| Dong et al. | Wavelet-Like Transform-Based Technology in Response to the Call for Proposals on Neural Network-Based Image Coding | |
| Marsh et al. | Removal of Blocking Artifacts from JPEG-Compressed Images Using a Neural Network | |
| Sole et al. | Joint sparsity-based optimization of a set of orthonormal 2-D separable block transforms | |
| Le et al. | Inr-mdsqc: Implicit neural representation multiple description scalar quantization for robust image coding | |
| US20240223762A1 (en) | A method, an apparatus and a computer program product for video encoding and video decoding | |
| Manjunatha et al. | An enhanced video compression approach through RLAH encoding and KDENN algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |