CN118264806A

CN118264806A - Image processing method and device, computer equipment and storage medium

Info

Publication number: CN118264806A
Application number: CN202211673500.2A
Authority: CN
Inventors: 潘翔
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-12-26
Filing date: 2022-12-26
Publication date: 2024-06-28

Abstract

The embodiment of the application provides an image processing method and device, computer equipment and a storage medium. The image processing method comprises the following steps: acquiring a code stream file; analyzing quantization value range information of N image components of an image to be decoded from a code stream file; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded; and decoding the N image components according to the quantized value range information of the N image components. By adopting the embodiment of the application, the proper quantization value range information can be respectively determined for each image component of the image to be decoded for decoding, and the image decoding effect is effectively improved.

Description

Image processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to the field of image encoding and decoding technology, and more particularly, to an image processing method, an image processing apparatus, a computer device, and a computer readable storage medium.

Background

With the rapid development of computer technology, image coding and decoding technology is widely applied to relevant business scenes of videos, images and the like; for example, encoding and decoding each image frame of a session video and a session image in a session service scene; as another example, encoding and decoding each image frame of a game video in a game service scene, and a game image; etc. The image encoding and decoding techniques include an image encoding technique and an image decoding technique; the image coding technology is a technology for compressing an image under the condition of meeting certain image quality requirements so as to reduce the image volume, and can effectively save the storage space of the image and improve the transmission efficiency of the image; image decoding refers to a technique of decoding an encoded image to reconstruct the image.

At present, in the image decoding process, the quantization and value range information (also called as the size of an alphabet) of the image can have a great influence on the image decoding effect; specifically, if the quantization value range information of the image is larger (i.e. if the alphabet of the image is larger), the image can be quantized into a wider symbol range, and the reconstruction quality of the image is higher, but in theory, the calculation complexity of image decoding also becomes larger, and a larger code rate is consumed; if the quantization value range information of the image is smaller (i.e. if the alphabet of the image is smaller), the image can be quantized into a narrower symbol range, the consumed code rate is lower, the calculation complexity of image decoding is lower in theory, but the reconstruction quality of the image is not high. Therefore, how to determine the proper quantitative value range information for the image becomes a current research hotspot.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, computer equipment and a storage medium, which can respectively determine proper quantization value range information for each image component of an image to be decoded for decoding, thereby effectively improving the image decoding effect.

In one aspect, an embodiment of the present application provides an image processing method, including:

acquiring a code stream file;

Analyzing quantization value range information of N image components of an image to be decoded from a code stream file; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded;

and decoding the N image components according to the quantized value range information of the N image components.

Accordingly, an embodiment of the present application provides an image processing apparatus including:

the acquisition unit is used for acquiring the code stream file;

The processing unit is used for analyzing the quantized value range information of N image components of the image to be decoded from the code stream file; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded;

and the processing unit is also used for decoding the N image components according to the quantized value range information of the N image components.

In one implementation manner, the processing unit is configured to, when parsing the quantized value range information of N image components of the image to be decoded from the code stream file, specifically perform the following steps:

and respectively analyzing the quantized value range information of each image component in N image components of the image to be decoded from the code stream file.

In one implementation manner, the processing unit is configured to, when parsing the quantized value range information of each image component of the N image components of the image to be decoded from the code stream file, specifically perform the following steps:

Analyzing the value range mark information of the target image component in the N image components from the code stream file, and calculating the value range mark information of the target image component to obtain quantized value range information of the target image component;

or analyzing the value range mark information of the target image component in the N image components from the code stream file, and calculating the basic information of the target image component and the value range mark information of the target image component to obtain the quantized value range information of the target image component.

analyzing quantization value range information of a target image component in N image components of an image to be decoded from a code stream file;

and determining the quantized value range information of the other image components according to the mapping relation between the quantized value range information of the target image component and the quantized value range information of the other image components in the N image components.

In one implementation, the determining process of the quantized value range information of the N image components includes:

And respectively determining the quantization value range information of each image component in the N image components based on the coding information of the image to be decoded.

determining quantization value range information of a target image component in N image components based on coding information of an image to be decoded;

In one implementation, the encoding information of the image to be decoded includes residual information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

And determining quantization value range information of the target image component based on residual information of the target image component.

In one implementation, the residual information corresponding to the target image component includes: residual values of each pixel point in the target image component; determining quantized value range information of the target image component based on residual information of the target image component, including:

determining a maximum residual value and a minimum residual value in the residual values of all pixel points in the target image component;

Generating a quantized information screening condition according to the maximum residual value and the minimum residual value;

And screening the quantized value range information of the target image component from the candidate value range information according to the quantized information screening condition.

In one implementation, the coding information of the image to be decoded includes code rate information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

and determining the quantization value range information of the target image component based on the code rate information of the target image component.

In one implementation, determining quantization value range information of a target image component based on code rate information of the target image component includes:

determining a target component type to which the target image component belongs;

Acquiring a value range prediction model matched with code rate information of a target image component from a plurality of value range prediction models under the target component type;

And calling a matched value range prediction model, and carrying out quantization value range prediction on the target image component to obtain quantization value range information of the target image component.

In one implementation, each value range prediction model under the target component type corresponds to respective preset code rate information;

In a plurality of value range prediction models under a target component type, acquiring a value range prediction model matched with code rate information of a target image component, wherein the value range prediction model comprises:

Determining reference code rate information matched with code rate information of a target image component in a plurality of pieces of preset code rate information under the target component type;

and taking the value range prediction model corresponding to the reference code rate information as a value range prediction model matched with the code rate information of the target image component.

In one implementation, a training process of a value range prediction model corresponding to reference code rate information includes:

acquiring an initial prediction model corresponding to the reference code rate information, and acquiring a sample image component of a training sample image under a target component type;

Invoking an initial prediction model, and predicting a quantized value range of the sample image component to obtain quantized value range information matched with the sample image component;

And updating model parameters of the initial prediction model according to the difference between the predicted quantized value range information and the quantized value range information marked under the reference code rate information to obtain a value range prediction model corresponding to the reference code rate information.

In one implementation, the reference code rate information is configured with a plurality of candidate value range information; the method further comprises the steps of:

Acquiring a plurality of marked sample images;

according to any candidate value range information configured by the reference code rate information, respectively encoding sample image components of the plurality of marked sample images under the target component type to obtain encoding performance information of the plurality of marked sample images under the candidate value range information;

Determining average coding performance information of candidate value range information according to coding performance information of a plurality of marked sample images under the candidate value range information;

And determining the highest coding performance information in the average coding performance information of the plurality of pieces of candidate value range information of which the reference code rate information is configured, and determining the candidate value range information corresponding to the highest coding performance information as quantized value range information marked under the reference code rate information.

Obtaining a value range prediction model corresponding to the target component type;

And calling a value range prediction model corresponding to the target component type, and carrying out quantization value range prediction on the target image component based on code rate information of the target image component to obtain quantization value range information of the target image component.

acquiring a value range configuration list corresponding to the type of the target component; the target component type is configured with a plurality of pieces of preset code rate information, and the value range configuration list comprises quantized value range information corresponding to each piece of preset code rate information;

Searching reference code rate information matched with code rate information of the target image component in the value range configuration list, and taking quantized value range information corresponding to the reference code rate information as quantized value range information matched with the target image component.

In one implementation, a configuration process of a value range configuration list corresponding to a target component type includes:

Acquiring candidate value range information corresponding to each preset code rate information in a plurality of preset code rate information configured by the target component type;

based on sample image components of the training sample image under the target component type, selecting intermediate value range information matched with corresponding preset code rate information from candidate value range information corresponding to each preset code rate information, and generating an intermediate configuration list according to a plurality of preset code rate information and the intermediate value range information matched with each preset code rate information;

Based on the sample image component of the optimized sample image under the target component type, selecting the middle value range information with high coding performance from the middle value range information corresponding to each piece of preset code rate information contained in the middle configuration column as quantized value range information corresponding to the corresponding preset code rate information, and optimizing the middle configuration list according to the plurality of pieces of preset code rate information and the quantized value range information corresponding to each piece of preset code rate information to obtain a value range configuration list corresponding to the target component type.

In one implementation, the number of training sample images is a plurality; based on sample image components of the training sample image under the target component type, selecting intermediate value range information matched with corresponding preset code rate information from candidate value range information corresponding to each preset code rate information, wherein the method comprises the following steps:

calculating residual information of sample image components of each training sample image in the plurality of training sample images under the target component type based on any one piece of preset code rate information;

According to the minimum average residual value and the maximum average residual value in residual information corresponding to the training sample images, determining a screening condition of preset code rate information;

And selecting candidate value range information meeting the screening condition of the preset code rate information from candidate value range information corresponding to the preset code rate information based on the screening condition of the preset code rate information, and taking the candidate value range information as intermediate value range information matched with the preset code rate information.

In one implementation, the processing unit is configured to perform the following steps when decoding the N image components according to the quantized value range information of the N image components:

analyzing quantization symbols of N image components from a code stream file;

based on the quantized value range information of the N image components, dequantizing the quantized symbols of the N image components to obtain residual information of the N image components;

Based on the residual information of the N image components, the N image components are reconstructed.

In another aspect, an embodiment of the present application provides an image processing method, including:

acquiring coding information of an image to be coded;

based on the coding information of the image to be coded, determining quantization value range information of N image components of the image to be coded; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component, and N is a positive integer;

and encoding the N image components according to the quantized value range information of the N image components.

an acquisition unit for acquiring encoding information of an image to be encoded;

The processing unit is used for determining quantization value range information of N image components of the image to be encoded based on the encoding information of the image to be encoded; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component, and N is a positive integer;

and the processing unit is also used for encoding the N image components according to the quantized value range information of the N image components.

In one implementation manner, the processing unit is configured to determine quantization value range information of N image components of the image to be encoded based on encoding information of the image to be encoded, and specifically is configured to perform the following steps:

Determining quantization value range information of a target image component in N image components based on coding information of an image to be coded;

In one implementation, the encoding information of the image to be encoded includes residual information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

In one implementation, the residual information corresponding to the target image component includes: residual values of each pixel point in the target image component; the processing unit is used for determining the quantization value range information of the target image component based on the residual information of the target image component, and is specifically used for executing the following steps:

In one implementation, the coding information of the image to be coded includes code rate information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

In one implementation, the processing unit is configured to, when determining the quantization range information of the target image component based on the code rate information of the target image component, specifically perform the following steps:

the processing unit is used for acquiring a value range prediction model matched with code rate information of the target image component from a plurality of value range prediction models under the target component type, and is specifically used for executing the following steps:

In one implementation, the reference code rate information is configured with a plurality of candidate value range information; the processing unit is further used for executing the following steps:

Acquiring a plurality of marked sample images;

In one implementation, the number of training sample images is a plurality; the processing unit is used for selecting the intermediate value range information matched with the corresponding preset code rate information from the candidate value range information corresponding to each preset code rate information based on the sample image component of the training sample image under the target component type, and is specifically used for executing the following steps:

In one implementation, the processing unit is further configured to perform the steps of:

and writing the quantized value range information of the N image components into the code stream file.

In one implementation, the processing unit is configured to write quantization value range information of N image components into a code stream file, and includes:

And respectively writing the quantized value range information of each image component in the N image components into the code stream file.

and writing the quantized value range information of the target image component in the N image components into the code stream file.

Accordingly, an embodiment of the present application provides a computer apparatus, including:

A processor adapted to implement a computer program;

A computer readable storage medium storing a computer program adapted to be loaded by a processor and to perform the image processing method described above.

Accordingly, an embodiment of the present application provides a computer-readable storage medium storing a computer program which, when read and executed by a processor of a computer device, causes the computer device to execute the above-described image processing method.

Accordingly, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the image processing method described above.

In the embodiment of the application, the quantization value range information matched with each image component in N image components of the image to be decoded can be analyzed from the code stream file, and the quantization value range information of the N image components is determined based on the coding information of the image to be decoded; that is, given the encoding information of the image to be decoded, the appropriate quantization value range information can be determined for each image component of the image to be decoded for decoding, so that the image decoding effect can be effectively improved.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an image encoding and decoding principle according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an architecture of an image processing system according to an embodiment of the present application;

fig. 3 is a schematic flow chart of an image processing method according to an embodiment of the present application;

FIG. 4 is a flowchart of another image processing method according to an embodiment of the present application;

FIG. 5 is a flowchart of another image processing method according to an embodiment of the present application;

fig. 6 is a schematic diagram of a configuration flow of quantization and value range information according to an embodiment of the present application;

FIG. 7 is a flowchart of another image processing method according to an embodiment of the present application;

FIG. 8 is a flowchart of another configuration of quantization range information provided by an embodiment of the present application;

FIG. 9 is a flowchart of another image processing method according to an embodiment of the present application;

Fig. 10 is a schematic structural view of an image processing apparatus according to an embodiment of the present application;

Fig. 11 is a schematic structural view of another image processing apparatus according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Embodiments of the present application relate to images. The image mentioned in the embodiments of the present application may be specifically an digital image, and the digital image may be referred to as a digital image or a digital image, which refers to: the two-dimensional image is represented by finite digital value pixels, and can be specifically represented by an array or a matrix, the illumination position and the intensity of the two-dimensional image are discrete, the digital image is obtained by digitizing an analog image, and the image can be stored and processed by a digital computer or a digital circuit by taking the pixels as basic elements.

The images may be classified into binary images, gray-scale images, and color images according to visual presentation effects of the images, and the like. The Binary Image ((Binary Image) has a Luminance value of 0 (the Luminance value of 0 indicates that the pixel is black) or1 (the Luminance value of 1 indicates that the pixel is white), that is, the Binary Image is a black-and-white Image, the gray Image (GRAY SCALE IMAGE) may also be referred to as a gray Image, the Luminance value of each pixel in the gray Image ranges from 0 (the Luminance value of 0 indicates that the pixel is black) to 255 (the Luminance value of 255 indicates that the pixel is white), the Luminance value of the pixel is between 0-255 indicates that the pixel is black), the color Image may typically include a plurality of Image components, for example, the color Image in YUV (one Image pixel format) may include a Luminance component (Luminance or Luma, Y component) and a chrominance component (Chrominance), and may be used for describing a Blue component (RGB component) and a chrominance component (Blue component, a Red component (RGB component) and a chrominance component (Blue component) as well as a color component (Blue component).

In summary, the image mentioned in the embodiments of the present application may include N image components, where N is a positive integer; when the image is a binary image or a gray scale image, the image may contain an image component which is a luminance component and which is the image itself; when the image is a color image, the image may contain a plurality of image components, for example, the image may contain a luminance component and a chrominance component, and for example, the image may contain a red component, a green component, and a blue component. In the following embodiments of the present application, the case where an image includes a luminance component and a chrominance component is described as an example, and the case where an image includes a red component, a green component, and a blue component is similar to the case where an image includes a luminance component and a chrominance component, the following embodiments of the present application will not describe the case where an image includes a red component, a green component, and a blue component as an example.

Before the image is stored or transmitted, the image is often required to be compressed and encoded by adopting an image encoding technology so as to reduce the volume of the image, thereby effectively saving the storage space of the image and improving the transmission efficiency of the image. The image coding technology refers to a technology for compressing an image under the condition of meeting certain image quality requirements so as to reduce the image volume; and the image decoding technique refers to a technique of decoding an encoded image to reconstruct the image. In particular, the embodiment of the present application provides an end-to-end image coding and decoding technique based on deep learning, and the following description will be made with reference to fig. 1 on the principle of the end-to-end image coding and decoding technique based on deep learning:

(1) Encoding side:

For the coding side, the encoder can comprise a plurality of trained image coding models, and each image coding model corresponds to a different code rate, wherein the code rate refers to the number of transmitted bits in unit time; the image coding model corresponds to the code rate and can be understood as: the image coding model is used for coding an image with a given code rate matched with the code rate corresponding to the image coding model, and the actual code rate after the image coding is close to the code rate corresponding to the image coding model; here, the given code rate may be understood as a desired code rate, that is, an actual code rate after the image is desired to be encoded is a code rate corresponding to the image encoding model.

That is, the logic of the encoder to encode the image can be described as follows: and acquiring the image to be encoded and a given code rate, and selecting an image encoding model with the code rate matched with the given code rate from a plurality of image encoding models contained in the encoder to encode the image. Here, the code rate of the image coding model is matched with the given code rate, which can be understood as that the code rate of the image coding model is the same as the given code rate; or it may be understood that the code rate of the image coding model is close to the given code rate, for example, the difference between the code rate corresponding to the selected image coding model and the given code rate is the smallest among the plurality of image coding models included in the encoder; under the condition that the code rate of the image coding model is close to the given code rate, the code rate of the image coding model can be the same as the given code rate by adjusting parameters of the image processing model.

In more detail, the selected image processing model encodes the luminance component (x _Y) and the chrominance component (x _UV) of the image, respectively, the encoding process indicated by the black arrow in fig. 1 corresponds to the luminance component and the encoding process indicated by the gray arrow corresponds to the chrominance component; the coding process of the luminance component and the chrominance component of an image is similar, and the coding process of the luminance component is described as an example.

The encoder inputs the brightness component (x _Y) into an analysis transformation network (Analysis Transform Net) in the selected image processing model to perform nonlinear transformation to obtain a transformation result (also called hidden variable, which can be expressed as y _Y) of the brightness component; secondly, the encoder subtracts the output (μ _Y) of the prediction fusion network (Prediction Fusion Net) in the selected image processing model with respect to the luminance component from the transform result (y _Y) of the luminance component, and can obtain residual information (r _Y) of the luminance component, which is typically a floating point type of value; then, the encoder may perform quantization processing on the residual information (r _Y) of the luminance component based on the quantized value range information, converting the residual information (r _Y) of the luminance component into an integer type quantized symbol; the encoder may then invoke quantization of the luma component by the entropy encoder (Lossless Coder) in the selected image processing modelPerforming entropy coding to generate a code stream file of a brightness component; the quantized value range information for performing quantization processing on the residual information r _Y of the luminance component is also encoded into the bitstream file of the luminance component.

Similarly, the selected image processing model may encode the chrominance component (x _UV) to obtain a code stream file of the chrominance component. The stream file of the luminance component and the stream file of the chrominance component may be transmitted to a decoder.

(2) Decoding side:

for the decoding side, the decoding flow indicated by the black arrow in fig. 1 corresponds to the luminance component, and the decoding flow indicated by the gray arrow corresponds to the chrominance component; the luminance component of an image is similar to the coding process of the chrominance component, and the decoding process of the luminance component is described as an example.

The decoder may invoke an entropy decoder (Lossless Decoder) to parse the quantized result of the luma component from the luma component's codestream fileQuantifying the value range information; second, the decoder may quantize the luminance component based on the quantized value range informationPerforming inverse quantization to recover residual information of the brightness component; then, the residual information of the restored luminance component is added with the output of the prediction fusion network (Prediction Fusion Net) about the luminance component to restore the transformation result of the luminance componentThe encoder may then invoke the transform result of the composite transform network (SYNTHESIS TRANSFORM NET) on the luma componentNon-linear transformation is carried out to recover the brightness component

Similarly, the decoder may recover the chrominance components from the code stream file of the chrominance componentsFrom the recovered luminance componentAnd chrominance componentThe image may be restored.

In the above description of the image codec process, the quantized value range information (alternatively referred to as Alphabet size (alphabet size)) may be used to indicate the value range of the quantized symbol (alternatively referred to as a letter); if the quantized value range information is denoted as α, the value range of the quantized symbol indicated by the quantized value range information may be denoted asFor example, if the quantized value range information is 64, the quantized symbol indicated by the quantized value range information may have a value range of [ -31, 33]. The residual information of the image component may include pixel values of each pixel point in the image component, and the result of quantizing the pixel values of each pixel point in the residual information of the image component does not exceed the value range indicated by the quantized value range information by setting quantized value range information; that is, the quantized symbol obtained by quantizing the residual value of each pixel in the residual information of the image component is within the range indicated by the quantized range information.

The quantization value range information of the image can have a great influence on the encoding and decoding effects of the image; specifically, if the quantization value range information of the image is larger (i.e. if the alphabet of the image is larger), the image can be quantized into a wider symbol range, and the reconstruction quality of the image is higher, but in theory, the calculation complexity of image decoding also becomes larger, and a larger code rate is consumed; if the quantization value range information of the image is smaller (i.e. if the alphabet of the image is smaller), the image can be quantized into a narrower symbol range, the consumed code rate is lower, the calculation complexity of image decoding is lower in theory, but the reconstruction quality of the image is not high. The statistical characteristic difference of the transformation result after nonlinear transformation of different image components of the image is large, for example, the statistical characteristic difference of the transformation result after nonlinear transformation of the luminance component of the image and the chrominance component of the image is large, therefore, the same quantization value range information is configured for different image components without distinguishing the image components, and the code rate may be excessively consumed and the performance may be reduced.

Based on this, the embodiment of the application provides an image processing method, which can set quantization value range information for different image components of an image respectively in an image coding stage, so that the setting mode is very flexible, and the adaptation degree of the quantization value range information and the image components is higher, so that the coding rate consumption of the image components can be reduced, the coding efficiency of the image components is improved, and the integral coding effect of the image components is improved. The quantized value range information of different image components is written into the code stream file, so that the quantized value range information of each image component of the image can be analyzed from the code stream file for decoding in the image decoding stage, the decoding efficiency of the image components can be improved, and the integral decoding effect of the image components can be improved.

An image processing system suitable for implementing the image processing method provided by the embodiment of the present application is described below with reference to fig. 2.

As shown in fig. 2, the image processing system may include an encoding device 201 and a decoding device 202; the encoding device 201 may be a terminal or a server, and the decoding device 202 may be a terminal or a server; the encoding device 201 and the decoding device 202 may establish a direct communication connection by means of wired communication or may establish an indirect communication connection by means of wireless communication.

Wherein, the encoding device 201 may be provided with an encoder, and the encoder may be configured to perform the image processing method provided by the embodiment of the present application, that is, the encoder may set matched quantization value range information for each image component of an image, and then may encode each image component based on the quantization value range information matched with each image component, and send a code stream file obtained by encoding to the decoding device 202, where the quantization value range information matched with each image is also written into the code stream file; wherein the encoder may be a VM (Verification Model, reference model) of the JPEG AI (Joint Photographic Experts Group AI) image coding standard. The decoding device 202 may be provided with a decoder, which may be configured to parse the quantized value range information of each image component from the code stream file, and decode each image component according to the quantized value range information of each image component, so as to reconstruct an image.

It should be noted that, in the image processing system provided in the embodiment of the present application, the mentioned terminal may include, but is not limited to, any of the following: smart phones, tablet computers, notebook computers, desktop computers, smart watches, smart home appliances, smart car terminals, aircraft, etc., but are not limited thereto; the servers mentioned may be independent physical servers, may be server clusters or distributed systems formed by a plurality of physical servers, and may also be cloud servers for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligence platforms, and the like, which are not limited by the embodiments of the present application.

It may be understood that, the image processing system described in the embodiment of the present application is for more clearly describing the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application, and those skilled in the art can know that, with the evolution of the system architecture and the appearance of the new service scenario, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

The image processing method provided by the embodiment of the application is described in more detail below with reference to the accompanying drawings.

The embodiment of the application provides an image processing method, which mainly introduces a mode of analyzing quantized value range information of different image components from a code stream file and a determining process of the quantized value range information of the different image components in an image decoding stage. The image processing method may be performed by a computer device, which may be the decoding device 202 in the image processing system shown in fig. 2 described above. As shown in fig. 3, the image processing method includes, but is not limited to, the following steps S301 to S303:

S301, obtaining a code stream file.

S302, analyzing quantization value range information of N image components of an image to be decoded from a code stream file.

In step S301 to step S302, a code stream file may be acquired, where the code stream file may be sent by the encoding device to the decoding device, and quantization value range information of N image components of the image to be decoded may be parsed from the code stream file; one image component is matched with one quantized value range information, and any quantized value range information is used for indicating the value range of a quantized symbol of the corresponding image component.

The method for resolving the quantization value range information of the N image components of the image to be decoded from the code stream file may include any one of the following:

(1) The quantized value range information of the N image components may be written into the code stream information, so that the quantized value range information of the N image components of the image to be decoded may be parsed from the code stream file, respectively. Specifically, taking any one of the N image components (may be referred to as a target image component) as an example, the method of analyzing the quantized value range information of the target image component from the code stream file may include any one of the following:

① The quantization value range information of the target image component can be directly analyzed from the code stream file.

② The value range flag information of the target image component can be resolved from the code stream file, and then the value range flag information of the target image component can be operated to obtain the quantized value range information of the target image component. The operations herein may be, for example, exponential operations, an exemplary procedure of which may be described as follows: the target image component is a luminance component, the value range flag information of the luminance component analyzed from the code stream file is α _{Y_}, and the value range flag information α _{Y_} of the luminance component may be subjected to an exponential operation to obtain quantized value range information α _Y＝m^αY _{_} ^f of the luminance component (where m is a constant, for example, m=2). In this way, the value range flag information in the code stream file is smaller than the quantized value range information, so that the analysis efficiency of the code stream file can be improved.

③ The value range flag information of the target image component in the N image components can be analyzed from the code stream file, and the basic information of the target image component and the value range flag information of the target image component are operated to obtain the quantized value range information of the target image component. Here, the basic information of the target image component may be set by default by the decoding device and the encoding device, and the operation herein may be, for example, performing an exponential operation on the sum of the basic information of the target image component and the value range flag information of the target image component, and an exemplary operation procedure may be described below: the target image component is a brightness component, the value range sign information of the brightness component analyzed from the code stream file is alpha _{Y_}, the basic information of the brightness component is alpha _{Y_}, and the sum (alpha _{Y_}+α_{Y_}) of the basic information of the brightness component and the value range sign information of the brightness component can be subjected to exponential operation to obtain the quantized value range information of the brightness component(Where m is a constant, e.g., m=2). In this way, the value range flag information in the code stream file is smaller than the quantized value range information, so that the analysis efficiency of the code stream file can be improved.

④ The value range compression information of the target image component in the N image components can be analyzed from the code stream file, and the value range flag information of the target image component is obtained by carrying out operation according to the value range compression information of the target image component and the value range flag information of the reference image component; the operation here may be, for example, subtraction, and the value range compression information of the target image component may be a difference between the value range flag information of the reference image component and the value range flag information of the target image component, so that the value range flag information of the reference image component is subtracted from the value range compression information of the target image component, and the value range flag information of the target image component may be obtained; the reference image component is any one of the N image components other than the target image component. Then, the quantized value range information of the target image component can be calculated according to the value range flag information of the target image component.

For example, the reference image component is a luminance component, the target image component is a chrominance component, the value range flag information of the luminance component is α _{Y_}, and the value range compression information (α _{Y_}-α_{UV_}) of the chrominance component may be written into a code stream file, the above two information may be parsed from the code stream file, and then the value range flag information of the luminance component is α _{Y_}, and subtracting the value range compression information (α _{Y_}-α_{UV_}) of the chrominance component may restore the value range flag information α _{UV_} of the chrominance component; thus, the value range flag information α _{UV_} of the chrominance component may be subjected to an exponential operation to obtain quantized value range information α _UV＝m^αUV _{_} of the chrominance component (where m is a constant, e.g., m=2); or the sum (alpha _{UV_}+α_{UV_}) of the basic information of the chrominance component and the value range mark information of the chrominance component can be subjected to exponential operation to obtain quantized value range information of the chrominance component(Where m is a constant, e.g., m=2). By the mode, compression information of the value range in the code stream file is smaller than that of the quantized value range, and analysis efficiency of the code stream file can be improved.

(2) The quantized value range information of the target image component in the N image components can be written into the code stream file, the quantized value range information of other image components except the target image component in the N image components has a mapping relation with the quantized value range information of the target image component, and the target image component can be any one of the N image components; in this case, the quantized value range information of the target image component of the N image components of the image to be decoded may be parsed from the code stream file, and then the quantized value range information of the other image components may be determined according to the mapping relationship between the quantized value range information of the target image component and the quantized value range information of the other image components of the N image components.

The method of resolving the quantized value range information of the target image component from the code stream file may refer to any one of the quantized value range information ①－③ of the N image components of the image to be decoded respectively resolved from the code stream file, which is not described herein. An exemplary mapping relationship between quantized span information of a target image component and quantized span information of other image components in the N image components can be seen in the following formula 1:

As shown in the above formula 1, the target image component is a luminance component, and the other image components are chrominance components; if the value range information α _Y of the luminance component is 32, 64, 128, 512, 1024, 2048, 4096 or 8192, the quantized value range information α _UV＝α_Y/2 of the chrominance component; if the luminance component value range information α _Y is 256, the chrominance component quantization value range information α _UV＝3α_Y/4.

The quantization value range information of the N image components is determined by the encoding apparatus based on the encoding information of the image to be decoded. In one implementation, the quantization range information of the N image components may be determined based on the encoding information of the image to be decoded, that is, for any one of the N image components (may be referred to as a target image component), the quantization range information of the target image component may be determined based on the encoding information of the image to be decoded. In another implementation, based on the encoding information of the image to be decoded, quantization value range information of a target image component of the N image components is determined, and the target image component may be any one of the N image components; then, the quantized value range information of the other image components can be determined according to the mapping relation between the quantized value range information of the target image component and the quantized value range information of the other image components in the N image components, and the mapping relation can be specifically referred to the above formula 1.

The method of determining the quantization range information of the N image components and the method of analyzing the quantization range information of the N image components from the code stream file may be combined. For example, when the quantization value range information of the N image components is determined based on the encoding information of the image to be decoded, the quantization value range information of the N image components may be written into the code stream file, respectively, so that the quantization value range information of the N image components of the image to be decoded may be parsed from the code stream file, respectively. For another example, when the quantization value range information of the target image component of the N image components is determined based on the encoding information of the image to be decoded, the quantization value range information of the target image component may be written into the code stream file when the quantization value range information of the other image component is determined according to the mapping relationship between the quantization value range information of the target image component and the quantization value range information of the other image component of the N image components, so that the quantization value range information of the target image component may be parsed from the code stream file, and then the quantization value range information of the other image component may be determined according to the mapping relationship between the quantization value range information of the target image component and the quantization value range information of the other image component of the N image components.

Whether the quantization value range information of the N image components is determined based on the coding information of the image to be decoded or the quantization value range information of the target image component in the N image components is determined based on the coding information of the image to be decoded, the quantization value range information of the other image components is determined according to the mapping relation between the quantization value range information of the target image component and the quantization value range information of the other image components in the N image components, and the quantization value range information of the target image component in the N image components is determined based on the coding information of the image to be decoded. The method for determining the quantization value range information of the target image component in the N image components based on the coding information of the image to be decoded may include any one of the following:

First, the encoded information of the image to be decoded may include information that the target image component actually needs to be compressed, where the information that the target image component actually needs to be compressed refers to residual information of the target image component, that is, the encoded information of the image to be decoded may include residual information of the target image component, and quantization value range information of the target image component may be determined based on the residual information of the target image component; this approach may be seen in particular in the description of the embodiment shown in fig. 4 below.

Second, the coding information of the image to be decoded may include the code rate information of the target image component, where the N image components may share the same target code rate information, that is, the coding information of the image to be decoded may include the same target code rate information, where the code rate information of the target image component is the same target code rate information, or the N image components correspond to respective code rate information; thus, the quantization value range information of the target image component can be determined based on the code rate information of the target image component. The method for determining the quantization value range information of the target image component based on the code rate information of the target image component may include any one of the following: inquiring a value range configuration list based on code rate information of the target image component to obtain quantized value range information of the target image component, wherein the mode can be specifically described with reference to an embodiment shown in the following FIG. 5; or predicting the quantized value range information of the target image component based on the value range prediction model of the code rate information of the target image component, which can be specifically described in the following embodiment shown in fig. 7.

S303, decoding the N image components according to the quantized value range information of the N image components.

After analyzing the quantized value range information of the N image components, the N image components may be decoded according to the quantized value range information of the N image components. Specifically, the quantized symbols of the N image components may be parsed from the code stream file; based on the quantized value range information of the N image components, dequantizing the quantized symbols of the N image components to obtain residual information of the N image components; based on the residual information of the N image components, reconstructing the N image components.

Taking any one of the N image components (may be referred to as a target image component) as an example, a quantization symbol of each pixel point in the target image component may be parsed from the code stream file, and then, based on quantization value range information of the target image component, the quantization symbol of each pixel point in the target image component may be dequantized to obtain a residual value of each pixel point in the target image component, so that the target image component may be reconstructed based on the residual value of each pixel point in the target image component.

In the embodiment of the application, the quantization value range information is respectively set for different image components of the image, so that the adaptation degree of the quantization value range information and the image components is higher, the image components are decoded based on the quantization value range information, the decoding efficiency of the image components can be improved, and the integral decoding effect of the image components is improved.

The embodiment of the application provides an image processing method, which mainly introduces a scheme that an encoding device configures quantization value range information based on actual compression information of image components. The image processing method may be performed by a computer device, which may be the encoding device 201 in the image processing system shown in fig. 2 described above. As shown in fig. 4, the image processing method includes, but is not limited to, the following steps S401 to S403:

s401, determining a maximum residual value and a minimum residual value in the residual values of all pixel points in the target image component.

The residual information of the target image component may include residual values of respective pixels in the target image component, where the residual values may specifically refer to unsigned residual values, and it may be understood that the original residual values of respective pixels in the image component may be signed, and the original residual values may be positive or negative, where the signs of the residual values are not considered, i.e. the residual values included in the residual information are magnitudes of the original residual values, i.e. the unsigned residual values. The maximum residual value and the minimum residual value may be determined among the residual values of the respective pixel points in the target image component.

S402, generating a quantized information screening condition according to the maximum residual value and the minimum residual value.

S403, according to the quantization information screening condition, the quantization value range information of the target image component is screened from the candidate value range information.

In steps S402 to S403, quantization information filtering conditions may be generated according to the maximum residual value and the minimum residual value, and quantization value range information of the target image component may be filtered out of the candidate value range information according to the quantization information filtering conditions. Taking the example that the target image component is a luminance component, the minimum residual value among the residual values of the respective pixels in the luminance component can be expressed asThe maximum residual value can be expressed asThe quantization information filtering condition can be specifically referred to as the following formula 2:

As shown in the above formula 2, α represents candidate value range information, where the candidate value range information is a positive integer (n+), that is, a value that satisfies the quantization information filtering condition may be selected from all positive integers and added to the set α _alphabet; then, further screening may be performed in the set α _alphabet, and finally, the minimum value in the set α _alphabet is determined as quantized value range information of the luminance component. It should be noted that, equation 2 may be an exemplary quantization information filtering condition, and may also be adopted in the actual determination process of the quantization value range information Or (b)The present embodiment is not limited to the quantitative information screening conditions of the same form.

In this way, the quantized value range information matched with the image component can be determined according to the data characteristics (i.e., the maximum residual value and the minimum residual value) of the residual information of the image component, so that the determined quantized value range information has a higher adaptation degree with the image component.

The above steps S401 to S403 describe a specific manner of determining the quantization range information of the target component based on the residual information of the target image component. For the manner of determining the quantization value range information of the N image components, the encoding information of the image to be decoded may include residual information of each of the N image components, and the quantization value range information of the N image components may be determined based on the residual information of each of the N image components.

In the embodiment of the application, the quantization value range information of the image component can be determined based on the data characteristics (namely, the maximum residual value and the minimum residual value) of the information (namely, the residual information) actually required to be compressed of the image component, so that the determined quantization value range information and the image component have higher adaptation degree.

The embodiment of the application provides an image processing method, which mainly introduces a scheme that an encoding device queries a value range configuration list based on code rate information of a target image component to obtain quantized value range information of the target image component. The image processing method may be performed by a computer device, which may be the encoding device 201 in the image processing system shown in fig. 2 described above. As shown in fig. 5, the image processing method may include the following steps S501 to S503:

s501, determining the type of the target component to which the target image component belongs.

S502, acquiring a value range configuration list corresponding to the target component type.

In step S501-step S502, different component types of the image correspond to different value range configuration lists, that is, each component type of the image may correspond to one value range configuration list, for example, a luminance component type corresponds to one value range configuration list, and a chrominance component type corresponds to one value range configuration list. Taking any one of the N image components (which may be referred to as a target image component) as an example, a target component type to which the target image component belongs may be determined, and then a value range configuration list corresponding to the target component type may be acquired; for example, if the target image component is a luminance component and belongs to a luminance component type, a value range configuration list corresponding to the luminance component type may be obtained; for another example, if the target image component is a chrominance component and belongs to the chrominance component type, the value range configuration list corresponding to the chrominance component type may be obtained. The target component type can be configured with a plurality of pieces of preset code rate information, and the value range configuration list corresponding to the target component type comprises quantized value range information corresponding to each piece of preset code rate information; an exemplary value range configuration list can be found in table 1 below:

TABLE 1

Preset code rate information	Quantized value range information
		0.06	64
0.12	64
		0.25	128
0.5	256
		0.75	256
1.0	512
		1.25	512
1.5	512
		1.75	1024
2.0	2048

S503, searching the reference code rate information matched with the code rate information of the target image component in the value range configuration list, and taking the quantized value range information corresponding to the reference code rate information as the quantized value range information matched with the target image component.

After the value range configuration list corresponding to the target component type is obtained, the reference code rate information matched with the code rate information of the target image component can be searched in the value range configuration list, and then the quantized value range information corresponding to the reference code rate information can be used as the quantized value range information matched with the target image component.

Specifically, the matching of the code rate information of the target image component with the reference code rate information may specifically refer to: the code rate information of the target image component is the same as the reference code rate information; or the code rate information of the target image component is similar to the reference code rate information, and the similarity can be understood as: the difference between the code rate information of the target image component and the reference code rate information is the smallest in a plurality of pieces of preset code rate information contained in the value range configuration list corresponding to the target component type; accordingly, the quantized value range information corresponding to the reference code rate information can be used as quantized value range information matched with the target image component. Taking the value range configuration list shown in table 1 as an example, if the code rate information of the target image component is 1.75, the reference code rate information 1.75 identical to the code rate information of the target image component can be found in the value range configuration list, and then the quantized value range information 1024 corresponding to the reference code rate information 1.75 can be used as the quantized value range information matched with the target image component; for another example, if the code rate information of the target image component is 1.47, the reference code rate information 1.5 similar to the code rate information of the target image component may be found in the value range configuration list, and then the quantized value range information 512 corresponding to the reference code rate information 1.5 may be used as the quantized value range information matched with the target image component.

As can be seen from the contents of steps S501 to S503, regarding the manner of determining the quantized span information of each image component, the scheme of configuring the quantized span information based on the span configuration list can be summarized in fig. 6: as shown in fig. 6, taking an example that an image to be decoded includes a luminance component and a chrominance component, a value range configuration list (α _{Y_}) corresponding to a luminance component type to which the luminance component belongs and a value range configuration list (α _{UV_}) corresponding to a chrominance component type to which the chrominance component belongs may be obtained; then, based on the code rate information of the given target image component, searching from a value range configuration list (alpha _{Y_}) corresponding to the brightness component type, and encoding the brightness component based on the quantized value range information matched with the brightness component; and can look up from the value range configuration list (alpha _{UV_}) corresponding to the chroma component type based on the code rate information of the given target image component, the quantization value range information matched with the chroma component, and encode the chroma component based on the quantization value range information matched with the chroma component; then, the encoded code stream file may be output, and the quantized value range information matched with the luminance component and the quantized value range information matched with the chrominance component may be written as encoding parameters into the code stream file.

In the configuration of the quantized value range information based on the value range configuration list described in step S501-step S503, different types of image components respectively correspond to different value range configuration lists, and based on given code rate information, the quantized value range information adapted under given code rate information can be found for the corresponding types of image components in the value range configuration list of the different types of image components.

The above description is mainly directed to a process of searching quantized value range information in a value range configuration list, and the following description is mainly directed to a configuration process of a value range configuration list corresponding to a target component type, where the configuration process of the value range configuration list may include two stages: an initial configuration stage and a configuration optimization stage, the initial configuration stage and the configuration optimization stage are described below by taking the example that the target component type is the luminance component type, respectively:

(1) An initial configuration stage:

For the luminance component type, the initial configuration stage of the value range configuration list (α _{Y_}) corresponding to the luminance component type may include: first, candidate value range information may be acquired, for example, candidate value range information may be α _candlist =64, 128, 256, 512, 1024, and 2048; secondly, 500 training sample images can be randomly extracted from a training data set of JPEG AI, and brightness components of the 500 training sample images are encoded under preset code rate information, for example, the preset code rate information can be bpp _list = {0.06,0.12,0.25,0.5,0.75,1.0,1.25,1.5,1.75,2.0}; then, for any preset code rate information (which may be referred to as current preset code rate information), an average value of maximum residual values (i.e., maximum average residual values, which may be expressed as ) And the average of the minimum residual values (i.e., the minimum average residual value, may be expressed as) ; Then, for the current preset code rate information, a screening condition of the current preset code rate information can be determined, and based on the screening condition of the current preset code rate information, candidate value range information meeting the screening condition is selected from the candidate value range information, wherein the screening condition can be seen in the following formula 3:

As shown in the above formula 3, the set α _{candlist_} may include candidate value range information satisfying the screening condition under the current preset code rate information; after that, the minimum candidate value range information can be selected from the candidate value range information meeting the screening condition under the current preset code rate information to be used as the intermediate value range information of the current preset code rate information Then, each piece of preset code rate information and the middle value range information corresponding to each piece of preset code rate information can be processedThe combination is performed to generate an intermediate configuration list α _{output__list} of the luminance component types.

It should be noted that, equation 3 may be an exemplary filtering condition, and may also be adopted in the actual determination process of the quantized value range information Or (b) The screening conditions of the same type are not limited in the examples of the present application.

The initial configuration phase of the above-described luminance component type can be summarized as the following steps: the candidate value range information corresponding to each preset code rate information can be the same or different; then, based on the sample image component of the training sample image under the target component type, selecting the intermediate value range information matched with the corresponding preset code rate information from the candidate value range information corresponding to each preset code rate information, and generating an intermediate configuration list according to the plurality of preset code rate information and the intermediate value range information matched with each preset code rate information.

Based on the sample image component of the training sample image under the target component type, selecting the intermediate value range information matched with the corresponding preset code rate information from the candidate value range information corresponding to each preset code rate information, wherein the method specifically can comprise the following steps:

First, residual information of an image component of each of a plurality of training sample images under a target component type may be calculated based on any one of preset code rate information (which may be referred to as current preset code rate information). Secondly, a screening condition of preset code rate information can be determined according to the minimum average residual value and the maximum average residual value in residual information corresponding to a plurality of training sample images; the residual information corresponding to each training sample image can comprise residual values of all pixel points, the maximum residual value in the residual information corresponding to each training sample image can be obtained, and the maximum average residual value can be obtained by taking an average value of the maximum residual values of a plurality of training sample images; similarly, the minimum residual value in the residual information corresponding to each training sample image can be obtained, and the minimum average residual value can be obtained by averaging the minimum residual values of the plurality of training sample images. Then, selecting candidate value range information meeting the screening condition of the current preset code rate information from candidate value range information corresponding to the current preset code rate information based on the screening condition of the current preset code rate information, and taking the candidate value range information as intermediate value range information matched with the current preset code rate information; in more detail, the minimum candidate value range information among the candidate value range information satisfying the screening condition of the current preset code rate information may be used as the intermediate value range information.

(2) Configuration optimization stage:

Taking the luminance component type as an example, the configuration optimization stage of the value range configuration list (α _{Y_}) corresponding to the luminance component type may include: firstly, 500 optimized sample images can be randomly extracted from a training data set of JPEG AI, and the intermediate value range information corresponding to each preset code rate information in the intermediate configuration list alpha _{output__list} is traversed in sequence to encode. Secondly, for any preset code rate information (which may be referred to as current preset code rate information), calculating average coding performance information of the luminance component of 500 optimized sample images under the intermediate value range information corresponding to the current preset code rate information, and then selecting the intermediate value range information with the highest average coding performance information as quantized value range information corresponding to the current preset code rate information. Then, the intermediate configuration list α _{outpput__} may be optimally updated according to each preset code rate information and the quantized value range information corresponding to each preset code rate information, so as to obtain a value range configuration list (α _{Y_}) corresponding to the luminance component type. Similarly, a value range configuration list (α _{UV_}) corresponding to the type of the chrominance component can be obtained.

The above-described configuration optimization phase of the luminance component type can be summarized as the following steps: the intermediate value range information with high coding performance can be selected as quantized value range information corresponding to the corresponding preset code rate information from the intermediate value range information corresponding to each preset code rate information contained in the intermediate configuration column based on the sample image component of the optimized sample image under the target component type, and the intermediate configuration list is optimized according to the plurality of preset code rate information and the quantized value range information corresponding to each preset code rate information to obtain a value range configuration list corresponding to the target component type.

Based on the sample image component of the optimized sample image under the target component type, the process of selecting the intermediate value range information with high coding performance from the intermediate value range information corresponding to each preset code rate information contained in the intermediate configuration column as the quantized value range information corresponding to the corresponding preset code rate information may include:

Firstly, for any preset code rate information (which may be referred to as current preset code rate information), sample image components of a plurality of optimized sample images under a target component type can be encoded according to any intermediate value range information corresponding to the current preset code rate information, so as to obtain encoding results of various optimized sample images under the intermediate value range information. Then, according to the coding performance information of the various optimized sample images under the middle value range information, the average coding performance information of the middle value range information can be determined; wherein the coding performance information may include any one or more of: PSNR (PEAK SIGNAL to Noise Ratio), R-D (Rate-Distortion) performance information, BD-RATE performance information, and so on; PSNR can be used to measure the quality difference between an image before encoding and a reconstructed image; the R-D performance information can be used for measuring the relation between the coding rate after the image coding and the image distortion degree; the BD-RATE performance information may be used to measure the relationship between the coding RATE and the coding quality after image coding. And then, determining the highest average coding performance information in the average coding performance information of a plurality of pieces of intermediate value range information corresponding to the current preset code rate information, and determining the intermediate value range information corresponding to the highest average coding performance information as the quantized value range information corresponding to the current preset code rate information.

It should be noted that, if any of the following situations occurs, the value range configuration list needs to be reconfigured again: ① A new quantization mode is adopted in the encoding and decoding process; ② New image components appear in the encoding and decoding process; ③ New image types (e.g., computer generated animated images or images of screen content) are processed in the codec process; ④ New code rate information is adopted in the coding and decoding process; and ⑤, a new performance enhancing tool module (tool) is adopted in the encoding and decoding process, for example, the image encoding and decoding architecture shown in fig. 1 does not contain the performance enhancing tool module, and if the new performance enhancing tool module is introduced into the image encoding and decoding framework shown in fig. 1, the value range configuration list needs to be reconfigured again; etc. By the method, the value range configuration list can be adapted to the newly-appearing coding and decoding conditions, so that the determined quantized value range information can be adapted to the newly-appearing coding and decoding conditions besides being adapted to the image components.

In the embodiment of the application, the value range configuration list corresponding to each component type contains the optimal quantized value range information corresponding to each preset code rate information, so that the quantized value range information matched with the image component can be found out from the value range configuration list corresponding to the component type to which any image component belongs based on the given code rate information.

The embodiment of the application provides an image processing method, which mainly introduces a scheme that a coding device predicts based on a value range prediction model of code rate information of a target image component to obtain quantized value range information of the target image component. The image processing method may be performed by a computer device, which may be the encoding device 201 in the image processing system shown in fig. 2 described above. As shown in fig. 7, the image processing method may include the following steps S701 to S703:

S701, determining a target component type to which the target image component belongs.

S702, acquiring a value range prediction model matched with code rate information of a target image component from a plurality of value range prediction models under the target component type.

In step S701-step S7023, the encoder may be configured with a plurality of preset code rate information, and under each preset code rate information, a value range prediction model of a plurality of different component types is included; for example, the encoder may be configured with a plurality of preset code rate information, and under each preset code rate information, a value range prediction model of a luminance component type and a value range prediction model of a chrominance component type may be included. Or it may be understood that the encoder may be configured with a plurality of different component types, and under each component type, the value range prediction model includes a plurality of different preset code rate information; for example, the encoder may be configured with a luminance component type and a chrominance component type, and the luminance component type may include a plurality of value range prediction models of preset code rate information, and the chrominance component type may include a plurality of value range prediction models of preset code rate information.

Based on this, for any one of the N image components (may be referred to as a target image component), a target component type to which the target image component belongs may be determined, and among a plurality of value range prediction models under the target component type, a value range prediction model that matches code rate information of the target image component is acquired. Specifically, each value range prediction model under the target component type corresponds to respective preset code rate information, and the reference code rate information matched with the code rate information of the target image component can be determined in a plurality of preset code rate information under the target component type; here, the matching of the code rate information of the target image component with the reference code rate information may specifically refer to: the code rate information of the target image component is the same as the reference code rate information; or the code rate information of the target image component is similar to the reference code rate information, and the similarity can be understood as: and the difference between the code rate information of the target image component and the reference code rate information is the smallest in a plurality of pieces of preset code rate information contained in the value range configuration list corresponding to the target component type. Then, the value range prediction model corresponding to the reference code rate information may be used as a value range prediction model matched with the code rate information of the target image component.

For example, the luminance component type includes preset code rate information of 0.06, 0.12, 0.25, 0.5, 0. 75. 1.0, 1.25, 1.5, 1.75, and 2.0; for example, the code rate information of the target image component is 1.75, the preset code rate information included in the brightness component type has the same reference code rate information 1.75 as the code rate information of the target image component, and the value range prediction model corresponding to the reference code rate information 1.75 can be determined as the value range prediction model matched with the code rate information of the target image component; for another example, the code rate information of the target image component is 1.47, the preset code rate information included in the brightness component type has the reference code rate information 1.5 similar to the code rate information of the target image component, and the value range prediction model corresponding to the reference code rate information 1.5 can be determined as the value range prediction model matched with the code rate information of the target image component.

S703, invoking a matched value range prediction model, and carrying out quantization value range prediction on the target image component to obtain quantization value range information of the target image component.

After the value range prediction model matched with the target code rate information is obtained, the matched value range prediction model can be called, and the quantized value range prediction is carried out on the target image component to obtain quantized value range information of the target image component.

As can be seen from the contents of steps S701-S703, the scheme for configuring the quantized value range information based on the machine-learned value range prediction model can be summarized in fig. 8: taking an example that an image to be decoded includes a luminance component and a chrominance component as shown in fig. 8, 2 value range prediction models which are matched with the code rate information of the target image component can be determined based on the code rate information of the given target image component, wherein the value range prediction models are respectively a value range prediction model corresponding to the luminance component and a value range prediction model corresponding to the chrominance component; then, a value range prediction model corresponding to the brightness component can be called to conduct quantization value range prediction on the brightness component, quantization value range information of the brightness component is obtained, and the brightness component is encoded based on the quantization value range information of the brightness component; and a value range prediction model corresponding to the chroma component can be called to carry out quantization value range prediction on the chroma component, quantization value range information of the chroma component is obtained, and the chroma component is encoded based on the quantization value range information of the chroma component; then, the encoded code stream file, the quantized value range information of the luminance component, and the quantized value range information of the chrominance component may be output, and may be written as encoding parameters into the code stream file.

In configuring the quantized value range information by the machine learning-based value range prediction model described in step S701-step S703, the configuration process considers specific content of the image component in addition to the code rate information, so that the quantized value range information obtained by model prediction can be more suitable for the image component.

The above description focuses on the process of predicting and quantifying the value range information based on the value range prediction model, and the following description focuses on the training process of the value range prediction model:

For the luminance component type, first, candidate value range information may be acquired, for example, the candidate value range information may be α _{cand_} = {64, 128, 256, 512, 1024, and 2048}; the candidate value range information may be mapped to 6 categories c _{cand_t} = {0, 1, 2,3,4, and 5}, respectively. Secondly, 500 marked sample images can be randomly extracted from the training data set of the JPEG AI, and the intermediate value range information corresponding to each preset code rate information in the intermediate configuration list alpha _{output_cfg_} is traversed in sequence to be encoded. Then, for any preset code rate information (which can be called as current preset code rate information), calculating average coding performance information of the brightness components of 500 marked sample images under the intermediate value range information corresponding to the current preset code rate information, and then selecting a category corresponding to the intermediate value range information with the highest average coding performance information as a marked category (namely marked quantized value range information) corresponding to the current preset code rate information; according to the annotation category corresponding to each preset code rate information under the luminance component type, an annotation list under the luminance component type (c _{Y_}) can be generated, and similarly, an annotation list under the chrominance component type (c _{UV_}) can be generated. After that, for any preset code rate information, 2 value range prediction models (quantized value range information for predicting luminance components and chrominance components, respectively) are trained respectively, the input of the input model is an image, and the output is quantized value range information of the image.

The training process of the value range prediction model under the above brightness component type can be summarized as follows: for a value range prediction model corresponding to any preset coding information (which can be reference code rate information) under a target component type, an initial prediction model corresponding to the reference code rate information can be obtained, and a sample image component of a training sample image under the target component type can be obtained; the initial prediction model can be called, and the quantized value range prediction is carried out on the sample image component to obtain quantized value range information matched with the sample image component; then, according to the difference between the predicted quantized value range information and the quantized value range information marked under the reference code rate information, the model parameters of the initial prediction model can be updated to obtain a value range prediction model corresponding to the reference code rate information.

The determining process of the quantized value range information marked under the reference code rate information can include: firstly, a plurality of marked sample images can be obtained; according to any candidate value range information configured by the reference code rate information, respectively encoding sample image components of the plurality of marked sample images under the target component type to obtain encoding performance information of the plurality of marked sample images under the candidate value range information; wherein the coding performance information may include any one or more of: PSNR (PEAK SIGNAL to Noise Ratio), R-D (Rate-Distortion) performance information, BD-RATE performance information, and so on; PSNR can be used to measure the quality difference between an image before encoding and a reconstructed image; the R-D performance information can be used for measuring the relation between the coding rate after the image coding and the image distortion degree; the BD-RATE performance information may be used to measure the relationship between the coding RATE and the coding quality after image coding. Secondly, determining average coding performance information of candidate value range information according to coding performance information of a plurality of marked sample images under the candidate value range information; and determining the highest coding performance information in the average coding performance information of the plurality of pieces of candidate value range information of which the reference code rate information is configured, and determining the candidate value range information corresponding to the highest coding performance information as quantized value range information marked under the reference code rate information.

It should be noted that, if any one of the following situations occurs, the value range prediction model needs to be trained again: ① A new quantization mode is adopted in the encoding and decoding process; ② New image components appear in the encoding and decoding process; ③ New image types (e.g., computer generated animated images or images of screen content) are processed in the codec process; ④ New code rate information is adopted in the coding and decoding process; and a new performance enhancing tool module (tool) is adopted in the ⑤ codec process, for example, the image codec architecture shown in fig. 1 does not include the performance enhancing tool module, and if the new performance enhancing tool module is introduced into the image codec framework shown in fig. 1, the value range prediction model needs to be trained again; etc. By the method, the value range prediction model can be adapted to the newly-appearing coding and decoding conditions, so that the predicted quantized value range information can be adapted to the newly-appearing coding and decoding conditions besides being adapted to the image components.

It should be further noted that the foregoing describes a first scheme for configuring quantized value range information based on a machine-learned value range prediction model, where a plurality of different component types may be configured in an encoder, and each component type includes a plurality of value range prediction models with different preset code rate information, so that after determining a target component type to which a target image component type belongs, it is necessary to determine, under the target component type, a quantized value range prediction model that matches code rate information of a given target image component. Optionally, a second scheme may exist based on the machine learning configuration quantization value range information, where a plurality of different component types may be configured in the encoder, and each component type may correspond to one value range prediction model, so after determining a target component type to which the target image component belongs, the value range prediction model corresponding to the target component type may be called, and quantization value range prediction may be performed on the target image component based on code rate information of the target image component, to obtain quantization value range information of the target image component. That is, the two schemes differ in that the value range prediction model in the first scheme is related to the code rate information, and thus, the input value range prediction model is an image; in the second scheme, the value range prediction model is irrelevant to the code rate information, so that the code rate information needs to be input in addition to the image of the value range prediction model.

In the embodiment of the application, the quantized value range information of the model prediction can be trained, and in the model prediction process, not only the code rate information but also the image content of the image component are considered, so that the quantized value range information obtained by the model prediction is more suitable for the image component.

The embodiment of the application provides an image processing method, which mainly introduces an image coding process. The image processing method may be performed by a computer device, which may be the encoding device 201 in the image processing system shown in fig. 2 described above. As shown in fig. 9, the image processing method may include, but is not limited to, the following steps S901 to S903:

S901, obtaining coding information of an image to be coded.

S902, based on coding information of the image to be coded, quantitative value range information of N image components of the image to be coded is determined.

In step S901 to step S902, the image to be encoded in the image encoding stage corresponds to the image to be decoded in the image decoding stage, where the correspondence means that the image to be encoded in the image encoding stage and the image to be decoded in the image decoding stage are different names of the same image in different stages. The coding information of the image to be coded can be obtained, and the quantization value range information of N image components of the image to be coded is determined based on the coding information of the image to be coded. Wherein, an image component is matched with a quantization value range information, any quantization value range information is used for indicating the value range of quantization symbols of the corresponding image component, and N is a positive integer.

Based on the coding information of the image to be coded, the method for determining the quantization value range information of the N image components of the image to be coded can comprise any one of the following steps:

In one implementation, the quantization range information of the N image components may be determined based on the encoding information of the image to be encoded, that is, for any one of the N image components (may be referred to as a target image component), the quantization range information of the target image component may be determined based on the encoding information of the image to be encoded.

In another implementation, based on the encoding information of the image to be encoded, quantization value range information of a target image component of the N image components is determined, where the target image component may be any one of the N image components; then, the quantized value range information of the other image components can be determined according to the mapping relation between the quantized value range information of the target image component and the quantized value range information of the other image components in the N image components, and the mapping relation can be specifically referred to the above formula 1.

Whether the quantization value range information of N image components is determined based on the coding information of the image to be coded or the quantization value range information of the target image component in the N image components is determined based on the coding information of the image to be coded, the quantization value range information of the target image component in the N image components is determined according to the mapping relation between the quantization value range information of the target image component and the quantization value range information of other image components in the N image components, and the quantization value range information of the other image components is determined based on the coding information of the image to be coded. The method for determining the quantization value range information of the target image component in the N image components based on the coding information of the image to be coded may include any one of the following:

First, the coding information of the image to be coded may include information that the target image component actually needs to be compressed, where the information that the target image component actually needs to be compressed refers to residual information of the target image component, that is, the coding information of the image to be coded may include residual information of the target image component, and quantization value range information of the target image component may be determined based on the residual information of the target image component; this approach can be seen in particular from the description of the embodiment shown in fig. 4 above.

Second, the coding information of the image to be coded may include code rate information of a target image component, where N image components may share the same target code rate information, that is, the coding information of the image to be coded may include the same target code rate information, where the code rate information of the target image component is the same target code rate information, or the N image components correspond to respective code rate information; thus, the quantization value range information of the target image component can be determined based on the code rate information of the target image component. The method for determining the quantization value range information of the target image component based on the code rate information of the target image component may include any one of the following: inquiring a value range configuration list based on code rate information of the target image component to obtain quantized value range information of the target image component, wherein the mode can be specifically described with reference to the embodiment shown in the above-mentioned figure 5; or predicting based on the value range prediction model of the code rate information of the target image component to obtain the quantized value range information of the target image component, which can be specifically referred to the description of the embodiment shown in fig. 7.

S903, coding the N image components according to the quantized value range information of the N image components.

After the quantization value range information of the N image components is determined, the N image components may be encoded according to the quantization value range information of the N image components. Specifically, the N image components may be quantized based on quantization value range information of the N image components, to obtain quantization symbols of the N image components; then, the quantized symbols of the N image components may be encoded to obtain a code stream file, which may be transmitted to a decoding apparatus.

Taking any one of the N image components (may be referred to as a target image component) as an example, quantizing the target image component may specifically refer to quantizing residual information of the target image component; the residual information of the target image component may include residual values of each pixel point in the target image component, and the residual values of each pixel point in the target image component may be quantized based on quantization value range information of the target image component to obtain quantization symbols of each pixel point in the target image component; then, the quantized symbols of each pixel point in the target image component may be encoded; the quantization symbol of each pixel point in the target image component is in the value range indicated by the quantization value range information of the target image component.

The residuals for each pixel point in the target image component may be of a floating point type, and the essence of quantization may be a process of converting the residual value of the floating point type into a quantized symbol of an integer type. For example, the target image component is a luminance component, residual information of the luminance component may be represented as r _Y, and a quantization result of residual information of the luminance component may be represented asThe quantization process can be seen in equation 4 below:

in the above formula 4, Q (r _Y) represents quantization operation, α _Y represents quantization value range information, CLIP represents truncation operation, and the process of truncation operation can be specifically referred to as the following formula 5:

As can be seen from the above equation 5, the purpose of the truncation operation is to: the quantization operation result beyond the value range (-alpha _Y,α_Y) is truncated, and the result is truncated within the value range (-alpha _Y,α_Y).

It should be noted that, as can be seen from the foregoing, the encoder may include a plurality of trained image coding models, each image coding model corresponds to a different code rate, and the image coding model whose code rate matches the code rate information of the target image component may be determined from the plurality of image coding models included in the encoder, and then the matched image coding model may be called to encode the target image component based on the quantized value range information of the target image component. The code rate information of the target image component may be target code rate information shared by the target image component and other image components in the N image components, or may be code rate information corresponding to the target image component alone, which is not limited in the embodiment of the present application.

It should be noted that, the determined quantization value range information of the N image components may be written into the encoded code stream file, and the manner of writing the quantization value range information of the N image components into the code stream file may include any one of the following:

(1) The quantized value range information of the N image components is written into the code stream file, so that the decoding device can parse the quantized value range information of the N image components from the code stream file. Taking any one of the N image components (may be referred to as a target image component) as an example, the manner of writing the quantization value range information of the N image components into the code stream file, respectively, may include any one of the following:

① And directly writing the quantization value range information of the target image component into the code stream file.

② And calculating the quantized value range information of the target image component to obtain the value range mark information of the target image component, and writing the quantized value range mark information of the target image component into the code stream file. The operations herein may be logarithmic, and an exemplary procedure may be described as follows: the target image component is a luminance component, logarithmic calculation (logarithmic calculation based on m, where m is a constant, for example, m=2) may be performed on quantized value range information α _Y of the luminance component, value range flag information α _{Y_} of the luminance component is obtained, and value range flag information α _{Y_} of the luminance component may be written into the code stream file. By the method, the value range mark information written into the code stream file is smaller than the quantized value range information, so that the coding efficiency can be improved.

③ And calculating the quantized value range information of the target image component and the basic information of the target image component to obtain the value range mark information of the target image component, and writing the quantized value range mark information of the target image component into the code stream file. The operation herein may specifically refer to that after performing a logarithmic operation (logarithmic operation based on m, where m is a constant, for example, m=2) on the quantized value range flag information of the target image component, the logarithmic operation result is subtracted from the base information of the target image component; an exemplary process of operation may be described as follows: the target image component is a luminance component, logarithmic operation (logarithmic operation based on m, where m is a constant, for example, m=2) may be performed on quantized value range information α _Y of the luminance component to obtain a logarithmic operation result, and then a difference value between the logarithmic operation result and basic information α _{Y_} of the target image component may be used as value range flag information α _{Y_} of the luminance component, and the value range flag information α _{Y_} of the luminance component may be written into the code stream file. By the method, the value range mark information written into the code stream file is smaller than the quantized value range information, so that the coding efficiency can be improved.

The value range marking information of the target image component and the value range marking information of the reference image component can be operated to obtain value range compression information of the target image component, and the value range compression information of the target image component is written into the code stream information; the operation here may be to subtract the value range flag information of the reference image component from the value range flag information of the target image component. The reference image component is any one of the N image components except the target image component, and the value range flag information of the reference image component may be calculated by writing the quantized value range information of the N image components into any one of ② or ③ of the code stream file. For example, the reference image component is a luminance component, the target image component is a chrominance component, the value range flag information of the luminance component is α _{Y_}, the value range flag information of the chrominance component is α _{UV_}, the value range flag information of the luminance component is α _{Y_} and then the value range flag information of the luminance component is α _{Y_}, and the difference (α _{Y_}-α_{UV_}) of the value range flag information of the chrominance component is α _{UV_} and is written as the value range compression information of the chrominance component. In this way, the compression information of the value range in the code stream file is smaller than the quantized value range information and the value range mark information, so that the coding efficiency can be improved.

(2) The quantized value range information of the target image component in the N image components can be written into the code stream file, the quantized value range information of other image components except the target image component in the N image components has a mapping relation with the quantized value range information of the target image component, and the target image component can be any one of the N image components; therefore, the decoding device can analyze lotus value range information of the target image component from the code stream file, and then can determine the quantized value range information of other image components according to the mapping relation between the quantized value range information of the target image component and the quantized value range information of other image components in the N image components. The method of writing the target image component into the code stream information may be any one of the methods ①－③ of writing the quantized value range information of the N image components into the code stream file.

It should be noted that, the manner of determining the quantization value range information of the N image components may be combined with the manner of writing the quantization value range information of the N image components into the code stream file. For example, when the quantized value range information of the N image components is determined based on the encoded information of the image to be decoded, the quantized value range information of the N image components may be written into the code stream file, respectively. For another example, when the quantized value range information of the target image component of the N image components is determined based on the encoded information of the image to be decoded, the quantized value range information of the target image component may be written into the code stream file when the quantized value range information of the other image component is determined according to the mapping relationship between the quantized value range information of the target image component and the quantized value range information of the other image component of the N image components.

In the embodiment of the application, the quantization value range information of each image component can be determined based on the coding information of the image to be coded, so that the determined quantization value range information and the image component have higher adaptation degree, thereby reducing code rate loss, improving coding efficiency and improving coding effect of the image component.

The foregoing details of the method of embodiments of the present application are provided for the purpose of better implementing the foregoing aspects of embodiments of the present application, and accordingly, the following provides an apparatus of embodiments of the present application.

Referring to fig. 10, fig. 10 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application, where the image processing apparatus may be disposed in a computer device provided in the embodiment of the present application, and the computer device may be a decoding device. The image processing apparatus shown in fig. 10 may be a computer program (comprising program code) running in a computer device, which image processing apparatus may be adapted to perform part or all of the steps of the method embodiments shown in fig. 3,4,5 or 7. Referring to fig. 10, the image processing apparatus may include the following units:

An obtaining unit 1001, configured to obtain a code stream file;

A processing unit 1002, configured to parse quantized value range information of N image components of an image to be decoded from a code stream file; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded;

the processing unit 1002 is further configured to decode the N image components according to the quantized value range information of the N image components.

In one implementation manner, the processing unit 1002 is configured to, when parsing the quantized value range information of N image components of the image to be decoded from the code stream file, specifically perform the following steps:

In one implementation manner, the processing unit 1002 is configured to, when parsing the quantized value range information of each of N image components of the image to be decoded from the code stream file, specifically perform the following steps:

Acquiring a plurality of marked sample images;

In one implementation, the processing unit 1002 is configured to perform, when decoding N image components according to the quantized value range information of the N image components, the following steps:

analyzing quantization symbols of N image components from a code stream file;

According to another embodiment of the present application, each unit in the image processing apparatus shown in fig. 10 may be separately or completely combined into one or several additional units, or some unit(s) thereof may be further split into a plurality of units having smaller functions, which may achieve the same operation without affecting the achievement of the technical effects of the embodiments of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the image processing apparatus may also include other units, and in practical applications, these functions may also be realized with assistance of other units, and may be realized by cooperation of a plurality of units.

According to another embodiment of the present application, an image processing apparatus as shown in fig. 10 may be constructed by running a computer program (including program code) capable of executing some or all of the steps involved in the method shown in fig. 3, 4, 5 or 7 on a general-purpose computing device such as a computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and a storage element, and implementing the image processing method of the embodiment of the present application. The computer program may be recorded on, for example, a computer-readable storage medium, and loaded into and executed by the computing device described above.

Referring to fig. 11, fig. 11 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present application, where the image processing apparatus may be disposed in a computer device provided in the embodiment of the present application, and the computer device may be an encoding device. The image processing apparatus shown in fig. 11 may be a computer program (comprising program code) running in a computer device, which image processing apparatus may be adapted to perform part or all of the steps of the method embodiments shown in fig. 4, 5, 7 or 9. Referring to fig. 11, the image processing apparatus may include the following units:

An acquisition unit 1101 for acquiring encoding information of an image to be encoded;

The processing unit 1102 is configured to determine quantization value range information of N image components of an image to be encoded based on encoding information of the image to be encoded; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component, and N is a positive integer;

the processing unit 1102 is further configured to encode the N image components according to quantization value range information of the N image components.

According to another embodiment of the present application, each unit in the image processing apparatus shown in fig. 11 may be configured by combining each unit into one or several other units, respectively, or some unit(s) thereof may be configured by splitting into a plurality of units having smaller functions, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present application, the image processing apparatus may also include other units, and in practical applications, these functions may also be realized with assistance of other units, and may be realized by cooperation of a plurality of units.

According to another embodiment of the present application, an image processing apparatus as shown in fig. 11 may be constructed by running a computer program (including program code) capable of executing some or all of the steps involved in the method shown in fig. 4, 5, 7 or 9 on a general-purpose computing device such as a computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), and the like, and a storage element, and implementing the image processing method of the embodiment of the present application. The computer program may be recorded on, for example, a computer-readable storage medium, and loaded into and executed by the computing device described above.

Based on the method and the device embodiments, the embodiment of the application provides a computer device. Referring to fig. 12, fig. 12 is a schematic structural diagram of a computer device according to an embodiment of the application. The computer device shown in fig. 12 includes at least a processor 1201, an input interface 1202, an output interface 1203, and a computer readable storage medium 1204. Wherein the processor 1201, the input interface 1202, the output interface 1203, and the computer readable storage medium 1204 may be connected by a bus or other means.

The computer readable storage medium 1204 may be stored in a memory of a computer device, the computer readable storage medium 1204 for storing a computer program comprising computer instructions, and the processor 1201 for executing the program instructions stored by the computer readable storage medium 1204. The processor 1201 (or CPU (Central Processing Unit, central processing unit)) is a computing core and a control core of a computer device, which is adapted to implement one or more computer instructions, in particular to load and execute one or more computer instructions to implement a corresponding method flow or a corresponding function.

The embodiment of the application also provides a computer readable storage medium (Memory), which is a Memory device in the computer device and is used for storing programs and data. It is understood that the computer readable storage medium herein may include both built-in storage media in a computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores an operating system of the computer device. Also stored in the memory space are one or more computer instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor. Note that the computer readable storage medium can be either a high-speed RAM Memory or a Non-Volatile Memory (Non-Volatile Memory), such as at least one magnetic disk Memory; optionally, at least one computer readable storage medium remotely located from the aforementioned processor.

In some embodiments, the computer device may be a decoding device that may be loaded by the processor 1201 and execute one or more computer instructions stored in the computer readable storage medium 1204 to implement the corresponding steps described above in connection with the image processing methods illustrated in fig. 3,4, 5, or 7. In particular implementations, computer instructions in computer-readable storage medium 1204 are loaded by processor 1201 and perform the steps of:

acquiring a code stream file;

In one implementation, when the computer instructions in the computer readable storage medium 1204 are loaded by the processor 1201 and execute the quantization range information of N image components of the image to be decoded, the method specifically includes the following steps:

In one implementation, the computer instructions in the computer readable storage medium 1204 are loaded by the processor 1201 and executed to parse the quantized value range information of each of the N image components of the image to be decoded from the code stream file, respectively, specifically for performing the following steps:

Acquiring a plurality of marked sample images;

In one implementation, the computer instructions in the computer readable storage medium 1204 are loaded by the processor 1201 and execute the steps of, when decoding N image components, specifically:

analyzing quantization symbols of N image components from a code stream file;

In these embodiments, the quantized value range information of each of the N image components of the image to be decoded that is matched with each other may be parsed from the code stream file, and the quantized value range information of the N image components is determined based on the encoding information of the image to be decoded; that is, given the encoding information of the image to be decoded, the appropriate quantization value range information can be determined for each image component of the image to be decoded for decoding, so that the image decoding effect can be effectively improved.

In other embodiments, the computer device may be an encoding device that may be loaded by the processor 1201 and execute one or more computer instructions stored in the computer readable storage medium 1204 to perform the corresponding steps described above with respect to the image processing methods illustrated in fig. 4, 5, 7, or 9. In particular implementations, computer instructions in computer-readable storage medium 1204 are loaded by processor 1201 and perform the steps of:

acquiring coding information of an image to be coded;

In these embodiments, the quantization value range information of each image component may be determined based on the encoding information of the image to be encoded, so that the determined quantization value range information and the image component have a higher degree of adaptation, thereby reducing the code rate loss, improving the encoding efficiency, and improving the encoding effect of the image component.

According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the image processing methods provided in the above-described various alternative manners.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

acquiring a code stream file;

Analyzing quantization value range information of N image components of an image to be decoded from the code stream file; wherein, one image component is matched with one quantization value range information, any one quantization value range information is used for indicating the value range of the quantization symbol of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded;

2. The method of claim 1, wherein parsing the quantized span information of N image components of the image to be decoded from the bitstream file comprises:

and respectively analyzing the quantized value range information of each image component in the N image components of the image to be decoded from the code stream file.

3. The method according to claim 2, wherein the parsing the quantized span information of each of the N image components of the image to be decoded from the code stream file includes:

4. The method of claim 1, wherein parsing the quantized span information of N image components of the image to be decoded from the bitstream file comprises:

Analyzing quantization value range information of a target image component in N image components of the image to be decoded from the code stream file;

5. The method of claim 1, wherein the determining of the quantized span information of the N image components comprises:

based on the coding information of the image to be decoded, respectively determining quantization value range information of each image component in the N image components; or alternatively

Determining quantization value range information of a target image component in the N image components based on the coding information of the image to be decoded; and determining the quantized value range information of the other image components according to the mapping relation between the quantized value range information of the target image component and the quantized value range information of the other image components in the N image components.

6. The method of any of claims 1-5, the encoding information of the image to be decoded comprising residual information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

And determining quantization value range information of the target image component based on the residual information of the target image component.

7. The method of claim 6, wherein the residual information corresponding to the target image component comprises: residual values of all pixel points in the target image component; the determining the quantization value range information of the target image component based on the residual information of the target image component includes:

And screening the quantized value range information of the target image component in the candidate value range information according to the quantized information screening condition.

8. The method according to any one of claims 1-5, wherein the coding information of the image to be decoded comprises code rate information of a target image component of the N image components; the quantization value range information determining process of the target image component comprises the following steps:

And determining quantization value range information of the target image component based on the code rate information of the target image component.

9. The method of claim 8, wherein the determining the quantized span information of the target image component based on the code rate information of the target image component comprises:

acquiring a value range prediction model matched with code rate information of the target image component from a plurality of value range prediction models under the target component type;

And calling the matched value range prediction model, and carrying out quantization value range prediction on the target image component to obtain quantization value range information of the target image component.

10. The method of claim 9, wherein each value range prediction model under the target component type corresponds to respective preset code rate information;

The obtaining a value range prediction model matched with the code rate information of the target image component in the multiple value range prediction models under the target component type comprises the following steps:

determining reference code rate information matched with the code rate information of the target image component in a plurality of pieces of preset code rate information under the target component type;

11. The method of claim 10, wherein the training process of the value range prediction model corresponding to the reference code rate information comprises:

Acquiring an initial prediction model corresponding to the reference code rate information, and acquiring a sample image component of a training sample image under the target component type;

invoking the initial prediction model, and predicting the quantized value range of the sample image component to obtain quantized value range information matched with the sample image component;

12. The method of claim 11, wherein the reference code rate information is configured with a plurality of candidate value range information; the method further comprises the steps of:

Acquiring a plurality of marked sample images;

According to any candidate value range information configured by the reference code rate information, respectively encoding sample image components of a plurality of marked sample images under the target component type to obtain encoding performance information of the plurality of marked sample images under the candidate value range information;

determining average coding performance information of the candidate value range information according to coding performance information of the plurality of marked sample images under the candidate value range information;

13. The method of claim 8, wherein the determining the quantized span information of the target image component based on the code rate information of the target image component comprises:

acquiring a value range prediction model corresponding to the target component type;

14. The method of claim 8, wherein the determining the quantized span information of the target image component based on the code rate information of the target image component comprises:

acquiring a value range configuration list corresponding to the target component type; the target component type is configured with a plurality of pieces of preset code rate information, and the value range configuration list comprises quantized value range information corresponding to each piece of preset code rate information;

Searching reference code rate information matched with the code rate information of the target image component in the value range configuration list, and taking the quantized value range information corresponding to the reference code rate information as quantized value range information matched with the target image component.

15. The method of claim 14, wherein the configuring of the value range configuration list corresponding to the target component type includes:

Based on sample image components of the training sample image under the target component type, selecting intermediate value range information matched with corresponding preset code rate information from candidate value range information corresponding to each preset code rate information, and generating an intermediate configuration list according to the plurality of preset code rate information and the intermediate value range information matched with each preset code rate information;

And selecting middle value range information with high coding performance from middle value range information corresponding to each preset code rate information contained in the middle configuration column based on sample image components of the optimized sample image under the target component type, and optimizing the middle configuration list according to the plurality of preset code rate information and the quantized value range information corresponding to each preset code rate information to obtain a value range configuration list corresponding to the target component type.

16. The method of claim 15, wherein the number of training sample images is a plurality of sheets; the selecting, based on the sample image component of the training sample image under the target component type, the intermediate value range information adapted to the corresponding preset code rate information from the candidate value range information corresponding to each preset code rate information, includes:

determining screening conditions of the preset code rate information according to the minimum average residual value and the maximum average residual value in residual information corresponding to the training sample images;

17. The method according to any one of claims 1-5, wherein decoding the N image components according to quantized span information of the N image components, comprises:

Analyzing the quantized symbols of the N image components from the code stream file;

Reconstructing the N image components based on residual information of the N image components.

18. An image processing method, comprising:

acquiring coding information of an image to be coded;

Determining quantization value range information of N image components of the image to be encoded based on the encoding information of the image to be encoded; wherein, one image component is matched with one quantized value range information, any quantized value range information is used for indicating the value range of quantized symbols of the corresponding image component, and N is a positive integer;

19. An image processing apparatus, comprising:

the acquisition unit is used for acquiring the code stream file;

the processing unit is used for analyzing the quantized value range information of N image components of the image to be decoded from the code stream file; wherein, one image component is matched with one quantization value range information, any one quantization value range information is used for indicating the value range of the quantization symbol of the corresponding image component; the quantization value range information of the N image components is determined based on the coding information of the image to be decoded;

The processing unit is further configured to decode the N image components according to quantization value range information of the N image components.

20. An image processing apparatus, comprising:

the processing unit is further configured to encode the N image components according to quantization value range information of the N image components.

21. A computer device, the computer device comprising:

A processor adapted to implement a computer program;

a computer readable storage medium storing a computer program adapted to be loaded by the processor and to perform the image processing method of any one of claims 1-17, or of claim 18.

22. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program adapted to be loaded by a processor and to perform the image processing method according to any one of claims 1-17 or claim 18.