CN115705614B - Image processing method, device, electronic equipment and storage medium - Google Patents
Image processing method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115705614B CN115705614B CN202110897405.XA CN202110897405A CN115705614B CN 115705614 B CN115705614 B CN 115705614B CN 202110897405 A CN202110897405 A CN 202110897405A CN 115705614 B CN115705614 B CN 115705614B
- Authority
- CN
- China
- Prior art keywords
- image
- type
- image processing
- processing module
- format
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 317
- 238000000034 method Methods 0.000 claims abstract description 55
- 230000008569 process Effects 0.000 claims abstract description 47
- 230000000694 effects Effects 0.000 abstract description 14
- 238000012937 correction Methods 0.000 description 17
- 230000009467 reduction Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Landscapes
- Studio Devices (AREA)
Abstract
The present disclosure relates to an image processing method, apparatus, electronic device, and storage medium, wherein the method is applied to an image signal processor, and includes: acquiring an image to be processed and the type of information contained in the image; determining an image processing module associated with the type; and processing the image by adopting an image processing module related to the type to obtain a processed image. Therefore, the image signal processor processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved.
Description
Technical Field
The disclosure relates to the technical field of image processing, and in particular relates to an image processing method, an image processing device, electronic equipment and a storage medium.
Background
With the continuous development of terminal technology, more and more users use electronic devices to capture images. Currently, after an image is captured by a camera in an electronic device, the captured image may be processed by an image signal Processor (IMAGE SIGNAL Processor, abbreviated as ISP) in a processing chip in the electronic device, so as to obtain an image in a format that can be displayed.
In the related art, an ISP in a processing chip is dedicated to processing an image containing a certain type of information, and when processing an image containing another type of information, there may be a case where an image processing is erroneous or not processed.
Disclosure of Invention
The present disclosure aims to solve, at least to some extent, one of the technical problems in the related art.
For this reason, the present disclosure proposes the following technical scheme:
an embodiment of a first aspect of the present disclosure proposes an image processing method, applied to an image signal processor, the method including:
Acquiring an image to be processed and the type of information contained in the image; determining an image processing module associated with the type; and processing the image by adopting an image processing module related to the type to obtain a processed image.
In one embodiment of the present disclosure, the acquiring the image to be processed, and the type of information contained in the image, includes:
acquiring the image and the format of the image;
and determining the type of information contained in the image according to the format.
In one embodiment of the present disclosure, the determining, according to the format, a type of information contained in the image includes:
determining that the type includes a brightness type and a color type when the format is a first format;
When the format is the second format, it is determined that the type includes only a brightness type.
In one embodiment of the disclosure, the determining the image processing module associated with the type includes:
Determining a configuration file corresponding to the type, wherein the configuration file comprises: the marks of the image processing modules comprise a first mark and a second mark, wherein the first mark represents that the corresponding image processing module is related to the type, and the second mark represents that the corresponding image processing module is not related to the type;
and determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the type includes a brightness type and a color type, the flags of all image processing modules in the image signal processor are first flags;
when the type includes only a brightness type, the image brightness processing module in the image signal processor is marked with a first mark, and the other modules except the image brightness processing module are marked with a second mark.
In one embodiment of the present disclosure, the configuration file further includes: image processing parameters of the image processing module corresponding to the first mark;
The image processing module for processing the image by adopting the image processing module related to the type to obtain a processed image, which comprises the following steps:
Acquiring image processing parameters in the configuration file for each image processing module related to the type;
And controlling the image processing module to process the image according to the image processing parameters so as to obtain a processed image.
An embodiment of a second aspect of the present disclosure proposes an image processing apparatus including:
The acquisition module is used for acquiring an image to be processed and the type of information contained in the image;
a determining module for determining an image processing module associated with the type;
and the processing module is used for processing the image by adopting the image processing module related to the type to obtain a processed image.
In one embodiment of the present disclosure, the acquiring module includes:
an acquisition unit configured to acquire the image, and a format of the image;
And the first determining unit is used for determining the type of the information contained in the image according to the format.
In one embodiment of the disclosure, the determining unit is specifically configured to:
determining that the type includes a brightness type and a color type when the format is a first format;
When the format is the second format, it is determined that the type includes only a brightness type.
In one embodiment of the present disclosure, the determining module includes:
A second determining unit, configured to determine a configuration file corresponding to the type, where the configuration file includes: the marks of the image processing modules comprise a first mark and a second mark, wherein the first mark represents that the corresponding image processing module is related to the type, and the second mark represents that the corresponding image processing module is not related to the type;
And the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the type includes a brightness type and a color type, the flags of all image processing modules in the image signal processor are first flags;
when the type includes only a brightness type, the image brightness processing module in the image signal processor is marked with a first mark, and the other modules except the image brightness processing module are marked with a second mark.
In one embodiment of the present disclosure, the configuration file further includes: image processing parameters of the image processing module corresponding to the first mark;
The processing module is specifically configured to:
Acquiring image processing parameters in the configuration file for each image processing module related to the type;
And controlling the image processing module to process the image according to the image processing parameters so as to obtain a processed image.
An embodiment of a third aspect of the present disclosure provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to invoke and execute the executable instructions stored in the memory to implement the image processing method according to the embodiment of the first aspect of the present disclosure.
An embodiment of a fourth aspect of the present disclosure proposes a non-transitory computer-readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the image processing method proposed by the embodiment of the first aspect of the present disclosure.
An embodiment of a fifth aspect of the present disclosure proposes a computer program product, which, when executed by a processor of an electronic device, enables the electronic device to perform the image processing method proposed by the embodiment of the first aspect of the present disclosure.
According to the technical scheme, the image to be processed and the type of information contained in the image are acquired through the ISP, and an image processing module related to the type is determined, so that the image is processed by adopting the image processing module related to the type, and the processed image is obtained. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
Additional aspects and advantages of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of an image processing method according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of an image processing method according to another embodiment of the present disclosure;
FIG. 3 is a schematic diagram of an image processing flow in an embodiment of the disclosure;
FIG. 4 is a flowchart of an image processing method according to yet another embodiment of the present disclosure;
FIG. 5 is a flowchart of an image processing method according to yet another embodiment of the present disclosure;
fig. 6 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic structural view of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present disclosure and are not to be construed as limiting the present disclosure.
Currently, an ISP in a processing chip in an electronic device performs image processing specifically for an image containing a certain type of information, that is, the ISP can only support a camera in one format, and for images collected by cameras in other formats, errors may be directly reported without processing the image. Or the ISP has the same processing flow for all the images, and does not perform differential processing on the images containing different types of information, so that image processing errors may be caused.
For example, taking an electronic device as a machine dog, an ISP in the machine dog only supports cameras in bayer format, but not other formats, such as cameras in mono raw format. ISP in the robot dog processes the images collected by the camera in the Bayer format better, and the ISP reports the images collected by the camera in the mono-row format by mistake, and does not process the images collected by the camera in the mono-row format.
In the related art, an image may be processed by a central processing unit (Central Processing Unit, abbreviated as CPU) or a graphic processor (Graphics Processing Unit, abbreviated as GPU) in a processing chip to improve image quality. That is, the CPU or GPU may design a new image processing algorithm for an image containing other types of information to improve the processing effect of the image.
In this way, more CPU and GPU resources are occupied.
Image processing methods, apparatuses, electronic devices, and storage media of embodiments of the present disclosure are described below with reference to the accompanying drawings.
Fig. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure.
In the embodiment of the disclosure, the image processing method can be applied to an ISP, wherein the ISP is arranged in a processing chip of the electronic device.
The electronic device may be an intelligent robot, a robot dog, or any terminal device provided with a camera and a processing chip. The terminal device may be a hardware device with various operating systems, touch screens and/or display screens, such as a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and the like.
The processing chip may include an image processor ISP, CPU, GPU and an image encoder, among others.
As shown in fig. 1, the image processing method may include the steps of:
step 101, acquiring an image to be processed and the type of information contained in the image.
In the embodiment of the disclosure, the image to be processed may be an image acquired by a camera in the electronic device.
In the embodiment of the present disclosure, the information contained in the image may include at least one of color information and brightness information.
In the embodiments of the present disclosure, the type of information contained in the image is determined by the camera that acquired the image, and may include a brightness type and/or a color type.
In the embodiment of the disclosure, the ISP may acquire an image acquired by the camera, which is referred to as an image to be processed in the present disclosure, and acquire the type of information included in the image to be processed.
Step 102, determining an image processing module associated with the type.
In an embodiment of the present disclosure, the image processing module refers to a related module in the ISP for processing information included in an image, and the image processing module may include at least one of: an exposure module (such as an automatic exposure (Automatic Exposure, abbreviated as AE) module), a correction module (such as a gamma correction module, a color correction module), a noise reduction (denoise) module, an interpolation (demosaic) module, an automatic white balance (Automatic White Balance, abbreviated as AWB) module, an auto focus (Automatic Focusing, abbreviated as AF) module, and the like.
In embodiments of the present disclosure, an ISP, after determining the type of information contained in an image, may determine an image processing module associated with the type, or determine an image processing module that matches the type.
As an example, the ISP determines that the type of information contained in the image includes a brightness type, and an image processing module related to the brightness type may be an image processing module for processing brightness information, which is referred to as an image brightness processing module in this disclosure, and the image brightness processing module may include at least one of an exposure module, a gamma correction module, and a noise reduction module.
As another example, the ISP determines that the type of information contained in the image includes a color type, an image processing module associated with the color type, which may be an image processing module for processing color information, referred to in this disclosure as an image color processing module, which may include at least one of an interpolation module, an automatic white balance module, a color correction module, an RGB color space to YUV color space conversion module, a chromaticity/saturation adjustment module, and the like.
As yet another example, the ISP determines that the type of information contained in the image includes a color type and a brightness type, and the image processing modules associated with the type may be all the image processing modules in the ISP.
And step 103, processing the image by adopting an image processing module related to the type to obtain a processed image.
In the embodiment of the disclosure, by classifying the image processing modules in the ISP according to different types, the ISP processes the images by only adopting the image processing modules related to or matched with the types for the images containing different types of information, so that the image containing different types of information can be processed in a targeted manner, and the processing effect of the images can be improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
As an application scenario, if the information contained in the image includes only color information, the image color processing module may be determined from all the image processing modules in the ISP, and the image may be processed only by the image color processing module.
As another application scenario, if the information contained in the image includes only luminance information, the image luminance processing module may be determined from all the image processing modules in the ISP, and the image may be processed only by the image luminance processing module.
As still another application scenario, if the information contained in the image includes both color information and luminance information, all the image processing modules in the ISP may be used to process the image.
According to the image processing method, the image to be processed and the type of information contained in the image are acquired through the ISP, and the image processing module related to the type is determined, so that the image is processed by the image processing module related to the type, and the processed image is obtained. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
In one possible implementation manner of the embodiment of the present disclosure, the type of information contained in the image is related to the format of the camera that collects the image, that is, in the present disclosure, the type of information contained in the image may be determined according to the format of the camera that collects the image. The above process will be described in detail with reference to fig. 2.
Fig. 2 is a flowchart of an image processing method according to another embodiment of the present disclosure.
As shown in fig. 2, the image processing method may include the steps of:
in step 201, an image to be processed is acquired, as well as the format of the image.
In the embodiment of the disclosure, the format of the image is related to the format of the camera that collects the image, and the formats of the collected images may be different for cameras of different formats. For example, the format of the image may include a first format, a second format.
Wherein each pixel in the image in the first format has both luminance information and color information, for example, the first format may be bayer format; each pixel in the image in the second format has only luminance information and no color information, for example, the second format may be a mono-row format. For example, the image output by a common camera may be in a first format (such as a bayer format), and the image output by a black-and-white camera may be in a second format (such as a mono format).
Step 202, determining the type of information contained in the image according to the format.
In the embodiment of the present disclosure, a correspondence between different formats and types may be preset, so after determining the format of an image, the type corresponding to the format of the image may be obtained as the type of information contained in the image by querying the correspondence according to the format of the image.
As an example, the correspondence between the first pattern and the luminance type and the color type, and the correspondence between the second format and the luminance type may be preset. If the format of the image is the second format, the type corresponding to the second format can be determined to be the brightness type by querying the correspondence relationship, so that the brightness type can be used as the type of the information contained in the image.
In step 203, an image processing module associated with the type is determined.
And 204, processing the image by adopting an image processing module related to the type to obtain a processed image.
Steps 203 and 204 may be referred to the execution of steps 102 and 103 in the above embodiments, and will not be described herein.
In the related art, an ISP hardware pipeline only supports a camera in a bayer format, but does not support processing an image acquired by the camera in a mono-raw format, and when a system starts the camera in the mono-raw format, if the ISP hardware pipeline detects that the image acquired by the camera is in the mono-raw format, the ISP hardware pipeline reports errors and does not process the image.
In the embodiment of the disclosure, the camera information in the mono-row format may be fixedly configured into the camera information in the bayer format by modifying the driving configuration file. After the driver software reads and writes the driver configuration file, the camera in the mono-row format is considered to be a camera in the bayer format, so that errors cannot be reported, and an image in the mono-row format is sent to an ISP hardware pipeline for processing.
When the image in the mono-raw format is sent to the ISP hardware pipeline for processing, the ISP hardware pipeline processes the image in the bayer format, so that the image in the mono-raw format is processed in disorder, the processed image is abnormal, and the output image is not available.
As an example, the ISP hardware pipeline may include image processing modules related to not only color types, but also brightness types, where the mono-raw format image has no color component, and if all the image processing modules in the ISP hardware pipeline are used to process the mono-raw format image, the image processing may be wrong, as shown in fig. 3.
In view of the above problems, in the present disclosure, an image quality debug file (tuning file) of an ISP may be modified, and for a bayer format image, as in the prior art, all image processing modules in an ISP hardware pipeline are used to process the image, while for a mono format image, only an image processing module related to a brightness type in the ISP hardware pipeline may be used to process the image, and an image processing module related to a color type may not be used, so that the mono format image is processed by an image processing module necessary for lens correction, dead point compensation, noise reduction, automatic exposure, edge enhancement, contrast adjustment, gamma correction, and the like, and is not processed by an image processing module related to a color type, thereby avoiding an error situation of a processed image picture, and improving a processing effect of the image.
It should be noted that, the above is only exemplified by one ISP hardware pipeline, and in practical application, a plurality of ISP hardware pipelines may be used to process images collected by a plurality of cameras, that is, images collected by a plurality of cameras may be processed by different ISP hardware pipelines at the same time, which is not limited in the disclosure.
According to the image processing method, the image to be processed and the format of the image are acquired, and the type of information contained in the image is determined according to the format, so that the reliability of a type determination result can be improved.
In one possible implementation manner of the embodiment of the present disclosure, each type has a corresponding configuration file, and each type of corresponding configuration file includes an image processing module related to the corresponding type, so that in the present disclosure, the ISP may obtain, from the configuration file corresponding to the type of information included in the image to be processed, the image processing module related to the type. The above process will be described in detail with reference to fig. 4.
Fig. 4 is a flowchart of an image processing method according to still another embodiment of the present disclosure.
As shown in fig. 4, the image processing method may include the steps of:
Step 401, acquiring an image to be processed and a type of information contained in the image.
In the embodiment of the present disclosure, step 401 may be implemented by any one of the embodiments of the present disclosure, which is not limited to this, and is not repeated herein.
Step 402, determining a configuration file corresponding to the type, wherein the configuration file comprises: the labels of the image processing modules comprise a first label and a second label, wherein the first label represents that the corresponding image processing module is related to the type, and the second label represents that the corresponding image processing module is not related to the type.
In the embodiment of the disclosure, each type has a corresponding configuration file, and each type of configuration file contains an image processing module related to the corresponding type and an image processing module unrelated to the corresponding type. Wherein the image processing modules associated with a type have a first signature that characterizes the corresponding image processing module as being associated with the type and the image processing modules not associated with the type have a second signature that characterizes the corresponding image processing module as not being associated with the type.
For example, the first flag is denoted by the numeral 1, the second flag is denoted by the numeral 0, the image processing module related to the type 1 has a flag of "1", the image processing module unrelated to the type 1 has a flag of "0", and for example, the image processing module related to the type 2 has a flag of "1", and the image processing module unrelated to the type 2 has a flag of "0".
It should be noted that, the first mark is only indicated by the number 1, and the second mark is indicated by the number 0, but the disclosure is not limited thereto, and for example, the first mark and the second mark may be different numbers, or the first mark and the second mark may be different characters, such as the first mark is indicated by the character a, and the second mark is indicated by the character B, which is not limited thereto by the disclosure.
In the embodiment of the present disclosure, a correspondence between each type and a configuration file may be preset, so in the present disclosure, after determining the type of information included in an image, the corresponding relationship may be queried according to the type, and the configuration file corresponding to the type may be determined.
In step 403, the image processing module with the first flag in the configuration file is determined as the image processing module related to the type.
And step 404, processing the image by adopting an image processing module related to the type to obtain a processed image.
In the embodiment of the disclosure, the image processing module with the first mark in the configuration file can be used as the image processing module related to the type of the information contained in the image, so that the image can be processed by adopting the image processing module related to the type to obtain the processed image. That is, the image processing module with the second tag in the configuration file may be disabled, and the image is processed with the image processing module with the first tag, resulting in a processed image. For example, for an image in the second format, the image processing module with the second mark may be an image color processing module, the image processing module with the first mark may be an image brightness processing module, the image color processing module may be disabled, and the image brightness processing module is used to process the image, so as to obtain a processed image.
It will be appreciated that, for an image in the second format (each pixel has only luminance information and no color information), if all the image processing modules are used to process the image, for example, an image color processing module (an interpolation module, an automatic white balance module, a color correction module, etc.) is used to process the image, where the image color processing module processes the color of each pixel in the image, and at the same time, the luminance of each pixel may be affected, because the luminance may be represented by the color information, etc., so that the luminance of the image may be low.
In the application, aiming at the image in the second format, only the image brightness processing module is adopted to process the image, so that the condition that the output image is darker can be avoided, and the processing effect of the image is improved. That is, in the application, aiming at the images containing different types of information, only the image processing module relevant to the corresponding type is adopted to process the images, so that the targeted processing of the images containing different types of information can be realized, and the processing effect of the images is improved.
According to the image processing method, the image to be processed and the type of information contained in the image are acquired through the ISP, and the image processing module related to the type is determined, so that the image is processed by the image processing module related to the type, and the processed image is obtained. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
In a possible implementation manner of the embodiment of the present disclosure, the configuration file may further include image processing parameters of the image processing modules, so that in order to improve an image processing effect, the image may be processed by using the image processing parameters corresponding to each image processing module, to obtain a processed image. The above process will be described in detail with reference to fig. 5.
Fig. 5 is a flowchart of an image processing method according to still another embodiment of the present disclosure.
As shown in fig. 5, the image processing method may include the steps of:
Step 501, an image to be processed and the type of information contained in the image are acquired.
In the embodiment of the present disclosure, step 501 may be implemented in any manner in the embodiments of the present disclosure, which is not limited thereto, and is not repeated herein.
Step 502, determining a configuration file corresponding to the type, wherein the configuration file comprises: the marks of the image processing modules comprise a first mark and a second mark, the first mark represents that the corresponding image processing module is related to the type, the second mark represents that the corresponding image processing module is not related to the type, and the configuration file further comprises: the first mark corresponds to an image processing parameter of the image processing module.
In the embodiment of the present disclosure, step 502 may be implemented in any manner in various embodiments of the present disclosure, which is not limited thereto, and is not repeated herein.
In one possible implementation of the embodiment of the disclosure, when the type of information contained in the image includes a brightness type and a color type, the labels of all the image processing modules in the ISP are the first labels. When the type of information contained in the image includes only the brightness type, the marks of the image brightness processing modules in the ISP are first marks, and the marks of the other modules than the image brightness processing modules are second marks.
That is, all image processing modules in the ISP may be included in the profile, and when the type of information included in the image includes a brightness type and a color type, the marks of all image processing modules in the profile corresponding to the type may be first marks, and when the type of information included in the image includes only a brightness type, only the marks of the image brightness processing modules in the profile corresponding to the type are first marks, and the marks of other modules than the image brightness processing modules are second marks.
As an example, the image processing module in the ISP includes an exposure module (such as an auto exposure AE module), a gamma correction module, a noise reduction denoise module, an interpolation removal module, an auto white balance AWB module, a color correction module, and an auto focus AF module, where the image processing module related to the brightness type is referred to as an image brightness processing module, the image brightness processing module may include at least one of an AE module, a gamma correction module, and a denoise module, the image processing module related to the color type is referred to as an image color processing module, and the image color processing module may include at least one of a removal module, an AWB module, and a color correction module.
Assuming that the format of the image is a mono raw format, in the configuration file corresponding to the type of the information contained in the image, the marks of the AE module, the gamma correction module and the denoise module are first marks, and the marks of the demosaic module, the AWB module and the color correction module are second marks. And assuming that the format of the image is a bayer format, in the configuration file corresponding to the type of information contained in the image, the marks of all the image processing modules are first marks.
It should be noted that, the above-mentioned image brightness processing module includes three modules, namely, an AE module, a gamma correction module, and a denoise module, and in practical application, the image brightness processing module may also include other image processing modules related to brightness, which is not limited in this disclosure. Similarly, the above-mentioned image color processing module includes three modules, namely a demosaic module, an AWB module, and a color correction module, and in practical application, the image color processing module may also include other image processing modules related to colors, which is not limited in this disclosure.
In the embodiment of the present disclosure, the configuration file corresponding to the type of information included in the image may further include an image processing parameter of the image processing module corresponding to the first mark, for example, the image processing parameter of the AE module may include a specific exposure time, for example, in a backlight scene, the image may be subjected to underexposure, a longer exposure time may be set, and in a forward light scene, the image may be subjected to overexposure, and a shorter exposure time may be set.
As a possible implementation manner, in order to improve the quality of the processed image, each image processing module may have multiple sets of image processing parameters, where each set of image processing parameters is related to one shooting scene (such as a backlight scene, a forward light scene, etc.), and when the image processing module is used to process the image, the image processing parameters related to the current shooting scene may be used to process the image, so as to improve the processing effect of the image.
In step 503, the image processing module with the first mark in the configuration file is determined as the image processing module related to the type.
In the embodiment of the present disclosure, step 503 may be implemented by any one of the embodiments of the present disclosure, which is not limited to this embodiment, and is not repeated herein.
Step 504, for each image processing module associated with a type, obtains image processing parameters in a configuration file.
In the embodiment of the disclosure, after determining the image processing module related to the type of the information contained in the image, the ISP may further obtain the image processing parameters of the image processing module related to the type, that is, obtain the image processing parameters of the image processing module with the first flag, from the configuration file.
In step 505, the image processing module is controlled to process the image according to the image processing parameters, so as to obtain a processed image.
In the embodiment of the disclosure, the ISP may process the image according to the image processing parameters corresponding to each image processing module, to obtain a processed image.
As an example, a demosaic module (for performing interpolation calculation on an image to obtain an image in RGB format), a gamma correction module, and a denoise module (for performing noise reduction processing on the image to obtain an image in YUV format for preview) may be disposed on the ISP. Before or after the demosaic module, the ISP may be further provided with an AE module, an AWB module (black-white balance, avoiding image redness, etc.), and an AF module.
The ISP may process the image in the first format, such as the bayer format, and the image processing modules related to the type corresponding to the image in the first format may be all the image processing modules in the ISP. For the image in the second format, such as the mono raw format, since the image only carries brightness information and does not carry color information, when the image is processed by the ISP, the image processing module related to the color in the ISP can be disabled, which is referred to as an image color processing module in the application, and only the image processing module related to the brightness in the ISP is used, which is referred to as an image brightness processing module in the application. The disabling means that the disabled module directly outputs the input data without processing the input data.
The ISP may have a plurality of interfaces, where one interface may be connected to the camera of the second format to process the image captured by the camera of the second format, and another interface may be connected to the camera of the first format to process the image captured by the camera of the first format using all the image processing modules in the ISP.
In a possible implementation manner of the embodiment of the present disclosure, JPEG encoding may also be used to encode the processed image to obtain an image in JPEG format to be displayed.
According to the image processing method, the image to be processed and the type of information contained in the image are acquired through the ISP, and the image processing module related to the type is determined, so that the image is processed by the image processing module related to the type, and the processed image is obtained. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
Corresponding to the image processing method provided by the embodiments of fig. 1 to 5, the present disclosure also provides an image processing apparatus, and since the image processing apparatus provided by the embodiments of the present disclosure corresponds to the image processing method provided by the embodiments of fig. 1 to 5, the implementation of the image processing method is also applicable to the image processing apparatus provided by the embodiments of the present disclosure, and will not be described in detail in the embodiments of the present disclosure.
Fig. 6 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure. Wherein the image processing apparatus may be configured in an ISP.
As shown in fig. 6, the image processing apparatus 600 includes: an acquisition module 610, a determination module 620, and a processing module 630, wherein:
an acquiring module 610 is configured to acquire an image to be processed and a type of information included in the image.
A determination module 620 for determining an image processing module associated with the type.
The processing module 630 is configured to process the image by using the image processing module related to the type, so as to obtain a processed image.
In one embodiment of the present disclosure, the obtaining module 610 may include:
An acquisition unit for acquiring the image, and a format of the image.
A first determining unit for determining the type of information contained in the image according to the format
In one embodiment of the present disclosure, the determining unit is specifically configured to: when the format is the first format, determining that the type comprises a brightness type and a color type; when the format is the second format, the determined type includes only the brightness type.
In one embodiment of the present disclosure, the determining module 620 may include:
The second determining unit is configured to determine a configuration file corresponding to the type, where the configuration file includes: the labels of the image processing modules comprise a first label and a second label, wherein the first label represents that the corresponding image processing module is related to the type, and the second label represents that the corresponding image processing module is not related to the type.
And the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the type includes a brightness type and a color type, the flags of all image processing modules in the image signal processor are first flags; when the type includes only the brightness type, the image brightness processing module in the image signal processor is marked with a first mark, and the other modules except the image brightness processing module are marked with a second mark.
In one embodiment of the present disclosure, the configuration file further includes: image processing parameters of the image processing module corresponding to the first mark; the processing module 630 is specifically configured to: acquiring image processing parameters in a configuration file aiming at each image processing module related to the type; and controlling the image processing module to process the image according to the image processing parameters so as to obtain a processed image.
The image processing device of the embodiment of the disclosure acquires an image to be processed and the type of information contained in the image through the ISP, and determines an image processing module related to the type so as to process the image by adopting the image processing module related to the type, thereby obtaining a processed image. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
Corresponding to the image processing method provided by the embodiments of fig. 1 to 5, the present disclosure also provides an electronic device, and since the electronic device provided by the embodiments of the present disclosure corresponds to the image processing method provided by the embodiments of fig. 1 to 5, the implementation of the image processing method is also applicable to the electronic device provided by the embodiments of the present disclosure, and will not be described in detail in the embodiments of the present disclosure.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.
As shown in fig. 7, the electronic device 700 includes: memory 710 and processor 720, bus 730 connecting the different components, including memory 710 and processor 720.
Wherein the memory 710 is configured to store executable instructions of the processor 720; the processor 720 is configured to invoke and execute the executable instructions stored in the memory 710 to implement the image processing method proposed by the above-described embodiments of the present disclosure.
Bus 730 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 700 typically includes a variety of electronic device readable media. Such media can be any available media that is accessible by electronic device 700 and includes both volatile and nonvolatile media, removable and non-removable media.
Memory 710 may also include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 740 and/or cache memory 750. Electronic device 700 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 760 may be used to read from or write to non-removable, non-volatile magnetic media (not shown in FIG. 7, commonly referred to as a "hard disk drive"). Although not shown in fig. 7, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 730 through one or more data medium interfaces. Memory 710 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the various embodiments of the disclosure.
A program/utility 780 having a set (at least one) of program modules 770 may be stored in, for example, memory 710, such program modules 770 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 770 typically carry out the functions and/or methods of the embodiments described in this disclosure.
The electronic device 700 may also communicate with one or more external devices 790 (e.g., keyboard, pointing device, display 791, etc.), one or more devices that enable a user to interact with the electronic device 700, and/or any device (e.g., network card, modem, etc.) that enables the electronic device 700 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 792. Also, the electronic device 700 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 793. As shown, the network adapter 793 communicates with other modules of the electronic device 700 over the bus 730. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 700, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processor 720 executes various functional applications and data processing by running programs stored in the memory 710.
In order to implement the above embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the image processing method proposed in any of the foregoing embodiments.
In order to implement the above-described embodiments, the disclosed embodiments also provide a computer program product that, when executed by a processor of an electronic device, enables the electronic device to perform the image processing method set forth in any one of the foregoing embodiments.
According to the technical scheme, the image to be processed and the type of information contained in the image are acquired through the ISP, and an image processing module related to the type is determined, so that the image is processed by adopting the image processing module related to the type, and the processed image is obtained. Therefore, the ISP processes the images containing different types of information by only adopting the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the processing effect of the images is improved. In addition, the ISP is used for processing the image, and the CPU or the GPU is not required to process the image, so that the occupation of resources of the CPU or the GPU can be avoided.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (12)
1. An image processing method, applied to an image signal processor, the image signal processor being connected to at least one camera, the method comprising:
acquiring an image acquired by the camera and the type of information contained in the image, wherein the type comprises a brightness type and/or a color type;
determining an image processing module associated with the type;
processing the image by adopting an image processing module related to the type to obtain a processed image, wherein when the type comprises a brightness type and a color type, the marks of all the image processing modules in the image signal processor are first marks, and the first marks represent that the corresponding image processing modules are related to the type; when the type only comprises a brightness type, the marks of the image brightness processing modules in the image signal processor are first marks, the marks of other modules except the image brightness processing modules are second marks, and the second marks represent that the corresponding image processing modules are irrelevant to the type.
2. The method of claim 1, wherein acquiring the image captured by the camera and the type of information contained in the image comprises:
acquiring the image and the format of the image;
and determining the type of information contained in the image according to the format.
3. The method of claim 2, wherein determining the type of information contained in the image based on the format comprises:
determining that the type includes a brightness type and a color type when the format is a first format;
When the format is the second format, it is determined that the type includes only a brightness type.
4. A method according to any one of claims 1 to 3, wherein determining an image processing module associated with the type comprises:
determining a configuration file corresponding to the type, wherein the configuration file comprises marks of each image processing module, and the marks comprise the first mark and the second mark;
and determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
5. The method of claim 4, wherein the configuration file further comprises: image processing parameters of the image processing module corresponding to the first mark;
The image processing module for processing the image by adopting the image processing module related to the type to obtain a processed image, which comprises the following steps:
Acquiring image processing parameters in the configuration file for each image processing module related to the type;
And controlling the image processing module to process the image according to the image processing parameters so as to obtain a processed image.
6. An image processing apparatus for application to an image signal processor, the image signal processor being coupled to at least one camera, the apparatus comprising:
The acquisition module is used for acquiring the image acquired by the camera and the type of information contained in the image, wherein the type comprises a brightness type and/or a color type;
a determining module for determining an image processing module associated with the type;
The processing module is used for processing the image by adopting the image processing module related to the type to obtain a processed image, wherein when the type comprises a brightness type and a color type, the marks of all the image processing modules in the image signal processor are first marks, and the first marks represent that the corresponding image processing modules are related to the type; when the type only comprises a brightness type, the marks of the image brightness processing modules in the image signal processor are first marks, the marks of other modules except the image brightness processing modules are second marks, and the second marks represent that the corresponding image processing modules are irrelevant to the type.
7. The apparatus of claim 6, wherein the acquisition module comprises:
an acquisition unit configured to acquire the image, and a format of the image;
And the first determining unit is used for determining the type of the information contained in the image according to the format.
8. The apparatus according to claim 7, wherein the determining unit is specifically configured to:
determining that the type includes a brightness type and a color type when the format is a first format;
When the format is the second format, it is determined that the type includes only a brightness type.
9. The apparatus according to any one of claims 6 to 8, wherein the determining module comprises:
A second determining unit configured to determine a configuration file corresponding to the type, where the configuration file includes a flag of each image processing module, and the flag includes the first flag and the second flag;
And the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
10. The apparatus of claim 9, wherein the configuration file further comprises: image processing parameters of the image processing module corresponding to the first mark;
The processing module is specifically configured to:
Acquiring image processing parameters in the configuration file for each image processing module related to the type;
And controlling the image processing module to process the image according to the image processing parameters so as to obtain a processed image.
11. An electronic device, comprising:
A processor;
a memory for storing executable instructions of the processor;
Wherein the processor is configured to invoke and execute the executable instructions stored in the memory to implement the image processing method of any of claims 1-5.
12. A non-transitory computer readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the image processing method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110897405.XA CN115705614B (en) | 2021-08-05 | 2021-08-05 | Image processing method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110897405.XA CN115705614B (en) | 2021-08-05 | 2021-08-05 | Image processing method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115705614A CN115705614A (en) | 2023-02-17 |
| CN115705614B true CN115705614B (en) | 2024-06-28 |
Family
ID=85178844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110897405.XA Active CN115705614B (en) | 2021-08-05 | 2021-08-05 | Image processing method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115705614B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107347137A (en) * | 2017-06-30 | 2017-11-14 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN108810406A (en) * | 2018-05-23 | 2018-11-13 | Oppo广东移动通信有限公司 | Portrait light efficiency processing method, device, terminal and computer readable storage medium |
| CN111901520A (en) * | 2020-06-26 | 2020-11-06 | 深圳蚂里奥技术有限公司 | Scene self-adaptive system, method and terminal based on image processing |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5338538B2 (en) * | 2008-09-11 | 2013-11-13 | 株式会社リコー | Image processing apparatus, control method thereof, and control program |
| WO2012059618A1 (en) * | 2010-11-01 | 2012-05-10 | Nokia Corporation | Tuning of digital image quality |
| CN103458173B (en) * | 2012-06-01 | 2017-04-26 | 华为技术有限公司 | Device, method and terminal for processing image relevant data |
| CN103472971B (en) * | 2013-09-03 | 2017-10-17 | 小米科技有限责任公司 | A kind of method, device and the terminal device of setting acquisition parameters |
| US10827140B2 (en) * | 2016-10-17 | 2020-11-03 | Huawei Technologies Co., Ltd. | Photographing method for terminal and terminal |
| CN106506962A (en) * | 2016-11-29 | 2017-03-15 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN108234933A (en) * | 2016-12-21 | 2018-06-29 | 上海杰图软件技术有限公司 | The method and system of real-time splicing panorama image based on multiway images signal processing |
| CN109923850B (en) * | 2017-07-07 | 2020-09-11 | 华为技术有限公司 | Image capturing device and method |
| CN107948511B (en) * | 2017-11-28 | 2019-10-25 | Oppo广东移动通信有限公司 | Brightness of image processing method, device, storage medium and brightness of image processing equipment |
| CN108632512A (en) * | 2018-05-17 | 2018-10-09 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and computer readable storage medium |
| CN108900819B (en) * | 2018-08-20 | 2020-09-15 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN111343386B (en) * | 2018-12-19 | 2022-03-25 | 北京小米移动软件有限公司 | Image signal processing method and device, electronic device and storage medium |
| CN111696039B (en) * | 2020-05-28 | 2023-07-28 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
| CN111614908B (en) * | 2020-05-29 | 2022-01-11 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
-
2021
- 2021-08-05 CN CN202110897405.XA patent/CN115705614B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107347137A (en) * | 2017-06-30 | 2017-11-14 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN108810406A (en) * | 2018-05-23 | 2018-11-13 | Oppo广东移动通信有限公司 | Portrait light efficiency processing method, device, terminal and computer readable storage medium |
| CN111901520A (en) * | 2020-06-26 | 2020-11-06 | 深圳蚂里奥技术有限公司 | Scene self-adaptive system, method and terminal based on image processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115705614A (en) | 2023-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11373275B2 (en) | Method for generating high-resolution picture, computer device, and storage medium | |
| CN109783178B (en) | Color adjusting method, device, equipment and medium for interface component | |
| KR102149187B1 (en) | Electronic device and control method of the same | |
| WO2020057199A1 (en) | Imaging method and device, and electronic device | |
| US9413951B2 (en) | Dynamic motion estimation and compensation for temporal filtering | |
| JP6615917B2 (en) | Real-time video enhancement method, terminal, and non-transitory computer-readable storage medium | |
| US20170099435A1 (en) | Image Generation Method Based On Dual Camera Module And Dual Camera Apparatus | |
| CN110855958B (en) | Image adjusting method and device, electronic equipment and storage medium | |
| CN111368587B (en) | Scene detection method, device, terminal equipment and computer readable storage medium | |
| US20200029006A1 (en) | Sensor auto-configuration | |
| CN108259754B (en) | Image processing method and device, computer readable storage medium and computer device | |
| JP2011081804A (en) | Method for classifying candidate red-eye object, computer readable medium, and image processor | |
| JP2007096437A (en) | Determination of image creator's intention | |
| CN108063933B (en) | Image processing method and device, computer readable storage medium and computer device | |
| CN115705614B (en) | Image processing method, device, electronic equipment and storage medium | |
| TWI825410B (en) | Image processing methods, devices, photographic equipment and storage media | |
| CN113315965B (en) | Optical anti-shake test method, device, equipment and medium | |
| CN111242087B (en) | Object identification method and device | |
| WO2021217428A1 (en) | Image processing method and apparatus, photographic device and storage medium | |
| KR101646673B1 (en) | An apparatus and a method for processing image, and a computer-readable medium storing a computer program for performing the method | |
| US20240029320A1 (en) | Image processing system and image object superimposition apparatus and method thereof | |
| CN113012051A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| WO2021259191A1 (en) | Video mirror processing method and apparatus, photographic device, and readable storage medium | |
| CN113988649B (en) | Display function testing method of display screen and related device | |
| EP4401417A1 (en) | Image signal processing driving method, control module, processing module, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |