CN116074640A

CN116074640A - Image processing method and related device

Info

Publication number: CN116074640A
Application number: CN202310136942.1A
Authority: CN
Inventors: 王秀花; 凌晨; 赵磊
Original assignee: Spreadtrum Communications Shanghai Co Ltd
Current assignee: Spreadtrum Communications Shanghai Co Ltd
Priority date: 2023-02-17
Filing date: 2023-02-17
Publication date: 2023-05-05

Abstract

The embodiment of the application provides an image processing method and a related device, wherein the method comprises the following steps: acquiring a first pixel and a plurality of second pixels in an image to be processed, wherein the distance between the first pixel and the second pixels is smaller than or equal to a first threshold value; performing color correction on the first pixel based on the hue value, the brightness value and the saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the second pixel is smaller than or equal to a fourth threshold value. The method and the device can effectively remove the false color in the image.

Description

Image processing method and related device

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to an image processing method and a related device.

Background

In daily cases, due to chromatic dispersion distortion of an optical lens, poor accuracy of an interpolation method, large color noise and the like, wrong colors which are inevitably present in an initial image acquired by electronic equipment are avoided. The false color existing in the image may be called a false color, for example, the true color of the word "flower" is black, but the edge of the "flower" appears in the captured image as false colors such as purple, green, yellow, and the like.

The false color not only affects the subjective visual perception of human eyes on the image, but also causes the subsequent research and application of the image to be affected to cause erroneous results, so how to effectively remove the false color in the image is a continuous direction for those skilled in the art.

Disclosure of Invention

The embodiment of the application provides an image processing method and a related device, which can effectively remove pseudo colors in an image.

In a first aspect, an embodiment of the present application provides an image processing method, including:

acquiring a first pixel and a plurality of second pixels in an image to be processed, wherein the distance between the first pixel and the second pixels is smaller than or equal to a first threshold value;

performing color correction on the first pixel based on the hue value, the brightness value and the saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the brightness value of the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the saturation value of the second pixel is smaller than or equal to a fourth threshold value.

With reference to the first aspect, in one possible implementation manner, the acquiring a first pixel and a plurality of second pixels in an image to be processed includes:

determining a boundary area based on segmentation information of the image to be processed, wherein the segmentation information represents segmentation conditions of different objects included in the image to be processed;

the first pixel and the plurality of second pixels are acquired from the boundary region.

With reference to the first aspect, in one possible implementation manner, the probability value corresponding to the first pixel is greater than or equal to a fifth threshold, and the probability value corresponding to the first pixel is determined based on a comparison result of hue values of the first pixel and the second pixel, a comparison result of brightness values of the first pixel and the second pixel, and a comparison result of saturation values of the first pixel and the second pixel.

With reference to the first aspect, in one possible implementation manner, the performing color correction on the first pixel based on the hue value, the brightness value and the saturation value of the first pixel and the second pixel to obtain a processed image includes:

performing color correction on the first pixel based on the colors of the plurality of third pixels to obtain the processed image; the probability value corresponding to the third pixel is smaller than or equal to a sixth threshold, and the distance between the third pixel and the first pixel is smaller than or equal to a seventh threshold.

With reference to the first aspect, in one possible implementation manner, performing color correction on the first pixel based on colors of the plurality of third pixels to obtain the processed image includes:

and correcting the color of the first pixel to the color of a third pixel nearest to the first pixel among the plurality of third pixels, thereby obtaining the processed image.

determining a target color based on a hue value and a corresponding weight value for each of the plurality of third pixels, the weight value corresponding to the third pixel being inversely proportional to a distance between the third pixel and the first pixel;

and correcting the color of the first pixel to the target color to obtain the processed image.

With reference to the first aspect, in one possible implementation manner, the method further includes:

determining a hue average value of a first pixel, wherein the hue average value is an average value of hue values of the first pixel and the plurality of second pixels in the image to be processed;

And modifying the color of the first pixel based on the hue average value and reference information when the absolute value of the difference value between the hue value after the correction of the first pixel and the hue average value is greater than or equal to an eighth threshold value, wherein the reference information comprises noise information and texture information of the image to be processed.

In a second aspect, embodiments of the present application provide a unit comprising means for performing the method of the first aspect or any possible implementation of the first aspect.

In a third aspect, embodiments of the present application provide an electronic device including a processor and a memory; the memory is used for storing data and computer execution instructions; the processor is configured to execute computer-executable instructions stored in the memory, to cause the method according to the first aspect or any possible implementation manner of the first aspect to be performed.

In a fourth aspect, embodiments of the present application provide a chip, including a logic circuit and an interface, where the logic circuit and the interface are coupled; the interface is for inputting and/or outputting code instructions and the logic circuitry is for executing the code instructions to cause the method of the first aspect or any possible implementation of the first aspect to be performed.

In a fifth aspect, the present application provides a module apparatus, where the module apparatus includes a communication module, a power module, a storage module, and a chip module, where the power module is configured to provide electric energy for the module apparatus; the storage module is used for storing data and instructions; the communication module is used for carrying out internal communication of the module equipment or carrying out communication between the module equipment and external equipment; the above-described chip module is for performing the method of the first aspect or any possible implementation of the first aspect.

In a sixth aspect, embodiments of the present application disclose a computer program product comprising program instructions which, when executed by a processor, cause the method of the first aspect or any of the possible implementations of the first aspect to be performed.

In a seventh aspect, embodiments of the present application provide a computer readable storage medium having a computer program stored therein, which when run on a processor, causes the method of the first aspect or any of the possible implementations of the first aspect to be performed.

Drawings

Fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present application;

FIG. 2 is a block diagram of an image processing system according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a data input module according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a preprocessing module provided in an embodiment of the present application;

fig. 5 is a schematic diagram of a false color detection module according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a pseudo color correction module according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of an aftertreatment module provided in an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a data output module according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of an auxiliary module provided in an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device 110 according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a module device according to an embodiment of the present application.

Detailed Description

The terminology used in the following embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates to the contrary. It should also be understood that the term "and/or" as used in this application is intended to encompass any or all possible combinations of one or more of the listed items.

It should be noted that the terms "first," "second," "third," and "fourth," etc. in the description, claims, and drawings of this application are used for distinguishing between different objects and not for describing a particular sequential order. It should also be understood that in the embodiments of the present application, the numbers before the steps are made for the convenience of understanding and describing the solution, and should not be construed as limiting the order in which the steps are performed.

In daily life, photos can be acquired through electronic equipment. Taking a mobile phone as an example, an optical lens in the mobile phone can reduce an object image to be projected onto a photosensitive element in the mobile phone, the photosensitive element generates current and voltage changes due to irradiation of light, and the current and voltage changes are transmitted to a chip for processing to obtain a photo of the object, wherein the chip can be an image signal processor (image signal processor, ISP) and a related chip module.

However, due to chromatic dispersion distortion of the optical lens, inaccurate interpolation method, large color noise, and the like, erroneous colors that may exist in an image acquired by the electronic device are unavoidable. False colors present in an image may be referred to as false colors, common false colors being purple-fringing, green-fringing, and the like. It will be appreciated that the false color, once generated, may be an unwanted and detrimental information, such as the false color may adversely affect the subjective visual perception of the image by the human eye, and such as the false color may affect the subsequent study and application of the image, and may even lead to erroneous study results. Therefore, removal of false colors in an image is of great importance.

Based on the above problems, embodiments of the present application provide an image processing method and related apparatus, by which false colors in an image can be effectively removed. The method provided in the embodiment of the present application may be executed by an electronic device, where the electronic device may be any electronic device capable of executing the technical solution disclosed in the embodiment of the method of the present application, and the electronic device may be a mobile phone, a tablet computer, a notebook computer, a monitor, a vehicle-mounted terminal, a mobile internet device (mobile Internet devices, MID), a wearable device or a scanning device, or may also be a terminal device, a server, or a server cluster formed by multiple servers, which is not limited in this application.

Optionally, the method embodiments of the present application may also be implemented by means of a processor executing computer program codes, where the processor may be an ISP, a related chip module, or the like. It will be appreciated that where embodiments of the methods of the present application are performed by a processor, the processor may be disposed within an electronic device, such as an electronic device.

Referring to fig. 1, fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present application.

As shown in fig. 1, the method includes:

101: and acquiring a first pixel and a plurality of second pixels in the image to be processed, wherein the distance between the first pixel and the second pixels is smaller than or equal to a first threshold value.

In the embodiments of the present application, the image to be processed may be understood as raw image data, such as an image acquired by a sensor and not subjected to compression processing. For example, the image format of the image to be processed may be RAW (RAW image format, a color coding method), RGB (red green blue), YUV (a color coding method, Y represents brightness, U and V represent chromaticity), and the like.

The image to be processed comprises a plurality of pixels, the first pixel can be understood as any pixel in the image to be processed, and the distance between the second pixel and the first pixel is smaller than or equal to a first threshold value. The distance may be a pixel distance or an absolute distance. The first threshold may be determined according to the type of the distance and an empirical value, which is not limited in this application, for example, in the case where the distance is a pixel distance, the first threshold may be 3, that is, a pixel distance between the first pixel and the second pixel is less than or equal to 3 pixels.

102: performing color correction on the first pixel based on the hue value, the brightness value and the saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the brightness value of the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the saturation value of the second pixel is smaller than or equal to a fourth threshold value.

In this step, the color correction is performed on the first pixel through the hue value, the brightness value and the saturation value of the first pixel and the second pixel, where the hue value of the pixel may be used to represent the color of the pixel, the brightness value of the pixel may be used to represent the brightness of the color of the pixel, and the saturation of the pixel may be used to represent the chroma of the color of the pixel.

In this step, the comparison result of the hue values of the first pixel and the second pixel, the comparison result of the brightness values of the first pixel and the second pixel, and the comparison result of the saturation values of the first pixel and the second pixel may be obtained by various comparison methods. For example, the comparison result may be an absolute value of a difference value of the numerical values, or may be a gradient value, a contrast, a mean square error, or the like, as long as the difference between the first pixel and the second pixel can be represented, which is not limited in the present application.

In this embodiment of the present application, specific values of the second threshold, the third threshold, and the fourth threshold are not limited, and may be limited according to an empirical value and information in an image to be processed. For example, the hue value in the image to be processed may be statistically analyzed, the second threshold may be set according to the range of the hue value, and for example, the image to be processed may be divided into regions, and different second thresholds may be set for different regions. It will be appreciated that the third threshold and the fourth threshold may be set similarly to the second threshold, and a single threshold may be set, or different thresholds may be set in regions. It should be understood that the setting of the threshold value may also be set according to the comparison mode, i.e. different threshold values may be set when calculating the comparison result using different comparison modes.

In this embodiment of the present application, when the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold, the electronic device performs color correction on the first pixel, where the color correction may be understood as modifying the color of the pixel, for example, may be understood as modifying the hue value of the color of the pixel.

For ease of understanding, pixels that satisfy the above conditions are referred to as pseudo-color pixels, and pixels that do not satisfy the above conditions are referred to as non-pseudo-color pixels. It will be appreciated that since the first pixel is any pixel in the image to be processed, that is, each pixel in each image to be processed will be processed identically. The method shown in fig. 1 can thus be understood as a loop, which is terminated after each pixel in the image to be processed has been determined to be the first pixel.

Generally, in an image, a pixel area where a pseudo color is located has a large hue value difference and a large brightness value difference but has a small saturation difference from surrounding pixel areas, and based on the above characteristics (referred to as a pseudo color characteristic), in this embodiment of the present application, an electronic device performs color correction on a first pixel based on the hue value, the brightness value and the saturation value of the first pixel and a second pixel, so as to obtain a processed image; the contrast result of the hue value of the first pixel and the second pixel is larger than or equal to the second threshold value, the contrast result of the brightness value of the first pixel and the second pixel is larger than or equal to the third threshold value, the contrast result of the saturation value of the first pixel and the second pixel is smaller than or equal to the fourth threshold value, and the distance between the first pixel and the second pixel is smaller than or equal to the first threshold value.

It should also be appreciated that, compared with other schemes in which a fixed color (such as a green or purple common false color) is selected to detect a false color pixel, the false color pixel detected based on the false color feature in the embodiment of the present application may include various different colors, so as to increase the types of the detected and removed false color types, so that the types of the removed false color are no longer single, and the flexibility of the removal of the false color is improved.

In some embodiments, step 101, acquiring a first pixel and a plurality of second pixels in an image to be processed includes:

1011: the boundary region is determined based on segmentation information of the image to be processed, the segmentation information representing the segmentation situation of different objects comprised by the image to be processed.

In this embodiment, the segmentation information represents the segmentation situation of different objects included in the image to be processed, for example, the segmentation situation of the foreground and the background in the image to be processed may be the segmentation situation. The foreground may also include a plurality of objects, and thus the segmentation information may also include segmentation situations of different objects in the plurality of objects. In one possible implementation, the electronic device may obtain the segmentation information from other devices; in another possible implementation manner, the electronic device may acquire the image to be processed first, and then perform segmentation processing on the image to be processed to obtain the segmentation information.

The above-mentioned segmentation information may be, for example, a segmentation line, such as a contour line of the object. The electronic device may determine the boundary area according to pixels around the dividing line, for example, a pixel having a distance to the pixel where the dividing line is located that is less than or equal to a threshold value a may be used as the boundary area, where the distance may be a pixel distance or an absolute distance. The threshold value a may be determined according to an empirical value, for example, may be 10 pixels, which is not limited in this application.

1012: a first pixel and the plurality of second pixels are acquired from the boundary region.

After determining the boundary region, the electronic device obtains the first pixel and the plurality of second pixels from the boundary region. Wherein the first pixel may be understood as any one of the pixels in the boundary region described above. It will be appreciated that each pixel in the boundary region will be subjected to the same process (i.e. the process of step 102 described above) and therefore the method comprising steps 1011 and 1012 may be understood as a loop, with the loop terminating after each pixel in the boundary region is identified as the first pixel.

It will be appreciated that the pseudocolor pixels are typically present at the boundary (near the contour) of the object in the image to be processed, and not inside the object. Therefore, the boundary region is determined based on the segmentation information of the image to be processed, and then the false color pixels are detected from the boundary region to perform color correction, so that the false color removal efficiency of the image to be processed can be improved.

In some embodiments, the probability value corresponding to the first pixel is greater than or equal to a fifth threshold, the probability value corresponding to the first pixel is determined based on a comparison of hue values of the first pixel and the second pixel, a comparison of luminance values of the first pixel and the second pixel, and a comparison of saturation values of the first pixel and the second pixel.

In this embodiment, the probability value corresponding to a pixel is used to characterize the probability that the pixel is a pseudo-color pixel, and the probability value is proportional to the probability that the pixel is a pseudo-color pixel. Taking the first pixel as an example, the probability value corresponding to the first pixel may be specifically determined according to the comparison result of the hue values of the first pixel and the second pixel, the comparison result of the brightness values of the first pixel and the second pixel, and the comparison result of the saturation values of the first pixel and the second pixel.

Illustratively, depending on the characteristics of the pseudo-color pixels, the greater the difference in hue values of the pixels from surrounding pixels, the greater the difference in luminance values but the greater the probability that the pixels with smaller saturation differences are pseudo-color pixels. Therefore, a plurality of thresholds can be set for the comparison result, different thresholds correspond to different probability values, and finally a final probability value is obtained based on each probability value. Alternatively, the probability values may be characterized by weight values.

Taking mean square error as an example in a comparison mode, when the mean square error result of the hue values of the first pixel and the second pixel (or a plurality of second pixels) is in the interval A, a probability value A is given; when the mean square error result is within the interval B, a probability value B is given. Giving a probability value C in case the mean square error result of the luminance values of the first pixel and the second pixel (or the plurality of second pixels) is within the interval C; when the mean square error result is within the interval D, a probability value D is given. Giving a probability value E in case the mean square error result of the saturation values of the first pixel and the second pixel (or the plurality of second pixels) is within a section E; when the mean square error result is within the interval F, a probability value F is given. For example, if the mean square error of the hue values of the first pixel and the second pixel (or the plurality of second pixels) is in the interval a, the mean square error of the luminance values is in the interval D, and the mean square error of the saturation values is in the interval F, the probability corresponding to the first pixel is obtained based on the probability value a, the probability value D, and the probability value F, and specifically, the average, the weighted average, or the direct multiplication may be adopted, which is not limited in the present application.

It will be appreciated that the above manner is merely an example, and more intervals may be divided in a specific implementation, and the start value and the end value of the interval may be set according to a specific situation, which is not limited in this application.

In one possible implementation manner, the electronic device may calculate a corresponding probability value for each pixel in the image to be processed, where the probability value is used to characterize the probability that each pixel is a pseudo-color pixel, and finally, pixels with the probability value greater than or equal to the fifth threshold may be understood as pseudo-color pixels, that is, pixels that need to be color corrected. The fifth threshold may be empirically set, for example, may be any value greater than or equal to 80%, which is not limited in this application. In another possible implementation manner, the electronic device may further calculate the corresponding probability value if it is determined that the first pixel meets a condition, where the condition includes: the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the brightness value of the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the saturation value of the second pixel is smaller than or equal to a fourth threshold value.

In this embodiment, the probability value is determined according to the hue value comparison result, the brightness value comparison result and the saturation comparison result between the first pixel and the second pixel, and the probability that the first pixel is a pseudo-color pixel can be intuitively represented by the probability value, so that the subsequent processing (such as color correction) flow is facilitated.

In some embodiments, step 102, performing color correction on the first pixel based on the hue value, the brightness value, and the saturation value of the first pixel and the second pixel to obtain a processed image, includes:

1021: performing color correction on the first pixels based on the colors of the plurality of third pixels to obtain a processed image; the probability value corresponding to the third pixel is less than or equal to the sixth threshold, and the distance between the third pixel and the first pixel is less than or equal to the seventh threshold.

In this embodiment, the sixth threshold may be set according to an empirical value or a specific case, for example, may be any value less than 30%, which is not limited in this application. The distance between the first pixel and the third pixel may be a pixel distance or an absolute distance; the seventh threshold may be set according to an empirical value or a specific case, which is not limited in this application. The determining manner of the probability value corresponding to the third pixel is similar to that of the probability value corresponding to the first pixel, and specifically, reference may be made to the foregoing embodiment, which is not repeated herein.

In this embodiment, the probability value corresponding to the third pixel is less than or equal to the sixth threshold value, and the distance between the third pixel and the first pixel is less than or equal to the seventh threshold value, so the third pixel can be understood as a non-pseudo-color pixel around the first pixel, then the color of the third pixel can be understood as a normal (or correct) color, and the color of the first pixel can be effectively corrected by performing color modification based on a plurality of third pixels around the first pixel.

In one possible implementation, step 1021, performing color correction on the first pixel based on the colors of the plurality of third pixels, to obtain a processed image, includes:

10211: and correcting the color of the first pixel to the color of a third pixel nearest to the first pixel in the plurality of third pixels to obtain a processed image.

In this embodiment of the present application, the electronic device may modify the color of the first pixel by modifying the hue value of the first pixel, that is, modifying the hue value of the first pixel to the hue value of a third pixel closest to the first pixel among the plurality of third pixels.

In another possible implementation, step 1021, performing color correction on the first pixel based on the colors of the plurality of third pixels, to obtain a processed image, includes:

10212: determining a target color based on a hue value and a corresponding weight value for each of the plurality of third pixels, the weight value corresponding to the third pixel being inversely proportional to a distance between the third pixel and the first pixel;

10213: and correcting the color of the first pixel to be the target color to obtain a processed image.

In this embodiment, the electronic device determines the target color by integrating the colors of the third pixels around the first pixel, so that the color transition between the first pixel after the color correction and the surrounding pixels is better.

It is understood that the weight value corresponding to the third pixel may be set according to the distance between the third pixel and the first pixel, and the weight value corresponding to the third pixel further from the first pixel is smaller. It will be appreciated that, in the case where there are a plurality of third pixels and the first pixel having the same distance, the weight value may be further set according to the number of pixels of the same color, for example, the third pixel a and the third pixel B have the same distance from the first pixel, the first pixel is the color a, the second pixel is the color B, and the number of pixels of the color a is greater than the number of pixels of the color B in the plurality of third pixels around the first pixel, so that the weight value corresponding to the third pixel a is greater than the weight value corresponding to the third pixel B.

It is understood that the target color determined by the electronic device may specifically be a hue value, i.e. the target hue value, and that correcting the color of the first pixel to the target color may be understood as correcting the hue value of the first pixel to the target hue value.

In some embodiments, the method shown in fig. 1 further comprises:

103: and determining a hue average value of the first pixel, wherein the hue average value is an average value of hue values of the first pixel and the plurality of second pixels in the image to be processed.

In this step, the hue average value of the first pixel may be understood as the hue average value before the pseudo-color processing, and specifically may be an average value of hue values of the first pixel and the plurality of second pixels.

104: and under the condition that the absolute value of the difference value between the hue value and the hue average value after the correction of the first pixel is larger than or equal to an eighth threshold value, modifying the color of the first pixel based on the hue average value and reference information, wherein the reference information comprises noise information and texture information of an image to be processed.

In this embodiment of the present application, the eighth threshold may be set according to a specific actual situation and an empirical value, which is not limited in this application, for example, may be set according to a hue value interval in the image to be processed, and for example, may be any value smaller than a maximum difference value of a hue value in the image to be processed.

It is understood that, in the case where the absolute value of the difference between the hue value after the first pixel correction and the hue average value is greater than or equal to the eighth threshold value, the color correction error of the first pixel can be considered. Generally, noise information, texture information, etc. in an image to be processed will cause fluctuation of data differences, for example, noise will make hue value differences larger, etc.

According to the method and the device, the color of the first pixel is further modified under the condition that the hue value difference before and after the color correction of the first pixel is large, so that the probability of false color correction errors can be reduced. For example, a new hue value may be determined by respectively assigning weights to the hue average value and the corrected hue value of the first pixel, for example, if the noise is greater than the intensity threshold and the texture information change is greater than the reference threshold, a first weight value is assigned to the hue average value of the first pixel, and a second weight value is assigned to the corrected hue value, where the first weight value is greater than the second weight value, and the new hue value is obtained by weighted average of the hue average value and the corrected hue value based on the first weight value and the second weight value.

In order to more clearly understand the method provided in the embodiments of the present application, the image processing method described above is described below in conjunction with the image processing system shown in fig. 2.

As shown in fig. 2, the electronic device may include an image processing system including a data input module, a preprocessing module, a pseudo-color processing module, a post-processing module, and a data output module, wherein the pseudo-color processing module includes a pseudo-color detection module, a pseudo-color correction module. Optionally, the pseudo color processing module may further include an auxiliary module and a fusion module. The description of the individual modules follows:

the data input module is used to obtain image data, such as the image to be processed in step 101 described above, optionally the data input module may also be used to obtain auxiliary information for use in the auxiliary module. As shown in fig. 3, the image data input from the data input module may be RAW data, RGB data, YUV data in image format. The auxiliary information input by the data input module can be environment information such as color temperature information, white balance information and color information, and can also be noise information such as signal-to-noise ratio information and color noise size; sensor information, lens information (e.g., angle of view), etc. are also possible.

As shown in fig. 4, the preprocessing module includes a color gamut conversion module for performing color gamut conversion on the image data and a color correction module for performing color correction on the image data. Illustratively, the gamut conversion module may be used for RAW to RGB, RGB to YUV, RAW to YUV, etc., which is not limited in this application. In the process of converting the RAW data into three channels, the RAW data can be interpolated into three-channel RGB data. The color correction module can perform color correction on the color gamut converted image, and reduces the probability of false color occurrence caused by inaccurate color.

The false color processing module is used for analyzing and integrating the input image data or the input image data and the auxiliary information so as to detect and correct the false color. The pseudo color processing module mainly comprises a pseudo color detection module and a pseudo color correction module, and optionally also comprises an auxiliary module and a fusion module, wherein the introduction of each module is as follows:

as shown in fig. 5, the false color detection module includes a latent feature simulation module and a true information simulation module. The hidden characteristic simulation module obtains characteristic information of pseudo color hidden through calculation of hue value, brightness value difference and saturation difference; the real information simulation module obtains real information to be verified of the local pixel area through calculation of the local hue information and the brightness saturation information, such as a local hue value, a local brightness value and a local saturation value.

For example, the latent feature simulation module may determine the probability that a pixel is a pseudo-color pixel based on the hue value, the brightness value difference, and the saturation difference, and may refer to the foregoing description regarding the probability value corresponding to the first pixel. The real information simulation module is used for screening based on probability values corresponding to the pixels, for example, pixels with probability values greater than 85% are used as pseudo-color pixels, so that pseudo-color correction is performed.

As shown in fig. 6, the pseudo color correction module includes a pseudo color information correction module and a transition information protection module. The pseudo color information correction module corrects the color of the pseudo color pixel by referring to the surrounding real colors. Meanwhile, the excessive information protection module is used for protecting local excessive information, such as hue continuity, so that the transition between the corrected pixels and the pixels of the surrounding normal area is better, and the image is more natural. Illustratively, the pseudo color information modification module may perform the foregoing step 10211, and the transition information protection module may perform the foregoing steps 10212 and 10213.

And the fusion module is used for fusing the corrected data with the original data. For example, by referring to the auxiliary information (such as noise information) and the difference degree of the data before and after correction, the data before and after correction are weighted respectively, so as to further modify the hue value of the pseudo-color pixel, the fusion module can help to manually adjust the degree of pseudo-color suppression, and the flexibility of the pseudo-color suppression of the system is improved.

As shown in fig. 7, the post-processing module is configured to perform post-processing, such as RAW data restoration, filtering processing, and color adjustment, on the image data after the de-pseudo-color processing. For example, for the output RAW data format, RAW data extraction and restoration can be performed on the image after the pseudo-color removal, and for the output RGB/YUV data format, filtering processing can be performed on pixels of the pseudo-color correction region, so that the image transitivity after the pseudo-color removal is more natural.

As shown in fig. 8, the data output module is configured to perform data output on the post-processed image data, where the output information includes the image data after the de-pseudo color processing and auxiliary information. The output auxiliary information can include color type, correction information (such as correction intensity) and the like, wherein the output of the auxiliary information related to the pseudo color is beneficial to backtracking and modification.

Alternatively, the probability of whether the current position pixel is a false color may be further determined by color information, noise information, and image feature information in the auxiliary information. As shown in fig. 9, the assistance module may include an artificial intelligence (artificial intelligence, AI) assistance module as well as a conventional assistance module.

The AI assistance module may be used to obtain scene segmentation information, semantic segmentation information, and edge detection information. The information obtained by the AI auxiliary module can be understood as the segmentation condition of different objects in the image data, such as the separation condition of the foreground and the background, and because the pseudo color generally appears at the boundary of the object, the region where the pseudo color pixel is located can be roughly positioned through the segmentation information, thereby improving the efficiency of removing the pseudo color and removing the pseudo color from the image more quickly.

The conventional auxiliary module may be used to acquire color information, noise information, defocus information, etc., for improving the accuracy of removing the false color from the image. Such as noise information, which can cause fluctuations in data variance, the noise can be combined to obtain a final variance result when calculating the variance between the current pixel and surrounding pixels. Also, for example, color temperature information, if image data is photographed in a yellow light (instead of an incandescent light) scene, it is possible to analyze what color is the most likely false color according to the yellow light scene, thereby helping to detect whether a pixel is a false color pixel. For example, the out-of-focus information (in-focus or out-of-focus) may reflect the definition of the image, if the out-of-focus information is low, the data difference may be smaller, and the threshold (for example, the second threshold) may be adjusted smaller.

Having described the method of the embodiments of the present application in detail, the following describes the apparatus provided by the embodiments of the present application.

Referring to fig. 10, fig. 10 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application. As shown in fig. 10, the electronic apparatus 100 includes an acquisition unit 1001, a color correction unit 1002. Optionally, a determining unit 1003 and a color modifying unit 1004 may also be included, which is described in detail as follows:

An obtaining unit 1001, configured to obtain a first pixel and a plurality of second pixels in an image to be processed, where a distance between the first pixel and the second pixels is less than or equal to a first threshold;

a color correction unit 1002 configured to perform color correction on the first pixel based on the hue value, the brightness value, and the saturation value of the first pixel and the second pixel, to obtain a processed image; the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the brightness value of the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the saturation value of the second pixel is smaller than or equal to a fourth threshold value.

In a possible implementation manner, the determining unit 1003 is configured to determine a boundary area based on segmentation information of the image to be processed, where the segmentation information indicates a segmentation situation of different objects included in the image to be processed;

the acquiring unit 1001 is specifically configured to acquire the first pixel and the plurality of second pixels from the boundary area.

In one possible implementation manner, the probability value corresponding to the first pixel is greater than or equal to a fifth threshold, and the probability value corresponding to the first pixel is determined based on a comparison result of hue values of the first pixel and the second pixel, a comparison result of brightness values of the first pixel and the second pixel, and a comparison result of saturation values of the first pixel and the second pixel.

In a possible implementation manner, the color correction unit 1002 is specifically configured to perform color correction on the first pixel based on the colors of the plurality of third pixels, so as to obtain the processed image; the probability value corresponding to the third pixel is smaller than or equal to a sixth threshold, and the distance between the third pixel and the first pixel is smaller than or equal to a seventh threshold.

In one possible implementation manner, the color correction unit 1002 is specifically configured to correct the color of the first pixel to the color of a third pixel closest to the first pixel among the plurality of third pixels, so as to obtain the processed image.

In a possible implementation manner, the color correction unit 1002 is specifically configured to determine the target color based on a hue value and a corresponding weighting value of each of the plurality of third pixels, where the weighting value corresponding to the third pixel is inversely proportional to a distance between the third pixel and the first pixel;

the color correction unit 1002 is specifically configured to correct the color of the first pixel to the target color, and obtain the processed image.

In a possible implementation manner, the determining unit 1003 is further configured to determine a hue average value of the first pixel, where the hue average value is an average value of hue values of the first pixel and the plurality of second pixels in the image to be processed;

A color modifying unit 1004, configured to modify, when an absolute value of a difference between the hue value after the correction of the first pixel and the hue average value is greater than or equal to an eighth threshold value, a color of the first pixel based on the hue average value and reference information, where the reference information includes noise information and texture information of the image to be processed.

Referring to fig. 11, fig. 11 is a schematic structural diagram of an electronic device 110 according to an embodiment of the present application. As shown in fig. 11, the electronic device 110 includes a memory 1101 and a processor 1102. Further optionally, a communication interface 1103 and a bus 1104 may be included, wherein the memory 1101, the processor 1102 and the communication interface 1103 are communicatively connected to each other via the bus 1104.

The memory 1101 is used to provide a storage space, and data such as an operating system and a computer program may be stored in the storage space. Memory 1101 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable read-only memory (compact disc read-only memory, CD-ROM).

The processor 1102 is a module for performing arithmetic operations and logical operations, and may be one or a combination of processing modules such as a central processing unit (central processing unit, CPU), a graphics card processor (graphics processing unit, GPU) or a microprocessor (microprocessor unit, MPU).

The memory 1101 stores a computer program therein, and the processor 1102 calls the computer program stored in the memory 1101 to execute the above-described image registration method. Illustratively, in the case where the electronic device 110 is the electronic device 100, the content acquired by the acquiring unit 1001 may be implemented by the communication interface 1103, and the steps performed by the color correcting unit 1002, the determining unit 1003, and the color modifying unit 1004 may be implemented by the processor 1102.

Optionally, the steps performed by the preprocessing module, the pseudo color processing module (such as an auxiliary module, a pseudo color detection module, and a pseudo color correction module therein), the fusing module, and the post processing module may be implemented by the processor 1102. The steps performed by the data input module and the data output module may be implemented by the communication interface 1103.

The embodiment of the application also provides a chip. The chip comprises: a processor and a memory. Wherein the number of processors may be one or more and the number of memories may be one or more. The processor may perform the methods described above, as well as the steps performed by the related embodiments, by reading instructions and data stored on the memory. Of course, there may be no memory in the chip.

Referring to fig. 12, fig. 12 is a schematic structural diagram of a module device according to an embodiment of the present application. The module device 120 may perform the steps associated with the electronic device in the method embodiments described above. The module apparatus 120 includes: a communication module 1201, a power module 1202, a memory module 1203, and a chip module 1204. Wherein the power module 1202 is configured to provide power to the module device; the storage module 1203 is used for storing data and instructions; the communication module 1201 is configured to perform module device internal communication or to perform module device communication with an external device, and is configured to perform the step performed by the acquisition unit 1001; the chip module 1204 may perform the steps performed by the color correction unit 1002, the determination unit 1003, and the color modification unit 1004 described above.

The present application also provides a computer readable storage medium having computer code stored therein, which when run on a computer causes the computer to perform the method of the above-described embodiments.

The present application also provides a computer program product comprising computer code or a computer program which, when run on a computer, causes the method in the above embodiments to be performed.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, the method comprising:

performing color correction on the first pixel based on the hue value, the brightness value and the saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the hue value of the second pixel is larger than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the second pixel is larger than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the second pixel is smaller than or equal to a fourth threshold value.

2. The method of claim 1, wherein the acquiring the first pixel and the plurality of second pixels in the image to be processed comprises:

3. The method of claim 1 or 2, wherein the probability value corresponding to the first pixel is greater than or equal to a fifth threshold, the probability value corresponding to the first pixel being determined based on a comparison of hue values of the first pixel and the second pixel, a comparison of luminance values of the first pixel and the second pixel, and a comparison of saturation values of the first pixel and the second pixel.

4. A method according to claim 3, wherein said performing color correction on said first pixel based on hue values, brightness values and saturation values of said first and second pixels to obtain a processed image comprises:

performing color correction on the first pixel based on the colors of the plurality of third pixels to obtain the processed image; the probability value corresponding to the third pixel is less than or equal to a sixth threshold, and the distance between the third pixel and the first pixel is less than or equal to a seventh threshold.

5. The method of claim 4, wherein color correcting the first pixel based on the colors of the plurality of third pixels results in the processed image, comprising:

and correcting the color of the first pixel to the color of a third pixel nearest to the first pixel in the plurality of third pixels to obtain the processed image.

6. The method of claim 4, wherein color correcting the first pixel based on the colors of the plurality of third pixels results in the processed image, comprising:

determining a target color based on a hue value and a corresponding weight value for each of the plurality of third pixels, the corresponding weight value for the third pixel being inversely proportional to a distance between the third pixel and the first pixel;

7. The method according to any one of claims 1-6, further comprising:

And under the condition that the absolute value of the difference value between the hue value after the correction of the first pixel and the hue mean value is larger than or equal to an eighth threshold value, modifying the color of the first pixel based on the hue mean value and reference information, wherein the reference information comprises noise information and texture information of the image to be processed.

8. An electronic device comprising means for performing the method of any of claims 1-7.

9. An electronic device, comprising: a processor and a memory; the memory is used for storing data and computer execution instructions; the processor configured to execute computer-executable instructions stored in the memory to cause the method of any one of claims 1-7 to be performed.

10. A chip comprising logic circuitry and an interface, the logic circuitry and interface coupled;

the interface being for inputting and/or outputting code instructions, the logic circuitry being for executing the code instructions to cause the method of any of claims 1-7 to be performed.

11. The module equipment is characterized by comprising a communication module, a power supply module, a storage module and a chip module, wherein the power supply module is used for providing electric energy for the module equipment; the storage module is used for storing data and instructions; the communication module is used for carrying out internal communication of module equipment or carrying out communication between the module equipment and external equipment; the chip module for performing the method of any of claims 1-7.

12. A computer readable storage medium comprising instructions which, when run on an electronic device, cause the method of any one of claims 1-7 to be performed.