CN112995633B

CN112995633B - Image white balance processing method and device, electronic equipment and storage medium

Info

Publication number: CN112995633B
Application number: CN202110427634.5A
Authority: CN
Inventors: 刘闯
Original assignee: Seashell Housing Beijing Technology Co Ltd
Current assignee: Seashell Housing Beijing Technology Co Ltd
Priority date: 2021-04-21
Filing date: 2021-04-21
Publication date: 2021-08-27
Anticipated expiration: 2041-04-21
Also published as: CN112995633A

Abstract

The embodiment of the disclosure discloses a white balance processing method and device for an image, an electronic device and a storage medium. The white balance processing method of the image comprises the following steps: acquiring an image to be processed; determining an exposure multiplying power for carrying out exposure processing on the image to be processed based on the gray-scale image corresponding to the image to be processed; carrying out exposure processing on an image to be processed by adopting an exposure multiplying power to generate a first exposed image; and performing white balance processing on the image to be processed based on the first exposed image to generate a white-balance-processed image. The embodiment of the disclosure can determine the exposure magnification based on the gray-scale image corresponding to the image to be processed, and perform white balance processing on the image to be processed by adopting the exposure magnification to obtain the exposed image, thereby improving the accuracy of the white balance processing of the image.

Description

Image white balance processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to image processing technologies, and in particular, to a method and an apparatus for processing white balance of an image, an electronic device, and a storage medium.

Background

White Balance (WB), an index describing the accuracy of White in an image. White balance is generated along with the reproduction of colors of electronic images, and is earlier applied in the field of professional photography. Home electronics (home video cameras, digital cameras) are also widely used.

In practice, many users encounter such a problem in capturing images using an image capturing apparatus: the image shot in the room of the fluorescent lamp appears green, the scenery shot under the indoor tungsten lamp light is yellowish, and the picture shot in the sunlight shadow is inexplicably bluish, which is caused by the processing mode of the white balance algorithm.

Currently, white balance is challenged by indoor environments represented by their rich colors, complex light source environments, large-area monochromatic scenes, and the like. Moreover, in the process of ISP (Internet Service Provider), various links have coupling effects of different degrees, so that under the complicated indoor environment shooting condition, the mutual influence and the mutual coupling effect are amplified. In addition, there is an influence of the distortion of the boundary data on the white balance for the indoor picture taken by the panoramic camera. Therefore, in the above-mentioned scenario, how to improve the accuracy of the image white balance processing is a significant technical problem.

Disclosure of Invention

The embodiment of the disclosure provides a white balance processing method and device for an image, an electronic device and a storage medium, so as to improve the accuracy of the white balance processing of the image.

According to a first aspect of the embodiments of the present disclosure, there is provided a white balance processing method for an image, including:

acquiring an image to be processed;

determining an exposure magnification for performing exposure processing on the image to be processed based on the gray-scale image corresponding to the image to be processed;

exposing the image to be processed by adopting the exposure multiplying power to generate a first exposed image;

and carrying out white balance processing on the image to be processed based on the first exposed image to generate a white-balance-processed image.

Optionally, in the method according to any embodiment of the present disclosure, the determining, based on a grayscale image corresponding to the image to be processed, an exposure magnification for performing exposure processing on the image to be processed includes:

sequentially selecting exposure multiplying powers according to the order from small to large aiming at the exposure multiplying powers in a predetermined exposure multiplying power set, and executing the following first exposure multiplying power determination operation based on the selected exposure multiplying powers:

carrying out exposure processing on the image to be processed by adopting the selected exposure multiplying power to generate a second exposed image of the image to be processed aiming at the exposure multiplying power;

converting the second exposed image of the image to be processed aiming at the exposure multiplying power into a gray image, and generating the gray image of the image to be processed aiming at the exposure multiplying power;

determining whether the mean value of the brightness of the pixels in the gray level image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value;

and in response to the fact that the mean value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is larger than or equal to the preset brightness threshold, taking the adopted exposure magnification as the exposure magnification for carrying out exposure processing on the image to be processed.

Optionally, in the method of any embodiment of the present disclosure, the method further includes:

and in response to that the average value of the brightness of the to-be-processed image for the pixel points in the gray-scale image of the exposure magnification is smaller than the preset brightness threshold, continuously reselecting the exposure magnification from the exposure magnification set according to the sequence from small to large, and executing the first exposure magnification determination operation based on the reselected exposure magnification.

selecting an exposure magnification from a predetermined exposure magnification interval, and performing a second exposure magnification determination operation based on the selected exposure magnification, the second exposure magnification determination operation being:

and in response to that the mean value of the brightness of the to-be-processed image for the pixel points in the gray-scale image of the exposure magnification is smaller than the preset brightness threshold, reselecting the unselected exposure magnification from the exposure magnification interval, and executing the second exposure magnification determination operation based on the reselected exposure magnification.

Optionally, in the method according to any embodiment of the present disclosure, the performing white balance processing on the image to be processed based on the first exposed image to generate a white-balance-processed image includes:

based on the size of the image to be processed, performing image division on the first exposed image to obtain an image area set;

and performing white balance processing on target pixels in the image areas in the image area set to generate an image after the white balance processing, wherein the target pixels are pixels to be subjected to the white balance processing.

Optionally, in the method according to any embodiment of the present disclosure, the acquiring an image to be processed includes:

acquiring an original image;

and carrying out corrosion operation on the image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed.

taking image pixels in the image to be processed as effective pixels;

determining at least one target image area from the image area set, wherein the product of the number of all pixels in the target image area and a preset percentage is less than or equal to the number of effective pixels in the target image area;

from the determined target image area, a target pixel for performing white balance processing is determined.

Optionally, in a method according to any embodiment of the present disclosure, the determining, from the determined target image region, a target pixel for performing white balance processing includes:

determining pixels meeting preset conditions in the determined target image area as target pixels for white balance processing;

wherein the preset conditions include: the single-channel numerical value of the pixel corresponding to the pixel in the original image is larger than a first preset threshold, and the sum of the color values of three channels of the pixel corresponding to the pixel in the image to be processed is smaller than a second preset threshold.

Optionally, in the method according to any embodiment of the present disclosure, the performing white balance processing on target pixels in image regions in the image region set to generate a white-balanced image includes:

for each of the determined target pixels, determining a value of a first coordinate and a value of a second coordinate of the target pixel based on the color values of three channels of the target pixel;

calculating the distance from the target pixel to a preset color temperature curve in a preset coordinate system based on the value of the first coordinate and the value of the second coordinate of the target pixel, wherein the preset coordinate system comprises the first coordinate and the second coordinate, and the preset color temperature curve represents the corresponding relation between the value of the first coordinate and the value of the second coordinate;

and carrying out white balance processing on the image to be processed based on the distance from the determined target pixel to the preset color temperature curve to generate an image after the white balance processing.

Optionally, in the method according to any embodiment of the present disclosure, the performing white balance processing on the image to be processed based on the determined distance between the target pixel and the predetermined color temperature curve to generate a white-balanced image includes:

determining the weight corresponding to each determined target pixel based on the distance from each determined target pixel to the predetermined color temperature curve;

determining color values of a red channel, a green channel and a blue channel of the target pixels based on the determined weights corresponding to the target pixels;

and generating a white balance processed image based on the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel and the image to be processed.

Optionally, in the method according to any embodiment of the present disclosure, a value of a first coordinate of a target pixel is a ratio of a color value of a blue channel of the target pixel to a color value of a green channel of the target pixel, and a value of a second coordinate of the target pixel is a ratio of a color value of a red channel of the target pixel to a color value of a green channel of the target pixel; and

the method further comprises the following steps:

aiming at each color temperature value in the preset color temperature value set, respectively shooting a preset number of pictures for each standard color card in a predetermined standard color card by adopting the color temperature value;

calculating a first coordinate value and a second coordinate value of a pixel point in a shot picture; the value of the first coordinate of the pixel point is the ratio of the color value mean value of the blue channel of the pixel point to the color value mean value of the green channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the red channel of the pixel point to the color value mean value of the green channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value;

and generating a preset color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel point.

According to a second aspect of the embodiments of the present disclosure, there is provided a white balance processing apparatus for an image, including:

an acquisition unit configured to acquire an image to be processed;

a determination unit configured to determine an exposure magnification for performing exposure processing on the image to be processed based on a grayscale image corresponding to the image to be processed;

a first generation unit configured to perform exposure processing on the image to be processed by using the exposure magnification to generate a first post-exposure image;

and the processing unit is configured to perform white balance processing on the image to be processed based on the first exposed image to generate a white balance processed image.

Optionally, in the apparatus according to any embodiment of the present disclosure, the determining unit includes:

a first selecting subunit configured to sequentially select, in order from small to large, exposure magnifications for exposure magnifications in a predetermined set of exposure magnifications, and perform the following first exposure magnification determination operation based on the selected exposure magnifications:

Optionally, in the apparatus of any embodiment of the present disclosure, the apparatus further includes:

and the first selection unit is configured to respond to the fact that the brightness mean value of the to-be-processed image for the pixel points in the gray-scale image of the exposure multiplying power is smaller than the preset brightness threshold value, continue to select the exposure multiplying power from the exposure multiplying power set from small to large in sequence, and execute the first exposure multiplying power determination operation based on the newly selected exposure multiplying power.

a second selecting subunit configured to select an exposure magnification from a predetermined exposure magnification interval, and perform a second exposure magnification determination operation based on the selected exposure magnification, the second exposure magnification determination operation being:

and the second selection unit is configured to reselect the unselected exposure multiplying power from the exposure multiplying power interval in response to the fact that the brightness mean value of the image to be processed for the pixel points in the gray-scale image of the exposure multiplying power is smaller than the preset brightness threshold, and execute the second exposure multiplying power determination operation based on the reselected exposure multiplying power.

Optionally, in the apparatus of any embodiment of the present disclosure, the processing unit includes:

the dividing unit is configured to divide the first exposed image based on the image size of the image to be processed to obtain an image area set;

and the generating subunit is configured to perform white balance processing on target pixels in the image areas in the image area set, and generate a white-balance-processed image, wherein the target pixels are pixels to be subjected to the white balance processing.

Optionally, in the apparatus according to any embodiment of the present disclosure, the obtaining unit includes:

an acquisition subunit configured to acquire an original image;

and the corroding subunit is configured to perform a corroding operation on the image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed.

Optionally, in the apparatus of any embodiment of the present disclosure, the corrosion subunit is further configured to:

taking image pixels in the image to be processed as effective pixels; and

the determining subunit includes:

a first determining module configured to determine at least one target image area from the image area set, wherein a product of a number of all pixels in the target image area and a preset percentage is less than or equal to a number of valid pixels in the target image area;

and a second determination module configured to determine a target pixel for performing white balance processing from the determined target image area.

Optionally, in the apparatus of any embodiment of the present disclosure, the second determining module includes:

a determination submodule configured to determine pixels, which meet a preset condition, in the determined target image region as target pixels for performing white balance processing;

Optionally, in the apparatus according to any embodiment of the present disclosure, the generating subunit includes:

a third determination module configured to determine, for each of the determined target pixels, a value of the first coordinate and a value of the second coordinate of the target pixel based on the color values of the three channels of the target pixel;

a calculation module configured to calculate a distance from the target pixel to a predetermined color temperature curve in a preset coordinate system based on a value of a first coordinate and a value of a second coordinate of the target pixel, wherein the preset coordinate system includes the first coordinate and the second coordinate, and the predetermined color temperature curve represents a correspondence between the value of the first coordinate and the value of the second coordinate;

and the generating module is configured to perform white balance processing on the image to be processed based on the determined distance between the target pixel and the predetermined color temperature curve, and generate a white-balance-processed image.

Optionally, in the apparatus according to any embodiment of the present disclosure, the generating module includes:

a first determining submodule configured to determine a weight corresponding to each determined target pixel based on a distance of each determined target pixel to the predetermined color temperature curve;

a second determining submodule configured to determine color values of a red channel, a green channel, and a blue channel of the target pixel based on the determined weights corresponding to the respective target pixels;

a generation submodule configured to generate a white balance processed image based on the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel, and the image to be processed.

Optionally, in the apparatus according to any embodiment of the present disclosure, a value of a first coordinate of a target pixel is a ratio of a color value of a blue channel of the target pixel to a color value of a green channel of the target pixel, and a value of a second coordinate of the target pixel is a ratio of a color value of a red channel of the target pixel to a color value of a green channel of the target pixel; and

the device further comprises:

the shooting unit is configured to shoot a preset number of pictures for each standard color card in a predetermined standard color card respectively by adopting the color temperature value aiming at each color temperature value in a preset color temperature value set;

the calculating unit is configured to calculate values of a first coordinate and a second coordinate of a pixel point in a shot picture; the value of the first coordinate of the pixel point is the ratio of the color value mean value of the blue channel of the pixel point to the color value mean value of the green channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the red channel of the pixel point to the color value mean value of the green channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value;

and a second generation unit configured to generate a predetermined color temperature curve based on the calculated values of the first and second coordinates of the pixel point.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory for storing a computer program;

a processor for executing the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the white balance processing method of the image of the first aspect of the present disclosure is realized.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable medium, which when executed by a processor, implements the method of any one of the embodiments of the white balance processing method for an image according to the first aspect described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program comprising computer readable code which, when run on a device, causes a processor in the device to execute instructions for implementing the steps in the method as in any one of the embodiments of the method for white balance processing of an image of the first aspect described above.

Based on the image white balance processing method, the image white balance processing device, the electronic device, and the storage medium provided by the embodiments of the present disclosure, an image to be processed may be acquired, an exposure magnification used for performing exposure processing on the image to be processed is determined based on a grayscale image corresponding to the image to be processed, the image to be processed is subjected to exposure processing using the exposure magnification to generate a first post-exposure image, and finally, the image to be processed is subjected to white balance processing based on the first post-exposure image to generate a post-white-balance image. The exposure multiplying power can be determined based on the gray-scale image corresponding to the image to be processed, white balance processing is carried out on the image to be processed by the exposed image obtained by carrying out exposure processing on the image to be processed by adopting the exposure multiplying power, different exposure multiplying powers used for carrying out exposure processing on the image to be processed can be determined according to different images to be processed, white balance processing is carried out on the image to be processed based on the exposed image, and therefore accuracy of the white balance processing of the image is improved.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.

The present disclosure may be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:

fig. 1 is a flowchart of a first embodiment of a white balance processing method of an image according to the present disclosure.

Fig. 2 is a flowchart of a white balance processing method of an image according to a second embodiment of the disclosure.

Fig. 3 is a flowchart of a white balance processing method of an image according to a third embodiment of the present disclosure.

Fig. 4 is a schematic diagram illustrating a generation process of a color temperature curve in an embodiment of the white balance processing method for an image according to the present disclosure.

Fig. 5 is a schematic structural diagram of an embodiment of a white balance processing apparatus for an image according to the present disclosure.

Fig. 6 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to at least one of a terminal device, a computer system, and a server, which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with at least one electronic device of a terminal device, computer system, and server include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above, and the like.

At least one of the terminal device, the computer system, and the server may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Referring to fig. 1, a flow 100 of a first embodiment of a method of white balance processing of an image according to the present disclosure is shown. The white balance processing method of the image comprises the following steps:

101, acquiring an image to be processed.

In the present embodiment, an execution subject (e.g., a server, a terminal device, an image processing unit having an image processing function, etc.) of the white balance processing method of an image may acquire an image to be processed from other electronic devices or locally by a wired connection manner or a wireless connection manner.

Wherein, the image to be processed is obtained. May be an image to be subjected to white balance processing. For example, the image to be processed may be an original image (e.g., a raw image file), a color image obtained based on the original image, or an image obtained after processing such as etching operation.

For example, in the case where the image to be processed is an image obtained after being subjected to a process such as a corrosion operation, the image to be processed may be obtained as follows: data of an original image corresponding to a point where a median value of an image mask (a binarized image) corresponding to the original image is 0 is removed, and calculation after the removal is not included. That is, the data of the original image corresponding to the point of the image mask whose median is 0 in the data from which the original image is to be removed is obtained, so as to obtain the image to be processed after the erosion operation.

And 102, determining an exposure magnification for exposing the image to be processed based on the gray-scale image corresponding to the image to be processed.

In this embodiment, the executing body may determine an exposure magnification for performing exposure processing on the image to be processed based on the grayscale image corresponding to the image to be processed acquired in 101.

Therein, the grayscale image corresponding to the image to be processed acquired in 101 may be a grayscale image into which the image to be processed is converted. Optionally, the grayscale image corresponding to the image to be processed acquired in 101 may also be acquired as follows: and performing exposure processing on the image to be processed by adopting a certain exposure magnification to generate an exposed image of the image to be processed, converting the exposed image into a gray image, and taking the gray image obtained after conversion as a gray image corresponding to the image to be processed acquired in the step 101. Here, the exposure magnification may be a predetermined exposure magnification, or an exposure magnification such that the average value of the luminance of each pixel in a grayscale image into which an exposed image after exposure processing is converted is greater than or equal to a preset threshold value.

And 103, performing exposure processing on the image to be processed by adopting the exposure magnification to generate a first post-exposure image.

In this embodiment, the executing body may perform exposure processing on the image to be processed acquired in 101 by using the exposure magnification determined in 102, and generate a first post-exposure image.

As an example, the executing body may multiply the value (for example, a value of a single channel or a value of three channels) of each pixel in the image to be processed acquired in 101 by the exposure magnification described above to perform exposure processing on the image to be processed acquired in 101, thereby obtaining a first post-exposure image.

As still another example, the executing entity may first determine an image area to be exposed from the image to be processed, and then multiply the value of each pixel (for example, a value of a single channel or a value of three channels) in the image area by the exposure magnification to perform exposure processing on the image to be processed acquired in 101, thereby obtaining the first post-exposure image.

And 104, performing white balance processing on the image to be processed based on the first exposed image to generate a white-balance-processed image.

In this embodiment, the executing body may perform white balance processing on the image to be processed acquired in 101 based on the first post-exposure image generated in 103, and generate a post-white-balance-processed image.

As an example, the executing entity may first determine the target pixel from the first post-exposure image generated in 103 by a white point detection algorithm. Then, a white balance algorithm (e.g., a gray world method, an automatic white balance method based on a dynamic threshold, a gray pixel, a gray index, etc.) is used to perform white balance processing on the determined target pixel in the first exposed image to determine a value of the target pixel, and a value of a pixel corresponding to the target pixel in the image to be processed is correspondingly updated to the determined value of the target pixel, so that white balance processing on the image to be processed acquired in 101 is achieved, and the image after white balance processing is generated. The target pixel is not limited to a pixel with RGB (red green blue ) values of 255, and the determination manner of the target pixel may be different according to the white point detection algorithm selected.

It should be noted that, the above embodiment of the present disclosure may execute the above 103 first and then execute 102, or may execute 102 first and then execute 103, and is not limited herein. For example, the execution body may first perform exposure processing on the image to be processed using the exposure magnification to generate a first post-exposure image. Then, the first exposed image is converted into a gray scale image, and a converted gray scale image is generated. And if the brightness mean value of the pixels in the gray-scale image is greater than or equal to a preset brightness threshold (for example, 0.5), taking the exposure magnification as the exposure magnification for exposing the image to be processed.

The white balance processing method for the image, provided by the above embodiment of the present disclosure, may obtain an image to be processed, determine an exposure magnification for performing exposure processing on the image to be processed based on a grayscale image corresponding to the image to be processed, perform exposure processing on the image to be processed by using the exposure magnification to generate a first post-exposure image, and perform white balance processing on the image to be processed based on the first post-exposure image to generate a white-balance-processed image. The embodiment of the disclosure can determine the exposure magnification based on the gray-scale image corresponding to the image to be processed, and perform white balance processing on the image to be processed by adopting the exposure magnification to obtain the exposed image, so that different exposure magnifications for performing exposure processing on the image to be processed can be determined for different images to be processed, and then perform white balance processing on the image to be processed based on the exposed image, thereby improving the accuracy of the white balance processing of the image.

In some optional implementations of the embodiment, the executing body may determine the exposure magnification for exposing the image to be processed, in the following manner:

selecting an exposure magnification from a predetermined exposure magnification interval, and performing a second exposure magnification determination operation based on the selected exposure magnification, that is, the following first to fourth steps:

the method comprises the following steps of firstly, carrying out exposure processing on an image to be processed by adopting a selected exposure multiplying power, and generating a second exposed image of the image to be processed aiming at the exposure multiplying power.

Here, the exposure magnification may be randomly selected from the above-described exposure magnification interval, or the selection of the exposure magnification may be performed in order of increasing or decreasing every predetermined value (for example, 1, 2, or the like).

As an example, the execution subject may multiply the value of each pixel (for example, the value of a single channel or the value of three channels) in the to-be-processed image acquired in 101 by the selected exposure magnification, thereby generating a second post-exposure image of the to-be-processed image for the exposure magnification.

As still another example, the executing entity may first determine an image area to be subjected to exposure processing from the image to be processed, and then multiply the value of each pixel in the image area (for example, the value of a single channel or the value of three channels) by the selected exposure magnification, thereby generating a second post-exposure image of the image to be processed for the exposure magnification.

And secondly, converting the second exposed image of the image to be processed aiming at the exposure multiplying power into a gray image, and generating the gray image of the image to be processed aiming at the exposure multiplying power.

Here, the execution subject may convert the second post-exposure image of the image to be processed for the exposure magnification into a grayscale image in various ways.

As an example, in the case where the image to be processed is an RGB three-channel image, the execution subject described above may generate a grayscale image of the image to be processed for the exposure magnification based on the result of weighted summation of color values of three channels of pixels of the image to be processed, or based on the average value of color values of three channels of pixels of the image to be processed, and convert it as a grayscale image into which the image to be processed is converted for the second post-exposure image of the exposure magnification.

And thirdly, determining whether the average value of the brightness of the pixel points in the gray level image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value. The brightness value of a pixel point in the gray image may be the gray value of the pixel point.

And fourthly, under the condition that the average value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is larger than or equal to a preset brightness threshold, taking the adopted exposure magnification as the exposure magnification for carrying out exposure processing on the image to be processed.

In some application scenarios in the above-described optional implementation manner, in a case that a luminance average of a pixel point in a gray-scale image of the exposure magnification of the image to be processed is smaller than a preset luminance threshold, the execution main body may further reselect an unselected exposure magnification from the exposure magnification interval, and execute a second exposure magnification determination operation based on the reselected exposure magnification, that is, the first step to the fourth step.

It can be understood that, in the above application scenario, when the mean luminance value of the pixels in the gray-scale image of the image to be processed for the exposure magnification is smaller than the preset luminance threshold, the exposure magnification is not used to perform the exposure processing on the image to be processed, but reselects the unselected exposure multiplying power until the exposure multiplying power which can enable the brightness mean value of the pixel points in the gray level image of the exposure multiplying power of the image to be processed to be larger than or equal to the preset brightness threshold value is selected, therefore, after the image to be processed is exposed by adopting the determined exposure multiplying factor, the obtained gray-scale image corresponding to the exposed image has higher brightness, and further, based on the first exposed image with larger pixel value, the white balance processing is performed on the image to be processed, so that the image after the white balance processing is generated, and the accuracy of the white balance processing can be further improved.

In some optional implementations of this embodiment, the executing main body may execute the step 101 as follows:

first, an original image is acquired. The raw image may include, among other things, image data processed from an image sensor of a digital camera, scanner, or motion picture film scanner. It will be appreciated that the original image is typically an image that has not been processed, printed or used for editing.

Then, carrying out corrosion operation on the image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed.

It can be understood that the image to be processed in the above alternative implementation manner is a corroded image after being processed by the corrosion operation, so that the influence of the outer boundary on the white balance processing can be eliminated, and the accuracy of the white balance processing is further improved.

In some optional implementations of this embodiment, the executing main body may execute the step 104 in the following manner:

the method comprises the following steps of firstly, carrying out image division on a first exposed image based on the size of an image to be processed to obtain an image area set.

As an example, the number of image divisions may be determined according to actual needs, for example, the number of image divisions may be positively or negatively correlated with the size of the image.

And secondly, determining a target pixel for white balance processing aiming at the target pixel in the image area set.

Here, for each image area in the image area set, the execution subject may first determine a target pixel for performing white balance processing from the image area by a white point detection algorithm.

And thirdly, carrying out white balance processing on target pixels in the image areas in the image area set to generate an image after the white balance processing.

Here, the executing entity may perform white balance processing on the determined target pixel by using a white balance algorithm (e.g., a gray world method, an automatic white balance method based on a dynamic threshold, etc.) to determine a value of the target pixel, and correspondingly update a value of a pixel corresponding to the target pixel in the image to be processed to the determined value of the target pixel, thereby performing white balance processing on the image to be processed, and generating the image after the white balance processing.

It can be understood that, in the above alternative implementation, the image after the first exposure is divided, and the target pixels for performing the white balance processing in each image region are respectively determined, so that the more targeted white balance processing is performed according to different characteristics of each image region, thereby further improving the accuracy of the image white balance processing.

In some application scenarios of the above optional implementation, in the case that the image to be processed in the above 101 is a corroded image obtained through a corrosion operation, an image pixel in the image to be processed may be used as an effective pixel.

Based on this, the execution subject may execute the second step to determine a target pixel for white balance processing for a target pixel in an image area in the set of image areas as follows:

the first step, from the set of image areas, determines at least one target image area. And the product of the number of all pixels in the target image area and the preset percentage is less than or equal to the number of effective pixels in the target image area.

And a second step of determining a target pixel for performing white balance processing from the determined target image region.

In some cases in the application scenario, the executing entity may execute the second step in such a manner as follows to determine a target pixel for white balance processing from the determined target image area:

and determining pixels meeting preset conditions in the determined target image area as target pixels for white balance processing. Wherein the preset conditions include: the single-channel numerical value of the pixel corresponding to the pixel in the original image is larger than a first preset threshold, and the sum of the color values of three channels of the pixel corresponding to the pixel in the image to be processed is smaller than a second preset threshold. The color values of three channels are red, green and blue.

It can be understood that, in the above case, the target pixel for performing the white balance processing is determined only based on the pixels meeting the preset condition, and the influence of the pixels not meeting the preset condition on the white balance processing of the image is eliminated, so that the influence of the dark noise and the excessively bright nonlinear region on the white balance can be avoided, and thus, the accuracy of the white balance processing of the image is further improved.

Optionally, the executing body may also execute the second step in the following manner to determine a target pixel for performing white balance processing from the determined target image area:

based on all the pixels in the determined target image area, a target pixel for performing white balance processing is determined.

It is understood that, in the application scenario, the target pixel for performing the white balance processing may be determined based on only the pixels (i.e., the valid pixels) in the target image region, and the influence of the target pixel in the non-target image region with more invalid pixels (i.e., non-valid pixels) on the image white balance processing is eliminated, thereby further improving the accuracy of the image white balance processing.

In some application scenarios of the above alternative implementation, the executing body may execute the third step in the following manner to perform white balance processing on the target pixels in the image regions in the image region set to generate a white-balanced image:

a first step of determining, for each of the determined target pixels, a value of a first coordinate and a value of a second coordinate of the target pixel based on the color values of three channels of the target pixel.

The color values of the three channels of the target pixel may include a color value of a red channel, a color value of a blue channel, and a color value of a green channel.

Here, how to calculate the value of the first coordinate and the value of the second coordinate of the target pixel may be determined according to actual needs.

As an example, the value of the first coordinate may be a ratio of a color value of the red channel to a color value of the blue channel, and the value of the second coordinate may be a ratio of a color value of the red channel to a color value of the green channel. Alternatively, the value of the first coordinate may also be a ratio of the color value of the blue channel to the color value of the red channel, and the value of the second coordinate may also be a ratio of the color value of the green channel to the color value of the red channel, and so on.

And a second step of calculating a distance from the target pixel to a predetermined color temperature curve in a preset coordinate system based on the value of the first coordinate and the value of the second coordinate of the target pixel. The preset coordinate system comprises a first coordinate and a second coordinate, and the preset color temperature curve represents the corresponding relation between the value of the first coordinate and the value of the second coordinate.

And a third step of performing white balance processing on the image to be processed based on the determined distance from the target pixel to the preset color temperature curve to generate an image after the white balance processing.

In some cases in the application scenario, the executing entity may execute the third step in a manner that, based on the determined distance between the target pixel and the predetermined color temperature curve, the executing entity performs white balance processing on the image to be processed to generate a white-balanced processed image:

firstly, based on the distance of each determined target pixel to a predetermined color temperature curve, the weight corresponding to each determined target pixel is determined.

For example, the weight corresponding to the target pixel may be positively or negatively correlated with the distance from the target pixel to the predetermined color temperature curve.

Then, based on the determined weights corresponding to the respective target pixels, color values of a red channel, a green channel, and a blue channel of the target pixels are determined.

Among them, the target pixel may be a pixel to perform white balance processing.

For example, the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel may be determined by the following formulas:

wherein R is the color value of the red channel of the target pixel, G is the color value of the green channel of the target pixel, B is the color value of the blue channel of the target pixel, i is used for uniquely identifying the determined target pixel, W_iCharacterizing the determined weight corresponding to the target pixel identified as i,

a color value characterizing the determined red channel of the target pixel identified as i,

a color value characterizing the determined green channel of the target pixel identified as i,

a color value characterizing the determined blue channel of the target pixel identified as i.

Optionally, the executing entity may further determine, based on the determined weight corresponding to each target pixel, a color value of a red channel, a color value of a green channel, and a color value of a blue channel of the target pixel by:

the color values of the blue channel characterizing the determined target pixels identified as i, N characterizing the total number of the determined respective target pixels.

And then, generating a white balance processed image based on the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel and the image to be processed.

As an example, the execution subject may first determine a color value with the largest numerical value among the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel. Then, the color value with the maximum value is divided by a matrix formed by the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel to obtain a first matrix. And then, taking the convolution result of the image matrix of the image to be processed and the first matrix as the white balance processed image.

Optionally, the executing body may further generate a white balance processed image based on the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel, and the image to be processed, as follows:

firstly, processing an image to be processed based on at least one of an etching operation and an exposure processing to obtain a processed image. Then, a color value with the largest numerical value among the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel is determined. Then, the color value with the maximum value is divided by a matrix formed by the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel to obtain a first matrix. And then, the result of the convolution operation between the image matrix of the processed image and the first matrix is used as the white balance processed image.

It can be understood that, in the above case, the white balance processing may be performed on the image to be processed based on the determined weight corresponding to each target pixel, and thus, the white balance processing may be performed based on the weight corresponding to each target pixel, and thus, the robustness and accuracy of the target pixel in the finally generated white-balanced processed image may be further improved.

Optionally, the executing main body may also execute the third step in the following manner, so as to perform white balance processing on the image to be processed based on the determined distance between the target pixel and the predetermined color temperature curve, and generate an image after the white balance processing:

and determining the color temperature when the image to be processed is shot based on the determined distance between the target pixel and the preset color temperature curve, and further performing white balance processing on the image to be processed based on the color temperature to generate an image after the white balance processing.

It can be understood that, in the above application scenario, the white balance processing may be performed on the image to be processed based on the determined distance from the target pixel to the predetermined color temperature curve, so that the white balance processing is assisted by the predetermined color temperature curve, and the accuracy of the white balance processing may be further improved.

With further reference to fig. 2, fig. 2 is a flowchart of a second embodiment of the white balance processing method of an image of the present disclosure. The flow 200 of the white balance processing method of the image comprises the following steps:

and 201, acquiring an image to be processed. Thereafter, 202 is performed.

In this embodiment, an execution subject (for example, a server, a terminal device, an image processing unit with an image processing function, or the like) of the white balance processing method of an image may acquire an image to be processed from another electronic device or locally by a wired connection manner or a wireless connection manner.

In this embodiment, 201 is substantially the same as 101 in the corresponding embodiment of fig. 1, and is not described here again.

202, the exposure magnifications in the predetermined exposure magnification set are sequentially selected in descending order of magnitude. Thereafter, 203 is performed.

In this embodiment, the execution subject may sequentially select exposure magnifications in an order from small to large for exposure magnifications in a predetermined exposure magnification set.

For example, if the following exposure magnifications are included in the set of exposure magnifications: 1. 4, 16, 64, then, when selecting for the first time, the execution body may select 1 as the selected exposure magnification; in the second selection, the execution main body may select 4 as the selected exposure magnification; in the third selection, the execution body may select 16 as the selected exposure magnification; in the fourth selection, the execution body may select 64 as the selected exposure magnification.

Here, 203-205 can be performed once every time the exposure magnification is selected.

And 203, performing exposure processing on the image to be processed by adopting the selected exposure multiplying power, and generating a second post-exposure image of the image to be processed aiming at the exposure multiplying power. Thereafter, 204 is performed.

In this embodiment, the executing body may perform exposure processing on the image to be processed by using the exposure magnification selected in 201, and generate a second post-exposure image of the image to be processed with respect to the exposure magnification.

As an example, the executing body may multiply the value (e.g., the value of a single channel or the value of three channels) of each pixel in the image to be processed acquired in 201 by the exposure magnification selected in 202 to perform the exposure processing on the image to be processed acquired in 201, thereby obtaining the second post-exposure image.

As still another example, the executing entity may first determine an image area to be subjected to exposure processing from the image to be processed, and then multiply the value of each pixel in the image area (for example, the value of a single channel or the value of three channels) by the exposure magnification selected in 202 to implement exposure processing on the image to be processed acquired in 201, thereby obtaining a second post-exposure image.

And 204, converting the second exposed image of the image to be processed corresponding to the exposure magnification into a gray image, and generating the gray image of the image to be processed corresponding to the exposure magnification. Thereafter, 205 is performed.

In this embodiment, the executing body may convert the second post-exposure image of the to-be-processed image at the exposure magnification into a grayscale image, and generate a grayscale image of the to-be-processed image at the exposure magnification.

As an example, in the case where the image to be processed is an RGB three-channel image, the execution body described above may generate a grayscale image of the image to be processed for the exposure magnification based on a result of weighted summation of color values of three channels of pixels of the image to be processed, or based on an average value of color values of three channels of pixels of the image to be processed.

And 205, determining whether the mean value of the brightness of the pixel points in the gray scale image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold. Then, if so, 206 is performed. Optionally, if not, 202 may be performed.

In this embodiment, the executing entity may determine whether an average value of the luminance of the pixel points in the gray-scale image of the exposure magnification of the image to be processed is greater than or equal to a preset luminance threshold.

The brightness of the pixel points in the gray image can be the gray of the pixel points in the gray image.

206, the exposure magnification used is taken as the exposure magnification for exposing the image to be processed. Thereafter, 207 is performed.

In the present embodiment, the execution body described above may take this exposure magnification adopted as the exposure magnification for exposing the image to be processed.

And 207, performing exposure processing on the image to be processed by adopting the exposure multiplying power determined in the step 206 to generate a first post-exposure image. Thereafter, 208 is performed.

In this embodiment, the executing entity may perform exposure processing on the image to be processed by using the exposure magnification determined in 206 to generate a first post-exposure image.

And 208, performing white balance processing on the image to be processed based on the first exposed image to generate a white-balance processed image.

In this embodiment, the execution subject may perform white balance processing on the image to be processed based on the first post-exposure image generated in 207, and generate a post-white-balance-processed image.

As an example, the executing entity may first determine the target pixel from the first post-exposure image generated in 207 by a white point detection algorithm. The target pixel is a pixel to be subjected to white balance processing. Then, a white balance algorithm (e.g., a gray world method, an automatic white balance method based on a dynamic threshold, etc.) is used to perform white balance processing on the determined target pixel in the first exposed image to determine a value of the target pixel, and a value of a pixel corresponding to the target pixel in the image to be processed is updated to the determined value of the target pixel, so that white balance processing on the image to be processed acquired in 201 is performed, and the image after white balance processing is generated.

It should be noted that, besides the above-mentioned contents, the embodiment of the present disclosure may also include the same or similar features and effects as the embodiment corresponding to fig. 1, and no further description is provided herein.

As can be seen from fig. 2, in the flow 200 of the image white balance processing method in this embodiment, the minimum exposure magnification in the exposure magnification set that makes the luminance average of the gray-scale image corresponding to the image to be processed larger can be selected, so that not only the effect of exposure processing can be achieved, but also the image to be processed can be prevented from being overexposed to a certain extent, and therefore, the accuracy and robustness of the image white balance processing can be further improved.

With continuing reference to fig. 3, fig. 3 is a flowchart illustrating a white balance processing method for an image according to a third embodiment of the present disclosure. The flow 300 of the method for processing white balance of an image comprises the following steps:

301, an original image is acquired.

In the present embodiment, an execution subject (e.g., a server, a terminal device, an image processing unit having an image processing function, etc.) of the white balance processing method of an image may acquire an original image from other electronic devices or locally by a wired connection manner or a wireless connection manner.

And 302, carrying out corrosion operation on the image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed.

In this embodiment, the executing entity may perform an erosion operation on the image mask of the original image to obtain an eroded image, and use an image area corresponding to the eroded image in the original image as the image to be processed.

And 303, determining an exposure magnification for exposing the image to be processed based on the gray-scale image corresponding to the image to be processed.

In this embodiment, the execution subject described above may determine an exposure magnification for performing exposure processing on an image to be processed based on a grayscale image corresponding to the image to be processed.

And 304, performing exposure processing on the image to be processed by adopting the exposure magnification to generate a first exposed image.

In this embodiment, the executing entity may perform exposure processing on the image to be processed by using the exposure magnification determined in 303, and generate a first post-exposure image.

And 305, performing image division on the first exposed image based on the image size of the image to be processed to obtain an image area set.

In this embodiment, the executing entity may perform image division on the first exposed image based on the image size of the image to be processed, so as to obtain the image area set.

At least one target image region is determined from the set of image regions 306.

In this embodiment, the execution subject may determine at least one target image region from the image region set.

307, determining the pixels meeting the preset conditions in the determined target image area as target pixels for performing white balance processing.

In this embodiment, the execution subject may determine, as a target pixel for performing white balance processing, a pixel that meets a preset condition in the determined target image region. The target pixel is a pixel to be subjected to white balance processing, wherein the preset conditions include: the single-channel numerical value of the pixel corresponding to the pixel in the original image is larger than a first preset threshold, and the sum of the color values of three channels of the pixel corresponding to the pixel in the image to be processed is smaller than a second preset threshold.

308, performing white balance processing on the target pixels in the image areas in the image area set to generate an image after the white balance processing.

In this embodiment, the executing entity may perform white balance processing on the target pixel in the image region set to generate a white-balanced image.

As an example, please further refer to fig. 4, fig. 4 is a schematic diagram illustrating a generation process of a color temperature curve in an embodiment of the white balance processing method for an image according to the present disclosure. In the case that the value of the first coordinate of the target pixel is the ratio of the color value of the blue channel of the target pixel to the color value of the green channel of the target pixel, and the value of the second coordinate of the target pixel is the ratio of the color value of the red channel of the target pixel to the color value of the green channel of the target pixel, the generating process of the color temperature curve includes:

401, for each color temperature value in the predetermined set of color temperature values, taking a predetermined number of pictures for each standard color card in the predetermined standard color cards respectively using the color temperature value.

And 402, calculating values of a first coordinate and a second coordinate of a pixel point in the shot picture.

The value of the first coordinate of the pixel point is the ratio of the color value mean value of the blue channel of the pixel point to the color value mean value of the green channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the red channel of the pixel point to the color value mean value of the green channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: and the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value.

And 403, generating a preset color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel point.

It can be understood that the color temperature curve generated by the method shown in fig. 4 can ensure that the shooting positions and angles of all images are completely the same, only the color temperature is changed, and a single variable is achieved by using the average value of the RGB three-channel color values of each pixel point corresponding to the pixel in a predetermined number of images of the same standard color card under the same color temperature value. In addition, the influence caused by random noise can be eliminated or weakened by taking a plurality of pictures at the same color temperature, so that the accuracy of the color temperature curve is improved.

It should also be understood that fig. 4 is only a schematic diagram of the generation process of the color temperature curve in one embodiment of the white balance processing method of the image of the present disclosure. Besides the color temperature curve generation method shown in fig. 4, the color temperature curve may be generated in other manners, for example, in a case that the value of the first coordinate of the target pixel is a ratio of the color value of the green channel of the target pixel to the color value of the blue channel of the target pixel, and the value of the second coordinate of the target pixel is a ratio of the color value of the green channel of the target pixel to the color value of the red channel of the target pixel, the color temperature curve generation process includes:

firstly, aiming at each color temperature value in a preset color temperature value set, a preset number of pictures are respectively shot for each standard color card in a predetermined standard color card by adopting the color temperature value.

And then, calculating the value of the first coordinate and the value of the second coordinate of the pixel point in the shot picture.

The value of the first coordinate of the pixel point is the ratio of the color value mean value of the green channel of the pixel point to the color value mean value of the blue channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the green channel of the pixel point to the color value mean value of the red channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: and the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value.

And finally, generating a preset color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel point.

As an application scenario of an embodiment of the white balance processing method for an image of the present disclosure, the white balance processing method for an image may be applied to a VR (Virtual Reality) viewing scene. Typically, to complete an online VR house visit involves the integration of multiple technologies, and the final view presented to the user is a ring that must not be left to processing the photos (e.g., ISP processing). Although devices such as cameras and mobile phones have become popular and involve various related photo processing techniques, various challenges exist with respect to the VR studio customization device. Firstly, the room shooting device is a customized hardware with high integration of various sensor devices and can only be developed autonomously. Secondly, such devices are not usually available in as large a volume as mobile phones, and therefore, are difficult to support by mature ISP manufacturers, and therefore, require self-built ISP capabilities at the same time. Finally, the ISP processing is different from the processing methods of other ISP manufacturers because the method is only limited to the special scene of the shooting room. A general white balance algorithm has a high success rate in a natural scene, but due to the characteristics of indoor environment represented by rich colors, complex light source environment, large-area monochromatic scene and the like, great challenges are brought to white balance. And because each link in the ISP processing process presents coupling effects of different degrees, the mutual coupling effects are superposed and amplified under the complex indoor environment shooting condition. In addition, for indoor pictures taken by a panoramic camera, there is also an influence of the distortion of the boundary data on the white balance.

The accuracy and robustness of image white balance processing can be improved, so that the method is suitable for being applied to VR house-watching scenes, for example, the method can be directly applied to ISP processing flows of indoor photographing scenes.

Specifically, the color temperature curve, that is, the predetermined color temperature curve described above, may be generated first by the method described above. .

For example, standard 24 color cards having color temperatures of 2850, 3100, 3350, 3600, 3850, 4100, 4350, 4600, 4850, 5000, 5250, 5500, 5750, 6000, 6250, 6500 and 10 standard white cards, respectively, can be photographed. Then, the average values of the color values of the three channels of the pixels in the 10 images are respectively calculated to reduce the influence of noise, and finally, calibration data is generated. Besides the change of the color temperature, the calibration scene including the position of the color card cannot be changed, and the uniqueness of the variable is ensured.

Then, the position of the white gray area in the calibration data picture is selected according to the 24-color chart. For example, the middle 80% area of each of the 3 grays at the lower left corner of the 24 color chart can be selected as the range for calculating the white point.

And then, respectively calculating the average value of RGB three-channel data of the selected area for all the calibration data to obtain the color value average value RMean of the red channel, the color value average value GMean of the green channel and the color value average value BMean of the blue channel, and calculating color coordinates (x, y). Wherein x = BMean/GMean, y = rmeean/GMean.

And finally, obtaining a final color temperature calibration curve by using the coordinates of the calculated points.

The white balance processing procedure of the image is entered as follows:

firstly, through corrosion operation, setting image position data corresponding to the boundary data of the panoramic camera to be 0, namely eliminating the influence of the outer boundary on white balance processing. The original calibration boundary pixel is binarized and only includes 0 and 255, the valid pixel may be 255, and the invalid pixel may be 0. And then, carrying out corrosion operation by using a filter kernel with KSize of 71 to obtain a corroded image.

Then, an automatic exposure process is performed.

Here, although the corroded image has a high dynamic range, the numerical values of most data (for example, the color values of RGB three channels) are low, which is not beneficial to subsequent white balance and other operations, so that the image needs to be automatically exposed first, and the numerical values of most data are improved, so that the subsequent algorithm can be processed more accurately.

The exposure magnification for exposing the image to be processed may be determined by the method described above, and details are not repeated here. After determining the exposure magnification and carrying out overexposure processing, an exposed image can be obtained.

And then, dividing the image, collecting the image areas, and removing the image areas with the effective data smaller than a threshold value in the image area collection. Here, the number of divided blocks may be decided for different sizes of original images. For example, for a 2774 x 3708 size image, the entire image may be divided into 9 small pictures, i.e., 1/3 with the number of rows and columns being the original data.

For each image area, if the number of effective data (i.e. pixel points with channel values not all 0) is found to be less than half of the total pixels of a single image area, the target pixel of the image area is not calculated.

Subsequently, the target pixels for the remaining image area may be calculated using GrayIndex or other white balance algorithms.

Before the algorithm is used for calculation, the over-bright data points with the single channel numerical value exceeding 245 and the over-dark data points with the sum of the color values of three channels being less than 10 in the original image can be removed, and the influence of dark noise and over-bright nonlinear areas on white balance can be avoided.

Next, B/G of each target pixel retained above is calculated as X for R/G, respectively_i,y_i. Wherein B represents the color value of the blue channel of the target pixel, G represents the color value of the green channel of the target pixel, R represents the color value of the red channel of the target pixel, and i is used for uniquely identifying each target pixel. Here, the coordinates of the target pixel may be referred to as (x)_i,y_i). Recalculating target pixel (x)_i,y_i) Distance d to the calibrated color temperature curve_iThen the corresponding weight W of each target pixel_iCan be as follows:

the color value R of the red channel, the color value G of the green channel, and the color value B of the blue channel of the target pixel (R, G, B) may be determined by the above formula, and are not described here again:

and finally, carrying out white balance processing on the picture.

I’=I*invRGB

Wherein I' represents the white-balanced processed image, I represents the original image, I represents the convolution operation,

。

As can be seen from fig. 3, in the flow 300 of the image white balance processing method in the present embodiment, the white balance processing of the image is implemented by combining the erosion operation, the exposure processing, and the image division, so that the influence of the boundary data, the nonlinear data, and the dark noise on the white balance processing can be removed, and in addition, by using the scheme of the image division, the target pixel in the image after the white balance processing can be made more robust and more accurate.

Referring now further to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an apparatus for white balance processing of an image, the apparatus embodiment corresponds to the method embodiment shown in fig. 1, and the apparatus embodiment may include the same or corresponding features as the method embodiment shown in fig. 1 in addition to the features described below, and produce the same or corresponding effects as the method embodiment shown in fig. 1. The device can be applied to various electronic equipment.

As shown in fig. 5, the white balance processing apparatus 500 of the image of the present embodiment includes: an acquisition unit 501 configured to acquire an image to be processed; a determination unit 502 configured to determine an exposure magnification for performing exposure processing on an image to be processed based on a grayscale image corresponding to the image to be processed; a first generation unit 503 configured to perform exposure processing on an image to be processed using an exposure magnification to generate a first post-exposure image; a processing unit 504 configured to perform white balance processing on the image to be processed based on the first post-exposure image, and generate a white-balance-processed image.

In the present embodiment, the acquisition unit 501 of the white balance processing apparatus 500 of an image can acquire an image to be processed.

In this embodiment, the above-described determination unit 502 may determine an exposure magnification for performing exposure processing on the image to be processed based on the grayscale image corresponding to the image to be processed acquired by the acquisition unit 501.

In this embodiment, the first generating unit 503 may perform exposure processing on the image to be processed acquired by the acquiring unit 501 by using the exposure magnification determined by the determining unit 502, so as to generate a first post-exposure image.

In this embodiment, the processing unit 504 may perform white balance processing on the image to be processed acquired by the acquisition unit 501 based on the first post-exposure image generated by the first generation unit 503 to generate a post-white-balance-processed image.

In some optional implementations of this embodiment, the determining unit 502 includes:

a first selecting subunit (not shown in the figure) configured to sequentially select exposure magnifications in descending order for the exposure magnifications in a predetermined exposure magnification set, and perform a first exposure magnification determination operation based on the selected exposure magnifications, the first exposure magnification determination operation including:

carrying out exposure processing on the image to be processed by adopting the selected exposure multiplying power to generate a second post-exposure image of the image to be processed aiming at the exposure multiplying power;

determining whether the mean value of the brightness of pixel points in the gray level image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value;

and in response to the fact that the average value of the brightness of the to-be-processed image for the pixel points in the gray-scale image of the exposure magnification is larger than or equal to a preset brightness threshold, taking the adopted exposure magnification as the exposure magnification for performing exposure processing on the to-be-processed image.

In some optional implementations of this embodiment, the apparatus 500 further includes:

and the first selecting unit (not shown in the figure) is configured to respond to the fact that the brightness mean value of the image to be processed for the pixel points in the gray-scale image of the exposure multiplying power is smaller than a preset brightness threshold value, continue to select the exposure multiplying power from the exposure multiplying power set from small to large in sequence, and execute the first exposure multiplying power determining operation based on the newly selected exposure multiplying power.

a second selecting subunit (not shown in the figure) configured to select an exposure magnification from a predetermined exposure magnification interval, and perform a second exposure magnification determination operation based on the selected exposure magnification, the second exposure magnification determination operation being:

and a second selecting unit (not shown in the figure) configured to reselect an unselected exposure magnification from the exposure magnification interval in response to that the average value of the brightness of the image to be processed for the pixels in the gray-scale image of the exposure magnification is smaller than a preset brightness threshold, and perform a second exposure magnification determination operation based on the reselected exposure magnification.

In some optional implementations of this embodiment, the processing unit 504 includes:

a dividing subunit (not shown in the figure) configured to perform image division on the first exposed image based on the image size of the image to be processed to obtain an image area set;

and a generation subunit (not shown in the figure) configured to perform white balance processing on the target pixels in the image areas in the image area set, and generate an image after the white balance processing. The target pixel is a pixel to be subjected to white balance processing.

In some optional implementations of this embodiment, the obtaining unit 501 includes:

an acquisition subunit (not shown in the figure) configured to acquire an original image;

and the etching subunit (not shown in the figure) is configured to perform etching operation on the image mask of the original image to obtain an etched image, and take an image area corresponding to the etched image in the original image as an image to be processed.

In some optional implementations of this embodiment, the corrosion subunit is further configured to:

taking image pixels in the image to be processed as effective pixels; and

the determining subunit includes:

a first determining module (not shown in the figures) configured to determine at least one target image area from the set of image areas, wherein the product of the number of all pixels in the target image area and a preset percentage is less than or equal to the number of valid pixels in the target image area;

and a second determination module (not shown in the figure) configured to determine a target pixel for performing white balance processing from the determined target image area.

In some optional implementations of this embodiment, the second determining module includes:

a determination submodule (not shown in the figure) configured to determine pixels, which meet a preset condition, in the determined target image area as target pixels for performing white balance processing;

In some optional implementations of this embodiment, the generating the subunit includes:

a third determining module (not shown in the figure) configured to determine, for each of the determined target pixels, a value of the first coordinate and a value of the second coordinate of the target pixel based on the color values of the three channels of the target pixel;

a calculation module (not shown in the figures) configured to calculate a distance from the target pixel to a predetermined color temperature curve in a preset coordinate system based on a value of a first coordinate and a value of a second coordinate of the target pixel, wherein the preset coordinate system includes the first coordinate and the second coordinate, and the predetermined color temperature curve represents a correspondence between the value of the first coordinate and the value of the second coordinate;

and a generating module (not shown in the figure) configured to perform white balance processing on the image to be processed based on the determined distance from the target pixel to the predetermined color temperature curve, and generate a white-balanced image.

In some optional implementations of this embodiment, the generating module includes:

a first determining submodule (not shown in the figure) configured to determine a weight corresponding to each determined target pixel based on a distance of each determined target pixel to a predetermined color temperature curve;

a second determining submodule (not shown in the figure) configured to determine color values of a red channel, a green channel, and a blue channel of the target pixel based on the determined weights corresponding to the respective target pixels;

and a generation submodule (not shown in the figure) configured to generate a white balance processed image based on the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel, and the image to be processed.

In some optional implementations of this embodiment, the value of the first coordinate of the target pixel is a ratio of a color value of a blue channel of the target pixel to a color value of a green channel of the target pixel, and the value of the second coordinate of the target pixel is a ratio of a color value of a red channel of the target pixel to a color value of a green channel of the target pixel; and

the apparatus 500 further comprises:

a photographing unit (not shown in the figure) configured to take a predetermined number of pictures for each of predetermined standard color cards respectively using a color temperature value for each of a predetermined set of color temperature values;

a calculation unit (not shown in the figure) configured to calculate values of a first coordinate and a second coordinate of a pixel point in a photographed picture; the value of the first coordinate of the pixel point is the ratio of the color value mean value of the blue channel of the pixel point to the color value mean value of the green channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the red channel of the pixel point to the color value mean value of the green channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value;

and a second generating unit (not shown in the figure) configured to generate a predetermined color temperature curve based on the calculated values of the first coordinates and the second coordinates of the pixel points.

In the white balance processing apparatus for an image according to the above embodiment of the present disclosure, the obtaining unit 501 may obtain an image to be processed, then the determining unit 502 may determine an exposure magnification for performing exposure processing on the image to be processed based on a grayscale image corresponding to the image to be processed, the first generating unit 503 may perform exposure processing on the image to be processed using the exposure magnification to generate a first post-exposure image, and finally, the processing unit 504 may perform white balance processing on the image to be processed based on the first post-exposure image to generate a white-balance-processed image. The exposure multiplying power can be determined based on the gray-scale image corresponding to the image to be processed, white balance processing is carried out on the image to be processed by the exposed image obtained by carrying out exposure processing on the image to be processed by adopting the exposure multiplying power, different exposure multiplying powers used for carrying out exposure processing on the image to be processed can be determined according to different images to be processed, white balance processing is carried out on the image to be processed based on the exposed image, and therefore accuracy of the white balance processing of the image is improved.

Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 6. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.

FIG. 6 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

As shown in fig. 6, the electronic device 6 includes one or more processors 601 and memory 602.

The processor 601 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

Memory 602 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 601 to implement the white balance processing method of the image of the various embodiments of the present disclosure described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device may further include: an input device 603 and an output device 604, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, when the electronic device is a first device or a second device, the input device 603 may be the microphone or the microphone array described above for capturing the input signal of the sound source. When the electronic device is a stand-alone device, the input means 603 may be a communication network connector for receiving the acquired input signals from the first device and the second device.

The input device 603 may also include, for example, a keyboard, a mouse, and the like. The output device 604 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 604 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device may include any other suitable components, depending on the particular application.

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method of white balance processing of images according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Additionally, embodiments of the present disclosure may also be a computer program, which may include computer readable code. When the above computer readable code is run on a device, a processor in the device performs the steps in the method of white balance processing of an image according to various embodiments of the present disclosure described in the above "exemplary methods" section of this specification.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method of white balance processing of an image, the method comprising:

acquiring an original image;

carrying out corrosion operation on an image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed;

performing white balance processing on a target pixel in a target image area in the image area set to generate a white-balance-processed image, wherein the target pixel is a pixel to be subjected to the white balance processing;

the method further comprises the following steps:

taking image pixels in the image to be processed as effective pixels;

2. The method according to claim 1, wherein determining an exposure magnification for exposing the image to be processed based on the grayscale image corresponding to the image to be processed comprises:

responding to the fact that the mean value of the brightness of the pixel points in the gray level image of the exposure magnification of the image to be processed is larger than or equal to the preset brightness threshold, and taking the adopted exposure magnification as the exposure magnification for carrying out exposure processing on the image to be processed; and

the method further comprises the following steps:

3. The method according to claim 1, wherein determining an exposure magnification for exposing the image to be processed based on the grayscale image corresponding to the image to be processed comprises:

the method further comprises the following steps:

4. The method according to claim 1, wherein determining a target pixel for white balance processing from the determined target image region comprises:

5. The method according to any one of claims 1 to 4, wherein the performing white balance processing on the target pixels in the target image areas in the image area set to generate a white-balance processed image comprises:

performing white balance processing on the image to be processed based on the distance from the determined target pixel to the preset color temperature curve to generate an image after the white balance processing; and

the white balance processing is performed on the image to be processed based on the determined distance between the target pixel and the preset color temperature curve, and an image after the white balance processing is generated, and the white balance processing comprises the following steps:

6. An apparatus for white balance processing of an image, the apparatus comprising:

an acquisition unit configured to acquire an image to be processed, the acquisition unit including: an acquisition subunit configured to acquire an original image; the etching subunit is configured to perform etching operation on an image mask of the original image to obtain an etched image, and take an image area corresponding to the etched image in the original image as an image to be processed;

a processing unit configured to perform white balance processing on the image to be processed based on the first post-exposure image, and generate a post-white balance processed image, the processing unit including: the dividing unit is configured to divide the first exposed image based on the size of the image to be processed to obtain an image area set; a generation subunit configured to perform white balance processing on a target pixel in a target image area in the image area set, and generate a white-balanced processed image; the target pixel is a pixel to be subjected to white balance processing;

a determining subunit comprising: a first determining module configured to determine at least one target image area from the image area set, wherein a product of a number of all pixels in the target image area and a preset percentage is less than or equal to a number of valid pixels in the target image area; and a second determination module configured to determine a target pixel for performing white balance processing from the determined target image area.

7. An electronic device, comprising:

a memory for storing a computer program;

a processor for executing a computer program stored in the memory, and when executed, implementing the method of any of the preceding claims 1-5.

8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of the preceding claims 1 to 5.