CN118233764A

CN118233764A - Image demosaicing method and device

Info

Publication number: CN118233764A
Application number: CN202410644593.9A
Authority: CN
Inventors: 张鹏辉; 沈晰强
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2024-05-23
Filing date: 2024-05-23
Publication date: 2024-06-21

Abstract

The application provides a demosaicing method and device for images, and relates to the technical field of imaging equipment, wherein the method comprises the following steps: obtaining pixel points to be interpolated, wherein the color channel value of each pixel point in the mosaic image is one of a green channel value, a red channel value and a blue channel value, and the color channel value of the pixel points to be interpolated is the red channel value or the blue channel value; determining a green channel value of a pixel point to be interpolated; correcting the green channel value of the pixel point to be interpolated at the edge of the mosaic image; determining a red channel value and a blue channel value of each pixel point; the mosaic image is converted into a demosaic image based on the green channel value, the red channel value, and the blue channel value of each pixel. The scheme can improve the sawtooth phenomenon, the false color phenomenon and the false color phenomenon of the edge area, which occur in interpolation in the oblique direction of the image in the demosaicing process.

Description

Image demosaicing method and device

Technical Field

The present application relates to the field of imaging devices, and in particular, to a method and apparatus for demosaicing an image.

Background

After a color filter array (color FILTER ARRAY, CFA) is placed on a single image sensor, each pixel point on the acquired image has only one channel color value, and the image is called a mosaic image. Whereas in the currently common RGB image, each pixel point actually has three channel color values. Demosaicing refers to filling in other channel color values missing for each pixel in the mosaic image.

In the prior art, the demosaicing method finds the color values of other channels missing from the pixel, and then fills the color values of other channels missing from the pixel.

However, the existing method has poor demosaicing effect, so that the image cannot accurately restore the color information of the real visual scene.

Disclosure of Invention

The application provides an image demosaicing method and device, which are used for solving the problem of poor demosaicing effect in the prior art.

In a first aspect, an embodiment of the present application provides a method for demosaicing an image, including:

obtaining pixel points to be interpolated in a mosaic image, wherein the color channel value of each pixel point in the mosaic image is one of a green channel value, a red channel value and a blue channel value, and the color channel value of the pixel point to be interpolated is the red channel value or the blue channel value;

Determining a green channel value of the pixel point to be interpolated based on the color channel values of the pixel point to be interpolated, the color channel values of the pixel point in a plurality of setting directions of the pixel point to be interpolated and a first interpolation formula;

Correcting a green channel value of a pixel to be corrected based on the color channel value of the pixel to be interpolated, the color channel value of the pixel in the preset direction of the pixel to be interpolated and a second interpolation formula, wherein the pixel to be corrected is the pixel to be interpolated at the edge of the mosaic image;

Determining a red channel value and a blue channel value of each pixel point based on a green channel value of each pixel point in the mosaic image;

the mosaic image is converted into a demosaic image based on the green channel value, the red channel value, and the blue channel value of each pixel point.

In a second aspect, an embodiment of the present application provides an image demosaicing apparatus, including:

The interpolation pixel point determining module is used for obtaining pixel points to be interpolated in the mosaic image, wherein the color channel value of each pixel point in the mosaic image is one of a green channel value, a red channel value and a blue channel value, and the color channel value of the pixel point to be interpolated is the red channel value or the blue channel value;

the first interpolation module is used for determining a green channel value of the pixel point to be interpolated based on the color channel value of the pixel point to be interpolated, the color channel values of the pixel point in a plurality of setting directions of the pixel point to be interpolated and a first interpolation formula;

The interpolation correction module is used for correcting the green channel value of the pixel to be corrected based on the color channel value of the pixel to be interpolated, the color channel value of the pixel in the preset direction of the pixel to be interpolated and a second interpolation formula, wherein the pixel to be corrected is the pixel to be interpolated at the edge of the mosaic image;

the channel value determining module is used for determining a red channel value and a blue channel value of each pixel point based on the green channel value of each pixel point in the mosaic image;

And the demosaicing module is used for converting the mosaic image into a demosaicing image based on the green channel value, the red channel value and the blue channel value of each pixel point.

According to the image demosaicing method and device provided by the embodiment of the application, the green component lacking in the pixel points is calculated based on the interpolation in multiple setting directions, so that the sawtooth and false color phenomena occurring in the interpolation in the oblique direction of the image in the demosaicing process can be improved, then the edge correction is carried out, the green component of the pixel point at the edge is corrected, the false color phenomenon of the edge of the image can be improved, and the image demosaicing effect is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application;

Fig. 1 is a schematic diagram of a mosaic image according to an embodiment of the present application;

fig. 2 is a schematic diagram of mosaic image generation according to an embodiment of the present application;

Fig. 3 is a schematic flow chart of an image demosaicing method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a pixel to be interpolated according to the present application;

Fig. 5 is a schematic diagram of 6 setting directions of pixel points to be interpolated according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a mask according to an embodiment of the present application;

FIG. 7 is a schematic diagram of color gradient calculation according to an embodiment of the present application;

FIG. 8 is a schematic diagram of green channel value correction according to an embodiment of the present application;

FIG. 9 is a graph showing a comparison of an ideal interpolation filter with a family of five-coefficient approximation functions;

fig. 10 is a schematic diagram of interpolation of red channel values of pixel points according to an embodiment of the present application;

fig. 11 is a schematic diagram of a demosaicing process according to another embodiment of the present application;

Fig. 12 is a schematic diagram of a demosaicing processing module according to an embodiment of the present application;

fig. 13 is a schematic process flow diagram of a green channel 8 direction interpolation module according to an embodiment of the present application;

FIG. 14 is a schematic diagram of a process flow of an edge correction module according to an embodiment of the present application;

fig. 15 is a schematic diagram of a processing flow of a red-blue channel interpolation module according to an embodiment of the present application;

fig. 16 is a schematic diagram of a processing flow of a high-low frequency division correction module according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of an image demosaicing device according to an embodiment of the present application;

Fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In order to reduce cost and complexity, the imaging element of the current camera is mainly a charge coupled device (Charge Coupled Device, CCD) and a CMOS, and the CMOS or the CCD is manufactured by placing a color filter array (color FILTER ARRAY, CFA) on the image sensor, and the CFA can filter light in different frequency bands, so that the data output by the imaging element is in a RAW format, and the RAW format is characterized in that each pixel point only provides one of three color channel values of RGB. Demosaicing means that the other two color channel values missing from the pixel are complemented, so that the pixel has three color channel values of RGB, and the complete RGB format is recovered. There is also a mosaic algorithm in the related art, in which the interpolation direction is determined based on only the green component to be reconstructed, i.e., the pixel points where the green channel value is not captured, so that a judgment needs to be made only in half the pixels of the image, while a frequency division operation is added at the end to further improve the generated reconstructed image. The algorithm has the defects that interpolation is only carried out based on the horizontal direction and the vertical direction, so that the judgment accuracy of the edge region in the oblique direction is not high, and the false color phenomenon exists in the high-frequency regions such as the image edge.

Aiming at the problems, the application provides a demosaicing scheme, which is used for calculating the green component with missing pixel points based on multi-direction interpolation, improving the sawtooth and false color phenomena of interpolation in the oblique direction of an image in the demosaicing process, then carrying out edge correction, correcting the green component with missing pixel points at the edge, and improving the false color phenomena of the edge of the image. After the correction is completed, a green component of each pixel point in the image can be obtained, then based on the green component, the color components of the red or blue channels missing from the pixel point are further reconstructed, and finally, a complete RGB format image is obtained, and the demosaicing effect of the image is improved.

Fig. 1 is a schematic diagram of a mosaic image according to an embodiment of the present application, as shown in fig. 1, taking a small square block in the image as an example of a pixel, G is used to indicate that the pixel has only a green channel value, R is used to indicate that the pixel has only a red channel value, and B is used to indicate that the pixel has only a blue channel value. And each pixel point in the RGB format image has R, G, B color channel values. The other two color channel values missing for each pixel in fig. 1 need to be filled in to enable demosaicing and restoration to a complete RGB image.

Among other things, such mosaic images in fig. 1 are caused by CFA filtering light of different frequency bands. Specifically, fig. 2 is a schematic diagram of mosaic image generation provided in the embodiment of the present application, and as shown in fig. 2, a CFA is laid above an image sensor. Wherein the CFA may be configured in a ratio of 1 red filter, 1 blue filter and 2 green filters. So that only light of a specific color can pass through the filter. In fig. 2, R represents red, G represents green, and B represents blue. Red light may pass through the red filter while green and blue light are blocked, so that a small square block with only red channel values appears in fig. 1; green light may pass through the green filter while red and blue light are blocked, such that small square blocks with only green channel values appear in fig. 1; blue light may pass through the blue filter while red and green light are blocked, so that small square squares with only blue channel values appear in fig. 1.

The demosaicing scheme provided by the application is described in detail below through specific embodiments. It should be noted that the following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 3 is a schematic flow chart of an image demosaicing method provided by the embodiment of the application, which can be applied to processing a mosaic image captured by a CCD or CMOS, and can improve the sawtooth and false color phenomena occurring in interpolation in the oblique direction of the image in the demosaicing process and the false color phenomena of the image edge, so that the demosaicing method provided by the embodiment has a better demosaicing effect in a shooting scene with a backlight and a bright-dark juncture. As shown in fig. 3, the method specifically includes the following steps:

step S310: and obtaining pixel points to be interpolated in the mosaic image.

The color channel value of each pixel point in the mosaic image is one of a green channel value, a red channel value and a blue channel value, and the color channel value of the pixel point to be interpolated is the red channel value or the blue channel value.

In the present embodiment, the characteristics of the mosaic image have been mentioned above, that is, each pixel has only one color channel value (see fig. 1 described above), and for convenience of description, herein, a red pixel is used to describe a pixel having only a red channel value, a blue pixel is used to describe a pixel having only a blue channel value, and a green pixel is used to describe a pixel having only a green channel value. According to different configuration proportions of filters in the CFA, the number of red pixel points, blue pixel points and green pixel points in the mosaic image is different. Illustratively, if the ratio of filters in the CFA is: 1 red filter, 1 blue filter and 2 green filters, the number ratio of red pixel points, blue pixel points and green pixel points in the corresponding mosaic image is 1:1:2, i.e. fig. 1, two small square blocks with green channel values only, one small square block with red channel values only and one small square block with blue channel values only appear in every four small square blocks.

The CFA is mainly formed by a plurality of Bayer filters, and each four adjacent small square blocks (i.e., four pixel points) in fig. 1 form a unit, and the unit is formed by laying the Bayer filters on the image sensor.

The CFA mainly consists of a plurality of Bayer filters, wherein the proportion of the filters in the Bayer filters is 1 red filter, 1 blue filter and 2 green filters, because human eyes are more sensitive to green, and the green light permeability is better than that of the other two colors.

In the present embodiment, since the bayer filter is widely used in the camera, the ratio of the numbers of red pixel, blue pixel, and green pixel in the mosaic image is 1:1:2. based on the consideration of the number proportion, the red pixel points and the blue pixel points are used as pixel points to be interpolated to fill the green components which are missing in the red pixel points and the blue pixel points, so that the method is equivalent to the situation that only 1/2 pixel points in the whole mosaic image need to be interpolated to fill the green components which are missing in the whole mosaic image, the interpolation number can be reduced, and the interpolation efficiency can be improved. If the blue pixel point and the green pixel point are assumed to be used as pixel points to be interpolated, red components missing in the blue pixel point and the green pixel point are filled, so that the interpolation is required to be carried out on 3/4 pixel points, the data volume is large, and the interpolation processing is not facilitated.

Step S320: and determining a green channel value of the pixel to be interpolated based on the color channel values of the pixel to be interpolated, the color channel values of the pixel to be interpolated in a plurality of setting directions and the first interpolation formula.

In this embodiment, it has been mentioned above that the pixel to be interpolated is a red pixel and a blue pixel, and if the pixel to be interpolated is a red pixel, the color channel value of the pixel to be interpolated refers to a red channel value, and if the pixel to be interpolated is a blue pixel, the color channel value of the pixel to be interpolated refers to a blue channel value.

Fig. 4 is a schematic diagram of a pixel to be interpolated according to the present application, as shown in fig. 4, if the pixel to be interpolated is a red pixel (denoted by R0 in fig. 4), there are other pixels in each direction around the pixel to be interpolated, for example, a green pixel G1 and a green pixel G-1, a green pixel G2 and a green pixel G-2, a red pixel R-2 and a red pixel R2, and the like, which are respectively located in different directions of the pixel to be interpolated.

The color channel values of the pixel points can be used as input parameters of a first interpolation formula, and the first interpolation formula obtains a calculation result based on the input parameters. The calculation result can be directly used as a green channel value of the pixel point to be interpolated, but the interpolation effect is not ideal. For this purpose, the calculation result may be further processed to improve the interpolation effect, which will be described in detail later herein.

Step S330: correcting the green channel value of the pixel to be corrected based on the color channel value of the pixel to be interpolated, the color channel value of the pixel in the preset direction of the pixel to be interpolated and the second interpolation formula.

The pixel points to be corrected are pixel points to be interpolated at the edge of the mosaic image.

The continuous edge is expressed as above, that is, the color channel value of the pixel to be interpolated refers to the red channel value or the blue channel value (at this time, the pixel to be interpolated has not yet completed interpolation filling, and only has one color channel value, that is, has no green channel value yet).

Continuing to use the description of fig. 4 above, the pixel points to be interpolated have corresponding pixel points in different directions, so that the preset direction can be preconfigured in combination with the actual situation, for example, a green pixel point G-2 is located above the pixel point to be interpolated, and a green pixel point G2 is located below the pixel point to be interpolated, which is used as the pixel point in the preset direction of the pixel point to be interpolated.

In this embodiment, the color channel value of the pixel point in the preset direction of the pixel point to be interpolated is used as an input of a second interpolation formula, and the second interpolation formula calculates a color value based on the input color channel value, where the color value can be directly used as a green channel value of the pixel point to be corrected. Similarly, the calculated color value can be further processed, and the processing result is used as the green channel value of the pixel to be corrected.

In this embodiment, the pixel points to be corrected refer to pixel points to be interpolated at the edge in the mosaic image. Which can determine which pixels to interpolate are at edges in the mosaic image by an edge detection algorithm. For example, the edge detection algorithm may include: laplace operator, sobel operator (Sobel operator), and the like.

The image edge may be a discontinuous portion of the distribution of characteristics (such as pixel gray level and texture) in the mosaic image, and the characteristics around the image have a step change or a ridge change.

In this embodiment, the green channel values of the pixels to be interpolated can be determined through the step S320, but due to the characteristics of the image edges, the green channel values of the pixels to be interpolated at the edges are revised again through the step S330, so that the occurrence of the false color phenomenon at the edges in the demosaicing image to be output later can be avoided.

Step S340: and determining a red channel value and a blue channel value of each pixel point based on the green channel value of each pixel point in the mosaic image.

In this embodiment, after the green channel value of the pixel to be interpolated is determined by the above-described interpolation method, all the pixels in the entire mosaic image have the green channel value. The pixel point to be interpolated has a red channel value or a blue channel value at the beginning, that is, the pixel point to be interpolated already has two color channel values (green+red or green+blue), and the original green pixel point (i.e., the pixel point not being interpolated) has only a green channel value. In this case, the red channel value or the blue channel value of the original green pixel point can be determined by a color difference value taking method, which is specifically as follows:

R₀-G₀=q（R₁-G₁+R_-1-G_-1）

In the above formula, R ₀ represents the red channel value of the green pixel, G ₀ represents the green channel value of the green pixel, R ₁ represents the red channel value of the first pixel, R _-1 represents the red channel value of the second pixel, G ₁ represents the green channel value of the first pixel, G _-1 represents the green channel value of the second pixel, q is a set value, and the calculated values of R ₀ are different according to the different set values.

Similarly, the blue channel value of the original green pixel point can be determined in the above manner.

Step S350: the mosaic image is converted into a demosaic image based on the green channel value, the red channel value, and the blue channel value of each pixel.

In this embodiment, after each pixel point has three color channel values of red, green and blue, the final interpolated RGB three-channel data is obtained, and the final interpolated RGB three-channel data can be converted into an RGB format image as a demosaicing image. The demosaicing image obtained by the scheme can accurately restore the color information of the real visual scene.

According to the embodiment of the application, the color channel values of other pixel points in multiple directions of the pixel point to be interpolated are utilized to fill the missing color channel values of the pixel point to be interpolated, and after the missing color channel values are filled, the color channel values filled by the pixel point to be interpolated at the edge are continuously corrected, so that the false color phenomenon and the sawtooth phenomenon can be effectively improved, and the demosaicing effect is improved.

The following describes in detail a plurality of setting directions of pixels to be interpolated by some embodiments.

Fig. 5 is a schematic diagram of 6 setting directions of a pixel to be interpolated according to an embodiment of the present application, and as shown in fig. 5, a red pixel R0 is taken as an example of the pixel to be interpolated, which includes directions (a), (b), (c), (d), (e), and (f). Wherein, (a) in fig. 5 shows the left and right sides of the pixel point to be interpolated; (b) above and below the pixel to be interpolated; (c) the left side and the upper side of the pixel point to be interpolated; (d) the left side and the lower side of the pixel point to be interpolated; (e) the right side and the upper side of the pixel point to be interpolated; and (f) represents the right side and the lower side of the pixel point to be interpolated.

In this embodiment, it has been mentioned above that the color filter array CFA is laid on the image sensor, and the color filter array CFA is formed by a plurality of filter (the filter ratio is 2 green filters, 1 red filter and 1 blue filter) arrays, and accordingly, it is possible to determine the row and column of each pixel point in the mosaic image, for example, R _i,j is a red pixel point, which is in the ith row and jth column in the mosaic image. By configuring each pixel with a corresponding row and column, the position of each pixel in the mosaic image can be located, and then the other pixels of each pixel in the set direction can be further determined based on the position of each pixel in the mosaic image.

Wherein, a mask matrix of three RGB color channels can be obtained according to the color filter array CFA, and each mask matrix represents the position of the pixel point under each channel. Fig. 6 is a schematic diagram of a mask provided by the embodiment of the present application, as shown in fig. 6, the small positive square with a value of "1" in fig. 6 (1) represents a red pixel point, and the small positive square with a value of "0" represents other pixel points (including a green pixel point and a blue pixel point). Similarly, a small positive square with a value of "1" in (2) of fig. 6 indicates a blue pixel, and a small positive square with a value of "1" in (3) of fig. 6 indicates a green pixel. After the mask is passed, the positions of the red pixel point, the blue pixel point and the green pixel point can be determined.

In this embodiment, N other pixels may be selected in each setting direction of the pixel point R0 to be interpolated in fig. 5, and then the color channel values of the N other pixels in the 6 setting directions are respectively input into the first interpolation formula, so that the first interpolation formula may output the calculation results corresponding to the 6 setting directions. For example, an optimal calculation result may be selected from the calculation results corresponding to the 6 setting directions, and the optimal calculation result may be used as the green channel value of the pixel point R0 to be interpolated. In addition, in some other embodiments for supporting the present solution, the average value of the calculation results corresponding to the 6 setting directions may be taken as the green channel value of the pixel point R0 to be interpolated. Wherein N is a positive integer.

According to the embodiment of the application, the color channel values of the pixel points in the 6 set directions of the pixel point to be interpolated are selected as the input of the first interpolation formula, so that 6 calculation results can be calculated, and the situation that the gradient change of the color difference values in the directions in the image is large can be judged, so that a proper calculation result is selected and used as the green channel value of the pixel point to be interpolated, and the interpolation effect is further improved.

Further, with continued reference to fig. 5, the following number of color channel values of the pixels may be selected for each set direction as the input parameters of the first interpolation formula:

the direction (a) in fig. 5 is: color channel values of at least two pixel points on the left side and color channel values of at least two pixel points on the right side of the pixel point to be interpolated;

direction (b) in fig. 5: color channel values of at least two pixel points above the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated;

direction (c) in fig. 5: color channel values of at least two pixel points on the left side of the pixel point to be interpolated and color channel values of at least two pixel points above the pixel point to be interpolated;

Direction (d) in fig. 5: color channel values of at least two pixel points on the left side of the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated;

direction (e) in fig. 5: color channel values of at least two pixel points on the right side of the pixel point to be interpolated and color channel values of at least two pixel points above the pixel point to be interpolated;

Direction (f) in fig. 5: color channel values of at least two pixel points on the right side of the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated.

Referring to fig. 5, for the direction (a), two pixels R-2 (red pixel) and G-1 (green pixel) on the left side of the pixel point R0 to be interpolated, and two pixels R2 (red pixel) and G1 (green pixel) on the right side of R0 may be selected as inputs of a first interpolation formula, and a first result may be calculated by the first interpolation formula.

For the direction (b), two pixel points R-2 (red pixel point) and G-1 (green pixel point) above the pixel point R0 to be interpolated, and two pixel points R2 (red pixel point) and G1 (green pixel point) below R0 may be selected as inputs of the first interpolation formula, and a second result is obtained through calculation of the first interpolation formula.

For the direction (c), two pixel points R2 (red pixel point) and G1 (green pixel point) above the pixel point R0 to be interpolated, two pixel points R-2 (red pixel point) and G-1 (green pixel point) on the left side of R0 may be selected as inputs of the first interpolation formula, and a third result is obtained through calculation of the first interpolation formula.

For the direction (d), two pixel points R-2 (red pixel point) and G-1 (green pixel point) on the left side of the pixel point R0 to be interpolated, and two pixel points R2 (red pixel point) and G1 (green pixel point) below R0 may be selected as inputs of the first interpolation formula, and a fourth result is calculated by the first interpolation formula.

For the direction (e), two pixel points R-2 (red pixel point) and G-1 (green pixel point) above the pixel point R0 to be interpolated, and two pixel points R2 (red pixel point) and G1 (green pixel point) on the right side of R0 may be selected as inputs of the first interpolation formula, and a fifth result is calculated by the first interpolation formula.

For the direction (f), two pixel points R-2 (red pixel point) and G-1 (green pixel point) below the pixel point R0 to be interpolated, and two pixel points R2 (red pixel point) and G1 (green pixel point) on the right side of R0 may be selected as inputs of the first interpolation formula, and a sixth result is calculated by the first interpolation formula.

Wherein the 6 directions correspond to one calculation result respectively, and total six calculation results are obtained. And acquiring color channel values of the pixel points to be interpolated based on the six calculation results.

It should be noted that, when there is no other pixel point on the left side, the upper side, the lower side, or the right side of a certain pixel point to be interpolated, for example, there is no other pixel point above the red pixel point R-2 in the direction (b) in fig. 5, at this time, the color channel value of the pixel point below R-2 (i.e., the green pixel point G-1 and the red pixel point R0) may be utilized to obtain the color channel value of the pixel point missing above the red pixel point R-2 through the mapping relationship.

Further, in some embodiments, the first interpolation formula is:

In the above-mentioned method, the step of, For the color channel value of the pixel point to be interpolated,/>For the color channel value of the first pixel point to the left or right or above or below the pixel point to be interpolated,/>For the color channel value of the second pixel to the left or right or above or below the pixel to be interpolated,/>For the green channel candidate values in the set direction, q1, q2, q3, q4, q5 are set values.

In this embodiment, q1, q2, q3, q4, q5 may be configured according to practical situations, and exemplary values of q1, q2, q3 may be 0.5, and values of q4, q5 may be 0.25.

The embodiment of the application collects four pixel points in each set direction of the pixel points to be interpolated、The color channel candidate values in all the setting directions are calculated through a first interpolation formula based on the color channel values of the pixel points to be interpolated and the color channel values of the four pixel points as input parameters, so that differences in all the setting directions can be compared, the optimal green channel value is selected from the green channel candidate values and is used as the green channel value of the pixel points to be interpolated, the interpolation effect is improved, and the occurrence of false color phenomenon and saw tooth phenomenon in the demosaicing process can be avoided.

Having described how to obtain green channel candidate values for each setting direction, it will be explained below by some embodiments how to determine green channel values for pixels to be interpolated based on these green channel candidate values. By way of example, there may be two ways:

Mode (1) selects one target candidate value from among the green channel candidate values in all the setting directions as the green channel value of the pixel to be interpolated.

Mode (2) configures different weighting coefficients for the green channel candidate values in each setting direction, and then weights and sums the green channel candidate values in each setting direction respectively to serve as the green channel value of the pixel to be interpolated.

Wherein, for the mode (1), the following steps are specifically implemented:

step (1): acquiring color difference values of pixel points to be interpolated in each setting direction based on green channel candidate values in each setting direction and color channel values of the pixel points to be interpolated;

Step (II): acquiring a color difference gradient in each set direction based on a color difference value of a pixel point to be interpolated in each set direction;

and step three, selecting a target candidate value from all green channel candidate values based on the color difference gradient in each setting direction.

In this embodiment, fig. 7 is a schematic diagram of color difference gradient calculation provided in the embodiment of the present application, as shown in fig. 7, fig. 7 (a) shows an original mosaic image, where there are a pixel point B to be interpolated (i.e., a blue pixel point) and a pixel point R to be interpolated (i.e., a red pixel point). After obtaining the green channel candidate of the pixel to be interpolated in the above manner, referring to (B) in fig. 7, subtracting (B) in fig. 7 from (a) in fig. 7, to obtain the color difference value of the pixel to be interpolated, where the specific calculation formula is as follows:

In the above-mentioned method, the step of, The color difference value is represented, i and j represent the row and column of the pixel point, H is the set direction, and the value is [1,6].

It should be noted that, each setting direction has a green channel candidate value, which is equivalent to that in each setting direction, the pixel point to be interpolated has a corresponding color difference value.

Further, after the color difference value is obtained by calculation, the color difference values of two adjacent pixel points to be interpolated can be subtracted to obtain a color difference gradient. The specific formula is as follows:

In the above-mentioned method, the step of, The color difference gradient is represented, i and j represent the row and column of the pixel point, H is the set direction, and the value is [1,6].

In this embodiment, after the color difference gradient of the pixel point to be interpolated in each set direction is calculated, the color difference gradient sum in the 5×5 neighborhood of the pixel point to be interpolated in each set direction may be calculated. Illustratively, each color difference gradient in a 5x5 neighborhood may be configured with the following weights:

And calculating to obtain a color difference gradient sum in a 5x5 neighborhood of the pixel point to be interpolated in each set direction (namely, one color difference gradient sum is arranged in each set direction, 6 color difference gradient sums are summed in 6 set directions), comparing the 6 color difference gradient sums, selecting an optimal color difference gradient sum from the 6 color difference gradient sums and a corresponding target set direction, and obtaining a green channel candidate value corresponding to the target set direction as a green channel value of the pixel point to be interpolated.

For example, among the 6 color difference gradient sums, the color difference gradient sum with the smallest value can be selected as the optimal color difference gradient sum.

According to the embodiment of the application, interpolation calculation is carried out through the color channel values of the pixel points in the 6 setting directions of the pixel points to be interpolated and the first interpolation formula, 6 green channel candidate values are obtained, and the optimal target candidate value is selected from the 6 green channel candidate values based on the color difference gradient sum and is used as the green channel value of the pixel points to be interpolated, so that the aliasing phenomenon and the false color phenomenon of oblique interpolation can be effectively improved, and the demosaicing effect of an image is further improved.

After obtaining the green channel value of the pixel to be interpolated, a further correction is required for the green channel value of the pixel to be interpolated at the edge, and the correction procedure is described in detail below through some embodiments.

In this embodiment, some pixels to be interpolated in the mosaic image may be located in an edge area, and for convenience of distinction, the pixels to be interpolated located in the edge area are expressed as pixels to be corrected.

In order to avoid the occurrence of the pseudo color phenomenon of the pixel points to be corrected, the color channel value of each pixel point to be interpolated (including the pixel point to be interpolated in the edge area and the pixel point to be interpolated not in the edge area) in other pixel points in the preset direction can be obtained on the basis of the most original mosaic image (i.e. the mosaic image before interpolation), and then the color channel value is input into a second interpolation formula, a numerical value is calculated, and if the pixel point to be interpolated is the pixel point to be corrected, the numerical value is taken as the green channel value of the pixel point to be corrected.

Fig. 8 is a schematic diagram for correcting a green channel value, as shown in fig. 8, where fig. 8 is a primitive mosaic image, and it is assumed that a red pixel point R0 in fig. 8 is a pixel point to be interpolated in an edge area, and the green channel value of the red pixel point R0 is L1 obtained through the above interpolation step. In this embodiment, a value L2 is calculated by continuously using the color channel values of the other pixels in the preset direction of the red pixel R0 and the second interpolation formula, and then the green channel value L1 determined in the previous interpolation step is corrected to the value L2.

The preset direction may refer to a horizontal direction and a vertical direction of the pixel to be interpolated (for example, the red pixel R0 in fig. 8), that is, left and right sides, upper and lower sides of the pixel to be interpolated. For example, with continued reference to fig. 8, two pixels on the left and right sides of the pixel to be interpolated (i.e., including a green pixel G1, a green pixel G-1, a red pixel R1, and a red pixel R-1) may be taken as the pixels in the horizontal direction of the pixel to be interpolated. In addition, two pixels above and below the pixel to be interpolated (i.e., including a green pixel G-2, a green pixel G2, a red pixel R2, and a red pixel R-2) may be taken as the pixels in the vertical direction of the pixel to be interpolated.

Further, in some embodiments, the second interpolation formula is:

In the above-mentioned method, the step of, For the color channel value of the pixel point to be interpolated,/>The color channel values of the pixels to be interpolated left and right or the color channel values of the pixels above and below the pixel to be interpolated,And correcting the green channel value for the pixel to be corrected. qx1, qx2, qx3, qx4, qx5 are set values

In this embodiment, from the perspective of fourier spectrum analysis and optimization algorithm design, an approximation function of the ideal interpolation filter under the condition of five coefficients can be deduced:

Wherein, The approximation degree of the approximation function and the ideal interpolation filter is determined by different values of the second interpolation formula, so that the calculation result of the second interpolation formula is influenced. The five coefficients are the five weight coefficients of qx1, qx2, qx3, qx4, and qx5 described above,/>The different values of (2) are related to five coefficients. Fig. 9 is a schematic diagram illustrating a comparison of an ideal interpolation filter with a family of five-coefficient approximation function curves. As shown in fig. 9, by adjusting/>The curve of the corresponding approximation function can be made to fit as closely as possible to the curve of the ideal interpolation filter (curve rect in fig. 9), and according to this deduction, an approximation function that more approximates the ideal interpolation filter is applied at the edges of the image. Wherein, at adjustment/>After the corresponding curve of the approximation function is fitted to the curve of the ideal interpolation filter (curve rect in fig. 9) as much as possible, the five coefficients obtained correspond to: -0.083,0.25,0.666,0.25, -0.083, i.e. qx1 and qx5 are-0.083, qx2 and qx4 are 0.25, qx3 is 0.666.

Specifically, if two pixels above and below the pixel to be interpolated (i.e., including the green pixel G-2, the green pixel G2, the red pixel R2, and the red pixel R-2 in fig. 8) are taken as the pixels in the vertical direction of the pixel to be interpolated, the second interpolation formula is as follows:

If two pixels on the left side and the right side of the pixel to be interpolated (i.e. including a green pixel G1, a green pixel G-1, a red pixel R1, and a red pixel R-1) are taken as the pixels in the horizontal direction of the pixel to be interpolated, the second interpolation formula is as follows:

Wherein, can choose according to the actual situation Or/>And the corrected green channel value of the pixel point to be corrected is used as the green channel value.

According to the embodiment of the application, the green channel value of the pixel point to be interpolated in the edge area is corrected by selecting the second interpolation formula which is more in line with the high-frequency area (namely the edge area), so that the problem of false color of the edge of the image can be effectively solved, and the demosaicing effect is further improved.

After the above interpolation and correction of the green channel values of all the pixels are completed, how the red channel value and the blue channel value of each pixel are determined based on the green channel value of each pixel after the interpolation and correction is described in detail below by way of one embodiment.

In this embodiment, after one pixel has two color channel values, the color difference value of the pixel can be obtained by subtracting, and then the missing color channel values of other pixels are obtained based on the principle that the color difference values of adjacent pixels are equal. The method can be concretely divided into the following modes:

mode a: acquiring a first color difference value of a left pixel point of a pixel point and a second color difference value of a right pixel point of the pixel point; and determining a red channel value and a blue channel value of the pixel point based on the first color difference value, the second color difference value and the green channel value of the pixel point.

The first color difference value is a difference value of two color channel values of the left pixel point, and the second color difference value is a difference value of two color channel values of the right pixel point.

Mode B: acquiring a third color difference value of a pixel above the pixel and a fourth color difference value of a pixel below the pixel; and determining a red channel value and a blue channel value of the pixel point based on the third color difference value, the fourth color difference value and the green channel value of the pixel point.

The third color difference value is a difference value of two color channel values of the upper pixel point, and the fourth color difference value is a difference value of two color channel values of the lower pixel point.

Fig. 10 is a schematic diagram of interpolation of red channel values of pixel points according to an embodiment of the present application, and as shown in fig. 10, the interpolation may be divided into the following two cases according to the positions of the pixel points:

case (1): the pixel is at the position in fig. 10 ①

Case (2): the pixel is at position ② or position in FIG. 10 ③

Wherein the R value (i.e., red channel value) or the B value (red channel value) at position ② is laterally bilinear interpolated; the R or B values at position ③ are interpolated bi-linearly in the longitudinal direction, the interpolation formula can be expressed as:

In the above-mentioned method, the step of, Is the R value at position ②,/>Is the G value at position ②,/>R value of pixel point at left side of position ②,/>R value of pixel point on right side of position ②,/>Is the G value of the pixel point at the left side of the position ②,/>The G value for the pixel to the right of location ②. The B value at position ② can be found in the same manner.

For the red channel, for the position ①, according to the color difference gradient sum obtained in the interpolation step, the interpolation calculation of the red channel value is performed on the current pixel point, and the calculation formula is as follows:

In the above-mentioned method, the step of, Red channel value for pixel point of ith row and jth column,/>The blue channel value of the pixel point of the ith row and the jth column. /(I)、/>The sum of the color difference gradients in the vertical direction and the color difference gradients in the horizontal direction of the pixel points are respectively.

According to the embodiment of the application, the color difference gradient sum used in the interpolation step is utilized to further determine the red channel value or the blue channel value of the pixel point, so that interpolation calculation of the red channel value and the blue channel value is not needed again, the calculation complexity can be reduced, and the image demosaicing efficiency is improved. Meanwhile, the demosaicing effect can be further improved by determining the red channel value or the blue channel value of the pixel point based on the green channel value of the pixel point obtained in the interpolation step.

The demosaicing process of the present solution is described in detail below with respect to a complete embodiment.

Fig. 11 is a schematic diagram of a demosaicing process according to another embodiment of the present application, as shown in fig. 11, which includes the following steps:

s1101: acquiring a two-dimensional bayer image;

S1102: acquiring RGB three-channel data;

s1103: interpolation is carried out on the G channel along 6 directions;

s1104: selecting an optimal interpolation direction according to the color difference gradient;

s1105: interpolating the G channel horizontally and vertically according to the high-frequency filter;

s1106: extracting an edge mask from the G channel by using a Laplacian operator;

s1107: correcting the edge area according to the edge mask;

s1108: interpolation is carried out on color difference components of the red and blue channels;

S1109: updating a high-frequency component of a green channel to be interpolated;

S1110: and updating the high-frequency component of the red and blue channels to be interpolated.

Correspondingly, fig. 12 is a schematic diagram of a demosaicing processing module according to an embodiment of the present application, and as shown in fig. 12, the demosaicing processing module is divided into four sub-modules:

1) Green channel 8 direction interpolation module: the method is used for processing the green channel, and rebuilding the position to be interpolated of the green channel along the 8 directions of up, down, left, right, left up, left down, right up and right down according to the designed basic interpolation filter. And then selecting the optimal interpolation direction according to the color difference gradient, thereby obtaining a reconstructed green channel. And carrying out interpolation based on 8 directions, designing a corresponding basic interpolation filter (namely corresponding to a first interpolation formula), and finally improving the sawtooth phenomenon and the pseudo-color phenomenon which occur in the oblique direction interpolation.

2) Edge correction module: reconstructing the position to be interpolated of the green channel along the horizontal and vertical directions according to the designed high-frequency interpolation filter (namely corresponding to a second interpolation formula); and then, processing the reconstructed green channel by using a Laplacian operator to obtain an edge mask, and correcting the edge region of the green channel according to the mask. Which can improve the false color phenomenon of the edge region.

3) Red and blue channel interpolation module: and interpolating the color difference value of the red and blue channels according to the classifier obtained by the green channel.

4) The high-low frequency division correction module: and correcting the high-frequency component of the reconstructed pixel point by using the high-frequency component of the current pixel point.

Specifically, fig. 13 is a schematic process flow diagram of a green channel 8 direction interpolation module according to an embodiment of the present application, as shown in fig. 13, including the following steps:

s1301: acquiring a two-dimensional bayer image;

s1302: acquiring RGB three-channel masks and extracting three-channel value data;

s1303: interpolation is carried out on the position to be interpolated of the G channel along the 8 direction;

S1304: calculating a color difference value of a position to be interpolated of the G channel;

s1305: calculating color difference gradients in 8 directions;

S1306: calculating a color difference gradient of a 5x5 neighborhood and acquiring a classifier;

s1307: and obtaining the pixel value of the position to be interpolated according to the classifier.

Wherein the basic interpolation filter fir= [ -0.25,0.5,0.5,0.5, -0.25], the first interpolation formula applied is:

The parameters associated with the first interpolation formula are described with particular reference to the description of fig. 5 above. In addition, the classifier may specifically refer to calculation of the green channel candidate value based on the 6 directions in fig. 5.

Specifically, fig. 14 is a schematic process flow diagram of an edge correction module according to an embodiment of the present application, as shown in fig. 14, including the following steps:

s1401: acquiring G channel data;

S1402: acquiring a high-frequency interpolation filter;

s1403: interpolating the G channel horizontally and vertically according to the high-frequency interpolation filter;

s1404: extracting an edge mask from the G channel by using a Laplacian operator;

s1405: and correcting the edge area according to the edge mask.

The specific flow is as follows:

1) And (3) applying a high-frequency interpolation filter to interpolate the position to be interpolated of the original G channel in the horizontal and vertical directions.

2) And processing the G channel data output by the previous stage by using the Laplacian, obtaining an edge image, and extracting an edge mask according to a set threshold value, wherein the threshold value is set to be 1/4 of the maximum value of the gray level number.

3) The edge region of the G channel data outputted from the previous stage is corrected according to the edge mask matrix obtained in 2) and the horizontal-vertical classifier obtained by the green channel 8 direction interpolation module ((refer to (a) and (b) in fig. 5). So far, the G channel data after the edge correction can be obtained. Wherein the high-frequency interpolation filter is FIR_hf= [ -0.083,0.25,0.666,0.25, -0.083].

Fig. 15 is a schematic process flow diagram of a red-blue channel interpolation module according to an embodiment of the present application, as shown in fig. 15, including the following steps:

S1501: processing R, B channel data by a sub-channel;

S1502: performing bilinear interpolation based on the color difference values for positions ② and ③ to be interpolated;

s1503: and interpolating according to the classifier aiming at the position ① to be interpolated.

The specific flow is as follows:

1) And carrying out channel separation processing on the RGB three-channel data and the RGB three-channel mask matrix.

2) As shown in fig. 10 above, for the red channel, a lateral bilinear interpolation is performed on the R value at position ②, and a longitudinal bilinear interpolation is performed on the R value at position ③; the same can be done for the blue channel. The interpolation formula can be expressed as:

3) According to the red-blue channel data calculated in the step 1), for the red channel, for the position ①, interpolation calculation can be performed on the current pixel point according to the classifier obtained in the previous green channel value interpolation step, and the calculation formula is as follows:

similarly, the same operation can be performed on the blue channel, and finally, the interpolated R, B color channel value can be obtained.

Fig. 16 is a schematic process flow diagram of a high-low frequency division correction module according to an embodiment of the present application, as shown in fig. 16, including the following steps:

s1601: RGB three-channel data;

S1602: dividing the frequency of the image;

s1603: updating a high-frequency component of a green channel to be interpolated;

s1604: and updating the high-frequency component of the red and blue channels to be interpolated.

The high-low frequency division correction module mainly corrects the image interpolated by the module, and replaces the high-frequency component of the reconstructed pixel point with the high-frequency component of the current pixel point. The specific process flow is as follows:

1) Carrying out channel separation processing on RGB three-channel data and RGB three-channel mask matrixes after the previous level interpolation;

2) And carrying out frequency division processing on the image, updating the green pixel value, and replacing the high-frequency component at the position to be interpolated of the G channel with the high-frequency component of the original R or B pixel.

3) The high frequency component of the R (B) pixel located at the G pixel position is replaced with the original G pixel high frequency component.

4) The high frequency component of the B (R) pixel located at the R (B) pixel position is replaced with the original R (B) pixel high frequency component. Thus, the final interpolated RGB three-channel data is obtained and converted into an (H, W, C) -format image. Wherein H is high, W is wide, and C is the number of color channels.

The following are examples of the apparatus of the present application that may be used to perform the method embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the method of the present application.

Fig. 17 is a schematic structural diagram of an image demosaicing device according to an embodiment of the present application, as shown in fig. 17, the demosaicing device 1700 includes: an interpolation pixel determination module 1710, a first interpolation module 1720, an interpolation correction module 1730, a channel value determination module 1740, and a demosaicing module 1750.

The interpolation pixel point determining module 1710 is configured to obtain a pixel point to be interpolated in the mosaic image.

The first interpolation module 1720 is configured to determine a green channel value of the pixel to be interpolated based on the color channel value of the pixel to be interpolated, the color channel values of the pixel in a plurality of setting directions of the pixel to be interpolated, and the first interpolation formula. The interpolation correction module 1730 is configured to correct a green channel value of the pixel to be corrected based on the color channel value of the pixel to be interpolated, the color channel value of the pixel in the preset direction of the pixel to be interpolated, and the second interpolation formula. The channel value determination module 1740 is configured to determine a red channel value and a blue channel value for each pixel point in the mosaic image based on the green channel value for each pixel point. The demosaicing module 1750 is configured to convert the mosaic image into a demosaicing image based on the green channel value, the red channel value, and the blue channel value of each pixel.

The color channel value of each pixel point in the mosaic image is one of a green channel value, a red channel value and a blue channel value, and the color channel value of the pixel point to be interpolated is the red channel value or the blue channel value. The pixel points to be corrected are pixel points to be interpolated at the edge of the mosaic image

Optionally, the setting direction includes: left and right sides of pixel points to be interpolated; left side and upper side of pixel point to be interpolated; left side and lower side of pixel point to be interpolated; the right side and the upper side of the pixel point to be interpolated; the right side and the lower side of the pixel point to be interpolated; above and below the pixel to be interpolated.

Optionally, the first interpolation module may specifically be configured to: acquiring a green channel candidate value under each set direction based on the color channel value of the pixel point to be interpolated, the color channel value of the pixel point in each set direction of the pixel point to be interpolated and the first interpolation formula; and selecting a target candidate value from all green channel candidate values as a green channel value of the pixel point to be interpolated.

Optionally, the first interpolation module may specifically be configured to: acquiring color difference values of pixel points to be interpolated in each setting direction based on green channel candidate values in each setting direction and color channel values of the pixel points to be interpolated; obtaining a color difference gradient in each set direction based on the color difference value of the pixel point to be interpolated in each set direction; and finally, selecting a target candidate value from all green channel candidate values based on the color difference gradient in each set direction.

Optionally, color channel values of the pixel points in a plurality of setting directions of the pixel point to be interpolated include: color channel values of at least two pixel points on the left side and color channel values of at least two pixel points on the right side of the pixel point to be interpolated; color channel values of at least two pixel points above the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated; color channel values of at least two pixel points on the left side of the pixel point to be interpolated and color channel values of at least two pixel points above the pixel point to be interpolated; color channel values of at least two pixel points on the left side of the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated; color channel values of at least two pixel points on the right side of the pixel point to be interpolated and color channel values of at least two pixel points above the pixel point to be interpolated; color channel values of at least two pixel points on the right side of the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated.

Optionally, the first interpolation formula is:

In the above-mentioned method, the step of, For the color channel value of the pixel point to be interpolated,/>、/>For the color channel value of the first pixel point to the left or right or above or below the pixel point to be interpolated,/>、/>For the color channel value of the second pixel to the left or right or above or below the pixel to be interpolated,/>For the green channel candidate values in the set direction, q1, q2, q3, q4, q5 are set values.

Optionally, the preset direction of the pixel point to be interpolated includes: left and right sides of the pixel to be corrected, and upper and lower sides of the pixel to be interpolated.

Optionally, the second interpolation formula is:

In the above-mentioned method, the step of, For the color channel value of the pixel point to be interpolated,/>、/>、/>、/>Color channel values of pixel points at the left side, the right side, the upper side and the lower side of the pixel point to be interpolated,/>, respectivelyAnd (3) for the green channel values after the pixel point to be corrected is corrected, wherein qx1, qx2, qx3, qx4 and qx5 are set values.

Optionally, the channel value determining module may specifically be configured to: acquiring a first color difference value of a left pixel point of a pixel point and a second color difference value of a right pixel point of the pixel point, wherein the first color difference value is a difference value of two color channel values of the left pixel point, and the second color difference value is a difference value of two color channel values of the right pixel point; then, based on the first color difference value, the second color difference value and the green channel value of the pixel point, determining the red channel value and the blue channel value of the pixel point;

Or, obtaining a third color difference value of the pixel above the pixel and a fourth color difference value of the pixel below the pixel, wherein the third color difference value is a difference value of two color channel values of the pixel above, and the fourth color difference value is a difference value of two color channel values of the pixel below; and then determining a red channel value and a blue channel value of the pixel point based on the third color difference value, the fourth color difference value and the green channel value of the pixel point.

The device provided by the embodiment of the application can be used for executing the method in the embodiment, and the implementation principle and the technical effect are similar, and are not repeated here.

It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the interpolation pixel point determining module may be a processing element which is set up separately, may be implemented as an integrated unit in one of the above-mentioned devices, or may be stored in a memory of the above-mentioned device in the form of a program code, and the function of the above-mentioned interpolation pixel point determining module may be called and executed by one of the above-mentioned processing elements of the above-mentioned device. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element here may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.

Fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 18, the electronic device 1800 includes: at least one processor 1801, memory 1802, bus 1803, and communication interfaces 1804. Wherein: the processor, communication interface and memory communicate with each other via the bus. The communication interface is used for communicating with other devices. The communication interface comprises a communication interface for data transmission, a display interface or an operation interface for human-computer interaction, and the like.

The processor is configured to execute the computer-executable instructions stored in the memory and may in particular perform the relevant steps of the methods described in the embodiments above.

The processor may be a central processing unit (cpu), or an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors comprised by the rechecking device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs. And the memory is used for storing computer execution instructions. The memory may comprise high speed RAM memory or may also comprise non-volatile memory, such as at least one disk memory.

The present embodiment also provides a readable storage medium having stored therein computer instructions which, when executed by at least one processor of an electronic device, perform the methods provided by the various embodiments described above.

The present embodiment also provides a program product comprising computer instructions stored in a readable storage medium. The computer instructions may be read from a readable storage medium by at least one processor of an electronic device, and executed by at least one processor, cause the electronic device to implement the methods provided by the various embodiments described above.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a alone, a and B together, and B alone, wherein a, B may be singular or plural. The character "/" generally indicates that the front and rear associated objects are an "or" relationship; in the formula, the character "/" indicates that the front and rear associated objects are a "division" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

It will be appreciated that the various numerical numbers referred to in the embodiments of the present application are merely for ease of description and are not intended to limit the scope of the embodiments of the present application. In the embodiment of the present application, the sequence number of each process does not mean the sequence of the execution sequence, and the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application in any way.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.

Claims

1. A method of demosaicing an image, comprising:

2. The method of claim 1, wherein the setting the direction comprises:

The left side and the right side of the pixel point to be interpolated;

the left side and the upper side of the pixel point to be interpolated;

the left side and the lower side of the pixel point to be interpolated;

the right side and the upper side of the pixel point to be interpolated;

the right side and the lower side of the pixel point to be interpolated;

Above and below the pixel point to be interpolated.

3. The method of claim 1, wherein the determining the green channel value for the pixel to be interpolated based on the color channel value for the pixel to be interpolated, the color channel values for the pixel in the plurality of set directions for the pixel to be interpolated, and the first interpolation formula comprises:

Acquiring a green channel candidate value in each set direction based on the color channel value of the pixel point to be interpolated, the color channel value of the pixel point in each set direction of the pixel point to be interpolated and the first interpolation formula;

And selecting a target candidate value from all green channel candidate values as the green channel value of the pixel point to be interpolated.

4. A method according to claim 3, wherein said selecting a target candidate value among all green channel candidate values comprises:

Acquiring a color difference value of the pixel point to be interpolated in each setting direction based on the green channel candidate value in each setting direction and the color channel value of the pixel point to be interpolated;

Acquiring a color difference gradient in each setting direction based on the color difference value of the pixel point to be interpolated in each setting direction;

The target candidate is selected from among all green channel candidates based on the color difference gradient in each set direction.

5. The method according to claim 1 or 2, wherein the color channel values of the pixels in the plurality of setting directions of the pixels to be interpolated include:

color channel values of at least two pixel points on the left side and color channel values of at least two pixel points on the right side of the pixel point to be interpolated;

Color channel values of at least two pixel points above the pixel point to be interpolated and color channel values of at least two pixel points below the pixel point to be interpolated;

the color channel values of at least two pixel points on the left side of the pixel point to be interpolated and the color channel values of at least two pixel points above the pixel point to be interpolated;

the color channel values of at least two pixel points on the left side of the pixel point to be interpolated and the color channel values of at least two pixel points below the pixel point to be interpolated;

The color channel values of at least two pixel points on the right side of the pixel point to be interpolated and the color channel values of at least two pixel points above the pixel point to be interpolated;

And the color channel values of at least two pixel points on the right side of the pixel point to be interpolated and the color channel values of at least two pixel points below the pixel point to be interpolated.

6. The method of claim 5, wherein the first interpolation formula is:

,

In the above-mentioned method, the step of, For the color channel value of the pixel point to be interpolated,/>、/>For the color channel value of the first pixel point at the left side or the right side or above or below the pixel point to be interpolated,/>、/>For the color channel value of the second pixel point at the left side or the right side or above or below the pixel point to be interpolated,/>For the green channel candidate values in the set direction, q1, q2, q3, q4, q5 are set values.

7. The method according to claim 1, wherein the pixel point to be interpolated preset direction includes:

the left side and the right side of the pixel point to be corrected;

Above and below the pixel point to be interpolated.

8. The method of claim 7, wherein the second interpolation formula is:

,

9. The method of claim 1, wherein determining the red channel value and the blue channel value for each pixel point in the mosaic image based on the green channel value for each pixel point comprises:

Acquiring a first color difference value of a left pixel point of the pixel point and a second color difference value of a right pixel point of the pixel point, wherein the first color difference value is a difference value of two color channel values of the left pixel point, and the second color difference value is a difference value of two color channel values of the right pixel point;

Determining a red channel value and a blue channel value of the pixel point based on the first color difference value, the second color difference value and the green channel value of the pixel point;

Or alternatively, the first and second heat exchangers may be,

Acquiring a third color difference value of a pixel above the pixel and a fourth color difference value of a pixel below the pixel, wherein the third color difference value is a difference value of two color channel values of the pixel above, and the fourth color difference value is a difference value of two color channel values of the pixel below;

and determining a red channel value and a blue channel value of the pixel point based on the third color difference value, the fourth color difference value and the green channel value of the pixel point.

10. An image demosaicing apparatus, comprising: