CN110493583A - Image processing method, device, electronic equipment and computer readable storage medium - Google Patents
Image processing method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN110493583A CN110493583A CN201810875810.XA CN201810875810A CN110493583A CN 110493583 A CN110493583 A CN 110493583A CN 201810875810 A CN201810875810 A CN 201810875810A CN 110493583 A CN110493583 A CN 110493583A
- Authority
- CN
- China
- Prior art keywords
- color channel
- original
- image
- data
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Color Image Communication Systems (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The embodiment of the invention provides a kind of image processing method, device, electronic equipment and computer readable storage medium, this programme can obtain the raw image data of imaging sensor acquisition;Wherein, the raw image data includes at least original all-pass channel data, original first color channel data and original second color channel data, any channel data all has empty pixel, and first color channel and second color channel are the two of them in R, G, B color channel;Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated;The all-pass channel image is merged with the RGB image, obtains fused image.Using scheme provided in an embodiment of the present invention, the preferable clear image of quality can be collected.
Description
Technical field
The present invention relates to image technique fields, more particularly to a kind of image processing method, device, electronic equipment and calculating
Machine readable storage medium storing program for executing.
Background technique
In the prior art, image capture device acquires image often through imaging sensor.It is, for example, possible to use Bayers
Visible images of the imaging sensor acquisition comprising three kinds of color channels in rgb color channel of format.However, environment can
It in the light-exposed lower situation of intensity of illumination, is limited by sensor photosensitive performance, this mode visible images collected are made an uproar
Sound is larger, not enough clearly.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of image processing method, device, electronic equipment and computer-readable
Storage medium, to collect the preferable clear image of quality.Specific technical solution is as follows:
In a first aspect, the embodiment of the invention provides a kind of image processing methods, which comprises
Obtain the raw image data of imaging sensor acquisition;Wherein, the raw image data includes at least original complete
Track data, original first color channel data and original second color channel data, any channel data all have cavity all
Pixel, first color channel and second color channel are the two of them in R, G, B color channel;
Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated;
The all-pass channel image is merged with the RGB image, obtains fused image.
Optionally, described the step of being based on the raw image data, generating the all-pass channel image of full resolution, packet
It includes:
Interpolation processing is carried out to the original all-pass channel data, obtains the all-pass channel image of the full resolution.
Optionally, described that interpolation processing is carried out to the original all-pass channel data, obtain the all-pass of the full resolution
The step of channel image, comprising:
Interpolation interpolation processing is carried out to the original all-pass channel data, is determined each in the original all-pass track data
Channel value at a cavity pixel, and then obtain the all-pass channel image of the full resolution.
Optionally, described that interpolation processing is carried out to the original all-pass channel data, obtain the all-pass of the full resolution
Channel image includes:
According to original first color channel data and original second color channel data, the first weight is determined;
According to the original all-pass channel data and first weight, interpolation is carried out to the original all-pass channel data
Processing determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution
The all-pass channel image of rate.
Optionally, described the step of being based on the raw image data, generating the RGB image of full resolution, comprising:
Interpolation processing is carried out to original first color channel data, obtains the first color channel figure of full resolution
Picture;
Interpolation processing is carried out to original second color channel data, obtains the second color channel figure of full resolution
Picture;
According to the all-pass channel image, first color channel images and second color channel images, determine
The third color channel image of full resolution, the third color channel are that first color is removed in R, G, B color channel
A color channel except color channel and second color channel;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
Optionally, described that interpolation processing is carried out to original first color channel data, obtain the first of full resolution
The step of color channel images, comprising:
Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel
The channel value at each empty pixel in data, and then obtain the first color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original second color channel data, the second weight is determined;According to
Second weight and original first color channel data carry out at interpolation original first color channel data
Reason is determined the channel value at each empty pixel in original first color channel data, and then is obtained described complete
First color channel images of resolution ratio.
Optionally, described that interpolation processing is carried out to original second color channel data, obtain the second of full resolution
The step of color channel images, comprising:
Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel
The channel value at each empty pixel in data, and then obtain the second color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original first color channel data, third weight is determined, according to
The third weight and original second color channel data carry out at interpolation original second color channel data
Reason is determined the channel value at each empty pixel in original second color channel data, and then is obtained described complete
Second color channel images of resolution ratio.
Optionally, described logical according to the all-pass channel image, first color channel images and second color
Road image, the step of determining the third color channel image of full resolution, comprising:
It determines the channel value of any pixel point in the third color channel image of the full resolution, and then is differentiated entirely
The third color channel image of rate;Wherein, the channel value of any pixel point is according to the all-pass channel image, described
In first color channel images and second color channel images with any pixel point same position pixel
What channel value determined.
Optionally, the channel value of any pixel point is determined by following formula in the third color channel image:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in the third color channel image, C1、S2And S3It respectively corresponds
Indicate in the all-pass channel image, first color channel images and second color channel images with any picture
Channel value of the vegetarian refreshments in the pixel of same position, α, β, K1And K2It is preset constant value.
Optionally, described the step of being based on the raw image data, generating the RGB image of full resolution, comprising:
According to the raw image data, third color channel data is determined, the third color channel is described R, G, B
A color channel in color channel in addition to first color channel and second color channel;
To original first color channel data, original second color channel data and the third color channel
Data carry out interpolation processing respectively, obtain the RGB image of the full resolution.
Optionally, described according to the raw image data, the step of determining third color channel data, comprising:
By the non-empty pixel in the all-pass channel data, the non-empty being determined as in the third color channel data
Hole pixel;
It determines the channel value of any non-empty pixel in the third color channel data, and then obtains third color
Channel data;Wherein, the channel value of any non-empty pixel in the third color channel data, is according to described original
The channel value of pixel in all-pass channel data with any non-empty pixel in same position, original first color
In color channel data with any non-empty pixel in the corresponding default field of pixel of same position each pixel
In the average channel value of point and second color channel data with any non-empty pixel same position picture
The average channel value of each pixel determines in the corresponding default field of vegetarian refreshments.
Optionally, the channel value of any non-empty pixel in the third color channel data is true by following formula
It is fixed:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in the third color channel data, C2Described in expression
The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, S5Described in expression
With any non-empty pixel in the corresponding default field of pixel of same position in original first color channel data
In each pixel average channel value, S6It indicates to exist in second color channel data with any non-empty pixel
The average channel value of each pixel in the corresponding default field of the pixel of same position;α,β,K1And K2It is preset
Constant value.
Optionally, described to original first color channel data, original second color channel data and described
Third color channel data carries out interpolation processing respectively, obtains the RGB image of the full resolution, comprising:
Interpolation processing is carried out to original first color channel data, determines original first color channel data
In each empty pixel at channel value, and then obtain the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, determines original second color channel data
In each empty pixel at channel value, and then obtain the second color channel images of full resolution;
Interpolation processing is carried out to the third color channel data, is determined each in the third color channel data
Channel value at empty pixel, and then obtain the third color channel image of full resolution;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
It is optionally, described that interpolation processing is carried out to original first color channel data, comprising:
Interpolation interpolation processing is carried out to original first color channel data;
Alternatively, according to the original all-pass channel data, original second color channel data and the third color
Channel data determines the 4th weight;According to the 4th weight and original first color channel data, to described original
One color channel data carries out interpolation processing;
It is described that interpolation processing is carried out to original second color channel data, comprising:
Interpolation interpolation processing is carried out to default second color channel data;
Alternatively, according to the original all-pass channel data, original first color channel data and the third color
Channel data determines the 5th weight;According to the 5th weight and original second color channel data, to described original
Two color channel datas carry out interpolation processing;
It is described that interpolation processing is carried out to the third color channel data, comprising:
Interpolation interpolation processing is carried out to the third color channel data;
Alternatively, according to the original all-pass channel data, original first color channel data and described original second
Color channel data determines the 6th weight;According to the 6th weight and the third color channel data, to the third color
Color channel data carries out interpolation processing.
Optionally, described to merge the all-pass channel image with the RGB image, obtain fused image
Step, comprising:
Obtain the color information in the RGB image;
Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image;
The color information, first luminance information and second luminance information are merged, described melt is obtained
Image after conjunction.
Second aspect, the embodiment of the invention provides a kind of image processing apparatus, described device includes:
Module is obtained, for obtaining the raw image data of image capture device acquisition;Wherein, the raw image data
Including at least original all-pass channel data, original first color channel data and original second color channel data, any channel
Data all have empty pixel, and first color channel and second color channel are its in R, G, B color channel
In two kinds;
Generation module generates the all-pass channel image and full resolution of full resolution for being based on the raw image data
The RGB image of rate;
Fusion Module obtains fused figure for merging the all-pass channel image with the RGB image
Picture.
Optionally, the generation module, comprising:
All-pass channel image generates submodule, for carrying out interpolation processing to the original all-pass channel data, obtains institute
State the all-pass channel image of full resolution.
Optionally, the all-pass channel image generates submodule, is specifically used for:
Interpolation interpolation processing is carried out to the original all-pass channel data, is determined each in the original all-pass track data
Channel value at a cavity pixel, and then obtain the all-pass channel image of the full resolution.
Optionally, the all-pass channel image generates submodule, is specifically used for:
According to original first color channel data and original second color channel data, the first weight is determined;
According to the original all-pass channel data and first weight, interpolation is carried out to the original all-pass channel data
Processing determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution
The all-pass channel image of rate.
Optionally, the generation module, comprising: the first RGB image generates submodule, is used for:
Interpolation processing is carried out to original first color channel data, obtains the first color channel figure of full resolution
Picture;
Interpolation processing is carried out to original second color channel data, obtains the second color channel figure of full resolution
Picture;
According to the all-pass channel image, first color channel images and second color channel images, determine
The third color channel image of full resolution, the third color channel are that first color is removed in R, G, B color channel
A color channel except color channel and second color channel;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
Optionally, first RGB image generates submodule and carries out at interpolation to original first color channel data
Reason, obtains the first color channel images of full resolution, specifically:
Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel
The channel value at each empty pixel in data, and then obtain the first color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original second color channel data, the second weight is determined;According to
Second weight and original first color channel data carry out at interpolation original first color channel data
Reason is determined the channel value at each empty pixel in original first color channel data, and then is obtained described complete
First color channel images of resolution ratio.
Optionally, first RGB image generates submodule and carries out at interpolation to original second color channel data
Reason, obtains the second color channel images of full resolution, specifically:
Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel
The channel value at each empty pixel in data, and then obtain the second color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original first color channel data, third weight is determined, according to
The third weight and original second color channel data carry out at interpolation original second color channel data
Reason is determined the channel value at each empty pixel in original second color channel data, and then is obtained described complete
Second color channel images of resolution ratio.
Optionally, first RGB image generates submodule according to the all-pass channel image, first color channel
Image and second color channel images, determine the third color channel image of full resolution, specifically:
It determines the channel value of any pixel point in the third color channel image of the full resolution, and then is differentiated entirely
The third color channel image of rate;Wherein, the channel value of any pixel point is according to the all-pass channel image, described
In first color channel images and second color channel images with any pixel point same position pixel
What channel value determined.
Optionally, the channel value of any pixel point in third color channel image is determined by following formula:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in the third color channel image, C1、S2And S3It respectively corresponds
Indicate in the all-pass channel image, first color channel images and second color channel images with any picture
Channel value of the vegetarian refreshments in the pixel of same position, α, β, K1And K2It is preset constant value.
Optionally, the generation module, comprising: the second RGB image generates submodule, is used for:
According to the raw image data, third color channel data is determined, the third color channel is described R, G, B
A color channel in color channel in addition to first color channel and second color channel;
To original first color channel data, original second color channel data and the third color channel
Data carry out interpolation processing respectively, obtain the RGB image of the full resolution.
Optionally, described second RGB image generates submodule according to the raw image data, determines that third color is logical
Track data specifically includes:
By the non-empty pixel in the all-pass channel data, the non-empty being determined as in the third color channel data
Hole pixel;
It determines the channel value of any non-empty pixel in the third color channel data, and then obtains the third
Color channel data, wherein the channel value of any non-empty pixel in the third color channel data, is basis
Channel value, the original of pixel in the original all-pass channel data with any non-empty pixel in same position
Begin in the first color channel data with any non-empty pixel in the corresponding default field of pixel of same position
With any non-empty pixel same in the average channel value of each pixel and second color channel data
The average channel value of each pixel determines in the corresponding default field of the pixel of position.
Optionally, the channel value of any non-empty pixel in third color channel data is determined by following formula:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in the third color channel data, C2Described in expression
The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, S5Described in expression
With any non-empty pixel in the corresponding default field of pixel of same position in original first color channel data
In each pixel average channel value, S6It indicates to exist in second color channel data with any non-empty pixel
The average channel value of each pixel in the corresponding default field of the pixel of same position;α,β,K1And K2It is preset
Constant value.
Optionally, second RGB image generates submodule to original first color channel data, described original the
Two color channel datas and the third color channel data carry out interpolation processing respectively, obtain the RGB figure of the full resolution
Picture, specifically:
Interpolation processing is carried out to original first color channel data, determines original first color channel data
In each empty pixel at channel value, and then obtain the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, determines original second color channel data
In each empty pixel at channel value, and then obtain the second color channel images of full resolution;
Interpolation processing is carried out to the third color channel data, is determined each in the third color channel data
Channel value at empty pixel, and then obtain the third color channel image of full resolution;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
Optionally, second RGB image generates submodule and carries out at interpolation to original first color channel data
Reason specifically includes:
Interpolation interpolation processing is carried out to original first color channel data;Alternatively, according to the original all-pass channel
Data, original second color channel data and the third color channel data, determine the 4th weight;According to the described 4th
Weight and original first color channel data carry out interpolation processing to original first color channel data;
Second RGB image generates submodule and specifically wraps to the original second color channel data progress interpolation processing
It includes:
Interpolation interpolation processing is carried out to default second color channel data;Alternatively, according to the original all-pass channel
Data, original first color channel data and the third color channel data, determine the 5th weight;According to the described 5th
Weight and original second color channel data carry out interpolation processing to original second color channel data;
Second RGB image generates submodule and specifically includes to third color channel data progress interpolation processing:
Interpolation interpolation processing is carried out to the third color channel data;Alternatively, according to the original all-pass channel data,
Original first color channel data and original second color channel data, determine the 6th weight;According to the described 6th
Weight and the third color channel data carry out interpolation processing to the third color channel data.
Optionally, the Fusion Module, is specifically used for:
Obtain the color information in the RGB image;
Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image;
The color information, first luminance information and second luminance information are merged, described melt is obtained
Image after conjunction.
The third aspect, the embodiment of the invention provides a kind of image processing systems, comprising:
Image capture device, including at least one imaging sensor, described image sensor include the colour in all-pass channel
Filter array, described image sensor, for acquiring raw image data, wherein the raw image data includes at least original
Beginning all-pass channel data, original first color channel data and original second color channel data, any channel data all have
Empty pixel, first color channel and second color channel are the two of them in R, G, B color channel;
Image processing equipment, for obtaining the raw image data of described image acquisition equipment acquisition;Based on described original
Image data generates the all-pass channel image of full resolution and the RGB image of full resolution;By the all-pass channel image and institute
It states RGB image to be merged, obtains fused image.
Fourth aspect, the embodiment of the invention provides a kind of electronic equipment, including processor, communication interface, memory and
Communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus;
Memory, for storing computer program;
Processor when for executing the program stored on memory, is realized described in any of the above-described image processing method
Method and step.
5th aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage
Dielectric memory contains computer program, and the computer program realizes any of the above-described image processing method institute when being executed by processor
The method and step stated.
Scheme provided in an embodiment of the present invention can obtain the original all-pass channel data, original of imaging sensor acquisition
The raw image data of first color channel data and original second color channel data;Based on raw image data, generate complete
The all-pass channel image of resolution ratio and the RGB image of full resolution;All-pass channel image is merged with RGB image, is obtained
Fused image.Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain
All-pass channel image is obtained to melt all-pass channel image with RGB image due to advantage of the all-pass channel image in sensitivity
Closing obtained blending image has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can adopt
Collect the preferable clear image of quality.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow diagram of image processing method provided in an embodiment of the present invention;
Fig. 2 be the present embodiments relate to spectral response schematic diagram;
Fig. 3 be the present embodiments relate to bayer array and RCCG array schematic diagram;
Fig. 4 is to merge all-pass channel image with RGB image in a kind of implementation, obtain the place of blending image
Manage block diagram;
Fig. 5 is at the first of the all-pass channel image provided in an embodiment of the present invention for generating full resolution and RGB image
Manage block diagram;
Fig. 6 be the present embodiments relate to interpolation processing schematic;
Fig. 7 is at the all-pass channel image of generation full resolution provided in an embodiment of the present invention and second of RGB image
Manage block diagram;
Fig. 8 be the present embodiments relate to the schematic diagram for supplying missing color channel;
Fig. 9 is at the third of the all-pass channel image provided in an embodiment of the present invention for generating full resolution and RGB image
Manage block diagram;
Figure 10 is a kind of structural schematic diagram of image processing apparatus provided in an embodiment of the present invention;
Figure 11 is a kind of structural schematic diagram of image processing system provided in an embodiment of the present invention;
Figure 12 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
First below to this specification relates to technical term simply introduced.
Raw image data, also referred to as RAW data refer to that the light signal captured is converted digital letter by imaging sensor
Number obtained initial data.In raw image data, if being separated into channel data, there are parts in single channel data
Empty pixel does not include any channel value at empty pixel, the cavity for example, left figure shown in fig. 6, in R channel data
Any R (Red) channel value is not present in pixel, and any G (Green) channel value is not present in the empty pixel in G channel data,
There is no any B (Blue) channel values etc. for empty pixel in channel B data.
The image of full resolution refers to for each channel in the presence of the image, in each pixel in the image
All with the channel value in the channel.For example, in embodiments of the present invention, each pixel in the all-pass channel image of full resolution
Point all includes all-pass channel value;Likewise, each pixel in the RGB image of full resolution includes R channel value, G
Channel value and channel B value.
In order to solve the problems, such as that mentioned by above-mentioned background technique, the embodiment of the invention provides provide a kind of image processing method
Method, device, electronic equipment and computer readable storage medium.
It describes in detail first below to a kind of image processing method provided in an embodiment of the present invention.
It should be noted that a kind of executing subject of image processing method provided by the embodiment of the present invention can be one kind
Image processing apparatus, wherein the image processing apparatus is run in electronic equipment.In a particular application, which can be with
Are as follows: the image capture device comprising imaging sensor is not limited thereto certainly.
A kind of image processing method provided in an embodiment of the present invention, as shown in Figure 1, the method comprising the steps of S101~
S103:
S101: the raw image data of imaging sensor acquisition is obtained.
Wherein, raw image data includes original all-pass channel data, original first color channel data and original second
Color channel data, any raw image data all have empty pixel, first color channel and second color
Channel is the two of them in R, G, B color channel.
It should be noted that including not only all-pass channel in raw image data, also includes rgb color channel, wrapped
The rgb color channel contained can be three kinds of channel R, the channel G and channel B color channels, or the channel R, the channel G and
Any two kinds in three kinds of color channels of channel B.For example, the raw image data can wrap containing the channel R, the channel G and complete
Circulation passage.
It is appreciated that the image capture device can wrap containing optical lens, optical filter and imaging sensor.Wherein,
The optical lens can be can be by visible light and the common lens of infrared light.In a kind of situation, which may include filter
Mating plate switching device, when ambient light illumination is more than preset threshold, switching device of optical fiber is switched to filtering infrared fileter;Work as ring
When border illumination is no more than preset threshold, such as night, switching device of optical fiber is switched to all-pass optical filter;In another case, should
Optical filter can also be directly filtering infrared fileter.
In one implementation, raw image data can be with are as follows: using containing all-pass channel road and at least two RGB colors
The image data of the imaging sensor acquisition of the color filter lens array in color channel.Above-mentioned imaging sensor can be include all-pass
The imaging sensor of channel (C-channel, also known as W channel) color filter lens array, as RGBW (Red-Green-Blue-White, four
Color framework) imaging sensor, RCCG (Red-Complete Color-Green, red green all-pass channel) imaging sensor, BCCG
(Green-Complete Color-Blue, bluish-green all-pass channel) imaging sensor RCCB (Red-Complete Color-
Blue, red indigo plant all-pass channel) imaging sensor etc..
Wherein, the channel photoreception of spectrum range ratio R in all-pass channel, the channel G, channel B and IR (infrared) channel are respective
Spectral region is wide, as shown in Fig. 2, to respectively correspond all-pass channel, the channel R, G logical by W, Red, Green, Blue and IR in figure
Road, channel B and the channel IR, wider array of photoreception of spectrum range can guarantee that more fully optical signal enters imaging sensor,
And then obtain the higher raw image data of quality.
Color filter lens array containing all-pass channel and tradition Bayer (Bayer) array are different, by taking Fig. 3 as an example, Fig. 3
In schematic diagram (a) be tradition Bayer array schematic diagram, schematic diagram (b) be RCCG array schematic diagram, it is found that compared to
Traditional Bayer array is substituted the channel B of traditional Bayer array in RCCG array using the channel G, and is substituted using C-channel
The channel G of traditional Bayer array.Using the rainbow filter array containing all-pass channel when Image Acquisition, guarantee more abundant
Light enter sensor, obtain higher-quality original image.
S102: it is based on raw image data, generates the all-pass channel image of full resolution and the RGB image of full resolution.
It is appreciated that in the raw image data, there are corresponding original all-pass channel data only on partial pixel,
On afterimage element and original all-pass channel data is not present, i.e. raw image data has cavity on all-pass channel, and of the invention
In embodiment in the all-pass channel image of full resolution, there is all-pass channel value in each pixel.
It is above-mentioned to be based on raw image data as the optional implementation of one of embodiment of the present invention, generate complete differentiate
The step of all-pass channel image of rate, may include:
Interpolation processing is carried out to original all-pass channel data, obtains the all-pass channel image of full resolution.
Specifically, in one implementation, original all-pass channel data can be based on, to original all-pass channel data into
Row interpolation processing, obtains the all-pass channel image of full resolution.
It is understood that being based on original all-pass channel data, the step of interpolation processing is carried out to original all-pass channel data
Suddenly, it is specifically as follows: based on original all-pass channel data, original all-pass channel data is carried out at interpolation using interpolation algorithm
Reason determines the channel value at each empty pixel in the original all-pass track data, and then obtains the complete of full resolution
Circulation passage image.
Wherein, used interpolation algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..It is exemplary
, for the pixel that all-pass channel value is each not present in original image, calculate all pixels in the predetermined neighborhood of the pixel
The average value of all-pass channel value, and using the average value as the all-pass channel value of the pixel, and then obtain the all-pass of full resolution
Channel image.Wherein, the predetermined neighborhood of pixel refers to centered on the pixel, the pixel set in preset range, for example, 3 × 3
Neighborhood.
In another implementation, be also based on original all-pass channel data, original first color channel data and
Original second color channel data carries out interpolation processing to original all-pass channel data, obtains the all-pass channel figure of full resolution
Picture.It should be understood that, not only with reference to original all-pass channel data, being also referred to when carrying out interpolation processing to all-pass channel data
Original first color channel data and original second color channel data, can be improved interpolation, obtain the complete of better quality
The all-pass channel image of resolution ratio.
Specifically, interpolation can be carried out to original all-pass channel data according to original all-pass channel data and the first weight
Processing, obtains the all-pass channel image of full resolution, wherein the first weight are as follows: according to original first color channel data and original
Begin what the second color channel data determined.
Illustratively, for each there is no the pixel of all-pass channel data (i.e. empty pixel), head in original image
First according to original first color channel data and original second color channel data of all pixels in the predetermined neighborhood of the pixel
The first weight is determined, then according to the all-pass channel of all pixels in the predetermined neighborhood of the identified first weight computing pixel
The average value of data, and using the average value as the all-pass channel data of the pixel, and then obtain the all-pass channel of full resolution
Image.
May include in step S101 raw image data obtained in the embodiment of the present invention all-pass channel data and
Two of them color channel data in rgb color channel.However, the full resolution to be generated in the embodiment of the present invention
In RGB image, there are R channel value, G channel value and channel B values in each pixel.
It should be understood that the first situation, when in raw image data including two kinds of color channel datas, in order to obtain
The RGB image of full resolution can first supply a kind of color channel data lacked, then to all colors
Channel data carries out interpolation, to obtain the RGB image of full resolution, or first to two kinds of existing color channel datas
Interpolation is carried out, then a kind of color channel data lacked is supplied, to obtain the RGB image of full resolution.
And second situation does not need then to carry out color when in raw image data including three kinds of color channel datas
Channel data supplies processing, but directly carries out interpolation processing to three color channel datas, and the RGB of full resolution can be obtained
Image.
In view of length problem, does not describe in detail herein to both the above situation temporarily, be discussed in detail and see below.
S103: all-pass channel image is merged with RGB image, obtains fused image.
In the embodiment of the present invention, all-pass channel image is carried out to merge used fusion with the RGB image of full resolution
Mode can be diversified, and the present invention does not limit specific amalgamation mode herein.
It is above-mentioned to melt all-pass channel image with RGB image as a kind of optional implementation of the embodiment of the present invention
The step of closing, obtaining blending image, may include following step c1~c3:
Step c1: the color information in RGB image is obtained.
When fusion treatment, need to extract color information to RGB image, then merged with the luminance information of step C2, with
Generate while having the output image of exact color and preferable low-light (level) effect.
The color information of pixel and the color space of image are related, for example, can execute to the RGB image of the full resolution
Then rgb color space will to the transformation of HSV (Hue, form and aspect, Saturation, saturation degree, Value, lightness) color space
The H channel value of each pixel and color of the channel S value as the pixel in blending image in the image obtained after colour space transformation
Multimedia message.
For another example, the transformation of rgb color space to YUV color space can be executed to the RGB image of the full resolution, then
Using the U channel value of pixel each in the image obtained after colour space transformation and V channel value as the pixel in blending image
Color information.
It is appreciated that image capture device calculate blending image in each pixel color information be not limited to it is above two
Implementation is also possible to the RGB image of full resolution being converted to other color spaces, as long as the color space can be drawn
It is divided into color information and luminance information.
Step c2: the luminance information in the luminance information and all-pass channel image in RGB image is obtained.
When fusion treatment, need that all-pass image and RGB image is combined to generate luminance information, since all-pass image is in sensitivity
On advantage so that the luminance picture picture quality with higher under low-light (level) environment generated.
In a kind of optional implementation, the luminance information of each pixel in blending image can directly use all-pass channel
Channel value possessed by the pixel of same position indicates in image, for example, the channel value of pixel a is x in all-pass channel image, then
The luminance information of pixel a is directly x in blending image.
In order to guarantee the generation effect of luminance information, another kind optionally in implementation, is based on all-pass channel image, meter
In calculation blending image the step of luminance information (c2) of each pixel, may include:
Based on all-pass channel image and RGB image, the luminance information of each pixel in blending image is calculated.
Specifically, the luminance information of the pixel in RGB image can be calculated for each pixel in blending image, make
For the first luminance information, and using the channel value of the pixel in all-pass channel image as the second luminance information, according still further to preset
The first luminance information of weight coefficient and the second luminance information weighted sum obtain the luminance information of the pixel in blending image.
Wherein, the luminance information of pixel can be obtained according to mode in RGB image: execute rgb color space to RGB image
To the transformation of YUV color space, believe the corresponding transformed V channel value of each pixel as the brightness of pixel in RGB image
Breath.
Illustratively, weight coefficient corresponding to the first luminance information and the second luminance information is all 0.5, then blending image
In the luminance information of each pixel can be with are as follows: the first luminance information of the pixel and the second luminance information average value.
Step c3: color information is merged with luminance information, obtains fused image.
Wherein, directly merge blending image in each pixel color information and the obtained blending image of luminance information simultaneously
Non- is the image under rgb color space, so the image that fusion obtains can also be converted to the image of rgb format at this time.
Illustratively, if step c1 color information calculated belongs under HSV color space, fusion is being calculated
In image after the luminance information of each pixel, using the luminance information of pixel each in blending image as the V channel value of the pixel,
Then H channel value, channel S value and the V channel value for combining blending image, obtain the blending image of HSV format, then by the fusion
Image is converted to the image of rgb format.
For another example, if step c1 color information calculated belongs under YUV color space, blending image is being calculated
In each pixel luminance information after, using the luminance information of pixel each in blending image as the Y channel value of the pixel, then
Y channel value, U channel value and the V channel value for combining blending image, obtain the blending image of yuv format, then by the blending image
Be converted to the image of rgb format.
It is shown in Figure 4 to merge all-pass channel image with RGB image in a kind of implementation, it is merged
The processing block diagram of image it is possible, firstly, to extract the color information in RGB image, obtains color image, together in above process
When available RGB image in and all-pass channel image in luminance information, obtain luminance picture, in turn, by color image with
Luminance picture is merged, and fused image is obtained.From the foregoing, it will be observed that scheme provided in an embodiment of the present invention, can obtain figure
The original of the original all-pass channel data, original first color channel data and original second color channel data that are acquired as sensor
Beginning image data;Based on raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated;It will
All-pass channel image is merged with RGB image, obtains fused image.It is original in scheme provided in an embodiment of the present invention
Include all-pass channel in image data, and then all-pass channel image can be obtained, since all-pass channel image is in sensitivity
Advantage, it is excellent that the blending image that all-pass channel image is merged with RGB image has that signal-to-noise ratio is high, clarity is high
Point, therefore, the preferable clear image of the available quality of scheme provided in an embodiment of the present invention.
In addition, the embodiment of the present invention obtains visible images using an imaging sensor, and obtained from visible images
The RGB image of full resolution and the all-pass channel image of full resolution, then by RGB image obtained and all-pass channel image
Fusion, so that the higher target image of clarity is obtained, so not needing in the case where the environment illumination intensities such as night are weaker
It is additional to carry out light filling, the scheme also preferable clear image of available quality provided by the embodiment of the present invention using infrared light.
Referring to Fig. 5, Fig. 7, it is shown in Fig. 9 generate full resolution all-pass channel image and RGB image processing block diagram,
In, processing block diagram when Fig. 5 and Fig. 7 correspond in raw image data comprising two kinds of color channel datas, Fig. 9 corresponds to original
Processing block diagram when including three kinds of color channel datas in image data.
In Fig. 5, Fig. 7, processing block diagram shown in Fig. 9, for generate full resolution all-pass channel image the step of,
It is by carrying out interpolation processing to original all-pass channel data, to obtain the all-pass channel image of full resolution.
The step of RGB channel image for generating full resolution, in processing block diagram shown in Fig. 5, first to original the
One color channel data and original second color channel data carry out interpolation processing, then according to the first color after interpolation processing
Channel image, the second color channel images and all-pass channel image supply the third color channel data lacked, from
And obtain the RGB image of full resolution;In processing block diagram shown in Fig. 7, first to the third lacked in raw image data
Kind color channel data is supplied, then to original first color channel data, original second color channel data and benefit
The third color channel data obtained after foot carries out difference processing respectively, to obtain the RGB image of full resolution.
In processing block diagram shown in Fig. 9, due to having existed three kinds of color channel datas in raw image data, no
It needs to carry out color channel and supplies processing, difference processing directly is carried out to three kinds of color channel datas, full resolution can be obtained
RGB image, optionally, when carrying out differences processing to three kinds of color channel datas and carrying out interpolation processing, see also all-pass
Channel data.
When the color channel to missing is supplied, from known original color channel data and original all-pass channel
Data supply the color channel of missing, it is ensured that accurate color rendition.When carrying out interpolation to color channel, need to refer to
The information of original all-pass channel data, so that interpolation is more accurate, the rgb color image of generation has image detail abundant.
Below by Fig. 5, Fig. 7, processing block diagram shown in Fig. 9, respectively for logical comprising rgb color in raw image data
Two kinds of color channel datas, three kinds of color channel data both of these case in road are generated to above-mentioned according to raw image data
The step of RGB image of full resolution, is described in detail.
For the first case, raw image data are as follows: comprising all-pass channel and original first color channel data, original
Second color channel data, wherein original first color channel data and original second color channel data are rgb color channel
In two kinds of color channels image data, that is to say, that: in raw image data other than including all-pass channel data, only
It also include two kinds of channel datas in R channel data, G channel data and channel B data, if raw image data includes R
Channel data and G channel data, but do not include channel B data, raw image data includes R channel data and channel B number for another example
According to, but do not include G channel data, for another example raw image data includes G channel data and channel B data, but does not include R port number
According to.It is appreciated that in this case it is desirable to lacked color channel is supplied, to guarantee accurate color rendition.
As the first optional implementation of the embodiment of the present invention, processing block diagram as shown in Figure 5, in original image number
It is above-mentioned to be based on raw image data in the case where being lacked in there are color channel, generate the step of the RGB image of full resolution
Suddenly, may include following step a1 and a2:
Step a1: it is carried out at interpolation respectively for original first color channel data and original second color channel data
Reason, obtains the first color channel images and the second color channel images of full resolution.
It in one implementation, can be to original first color channel number by taking original first color channel data as an example
According to interpolation processing is carried out, the channel value at each empty pixel in original first color channel data is determined, and then obtain
To the first color channel images of full resolution.Wherein, used interpolation algorithm can be bilinear interpolation method, bicubic
Interpolation Interpolation method etc..
Illustratively, the raw image data be RCCG imaging sensor imaging obtain, then the first color channel and
Second color channel is respectively the channel R and the channel G, and the third color channel lacked is channel B.Then at this point, step a1 needs
For in raw image data the channel R and the channel G carry out interpolation processing respectively, obtain full resolution R channel image and
The G channel image of full resolution.
By taking Fig. 6 as an example, Fig. 6 indicates that the interpolation processing schematic in the channel R has five before interpolation processing in array on the right side of Fig. 6
Asterisk note pixel do not have R channel value, then interpolation handle when, should with five-pointed star label pixel R channel value be its 3 × 3
The average value of all R channel values in neighborhood.
In another implementation, by taking original first color channel data as an example, it is also based on original all-pass channel
Data and original first color channel data and original second color channel data carry out original first color channel data
Interpolation processing obtains the first color channel data image of full resolution.It should be understood that any rgb color port number
When according to carrying out interpolation processing, not only with reference to the rgb color all-pass channel data, other RGB in raw image data are also referred to
Color channel data and full tunnel channel data, can be improved interpolation, obtain the first color channel number of better quality
According to image and the second color channel data image.
Specifically, it is directed to original first color channel data, it can be based on original first color channel data and the second power
Value carries out interpolation processing to original first color channel data, obtains the first color channel data image, wherein the second weight
Are as follows: it is determined according to original all-pass channel data and original second color channel data.
Illustratively, for each rgb color channel, for the empty pixel in the channel, first according to the sky
In the predetermined neighborhood of hole pixel another rgb color channel data of all pixels point in the presence of raw image data with
And original full tunnel channel data determines the second weight, then according to the pre- of the identified second weight computing cavity pixel
Determine the average value of the channel value of all pixels in neighborhood, and using the average value as the channel value of the cavity pixel.
Step a2: according to all-pass channel image, the first color channel images and the second color channel images, complete differentiate is determined
The third color channel image of rate, and then obtain the RGB image of full resolution.
For the third color channel image for not including in raw image data, each pixel is in third color channel
Channel value on image can use original all-pass channel image and the first color channel images, the second color channel images
In calculate and obtain with the channel value that the pixel of the pixel same position is respectively provided with, the embodiment of the present invention does not limit this
The specific implementation of step a2.
It is above-mentioned according to all-pass channel image, first as a kind of specific implementation of step a2 in the embodiment of the present invention
Color channel images and the second color channel images, determine the third color channel image of full resolution, and then are differentiated entirely
The step of RGB image of rate, may include:
The channel value of any pixel point in third color channel image, predetermined formula are calculated according to predetermined formula are as follows:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in third color channel image, C1、S2And S3Respectively correspond expression
In all-pass channel image, the first color channel images and the second color channel images with any pixel point same position picture
The channel value of vegetarian refreshments;α,β,K1And K2It is preset constant value.
Above-mentioned α, β, K1And K2It is preset constant value, the embodiment of the present invention does not limit α, β, K1And K2Tool
Body value, those skilled in the art can be set as needed, for example, presetting α, β, K1And K2Value be respectively 0,
0,1 and 1.
Illustratively, above-mentioned S1、S2And S3Respectively channel B value, R channel value and G channel value, then need polishing channel B
Channel data, above-mentioned predetermined formula can be at this time are as follows:
B=[C- (R+G- α) * K1+β]*K2;
Wherein, B represents the channel value of the channel B of a certain pixel, and C represents channel of the pixel in all-pass channel image
Value, R represent channel value of the pixel in R channel image, and G represents channel value of the pixel in G channel image.
As second of optional implementation of the embodiment of the present invention, processing block diagram as shown in Figure 7, in original image number
It is above-mentioned to be based on raw image data in the case where being lacked in there are color channel, generate the step of the RGB image of full resolution
Suddenly, may include following step b1 and b2:
Step b1: according to raw image data, third color channel data is determined.
In one implementation, can for the pixel in raw image data each including all-pass channel value,
The first color channel of each pixel is logical in the default neighborhood of all-pass channel value and the pixel based on the pixel
The channel value of road value and the second color channel calculates the channel value in the third color channel of the pixel.
Identical with the concept of aforementioned predetermined neighborhood is that the default neighborhood of pixel refers to centered on the pixel, in advance
If the pixel set in range, for example, 3 × 3 neighborhoods of pixel.By taking Fig. 8 as an example, the pixel institute of array center is located in Fig. 8
The channel value having is C-channel value, it is clear that, above-mentioned first color channel values and the second color channel be respectively R channel value with
And G channel value, third color channel value are channel B value, it is assumed that the default neighborhood is 3 × 3 neighborhoods, then for being located at battle array in Fig. 8
The channel B value of the pixel at column center, can according in the C-channel value of the pixel, Fig. 8 two R channel values and two channels G
Value, which calculates, to be obtained.
It is diversified, this hair that the specific implementation of the third color channel value of pixel is calculated in the embodiment of the present invention
Bright embodiment does not limit herein.It is above-mentioned to be directed to raw image data as a kind of optional implementation of the embodiment of the present invention
In each include all-pass channel value pixel, it is each in the default neighborhood of all-pass channel value and the pixel based on the pixel
The first color channel values and the second color channel values of pixel, can wrap the step of calculating the third color channel value of the pixel
It includes:
By the non-empty pixel in all-pass channel data, the non-empty pixel that is determined as in third color channel data
Point determines the channel value of any non-empty pixel in third color channel data, wherein any non-empty pixel in turn
The channel value of point be according in original all-pass channel data with any non-empty pixel same position pixel it is logical
In road value, original first color channel data with any non-empty pixel same position the corresponding default neck of pixel
In domain in the average channel value of each pixel and the second color channel data with any non-empty pixel in same position
The average channel value of each pixel determines in the corresponding default field of the pixel set.
Specifically, for the pixel in raw image data each including all-pass channel value, according to following formula meter
Calculate the third color channel value of the pixel:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in third color channel data, C2Indicate original all-pass
The channel value of pixel in channel data with any non-empty pixel in same position, S5Indicate original first color channel
The average of each pixel is led in the corresponding default field of pixel of same position with any non-empty pixel in data
Road value, S6It indicates in the second color channel data with any non-empty pixel in the corresponding default neck of the pixel of same position
The average channel value of each pixel in domain;α,β,K1And K2It is preset constant value.
It is understood that the type of third color channel value belongs to any of R channel value, G channel value and channel B value
One kind, but there are two types of the type of the first color channel values and the second color channel values includes, therefore above-mentioned S5And S6Respectively correspond
One color channel values and the second color channel values, such as when the type of third color channel value belongs to R channel value, the first color
The type of channel value and the second color channel values is respectively G channel value and channel B value, then S5It can indicate the pre- of the pixel
If the average value of the G channel value of each pixel, S in neighborhood6It can indicate the B of each pixel in the default neighborhood of the pixel
The average value of channel value.
Likewise, above-mentioned α, β, K1And K2It is preset constant value, the embodiment of the present invention does not limit α, β, K1And
K2Specific value, those skilled in the art can be set as needed, for example, presetting α, β, K1And K2Value point
It Wei 0,0,1 and 1.
For this implementation, for the target pixel points of the array center shown in Fig. 8, it is assumed that the default neighborhood is 3
× 3 neighborhoods, then above-mentioned S4Indicate the channel B value of the target pixel points, C2Indicate the C-channel value of the target pixel points, S5It can be with table
Show the average value of two G channel values shown in fig. 7, S6It can indicate the average value of two R channel values shown in fig. 7.
Step b2: to original first color channel data, original second color channel data and third color channel data
Interpolation processing is carried out respectively, obtains the RGB image of full resolution.
It include the channel R, the channel G and channel B, institute in raw image data after image capture device executes step b1
Interpolation processing can be carried out respectively to the channel R, the channel G and the channel B in raw image data, full resolution is obtained
RGB image.
In one implementation, for original first color channel data, original second color channel data and third
Totally three kinds of rgb color channel datas for each rgb color channel can be based on the rgb color channel to color channel data
Data carry out interpolation processing to the rgb color channel data, to obtain the RGB image of full resolution.
Specifically, each rgb color channel can be directed to, it is based on the rgb color channel data, it is logical to the rgb color
Track data carries out interpolation processing using interpolation algorithm, obtains the corresponding full resolution image of the color channel.Wherein, used
Interpolation algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..
In another implementation, each rgb color channel, the all-pass based on raw image data can also be directed to
Channel data and other two rgb color channel data carry out interpolation processing to the rgb color channel, obtain comprising three kinds of RGB
The RGB image of the full resolution of color channel.It should be understood that when carrying out interpolation processing to any rgb color channel data,
Not only with reference to the rgb color channel data, other rgb color channel datas and full tunnel channel data, Ke Yiti are also referred to
High interpolation obtains the RGB image of the full resolution comprising three kinds of rgb color channels of better quality.
Specifically, each rgb color channel can be directed to, based on the rgb color channel data in raw image data
With the 4th weight, interpolation processing is carried out to the color channel data, obtains the full resolution comprising three kinds of rgb color channels
RGB image, wherein the 4th weight are as follows: according to original all-pass channel data, original first color channel data and original second color
What color channel data determined.
Illustratively, for each rgb color channel, for the empty pixel in the channel, first according to the sky
The other two rgb color channel data of all pixels point and full tunnel channel data determine in the predetermined neighborhood of hole pixel
Then four weights are averaged according to the channel value of all pixels point in the predetermined neighborhood of the identified third weight computing pixel
Value, and using the average value as the channel value of the pixel, and then the RGB image of full resolution.
It should be noted that all-pass port number is all utilized in the optional implementation of aforementioned two kinds of embodiment of the present invention
According to supplying the color channel data lacked in raw image data, and then the interpolation processing of all color channels is completed, guaranteed
The accuracy of color channel interpolation, so that the RGB image of the full resolution generated has image detail abundant.
For second situation, raw image data can be with are as follows: includes three kinds of colors in all-pass channel and rgb color channel
The image data in color channel, if the raw image data is the image data that the imaging of RGBW imaging sensor obtains, then, such as Fig. 9
Shown in handle block diagram, above-mentioned to be based on raw image data, the step of generating the RGB image of full resolution, can be with are as follows:
Interpolation processing is carried out respectively for three kinds of color channels in the presence of raw image data, is obtained comprising three kinds
The RGB image of the full resolution in rgb color channel.
Specifically, in one implementation, it can be logical for each rgb color in the presence of raw image data
Road is based on the rgb color channel data, carries out interpolation processing to the rgb color channel data, obtains comprising three kinds of rgb colors
The RGB image of the full resolution in channel.
It is understood that being based on the RGB color for each rgb color channel in the presence of raw image data
Color channel data, is specifically as follows: for raw image data the step of carrying out interpolation processing to the rgb color channel data
In the presence of each rgb color channel, be based on the rgb color channel data, to the rgb color channel data use interpolation
Algorithm carries out interpolation processing, obtains the RGB image of the full resolution comprising three kinds of rgb color channels.Wherein, used interpolation
Algorithm can be bilinear interpolation method, bicubic interpolation interpolation method etc..
In another implementation, it can also be directed to each rgb color channel in the presence of raw image data,
Rgb color channel data in the presence of all-pass channel data based on raw image data and raw image data, to this
Rgb color channel carries out interpolation processing, obtains the RGB image of the full resolution comprising three kinds of rgb color channels.It is understood that
, when carrying out interpolation processing to any rgb color channel data, not only with reference to the rgb color all-pass channel data, also refer to
Other rgb color channel datas and full tunnel channel data in raw image data, can be improved interpolation, obtain matter
Amount preferably includes the RGB image of the full resolution in three kinds of rgb color channels.
Specifically, original image number can be based on for each rgb color channel in the presence of raw image data
The rgb color channel data and third weight in carry out interpolation processing to the color channel data of raw image data, obtain
To the RGB image of the full resolution comprising three kinds of rgb color channels, wherein third weight are as follows: according to complete in raw image data
The other two rgb color channel data in the presence of track data and raw image data determines all.
Illustratively, first for the empty pixel in the channel in original image for each rgb color channel
First according to the other two rgb color channel data of all pixels point and full tunnel channel in the predetermined neighborhood of the cavity pixel
Data determine third weight, then according to all channels in the predetermined neighborhood of the identified third weight computing cavity pixel
The average value of value, and using the average value as the channel value of the cavity pixel, and then obtain including three kinds of rgb color channels
The RGB image of full resolution.
Corresponding to embodiment of the method shown in Fig. 1, the embodiment of the invention also provides a kind of image processing apparatus, such as Figure 10 institute
Show, described device includes:
Module 110 is obtained, for obtaining the raw image data of image capture device acquisition;Wherein, the original image
Data include at least original all-pass channel data, original first color channel data and original second color channel data, any
Channel data all has empty pixel, and first color channel and second color channel are in R, G, B color channel
Two of them;
Generation module 120, for being based on the raw image data, generating the all-pass channel image of full resolution and dividing entirely
The RGB image of resolution;
Fusion Module 130 obtains fused for merging the all-pass channel image with the RGB image
Image.
In a kind of implementation, the generation module 120, comprising:
All-pass channel image generates submodule 121, for carrying out interpolation processing to the original all-pass channel data, obtains
The all-pass channel image of the full resolution.
In a kind of implementation, the all-pass channel image generates submodule 121, is specifically used for:
Interpolation interpolation processing is carried out to the original all-pass channel data, is determined each in the original all-pass track data
Channel value at a cavity pixel, and then obtain the all-pass channel image of the full resolution.
In a kind of implementation, the all-pass channel image generates submodule 121, is specifically used for:
According to original first color channel data and original second color channel data, the first weight is determined;
According to the original all-pass channel data and first weight, interpolation is carried out to the original all-pass channel data
Processing determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution
The all-pass channel image of rate.
In a kind of implementation, the generation module 120, comprising: the first RGB image generates submodule 122, is used for:
Interpolation processing is carried out to original first color channel data, obtains the first color channel figure of full resolution
Picture;
Interpolation processing is carried out to original second color channel data, obtains the second color channel figure of full resolution
Picture;
According to the all-pass channel image, first color channel images and second color channel images, determine
The third color channel image of full resolution, the third color channel are that first color is removed in R, G, B color channel
A color channel except color channel and second color channel;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
In a kind of implementation, first RGB image generates submodule 122 to original first color channel data
Interpolation processing is carried out, the first color channel images of full resolution are obtained, specifically:
Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel
The channel value at each empty pixel in data, and then obtain the first color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original second color channel data, the second weight is determined;According to
Second weight and original first color channel data carry out at interpolation original first color channel data
Reason is determined the channel value at each empty pixel in original first color channel data, and then is obtained described complete
First color channel images of resolution ratio.
In a kind of implementation, first RGB image generates submodule 122 to original second color channel data
Interpolation processing is carried out, the second color channel images of full resolution are obtained, specifically:
Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel
The channel value at each empty pixel in data, and then obtain the second color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original first color channel data, third weight is determined, according to
The third weight and original second color channel data carry out at interpolation original second color channel data
Reason is determined the channel value at each empty pixel in original second color channel data, and then is obtained described complete
Second color channel images of resolution ratio.
In a kind of implementation, first RGB image generates submodule 122 according to the all-pass channel image, described
First color channel images and second color channel images, determine the third color channel image of full resolution, specifically:
It determines the channel value of any pixel point in the third color channel image of the full resolution, and then is differentiated entirely
The third color channel image of rate;Wherein, the channel value of any pixel point is according to the all-pass channel image, described
In first color channel images and second color channel images with any pixel point same position pixel
What channel value determined.
In a kind of implementation, the channel value of any pixel point in third color channel image is determined by following formula:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in the third color channel image, C1、S2And S3It respectively corresponds
Indicate in the all-pass channel image, first color channel images and second color channel images with any picture
Channel value of the vegetarian refreshments in the pixel of same position, α, β, K1And K2It is preset constant value.
In a kind of implementation, the generation module 120, comprising: the second RGB image generates submodule 123, is used for:
According to the raw image data, third color channel data is determined, the third color channel is described R, G, B
A color channel in color channel in addition to first color channel and second color channel;
To original first color channel data, original second color channel data and the third color channel
Data carry out interpolation processing respectively, obtain the RGB image of the full resolution.
In a kind of implementation, described second RGB image generates submodule 123 according to the raw image data, really
Determine third color channel data, specifically include:
By the non-empty pixel in the all-pass channel data, the non-empty being determined as in the third color channel data
Hole pixel;
It determines the channel value of any non-empty pixel in the third color channel data, and then obtains the third
Color channel data, wherein the channel value of any non-empty pixel in the third color channel data, is basis
Channel value, the original of pixel in the original all-pass channel data with any non-empty pixel in same position
Begin in the first color channel data with any non-empty pixel in the corresponding default field of pixel of same position
With any non-empty pixel same in the average channel value of each pixel and second color channel data
The average channel value of each pixel determines in the corresponding default field of the pixel of position.
In a kind of implementation, any non-empty pixel in third color channel data is determined by following formula
Channel value:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in the third color channel data, C2Described in expression
The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, S5Described in expression
With any non-empty pixel in the corresponding default field of pixel of same position in original first color channel data
In each pixel average channel value, S6It indicates to exist in second color channel data with any non-empty pixel
The average channel value of each pixel in the corresponding default field of the pixel of same position;α,β,K1And K2It is preset
Constant value.
In a kind of implementation, second RGB image generates submodule 123 to the original first color channel number
Interpolation processing is carried out respectively according to, original second color channel data and the third color channel data, is obtained described complete
The RGB image of resolution ratio, specifically:
Interpolation processing is carried out to original first color channel data, determines original first color channel data
In each empty pixel at channel value, and then obtain the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, determines original second color channel data
In each empty pixel at channel value, and then obtain the second color channel images of full resolution;
Interpolation processing is carried out to the third color channel data, is determined each in the third color channel data
Channel value at empty pixel, and then obtain the third color channel image of full resolution;
According to first color channel images, second color channel images and the third color channel image,
Obtain the RGB image of the full resolution.
In a kind of implementation, second RGB image generates submodule 123 to original first color channel data
Interpolation processing is carried out to specifically include:
Interpolation interpolation processing is carried out to original first color channel data;Alternatively, according to the original all-pass channel
Data, original second color channel data and the third color channel data, determine the 4th weight;According to the described 4th
Weight and original first color channel data carry out interpolation processing to original first color channel data;
Second RGB image generates submodule 123 and carries out interpolation processing tool to original second color channel data
Body includes:
Interpolation interpolation processing is carried out to default second color channel data;Alternatively, according to the original all-pass channel
Data, original first color channel data and the third color channel data, determine the 5th weight;According to the described 5th
Weight and original second color channel data carry out interpolation processing to original second color channel data;
Second RGB image generates submodule 123 and specifically wraps to third color channel data progress interpolation processing
It includes:
Interpolation interpolation processing is carried out to the third color channel data;Alternatively, according to the original all-pass channel data,
Original first color channel data and original second color channel data, determine the 6th weight;According to the described 6th
Weight and the third color channel data carry out interpolation processing to the third color channel data.
In a kind of implementation, the Fusion Module 130 is specifically used for:
Obtain the color information in the RGB image;
Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image;
The color information, first luminance information and second luminance information are merged, described melt is obtained
Image after conjunction.
Scheme provided in an embodiment of the present invention, can obtain imaging sensor acquisition includes all-pass channel and rgb color
The raw image data of at least two color channels in channel;Based on raw image data, the all-pass for generating full resolution is logical
The RGB image of road image and full resolution;All-pass channel image is merged with RGB image, obtain with exact color and
The good blending image of low-light (level) effect, as collected target image.In scheme provided in an embodiment of the present invention, original image
Include all-pass channel in data, and then all-pass channel image can be obtained, since all-pass channel image is excellent in sensitivity
The blending image that all-pass channel image is merged with RGB image is had the advantages that signal-to-noise ratio is high, clarity is high by gesture,
Therefore, scheme provided in an embodiment of the present invention can collect the preferable clear image of quality.
Corresponding to embodiment of the method shown in Fig. 1, the embodiment of the invention also provides a kind of image processing systems, such as Figure 11 institute
Show, the system comprises:
Image capture device 210, including at least one imaging sensor 211, described image sensor include all-pass channel
Color filter lens array, described image sensor, for acquiring raw image data, wherein the raw image data is at least
Including original all-pass channel data, original first color channel data and original second color channel data, any channel data
All have empty pixel, first color channel and second color channel are wherein two in R, G, B color channel
Kind;
Image processing equipment 220, for obtaining the raw image data of described image acquisition equipment acquisition;Based on the original
Beginning image data generates the all-pass channel image of full resolution and the RGB image of full resolution;By the all-pass channel image with
The RGB image is merged, and fused image is obtained.
Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain complete
All-pass channel image merge with RGB image by circulation passage image due to advantage of the all-pass channel image in sensitivity
The blending image arrived has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can collect
The preferable clear image of quality.
The embodiment of the invention also provides a kind of electronic equipment, as shown in figure 12, including processor 301, communication interface
302, memory 303 and communication bus 304, wherein processor 301, communication interface 302, memory 303 pass through communication bus
304 complete mutual communication,
Memory 303, for storing computer program;
Processor 301 when for executing the program stored on memory 303, realizes following steps:
Obtain the raw image data of imaging sensor acquisition;Wherein, the raw image data are as follows: include all-pass channel
With the image data of at least two color channels in rgb color channel;
Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated;
The all-pass channel image is merged with the RGB image, blending image is obtained, as collected target
Image.
The method that specific implementation and relevant explanation content about each step of this method may refer to above-mentioned Fig. 1 is implemented
Example, this will not be repeated here.
In addition, processor 301 executes other realities of the program stored on memory 303 and the image processing method realized
Existing mode, it is identical as implementation mentioned by preceding method embodiment part, it also repeats no more here.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component
Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just
It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy
The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit,
CPU), network processing unit (Network Processor, NP) etc.;It can also be digital signal processor (Digital Signal
Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing
It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete
Door or transistor logic, discrete hardware components.
Scheme provided in an embodiment of the present invention can obtain the original all-pass channel data, original of imaging sensor acquisition
The raw image data of first color channel data and original second color channel data;Based on raw image data, generate complete
The all-pass channel image of resolution ratio and the RGB image of full resolution;All-pass channel image is merged with RGB image, is obtained
Fused image.Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain
All-pass channel image is obtained to melt all-pass channel image with RGB image due to advantage of the all-pass channel image in sensitivity
Closing obtained blending image has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can adopt
Collect the preferable clear image of quality.
In another embodiment provided by the invention, a kind of computer readable storage medium is additionally provided, which can
It reads to be stored with instruction in storage medium, when run on a computer, so that computer executes any institute in above-described embodiment
The image processing method stated.
Scheme provided in an embodiment of the present invention can obtain the original all-pass channel data, original of imaging sensor acquisition
The raw image data of first color channel data and original second color channel data;Based on raw image data, generate complete
The all-pass channel image of resolution ratio and the RGB image of full resolution;All-pass channel image is merged with RGB image, is obtained
Fused image.Include all-pass channel in scheme provided in an embodiment of the present invention, in raw image data, and then can obtain
All-pass channel image is obtained to melt all-pass channel image with RGB image due to advantage of the all-pass channel image in sensitivity
Closing obtained blending image has the advantages that signal-to-noise ratio is high, clarity is high, and therefore, scheme provided in an embodiment of the present invention can adopt
Collect the preferable clear image of quality.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For electronic equipment and computer readable storage medium embodiment, since it is substantially similar to the method embodiment, so description
It is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (33)
1. a kind of image processing method, which is characterized in that the described method includes:
Obtain the raw image data of imaging sensor acquisition;Wherein, the raw image data is logical including at least original all-pass
Track data, original first color channel data and original second color channel data, any channel data all have empty pixel
Point, first color channel and second color channel are the two of them in R, G, B color channel;
Based on the raw image data, the all-pass channel image of full resolution and the RGB image of full resolution are generated;
The all-pass channel image is merged with the RGB image, obtains fused image.
2. generating complete differentiate the method according to claim 1, wherein described be based on the raw image data
The step of all-pass channel image of rate, comprising:
Interpolation processing is carried out to the original all-pass channel data, obtains the all-pass channel image of the full resolution.
3. according to the method described in claim 2, it is characterized in that, described carry out at interpolation the original all-pass channel data
The step of managing, obtaining the all-pass channel image of the full resolution, comprising:
Interpolation interpolation processing is carried out to the original all-pass channel data, determines each sky in the original all-pass track data
Channel value at the pixel of hole, and then obtain the all-pass channel image of the full resolution.
4. according to the method described in claim 2, it is characterized in that, described carry out at interpolation the original all-pass channel data
Reason, the all-pass channel image for obtaining the full resolution include:
According to original first color channel data and original second color channel data, the first weight is determined;
According to the original all-pass channel data and first weight, the original all-pass channel data is carried out at interpolation
Reason determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution
All-pass channel image.
5. method according to claim 1 to 4, which is characterized in that it is described to be based on the raw image data,
The step of generating the RGB image of full resolution, comprising:
Interpolation processing is carried out to original first color channel data, obtains the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, obtains the second color channel images of full resolution;
According to the all-pass channel image, first color channel images and second color channel images, complete point is determined
The third color channel image of resolution, the third color channel are in R, G, B color channel except first color is logical
A color channel except road and second color channel;
According to first color channel images, second color channel images and the third color channel image, obtain
The RGB image of the full resolution.
6. according to the method described in claim 5, it is characterized in that, it is described original first color channel data is carried out it is slotting
The step of value is handled, and obtains the first color channel images of full resolution, comprising:
Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel data
In each empty pixel at channel value, and then obtain the first color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original second color channel data, the second weight is determined;According to described
Second weight and original first color channel data carry out interpolation processing to original first color channel data, really
The channel value at each empty pixel in original first color channel data is made, and then obtains the full resolution
The first color channel images.
7. according to the method described in claim 5, it is characterized in that, it is described original second color channel data is carried out it is slotting
The step of value is handled, and obtains the second color channel images of full resolution, comprising:
Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel data
In each empty pixel at channel value, and then obtain the second color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original first color channel data, third weight is determined, according to described
Third weight and original second color channel data carry out interpolation processing to original second color channel data, really
The channel value at each empty pixel in original second color channel data is made, and then obtains the full resolution
The second color channel images.
8. according to the method described in claim 5, it is characterized in that, described according to the all-pass channel image, first color
Color channel image and second color channel images, the step of determining the third color channel image of full resolution, comprising:
It determines the channel value of any pixel point in the third color channel image of the full resolution, and then obtains full resolution
Third color channel image;Wherein, the channel value of any pixel point is according to the all-pass channel image, described first
The channel of pixel in color channel images and second color channel images with any pixel point in same position
Value determination.
9. according to the method described in claim 8, it is characterized in that, in the third color channel image any pixel point it is logical
Road value is determined by following formula:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in the third color channel image, C1、S2And S3Respectively corresponding indicates institute
It states in all-pass channel image, first color channel images and second color channel images and exists with any pixel point
The channel value of the pixel of same position, α, β, K1And K2It is preset constant value.
10. method according to claim 1 to 4, which is characterized in that described to be based on the original image number
According to the step of generating the RGB image of full resolution, comprising:
According to the raw image data, third color channel data is determined, the third color channel is R, G, B color
A color channel in channel in addition to first color channel and second color channel;
To original first color channel data, original second color channel data and the third color channel data
Interpolation processing is carried out respectively, obtains the RGB image of the full resolution.
11. according to the method described in claim 10, determining third it is characterized in that, described according to the raw image data
The step of color channel data, comprising:
By the non-empty pixel in the all-pass channel data, the non-empty picture that is determined as in the third color channel data
Vegetarian refreshments;
It determines the channel value of any non-empty pixel in the third color channel data, and then obtains third color channel
Data;Wherein, the channel value of any non-empty pixel in the third color channel data, is according to the original all-pass
The channel value of pixel in channel data with any non-empty pixel in same position, original first color are logical
In track data with any non-empty pixel in the corresponding default field of pixel of same position each pixel
In average channel value and second color channel data with any non-empty pixel same position pixel
The average channel value of each pixel determines in corresponding default field.
12. according to the method for claim 11, which is characterized in that any non-cavity in the third color channel data
The channel value of pixel is determined by following formula:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in the third color channel data, C2Indicate described original
The channel value of pixel in all-pass channel data with any non-empty pixel in same position, S5Indicate described original
It is each in the corresponding default field of pixel of same position with any non-empty pixel in first color channel data
The average channel value of a pixel, S6It indicates in second color channel data with any non-empty pixel same
The average channel value of each pixel in the corresponding default field of the pixel of position;α,β,K1And K2It is preset constant
Value.
13. according to the method described in claim 10, it is characterized in that, described to original first color channel data, institute
It states original second color channel data and the third color channel data carries out interpolation processing respectively, obtain the full resolution
RGB image, comprising:
Interpolation processing is carried out to original first color channel data, is determined in original first color channel data
Channel value at each cavity pixel, and then obtain the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, is determined in original second color channel data
Channel value at each cavity pixel, and then obtain the second color channel images of full resolution;
Interpolation processing is carried out to the third color channel data, determines each cavity in the third color channel data
Channel value at pixel, and then obtain the third color channel image of full resolution;
According to first color channel images, second color channel images and the third color channel image, obtain
The RGB image of the full resolution.
14. according to the method for claim 13, which is characterized in that described to be carried out to original first color channel data
Interpolation processing, comprising:
Interpolation interpolation processing is carried out to original first color channel data;
Alternatively, according to the original all-pass channel data, original second color channel data and the third color channel
Data determine the 4th weight;According to the 4th weight and original first color channel data, to original first color
Color channel data carries out interpolation processing;
It is described that interpolation processing is carried out to original second color channel data, comprising:
Interpolation interpolation processing is carried out to default second color channel data;
Alternatively, according to the original all-pass channel data, original first color channel data and the third color channel
Data determine the 5th weight;According to the 5th weight and original second color channel data, to original second color
Color channel data carries out interpolation processing;
It is described that interpolation processing is carried out to the third color channel data, comprising:
Interpolation interpolation processing is carried out to the third color channel data;
Alternatively, according to the original all-pass channel data, original first color channel data and original second color
Channel data determines the 6th weight;It is logical to the third color according to the 6th weight and the third color channel data
Track data carries out interpolation processing.
15. according to claim 1 to method described in any one of 14, which is characterized in that described by the all-pass channel image
The step of being merged with the RGB image, obtain fused image, comprising:
Obtain the color information in the RGB image;
Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image;
The color information, first luminance information and second luminance information are merged, after obtaining the fusion
Image.
16. a kind of image processing apparatus, which is characterized in that described device includes:
Module is obtained, for obtaining the raw image data of image capture device acquisition;Wherein, the raw image data is at least
Including original all-pass channel data, original first color channel data and original second color channel data, any channel data
All have empty pixel, first color channel and second color channel are wherein two in R, G, B color channel
Kind;
Generation module generates the all-pass channel image and full resolution of full resolution for being based on the raw image data
RGB image;
Fusion Module obtains fused image for merging the all-pass channel image with the RGB image.
17. device according to claim 16, which is characterized in that the generation module, comprising:
All-pass channel image generates submodule, for carrying out interpolation processing to the original all-pass channel data, obtains described complete
The all-pass channel image of resolution ratio.
18. device according to claim 17, which is characterized in that the all-pass channel image generates submodule, specific to use
In:
Interpolation interpolation processing is carried out to the original all-pass channel data, determines each sky in the original all-pass track data
Channel value at the pixel of hole, and then obtain the all-pass channel image of the full resolution.
19. device according to claim 17, which is characterized in that the all-pass channel image generates submodule, specific to use
In:
According to original first color channel data and original second color channel data, the first weight is determined;
According to the original all-pass channel data and first weight, the original all-pass channel data is carried out at interpolation
Reason determines the channel value at each empty pixel in the original all-pass track data, and then obtains the full resolution
All-pass channel image.
20. device described in any one of 6 to 19 according to claim 1, which is characterized in that the generation module, comprising: first
RGB image generates submodule, is used for:
Interpolation processing is carried out to original first color channel data, obtains the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, obtains the second color channel images of full resolution;
According to the all-pass channel image, first color channel images and second color channel images, complete point is determined
The third color channel image of resolution, the third color channel are in R, G, B color channel except first color is logical
A color channel except road and second color channel;
According to first color channel images, second color channel images and the third color channel image, obtain
The RGB image of the full resolution.
21. device according to claim 20, which is characterized in that first RGB image generates submodule to the original
Begin the first color channel data progress interpolation processing, obtains the first color channel images of full resolution, specifically:
Interpolation interpolation processing is carried out to original first color channel data, determines original first color channel data
In each empty pixel at channel value, and then obtain the first color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original second color channel data, the second weight is determined;According to described
Second weight and original first color channel data carry out interpolation processing to original first color channel data, really
The channel value at each empty pixel in original first color channel data is made, and then obtains the full resolution
The first color channel images.
22. device according to claim 20, which is characterized in that first RGB image generates submodule to the original
Begin the second color channel data progress interpolation processing, obtains the second color channel images of full resolution, specifically:
Interpolation interpolation processing is carried out to original second color channel data, determines original second color channel data
In each empty pixel at channel value, and then obtain the second color channel images of the full resolution;Alternatively,
According to the original all-pass channel data and original first color channel data, third weight is determined, according to described
Third weight and original second color channel data carry out interpolation processing to original second color channel data, really
The channel value at each empty pixel in original second color channel data is made, and then obtains the full resolution
The second color channel images.
23. device according to claim 20, which is characterized in that first RGB image generates submodule according to
All-pass channel image, first color channel images and second color channel images, determine the third color of full resolution
Color channel image, specifically:
It determines the channel value of any pixel point in the third color channel image of the full resolution, and then obtains full resolution
Third color channel image;Wherein, the channel value of any pixel point is according to the all-pass channel image, described first
The channel of pixel in color channel images and second color channel images with any pixel point in same position
Value determination.
24. device according to claim 23, which is characterized in that
The channel value of any pixel point in third color channel image is determined by following formula:
S1=[C1-(S2+S3-α)*K1+β]*K2;
In formula, S1Indicate the channel value of any pixel point in the third color channel image, C1、S2And S3Respectively corresponding indicates institute
It states in all-pass channel image, first color channel images and second color channel images and exists with any pixel point
The channel value of the pixel of same position, α, β, K1And K2It is preset constant value.
25. device described in any one of 6 to 19 according to claim 1, which is characterized in that the generation module, comprising: second
RGB image generates submodule, is used for:
According to the raw image data, third color channel data is determined, the third color channel is R, G, B color
A color channel in channel in addition to first color channel and second color channel;
To original first color channel data, original second color channel data and the third color channel data
Interpolation processing is carried out respectively, obtains the RGB image of the full resolution.
26. device according to claim 25, which is characterized in that described second RGB image generates submodule according to institute
Raw image data is stated, third color channel data is determined, specifically includes:
By the non-empty pixel in the all-pass channel data, the non-empty picture that is determined as in the third color channel data
Vegetarian refreshments;
It determines the channel value of any non-empty pixel in the third color channel data, and then obtains the third color
Channel data, wherein the channel value of any non-empty pixel in the third color channel data, is according to
The channel value of pixel in original all-pass channel data with any non-empty pixel in same position, described original the
It is each in the corresponding default field of pixel of same position with any non-empty pixel in one color channel data
With any non-empty pixel in same position in the average channel value of pixel and second color channel data
The corresponding default field of pixel in each pixel average channel value determine.
27. device according to claim 26, which is characterized in that
The channel value of any non-empty pixel in third color channel data is determined by following formula:
S4=[C2-(S5+S6-α)*K1+β]*K2;
In formula, S4Indicate the channel value of any non-empty pixel in the third color channel data, C2Indicate described original
The channel value of pixel in all-pass channel data with any non-empty pixel in same position, S5Indicate described original
It is each in the corresponding default field of pixel of same position with any non-empty pixel in first color channel data
The average channel value of a pixel, S6It indicates in second color channel data with any non-empty pixel same
The average channel value of each pixel in the corresponding default field of the pixel of position;α,β,K1And K2It is preset constant
Value.
28. device according to claim 25, which is characterized in that second RGB image generates submodule to the original
Begin the first color channel data, original second color channel data and the third color channel data carries out interpolation respectively
Processing, obtains the RGB image of the full resolution, specifically:
Interpolation processing is carried out to original first color channel data, is determined in original first color channel data
Channel value at each cavity pixel, and then obtain the first color channel images of full resolution;
Interpolation processing is carried out to original second color channel data, is determined in original second color channel data
Channel value at each cavity pixel, and then obtain the second color channel images of full resolution;
Interpolation processing is carried out to the third color channel data, determines each cavity in the third color channel data
Channel value at pixel, and then obtain the third color channel image of full resolution;
According to first color channel images, second color channel images and the third color channel image, obtain
The RGB image of the full resolution.
29. device according to claim 28, which is characterized in that second RGB image generates submodule to the original
The first color channel data progress interpolation processing that begins specifically includes:
Interpolation interpolation processing is carried out to original first color channel data;Alternatively, according to the original all-pass channel data,
Original second color channel data and the third color channel data, determine the 4th weight;According to the 4th weight
With original first color channel data, interpolation processing is carried out to original first color channel data;
Second RGB image generates submodule and specifically includes to the original second color channel data progress interpolation processing:
Interpolation interpolation processing is carried out to default second color channel data;Alternatively, according to the original all-pass channel data,
Original first color channel data and the third color channel data, determine the 5th weight;According to the 5th weight
With original second color channel data, interpolation processing is carried out to original second color channel data;
Second RGB image generates submodule and specifically includes to third color channel data progress interpolation processing:
Interpolation interpolation processing is carried out to the third color channel data;Alternatively, according to the original all-pass channel data, described
Original first color channel data and original second color channel data, determine the 6th weight;According to the 6th weight
With the third color channel data, interpolation processing is carried out to the third color channel data.
30. device described in any one of 6 to 29 according to claim 1, which is characterized in that the Fusion Module is specifically used for:
Obtain the color information in the RGB image;
Obtain the second luminance information in the first luminance information and the all-pass channel image in the RGB image;
The color information, first luminance information and second luminance information are merged, after obtaining the fusion
Image.
31. a kind of image processing system characterized by comprising
Image capture device, including at least one imaging sensor, described image sensor include the color filter in all-pass channel
Array, described image sensor, for acquiring raw image data, wherein the raw image data includes at least original complete
Track data, original first color channel data and original second color channel data, any channel data all have cavity all
Pixel, first color channel and second color channel are the two of them in R, G, B color channel;
Image processing equipment, for obtaining the raw image data of described image acquisition equipment acquisition;Based on the original image
Data generate the all-pass channel image of full resolution and the RGB image of full resolution;By the all-pass channel image with it is described
RGB image is merged, and fused image is obtained.
32. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing
Device, communication interface, memory complete mutual communication by communication bus;
Memory, for storing computer program;
Processor when for executing the program stored on memory, realizes any method step in claim 1 to 15
Suddenly.
33. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium
Program realizes any method and step in claim 1 to 15 when the computer program is executed by processor.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210094591.8A CN114422766B (en) | 2018-08-03 | 2018-08-03 | Image acquisition equipment |
CN201810875810.XA CN110493583B (en) | 2018-08-03 | 2018-08-03 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810875810.XA CN110493583B (en) | 2018-08-03 | 2018-08-03 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210094591.8A Division CN114422766B (en) | 2018-08-03 | 2018-08-03 | Image acquisition equipment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110493583A true CN110493583A (en) | 2019-11-22 |
CN110493583B CN110493583B (en) | 2021-12-17 |
Family
ID=68545492
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210094591.8A Active CN114422766B (en) | 2018-08-03 | 2018-08-03 | Image acquisition equipment |
CN201810875810.XA Active CN110493583B (en) | 2018-08-03 | 2018-08-03 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210094591.8A Active CN114422766B (en) | 2018-08-03 | 2018-08-03 | Image acquisition equipment |
Country Status (1)
Country | Link |
---|---|
CN (2) | CN114422766B (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111062378A (en) * | 2019-12-23 | 2020-04-24 | 重庆紫光华山智安科技有限公司 | Image processing method, model training method, target detection method and related device |
CN111861964A (en) * | 2020-08-06 | 2020-10-30 | 深圳市汇顶科技股份有限公司 | Image processing method, apparatus and storage medium |
CN112016478A (en) * | 2020-08-31 | 2020-12-01 | 中国电子科技集团公司第三研究所 | Complex scene identification method and system based on multispectral image fusion |
CN112261391A (en) * | 2020-10-26 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
CN112351172A (en) * | 2020-10-26 | 2021-02-09 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
CN112738493A (en) * | 2020-12-28 | 2021-04-30 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
WO2022027469A1 (en) * | 2020-08-06 | 2022-02-10 | 深圳市汇顶科技股份有限公司 | Image processing method and apparatus, and storage medium |
CN111861964B (en) * | 2020-08-06 | 2024-07-05 | 深圳市汇顶科技股份有限公司 | Image processing method, device and storage medium |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020101524A1 (en) * | 1998-03-04 | 2002-08-01 | Intel Corporation | Integrated color interpolation and color space conversion algorithm from 8-bit bayer pattern RGB color space to 12-bit YCrCb color space |
CN101233763A (en) * | 2005-07-28 | 2008-07-30 | 伊斯曼柯达公司 | Processing color and panchromatic pixels |
CN101652798A (en) * | 2007-03-30 | 2010-02-17 | 伊斯曼柯达公司 | Use the edge map of panchromatic pixels |
CN101375610B (en) * | 2006-01-27 | 2011-09-21 | 全视技术有限公司 | Interpolation of panchromatic and color pixels |
CN102369721A (en) * | 2009-03-10 | 2012-03-07 | 美商豪威科技股份有限公司 | Cfa image with synthetic panchromatic image |
CN102461175A (en) * | 2009-06-09 | 2012-05-16 | 全视科技有限公司 | Interpolation for four-channel color filter array |
CN105070270A (en) * | 2015-09-14 | 2015-11-18 | 深圳市华星光电技术有限公司 | RGBW panel subpixel compensation method and device |
CN108024106A (en) * | 2016-11-04 | 2018-05-11 | 上海富瀚微电子股份有限公司 | Support the color correction device and method of RGBIR and RGBW forms |
Family Cites Families (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8264576B2 (en) * | 2007-03-05 | 2012-09-11 | DigitalOptics Corporation Europe Limited | RGBW sensor array |
JP3942569B2 (en) * | 2003-09-04 | 2007-07-11 | オリンパス株式会社 | Imaging apparatus and image data conversion method |
US7876956B2 (en) * | 2006-11-10 | 2011-01-25 | Eastman Kodak Company | Noise reduction of panchromatic and color image |
CN101527033A (en) * | 2008-03-04 | 2009-09-09 | 河海大学 | Industrial CCD color imaging system based on super-resolution reconstruction and automatic registration |
JP5033700B2 (en) * | 2008-04-02 | 2012-09-26 | 富士フイルム株式会社 | Imaging device |
KR20090120159A (en) * | 2008-05-19 | 2009-11-24 | 삼성전자주식회사 | Apparatus and method for combining images |
US8045024B2 (en) * | 2009-04-15 | 2011-10-25 | Omnivision Technologies, Inc. | Producing full-color image with reduced motion blur |
US8203633B2 (en) * | 2009-05-27 | 2012-06-19 | Omnivision Technologies, Inc. | Four-channel color filter array pattern |
US8237831B2 (en) * | 2009-05-28 | 2012-08-07 | Omnivision Technologies, Inc. | Four-channel color filter array interpolation |
US8125546B2 (en) * | 2009-06-05 | 2012-02-28 | Omnivision Technologies, Inc. | Color filter array pattern having four-channels |
US9191635B2 (en) * | 2012-03-19 | 2015-11-17 | Semiconductor Components Industries, Llc | Imaging systems with clear filter pixels |
RU2014138087A (en) * | 2012-03-27 | 2016-04-10 | Сони Корпорейшн | IMAGE PROCESSING DEVICE, IMAGE FORMING DEVICE, IMAGE PROCESSING METHOD AND PROGRAM |
US9348464B2 (en) * | 2012-06-06 | 2016-05-24 | Semiconductor Components Industries, Llc | Imaging systems and methods for user input detection |
KR101695252B1 (en) * | 2012-06-07 | 2017-01-13 | 한화테크윈 주식회사 | Camera system with multi-spectral filter array and image processing method thereof |
FR3004882B1 (en) * | 2013-04-17 | 2015-05-15 | Photonis France | DEVICE FOR ACQUIRING BIMODE IMAGES |
US9497427B2 (en) * | 2013-07-15 | 2016-11-15 | Semiconductor Components Industries, Llc | Method and apparatus for image flare mitigation |
CN103888672A (en) * | 2014-03-31 | 2014-06-25 | 宇龙计算机通信科技(深圳)有限公司 | Terminal and terminal shooting method |
WO2016026072A1 (en) * | 2014-08-18 | 2016-02-25 | Nokia Technologies Oy | Method, apparatus and computer program product for generation of extended dynamic range color images |
CN104581100A (en) * | 2015-02-12 | 2015-04-29 | 张李静 | Color filter array and image processing method |
US9654756B1 (en) * | 2015-11-16 | 2017-05-16 | Motorola Mobility Llc | Method and apparatus for interpolating pixel colors from color and panchromatic channels to color channels |
KR102287944B1 (en) * | 2015-12-22 | 2021-08-09 | 삼성전자주식회사 | Apparatus for outputting image and method thereof |
EP3301911A1 (en) * | 2016-09-29 | 2018-04-04 | Conti Temic microelectronic GmbH | Device for white balance correction |
DE102016221336A1 (en) * | 2016-10-28 | 2018-05-03 | Conti Temic Microelectronic Gmbh | IMAGE SENSOR WITH A COLOR FILTER SUITABLE FOR PIXEL BINNING |
JP7068796B2 (en) * | 2016-11-25 | 2022-05-17 | ソニーセミコンダクタソリューションズ株式会社 | Solid-state image sensor, drive method, and electronic equipment |
CN109035192A (en) * | 2018-08-17 | 2018-12-18 | 凌云光技术集团有限责任公司 | A kind of visible images and full-colour image synthetic method and device |
CN112104847B (en) * | 2020-09-17 | 2021-07-23 | 北京理工大学 | SONY-RGBW array color reconstruction method based on residual error and high-frequency replacement |
-
2018
- 2018-08-03 CN CN202210094591.8A patent/CN114422766B/en active Active
- 2018-08-03 CN CN201810875810.XA patent/CN110493583B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020101524A1 (en) * | 1998-03-04 | 2002-08-01 | Intel Corporation | Integrated color interpolation and color space conversion algorithm from 8-bit bayer pattern RGB color space to 12-bit YCrCb color space |
CN101233763A (en) * | 2005-07-28 | 2008-07-30 | 伊斯曼柯达公司 | Processing color and panchromatic pixels |
CN101375610B (en) * | 2006-01-27 | 2011-09-21 | 全视技术有限公司 | Interpolation of panchromatic and color pixels |
CN101652798A (en) * | 2007-03-30 | 2010-02-17 | 伊斯曼柯达公司 | Use the edge map of panchromatic pixels |
CN102369721A (en) * | 2009-03-10 | 2012-03-07 | 美商豪威科技股份有限公司 | Cfa image with synthetic panchromatic image |
CN102461175A (en) * | 2009-06-09 | 2012-05-16 | 全视科技有限公司 | Interpolation for four-channel color filter array |
CN105070270A (en) * | 2015-09-14 | 2015-11-18 | 深圳市华星光电技术有限公司 | RGBW panel subpixel compensation method and device |
CN108024106A (en) * | 2016-11-04 | 2018-05-11 | 上海富瀚微电子股份有限公司 | Support the color correction device and method of RGBIR and RGBW forms |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111062378B (en) * | 2019-12-23 | 2021-01-26 | 重庆紫光华山智安科技有限公司 | Image processing method, model training method, target detection method and related device |
CN111062378A (en) * | 2019-12-23 | 2020-04-24 | 重庆紫光华山智安科技有限公司 | Image processing method, model training method, target detection method and related device |
WO2022027469A1 (en) * | 2020-08-06 | 2022-02-10 | 深圳市汇顶科技股份有限公司 | Image processing method and apparatus, and storage medium |
CN111861964A (en) * | 2020-08-06 | 2020-10-30 | 深圳市汇顶科技股份有限公司 | Image processing method, apparatus and storage medium |
CN111861964B (en) * | 2020-08-06 | 2024-07-05 | 深圳市汇顶科技股份有限公司 | Image processing method, device and storage medium |
CN112016478A (en) * | 2020-08-31 | 2020-12-01 | 中国电子科技集团公司第三研究所 | Complex scene identification method and system based on multispectral image fusion |
CN112016478B (en) * | 2020-08-31 | 2024-04-16 | 中国电子科技集团公司第三研究所 | Complex scene recognition method and system based on multispectral image fusion |
WO2022088310A1 (en) * | 2020-10-26 | 2022-05-05 | Oppo广东移动通信有限公司 | Image processing method, camera assembly, and mobile terminal |
CN112351172B (en) * | 2020-10-26 | 2021-09-17 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
CN112351172A (en) * | 2020-10-26 | 2021-02-09 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
CN112261391A (en) * | 2020-10-26 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
CN112738493A (en) * | 2020-12-28 | 2021-04-30 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
CN112738493B (en) * | 2020-12-28 | 2023-03-14 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN114422766B (en) | 2024-06-04 |
CN110493583B (en) | 2021-12-17 |
CN114422766A (en) | 2022-04-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110493583A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
CN107534735B (en) | Image processing method, device and the terminal of terminal | |
CN105430359A (en) | Imaging method, image sensor, imaging device and electronic device | |
CN105409211B (en) | For the automatic white balance positive with skin-color adjustment of image procossing | |
CN107424198A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
CN106375740B (en) | Generate the methods, devices and systems of RGB image | |
CN108154514B (en) | Image processing method, device and equipment | |
CN105611185B (en) | image generating method, device and terminal device | |
CN105592303B (en) | imaging method, imaging device and electronic device | |
CN107967668A (en) | A kind of image processing method and device | |
WO2013101642A1 (en) | Spectral image processing | |
CN106911876A (en) | For the method and apparatus of output image | |
CN105430361A (en) | Imaging method, image sensor, imaging device and electronic device | |
CN105578080B (en) | Imaging method, imaging device and electronic device | |
CN108024054A (en) | Image processing method, device and equipment | |
CN112767291B (en) | Visible light image and infrared image fusion method, device and readable storage medium | |
CN107396079B (en) | White balance adjustment method and device | |
CN105578081A (en) | Imaging method, image sensor, imaging device and electronic device | |
CN105611257B (en) | Imaging method, imaging sensor, imaging device and electronic device | |
CN108230407A (en) | A kind for the treatment of method and apparatus of image | |
JP4941482B2 (en) | Pseudo color image generation apparatus and program | |
CN107517348A (en) | The rendering intent and device of image | |
CN105430362A (en) | Imaging method, imaging device and electronic device | |
CN107317967A (en) | Image processing method, device, mobile terminal and computer-readable recording medium | |
CN105554485B (en) | Imaging method, imaging device and electronic device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |