CN101098399A - Apparatus and method for recovering high pixel image processing - Google Patents

Apparatus and method for recovering high pixel image processing Download PDF

Info

Publication number
CN101098399A
CN101098399A CNA2007101126736A CN200710112673A CN101098399A CN 101098399 A CN101098399 A CN 101098399A CN A2007101126736 A CNA2007101126736 A CN A2007101126736A CN 200710112673 A CN200710112673 A CN 200710112673A CN 101098399 A CN101098399 A CN 101098399A
Authority
CN
China
Prior art keywords
image
original
color
pixel
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA2007101126736A
Other languages
Chinese (zh)
Inventor
成基荣
朴斗植
金昌容
李皓荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electro Mechanics Co Ltd
Original Assignee
Samsung Electro Mechanics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electro Mechanics Co Ltd filed Critical Samsung Electro Mechanics Co Ltd
Publication of CN101098399A publication Critical patent/CN101098399A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4015Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0007Image acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening
    • HELECTRICITY
    • H01ELECTRIC ELEMENTS
    • H01LSEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144Devices controlled by radiation
    • H01L27/146Imager structures
    • H01L27/14601Structural or functional details thereof
    • H01L27/1462Coatings
    • H01L27/14621Colour filter arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Power Engineering (AREA)
  • Electromagnetism (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Computer Hardware Design (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

An apparatus and method of recovering a high pixel image are provided. The apparatus includes a camera module including a plurality of lenses, and a plurality of sub image sensors corresponding to the plurality of lenses, the plurality of sub image sensors each including a color filter having a single color, an original image generation module receiving a plurality of original color-separated images, an intermediate image generation module rearranging pixel information of pixels at identical positions at the plurality of original color-separated images provided from the original image generation module and generating an intermediate image having a higher resolution than each of the original color-separated images, and a final image generation module performing demosaicing on the intermediate image, performing deblurring on the demosaiced intermediate image, and generating a final image.

Description

Recover the equipment and the method for high pixel image processing
The application requires the priority of the 10-2006-0105348 korean patent application submitted in Korea S Department of Intellectual Property in the 10-2006-0057660 korean patent application submitted in Korea S Department of Intellectual Property on June 26th, 2006 and on October 27th, 2006 respectively, and this application all is disclosed in this for reference.
Technical field
The present invention relates to a kind of equipment and method of recovering high pixel image processing, more particularly, relate to and a kind ofly recover the equipment and the method for high pixel image processing from the image that obtains by the compact camera module that is installed in the small-sized digital device (such as mobile phone, PDA(Personal Digital Assistant), MP3 player etc.).
Background technology
As most of digital devices, digital camera has been opened up the new world to people.As substituting of traditional camera based on film, the advantage of digital camera is simple to operate, and the person compares with those Professional Photographies, and domestic consumer also can take pictures, and under the situation that need not wash and print, can watch the photo of shooting immediately after catching.In addition, because digital camera is recorded as digital document with the image of catching in the memory of camera, therefore the photo of the high picture quality taken can be uploaded to personal computer (PC) as permanent or semipermanent digital image file, for example, be used for storage, image processing and/or printing in any time of necessity.
In addition, with the digital camera miniaturization, and digital camera become lighter, makes them can be embedded in the small-sized digital device (such as mobile phone, PDA(Personal Digital Assistant), MP3 player etc.).Consequently, taking pictures and appreciating photo has become the part of routine matter, and whether digital camera can be embedded into and become extremely important in the small-sized digital device when buying digital device.
Recently, the digital device miniaturization more that become needs personalized device and convenience in addition, and the consumer need to have begun less and lighter digital device product.
Therefore, be apparent that: less and lighter for the small-sized digital device that makes the digital camera with embedding, it is less and lighter that the digital camera of embedding itself should be done.
Fig. 1 schematically illustrates the operating principle of incorporating the digital camera in traditional compact camera module into.
With reference to Fig. 1, the user is by diameter D aCamera lens 101a and diameter D bThe image of the predetermined object 101 taken of camera lens 101b on imageing sensor 102a and imageing sensor 102b, form image A and image B respectively.
Although see to have big relatively diameter D from the angle of resolution bCamera lens 101b have superiority, but such problem may appear: because long relatively focal distance f bAnd make the camera model of incorporating in the decimal fractions device become heavy, be desired characteristics not camera model being installed in the decimal fractions device this.That is to say that macro lens size and long-focus make and realize that miniaturization, light digital camera are very difficult.
On the contrary, has relatively little diameter D aCamera lens 101a have short focal distance f a, to form the image A of object 101.Therefore, although can be, can not obtain to have the high-resolution image of one of most important characteristics as in the digital camera with the camera model miniaturization.This camera model can not satisfy the demand of consumer to high resolution pictures fully.
In order to address this is that, a lot of inventions have been proposed, for example, the early stage disclosed application of 2003-0084343 Korean Patent, but also do not solve described problem.
Summary of the invention
Therefore, an aspect of of the present present invention is to provide a kind of equipment and method of recovering high pixel image processing, and described equipment and method can realize small-sized digital device by the size that reduces to be installed in the digital camera in the small-sized digital device.
Another aspect of the present invention is to provide a kind of equipment and method of recovering high pixel image processing, and described equipment and method can reduce to be installed in the size of the digital camera in the small-sized digital device.
Another aspect of the present invention is to provide a kind of equipment and method of recovering high pixel image processing, and described equipment and method can easily be proofreaied and correct the inconsistent of optical aberration and susceptibility in the performance high pixel image processing.
Another aspect of the present invention is to provide a kind of equipment and method of recovering high pixel image processing, also can not realize high sensitive image sensing and low sensitivity image sensing simultaneously even the color filter that described equipment and method have different transmissivities by use does not change the structure of imageing sensor.
Another aspect of the present invention is to provide a kind of equipment and method of recovering high pixel image processing in the digital camera module with embedded digital camera, wherein, when design and installation during the small-sized digital device of digital camera, more the multiselect item is available.
Will be in ensuing description part set forth the present invention other aspect and/or advantage, some will be clearly by describing, and perhaps can learn through enforcement of the present invention.
Therefore, an aspect of of the present present invention is to provide a kind of equipment that recovers high pixel image processing, described equipment comprises: camera model, comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of subimage transducers; The original image generation module receives a plurality of original color separated image; The intermediate image generation module rearranges the Pixel Information of the pixel on the same position of a plurality of original color separated image that the original image generation module provides, and produces the intermediate image with the resolution that is higher than each original color separated image; And the final image generation module, mosaic is removed in middle image execution, carry out the intermediate image sharpening that makes mosaic, and produce final image.
According to a further aspect in the invention, a kind of equipment that recovers high pixel image processing is provided, described equipment comprises: camera model, comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of a plurality of camera lenses, each all comprises color filter described a plurality of subimage transducer, and described color filter is divided into a plurality of color regions with different colours; The original image generation module is divided into a plurality of pixel groups with a plurality of original color separated image; The intermediate image generation module, the Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image with the resolution that is higher than each original color separated image; And the final image generation module, use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes reception, and produce final image.
According to a further aspect in the invention, a kind of equipment that recovers high pixel image processing is provided, described equipment comprises: camera model, comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, wherein, in described a plurality of color camera lens each has monochrome, and obtains a plurality of original color separated image by described a plurality of subimage transducers; The original image generation module receives a plurality of original color separated image; The intermediate image generation module rearranges the Pixel Information of the pixel on the same position of a plurality of original color separated image that the original image generation module provides, and produces the intermediate image with the resolution that is higher than each original color separated image; And the final image generation module, mosaic is removed in middle image execution, carry out the intermediate image sharpening that makes mosaic, and produce final image.
According to a further aspect in the invention, a kind of equipment that recovers high pixel image processing is provided, described equipment comprises: camera model, comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, wherein, in described a plurality of color camera lens each has monochrome, and obtains a plurality of original color separated image by described a plurality of subimage transducers; The original image generation module is divided into a plurality of pixel groups with a plurality of original color separated image; The intermediate image generation module is mapped to each pixel with the corresponding pixel groups of described same position with the Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups, and produces intermediate image; And the final image generation module, use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes recovery, and produce final image.
According to a further aspect in the invention, a kind of method of recovering high pixel image processing in camera model is provided, described camera model comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of camera lens, and described method comprises: obtain a plurality of original images by each of a plurality of subimage transducers; Receive a plurality of original images, and produce a plurality of original color separated image; The Pixel Information of the pixel on the same position of a plurality of original color separated image that provide from the original image generation module of camera model is provided, and produces intermediate image with the resolution that is higher than each original color separated image; And middle image carried out remove mosaic, carry out the intermediate image sharpening that makes mosaic, and produce final image from the image of sharpening.
According to a further aspect in the invention, a kind of method of recovering high pixel image processing in camera model is provided, described camera model comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of camera lens, and described method comprises: obtain a plurality of original color separated image by each of a plurality of subimage transducers; The a plurality of original color separated image that obtains is divided into a plurality of pixel groups; The Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image; And use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes interpolation, and produce final image from the image of sharpening.
According to a further aspect in the invention, a kind of method of recovering high pixel image processing in camera model is provided, described camera model comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, each all comprises having monochromatic color filter described a plurality of color camera lens, and described method comprises: obtain a plurality of original color separated image by each of a plurality of subimage transducers; Receive a plurality of original color separated image; The Pixel Information of the pixel on the same position of a plurality of original color separated image that provide from the original image generation module of camera model is provided, and produces intermediate image with the resolution that is higher than each original color separated image; And middle image carried out remove mosaic, carry out the intermediate image sharpening that makes mosaic, and produce final image from the image of sharpening.
According to a further aspect in the invention, a kind of method of recovering high pixel image processing in camera model is provided, described camera model comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, each all has monochrome described a plurality of color camera lens, and described method comprises: obtain a plurality of original color separated image by each of subimage transducer; A plurality of original color separated image are divided into a plurality of pixel groups; The Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image; And use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes interpolation, and produce final image.
Description of drawings
By the detailed description of the preferred embodiments of the present invention being carried out below in conjunction with accompanying drawing, above-mentioned and other characteristics of the present invention and advantage will become apparent, wherein:
Fig. 1 is the diagrammatic sketch that the operating principle that is installed in the digital camera in traditional decimal fractions device is shown;
Fig. 2 A is the diagrammatic sketch that the basic structure of conventional digital camera is shown;
Fig. 2 B is the cutaway view that forms the unit picture element of the image sensor cell shown in Fig. 2 A;
Fig. 3 is the block diagram of the equipment (300) that recovers high pixel image processing according to an embodiment of the invention;
Fig. 4 A and Fig. 4 B illustrate the diagrammatic sketch of the structure of digital camera module according to an embodiment of the invention;
Fig. 4 C is the cutaway view of the unit picture element of the formation image sensor cell shown in Fig. 4 A;
Fig. 5 A and Fig. 5 B illustrate the diagrammatic sketch of the method for coating colour filter according to an embodiment of the invention;
Fig. 6 illustrates the diagrammatic sketch of the inconsistent processing of the susceptibility of original image generation module correction position deviation and original image according to an embodiment of the invention;
Fig. 7 A and Fig. 7 B are the diagrammatic sketch that the processing that produces intermediate image according to an embodiment of the invention is shown;
Fig. 8 illustrates the diagrammatic sketch of the processing of image generation module recovery high pixel image processing according to an embodiment of the invention;
Fig. 9 illustrates the diagrammatic sketch of the structure of digital camera module according to an embodiment of the invention;
Figure 10 illustrates with the camera model with the structure shown in Fig. 4 A to use the first image generation module shown in Fig. 7 B to recover the flow chart of the method for high pixel image processing; And
Figure 11 illustrates with the camera model with the structure shown in Fig. 4 A to use the second image generation module shown in Figure 8 to recover the flow chart of the method for high pixel image processing.
Embodiment
Now will describe embodiments of the invention in detail, example of the present invention shown in the drawings, wherein, identical label is indicated identical parts all the time.Embodiment is described with reference to the accompanying drawings to explain the present invention.
Usually, because the quantity of unit picture element is big more, the image of acquisition is just clear more and neat more, so can determine the performance of digital camera by the quantity of unit picture element.
Except the quantity of unit picture element, the luminous intensity of camera lens also influences the performance of digital camera.The luminous intensity of camera lens is called as F number or f-number.By (that is, f/D) obtain the F number, described F number is the expression of light quantity in per unit zone that arrives the imageing sensor of digital camera divided by diameter D with the focal distance f of camera lens.The F number is big more, and the light quantity in the per unit zone of the imageing sensor of arrival digital camera is more little.The F number is more little, and the light quantity in the per unit zone of the imageing sensor of arrival digital camera is big more, thereby obtains to have high-resolution bright image.As mentioned above, the light quantity in the per unit zone of the imageing sensor of the resolution of the image of F number and acquisition and arrival digital camera is closely related.
Suppose that two digital cameras with different Lens, focal length and unit picture element have identical F number.Because the light quantity in the per unit zone of each imageing sensor of two digital cameras of arrival is identical, therefore obtains image and will have identical luminous intensity from each digital camera.
Based on above-mentioned principle, embodiments of the invention propose a kind of method that can be applicable to the recovery high pixel image processing of digital camera module, and described digital camera module has a plurality of camera lenses that can show high pixel image processing when reducing diameter of lens and focal length.
Fig. 2 A is the diagrammatic sketch that the basic structure of conventional digital camera is shown.With reference to Fig. 2, conventional digital camera mainly comprises: have diameter D 2Camera lens 201, concentrate from the light of predetermine one reflection; With imageing sensor 202, the light of concentrating in response to camera lens 201 produces and the corresponding electrical picture signal of pixel class.Also show the focal length of camera lens 201.
In imageing sensor 202, comprise color filter array, being embodied as original panchromatic (full-color) image at the light that camera lens 201 receives according to the Bayer pattern.Fig. 2 A illustrates the top view of imageing sensor 202.
Even the Bayer pattern comes from the in esse image of occurring in nature not by forming, also realize the principle of digital picture inevitably by point.
For by forming digital picture, collect brightness (or luminosity (brightness)) and colourity (or color) composition, and the point of each brightness of on two-dimentional panel, distribute reception red (R) look, green (G) look and indigo plant (B) colour content.
In the array of Bayer pattern, the sensitive G colour content of human eye is 50%, and each all is 25% for R look and B colour content, thereby forms two-dimensional matrix, is called as Bayer pattern color filter.
Bayer pattern color filter is only discerned the color component that is assigned to wherein from each R look, G look and the B colour content that form matrix, and the panchromatic composition of nonrecognition, and insert Unidentified color component to reproduce panchromatic composition.
Fig. 2 B is the cutaway view of unit picture element that is used to form the imageing sensor of Fig. 2 A
Fig. 2 B with reference to the part that the unit picture element 202a to 202d that forms imageing sensor (Fig. 2 A 202) is shown comprises Bayer pattern color filter 203 in imageing sensor 202.At the further feature of legacy equipment shown in Fig. 2 B, but described further feature will be discussed, because this discussion is dispensable to understanding the application at this.
Fig. 3 is the block diagram that recovers the equipment 300 of high pixel image processing according to an embodiment of the invention.
Equipment 300 comprises: camera model 301, and concentrate incident light and produce a plurality of color separations (color-separated) image; Image generation module 302, the color separated image that provides based on camera model 301 produces final image; And display module 303, the final image that display image generation module 302 provides.
Have the structure of the digital camera module of a plurality of camera lenses with reference to Fig. 4 A and Fig. 4 B description, described digital camera module can be used the method for recovering high pixel image processing according to an embodiment of the invention.
Fig. 4 A to Fig. 4 C is the structure that the digital camera module with a plurality of camera lenses is shown, and described digital camera module can be used the method for recovering high pixel image processing according to an embodiment of the invention.
Camera model 301 comprise a plurality of camera lenses 401 and with a plurality of camera lens 401 corresponding a plurality of subimage transducers 402.Here, a plurality of subimage transducers 402 have color filter, and obtain the original image of each color from each subimage transducer 402, and described color filter has the zone that scribbles single color.
Digital camera module comprises: a plurality of camera lenses 401, to concentrate from the incident light of predetermine one reflection, and described a plurality of camera lenses 401 have identical diameter; And a plurality of subimage transducer 402a to 402d, produce electrical picture signal in response to light from the object reflection.That is to say, in a plurality of subimage transducer 402a to 402d, comprise the color filter similar to the color filter of prior art, described color filter has the space that is divided into a plurality of color regions, being embodied as original panchromatic by the light that camera lens 401 is concentrated.
Fig. 4 B is the end view of the digital camera module shown in Fig. 4 A, wherein, supposes that scope is from having same diameter D 3A plurality of camera lenses 401 (401a-401d) form the focal distance f of transducer 402a-402d to a plurality of object images 3Identical.
Therefore, has same diameter D 3A plurality of camera lenses 401 (401a-401d) be arranged on same plane.
For camera lens pattern of rows and columns, although the embodiment that illustrates shown according to up and down and a plurality of camera lenses 401 of left and right directions symmetric arrays (401a-401d), the invention is not restricted to this.Can be according to horizontal direction or the linear a plurality of camera lenses 401 (401a-401d) of arranging of vertical direction.Under the situation that the odd number camera lens can be used, can be around the camera lens of center the arranged radially camera lens, and can adopt other various camera lens pattern of rows and columns.
A plurality of camera lenses 401 (401a-401d) can the camera lens appointed positions or based on the offset of predetermined camera lens in a plurality of camera lenses of predetermined number of pixels except this position of the camera lens the predetermined camera lens, concentrate from the light of predetermine one reflection.
Below, explain for convenience, will explain the embodiment that four camera lenses are arranged with the form of 2 * 2 (wide * height) matrix in the mode of example.
Two digital cameras that have different Lens, focal length and unit picture element based on above-mentioned hypothesis have the principle of identical F number, because the light quantity in the per unit zone of each imageing sensor of two digital cameras of arrival is identical, therefore obtains image and will have identical luminous intensity from each digital camera.In this embodiment, use each all to have a plurality of subimage transducer 402a to 402d that lack than the quantity of the unit picture element of image sensor cell shown in Figure 2 202 than the little a plurality of camera lenses 401 (401a-401d) of camera lens shown in Figure 2 201 and each.In addition, the focal distance f of Fig. 4 B 3Focal distance f than Fig. 2 2Short.
Suppose that imageing sensor shown in Figure 2 202 forms 4,000,000 pixels, and each of a plurality of subimage transducer 402a to 402d shown in Fig. 4 A forms 1,000,000 pixels respectively.
Therefore, a plurality of subimage transducer 402a to 402d shown in Fig. 4 A are corresponding with four imageing sensors that each all has a mega pixel.Each all has same diameter D 3 Four camera lens 401a-401d producing image respectively with on the corresponding subimage transducer of the camera lens 401 a-401d 402a to 402d.
With each quartering all of the color filter among the subimage transducer 402a to 402d, so that four zones are fit to the size of subimage transducer 402a to 402d, then each zone is scribbled monochrome.
Fig. 4 C is the cutaway view of unit picture element 402d-1 to 402d-4 that is used to form the subimage transducer 402a to 402d of Fig. 4 A.Owing in identical subimage transducer 402d, have unit picture element 402d-1 to 402d-4, then in the zone of unit picture element 402d-1 to 402d-4, comprise identical color filter 403.
Here, the transmissivity according to the color that is coated with on it is divided into first filtration zone and second filtration zone with color filter 403.Make by first filtration zone by the transmissivity that changes the color that is coated with on first filtration zone and second filtration zone different with the light quantity of second filtration zone, thereby realize high sensitive sensing and low sensitivity sensing simultaneously.In the shades of colour zone, the color region with high-transmission rate is grouped in first filtration zone, and other color region is grouped in second filtration zone.
Below, explain for convenience, will explain the embodiment that represents a plurality of color regions and subimage transducer by same numeral.
For example, each color region that formation is included in the color filter among the subimage transducer 402a to 402d shown in Fig. 4 A (promptly, green (G) look zone, red (R) look zone, indigo plant (B) look zone and ash (Gr) look zone) by representing with the identical label of subimage transducer respectively, that is, 402a, 402b, 402c and 402d.Ash (Gr) look zone 402d with high-transmission rate is grouped into second filtration zone, and other color region (that is, green (G) look zone 402a, red (R) look zone 402b and indigo plant (B) look regional 402c) is grouped into first filtration zone.
In another embodiment, can in second filtration zone, form the color filter of the color except ash.For example, can form white (W), yellow (Y), any one color filter in blue or green and purplish red.Yet, the color of the color filter that forms in second filtration zone is not limited to these examples, and the color filter with any color of the high transmissivity of the transmissivity of the color filter that forms in the filtration zone than first filter all can be believed to comprise within the scope of the invention.
For the transmissivity of the color filter shown in Fig. 4 A, in blue (B) chromogenic filter device, green (G) chromogenic filter device, red (R) chromogenic filter device and ash (Gr) chromogenic filter device, ash (Gr) chromogenic filter utensil has the highest transmissivity.
Like this, if be higher than transmissivity with the corresponding color of the first filtration zone 402a to 402c, then different with the light quantity of second filtration zone by first filtration zone with the transmissivity of the corresponding color of the second filtration zone 402d.
This means that the light quantity that arrives each the number of sub images transducer 402a to 402d that mates with each color region is different, and in each number of sub images transducer 402a to 402d, can realize high sensitive sensing function and low sensitivity sensing function simultaneously.
Can be coated with above-mentioned color filter by photographic printing (photo-lithography) method or ink-jet (inkjet) method that illustrates respectively with reference to Fig. 5 A and Fig. 5 B.
Under the situation of coating monochromatic blue (B), green (G), red (R) and ash (Gr) according to the photographic printing method, there is not the original white transducer (501) of color to be coated with green (502).Then, belong to the green pattern except scribbling 1/4th (1/4) of green image sensor area, scribbling green image sensor area has and scribbles 3/4ths (3/4) of green image sensor area (503) and be removed.
The imageing sensor of removing is coated with redness (504).Then, scribbling red 2/4ths (2/4) image sensor area is removed.
Consequent subimage transducer is corresponding with the imageing sensor with the image sensor area that scribbles 1/4th greens and 1/4th redness respectively, covers 2/4ths (2/4) green zones in the described subimage transducer and is coated with blue and grey (506~508).
Compare with the processing of Bayer pattern color filtering, current photographic printing method is carried out easily.
Fig. 5 B illustrates ink ejecting method.
This operation is from white transducer 509.According to ink ejecting method, at first carry out partition wall and form operation (510), wherein, (for example in imageing sensor, form the partition wall identical with the quantity of camera lens among the embodiment that illustrates, 4), and the look China ink (colorink) of expectation is coated in 4 zones that the next door produces respectively, that is, and green ink, red China ink, Lan Mo and ash China ink (511~514).
Ink ejecting method is quite simply to handle, and can help saving the use amount of China ink, thereby finally reduces the manufacturing cost of transducer.
The color filter that is coated with in the above described manner respectively with a plurality of camera lenses and a plurality of subimage sensor matching.
For example, with reference to Fig. 4 A, in subimage transducer 402a to 402d, comprise under the situation of green (G) chromogenic filter device, red (R) chromogenic filter device, indigo plant (B) chromogenic filter device and ash (Gr) chromogenic filter device, by being included in green (G) chromogenic filter device among the subimage transducer 402a that mates with the first camera lens 401a, form green (G) color image from the incident ray that object reflects by the first camera lens 401a in four camera lenses 401 (401a-401d).By being included in red (R) chromogenic filter device among the subimage transducer 402b that mates with the second camera lens 401b, form red (R) image from the incident ray that object reflects by the second camera lens 401b in four camera lenses 401 (401a-401d).
Similarly, by being included in indigo plant (B) the chromogenic filter device and ash (Gr) the chromogenic filter device that is included among the subimage transducer 402d that mates with four-barrel 401d among the subimage transducer 402c that mates with three-lens 401c, form indigo plant (B) color image and ash (Gr) color image from the incident ray that object reflects respectively by three-lens 401c in four camera lenses 401 (401a-401d) and four-barrel 401d.
In other words, form the image with the color that is included in the respective color filter each number of sub images transducer 402a to 402d, i.e. four width of cloth images of same size different colours by four camera lens 401a-401d from the incident ray that object reflects.
Simultaneously, image generation module 302 receives a plurality of color separated image from camera model 301, and produces final image based on the color separated image that provides from camera model 301.
For this reason, image generation module 302 comprises: original image generation module 302a, intermediate image generation module 302b and final image generation module 302c.
In the image generation module 302 according to current embodiment of the present invention, the input of a plurality of original color separated image that provide from camera model 301 is provided original image generation module 302a.
That is to say, shown in Fig. 4 A, the gray image that the red image that the subimage transducer 402b that original image generation module 302a receives green image that the subimage transducer 402a comprise green filters obtains, comprise red filter obtains, blue image that the subimage transducer 402c that comprises blue filter obtains and the subimage transducer 402d that comprises the grey filter obtain.
Here, green image, red image and blue image provide by final image generation module 302c and produce the required colouring information of final image, after a while with described.On the contrary, gray image provides and produces the required monochrome information of final image.
In optional embodiment of the present invention, when the original image that obtains was not arranged with the appointed positions of corresponding subimage transducer 402a to 402d, original image generation module 302a proofreaied and correct the position of original image.In addition, when the original image susceptibility that obtains was inconsistent, original image generation module 302a proofreaied and correct the sensitivity grade of the original image except the original image with minimum sensitivity grade based on the susceptibility of the original image with minimum sensitivity grade.
Fig. 6 illustrates the diagrammatic sketch of the inconsistent processing of the susceptibility of original image generation module (302a) correction position deviation and original image according to an embodiment of the invention.
By being included in the color filter among each the number of sub images transducer 402a to 402d that mates with camera lens 401a to 401d, transmission is incided the light of predetermine one by four camera lens 401a to 401d that are arranged in the fixed position, has image with the corresponding color of color filter with formation.Then, original image generation module 302a checks the position of each image that forms in subimage transducer 402a to 402d.
With reference to Fig. 6, be illustrated in each image that forms on the position among the subimage transducer 402a to 402d by label 601 to 604.
Label 601 is illustrated in subimage transducer 402a and goes up the image that forms.In an embodiment of the present invention, unless fixed-site, otherwise with the pixel of four camera lens 401a to 401d skew predetermined quantities, the image that is positioned at subimage transducer 402a center can think that image is positioned at normal place, just, shown in dashed rectangle 605, the assigned address of image.
Label 602 expression subimage transducer 402b go up the image that forms, and described image is from the assigned address pixel that moves to left.Label 603 expression subimage transducer 402c go up the image that forms, and described image moves down a pixel from assigned address.Label 604 expression subimage transducer 402d go up the image that forms, and described image is pressed pixel of diagonal skew from assigned address, that is to say, all is offset a pixel to the right with downward direction from specify image.
As shown in Figure 6, general owing to optical aberration, the image of each number of sub images transducer 402a to 402d may depart from their assigned address.This optical aberration makes each number of sub images transducer 402a to 402d image depart from their assigned address usually, thereby is difficult to obtain neat and distinct image.
According to embodiments of the invention, the picture position of departing from that original image generation module 302a can cause by software calibrating optical deviation.
At length, original image generation module 302a proofreaies and correct in the following manner because the picture position of departing from that optical aberration causes.For example, for image 602, in case image departs from a pixel left from assigned address, a pixel just moves to right image.In image 603, a pixel will be moved on the image.In image 602, with image move to left a pixel and on move a pixel.Perhaps, in image 604, image can by on move a pixel, a pixel then moves to left.
In addition, original image generation module 302a corrects the inconsistent of susceptibility, with reference to Fig. 6 this is described.
The sensitivity grade of supposing each image in 601,602,603 and 604 is respectively 10,9,8,7, original image generation module 302a is based on the susceptibility of the image 604 with sensitivity grade 7, proofread and correct inconsistent sensitivity grade by adjustment except the sensitivity grade of the image outside the image 604 with minimum sensitivity grade (that is, 7).
By above-mentioned processing, the image of each color is offered intermediate image generation module 302b.
The Pixel Information of pixel of the same position of each original image that provides at original image generation module 302a is provided intermediate image generation module 302b, thereby can produce the intermediate image with resolution higher than the resolution of the original image of each color.
Here, the image that obtains of each bar Pixel Information of the pixel of term " intermediate image " expression that is used for distinguishing each image that forms on the subimage transducer by rearranging each original image same position with resolution higher than the resolution of original image.Yet it is to have high-resolution relatively final image that term " intermediate image " there is no need.Can go mosaic (demosaicing) or sharpening to produce final image by middle image is carried out.Simultaneously, the intermediate image of generation can have the resolution identical with the resolution of imageing sensor 402, has arranged a plurality of subimage transducer 402a to 402d in described imageing sensor 402.
For example, if each among a plurality of subimage transducer 402a to 402d all has 4 * 4 resolution, then intermediate image can have 8 * 8 resolution, and this resolution with imageing sensor 402 is identical.Then intermediate image is offered final image generation module 302c.Final image generation module 302c carries out by the intermediate image that middle image generation module 302b is provided and removes mosaic and make the image sharpening of mosaic produce final image.
Display module 303 shows the final image that final image generation module 302c provides.
For example the form of flat-panel monitor or touch-screen realizes display module 303, but is not limited to these forms.
Fig. 7 A and Fig. 7 B are the diagrammatic sketch that the processing that produces intermediate image according to an embodiment of the invention is shown.
Before explaining the method for recovering high pixel image processing, suppose to arrange the camera lens of camera model 301 by this way in display device shown in Figure 3 300: the camera lens except the predetermined camera lens that is arranged in the fixed position is from the pixel of the offset predetermined quantity of predetermined camera lens.
Here, the pixel that camera lens is offset predetermined quantity can comprise based on displacement and moving direction and is offset camera lens.
Shown in Fig. 7 A, when using camera model 301 to take the predetermine one 701 of the rectangle that comprises four same sizes, suppose at fixed position arrangement camera lens 401a to take rectangle A (701a), and based on the position of camera lens 401a pixel, to take rectangle B, C and D (701b, 701c and 701d) with remaining camera lens 401b to 401d skew predetermined quantity.
In being displaced to the precalculated position, based on the position of the camera lens 401a that takes rectangle A (701a), the camera lens of taking rectangle B (701b) is shifted to the right to the precalculated position.Based on the position of the camera lens 401a that takes rectangle A (701a), the camera lens of taking rectangle C (701c) is moved down into the precalculated position.
In addition, based on the position of the camera lens 401a that takes rectangle A (701a), the camera lens of taking rectangle D (701d) is displaced to the precalculated position by diagonal (to the right with downward method).
Rectangle A (701a), B (701b), C (701c) and D (701d) are corresponding with the image that is formed on by camera lens 401a to 401d on each number of sub images transducer 402a to 402d respectively.
Suppose to form a mega pixel respectively with a plurality of subimage transducer 402a to 402d of corresponding camera lens coupling, arrange the image of rectangle A (701a), the B (701b), C (701c) and the D (701d) that take according to the shape of object, to form the single image that symbol "+" is represented among Fig. 7 A.By this way, can form image 703 with resolution identical with the resolution of all images 704 of the object of taking with the camera that can carry out four mega pixel resolution imagings.
Fig. 7 B illustrates the method that produces intermediate image according to an embodiment of the invention.
Based on the top identical principle of describing with reference to Fig. 7 A, in four camera lens 401a to 401d of the camera model 301 of taking predetermine one 705, at camera lens 401a of fixed position arrangement, and other camera lens 401b to 401d is from the position individual element skew of camera lens 401a.
In other words, the camera lens 401a that position-based is fixing with camera lens 401b to the 401d pixel that moves to right, moves down a pixel and by a diagonal skew pixel (that is, being offset a pixel to the right with downward direction).
Use above-mentioned camera model 301, the intermediate image generation module 302b (see figure 3) of image generation module 302 rearranges the Pixel Information of the pixel on the same position of each original image 706 that original image generation module 302a provides, thereby can produce the intermediate image 707 with resolution higher than the resolution of the original image of each color.
The intermediate image 707 that final image generation module 302c produces middle image generation module 302b is carried out and is removed mosaic, and makes the intermediate image sharpening of mosaic, thereby original image is reverted to the final image with high-resolution.
For the sake of simplicity, the existing image generation module of describing with reference to Fig. 7 A 302 is called the first image generation module.
Fig. 8 is that the image generation module that illustrates according to another embodiment of the present invention recovers the diagrammatic sketch of the processing of high pixel image processing.The camera model that is used for current embodiment is with above-described identical, thereby will not carry out the explanation of repetition to it.
Simultaneously, the original image generation module 302a of image generation module 302 is divided into a plurality of pixel groups with the original color separated image that camera model 301 provides.Intermediate image generation module 302b produces first intermediate image 802 from original image, and it has and the identical resolution of subimage transducer 402a to 402d shown in Figure 8.
Here, first intermediate image 802 can be divided into a plurality of pixel groups 803,804 and 805, and each pixel groups is formed by 2 * 2 virtual pixels (wide * height).
In a plurality of pixel groups 803,804 and 805 each, pixel can be divided into color and monochrome information and be mapped to main pixel 803a, 804a and 805a on it and sub-pixel 803b, the 804b and the 805b that are positioned near the of main pixel 803a, 804a and 805a and do not have information.
The position of main pixel 803a, 804a and 805a can be set at a plurality of positions of each pixel groups 803,804 and 805.
For example, in each pixel groups 803,804 and 805 that forms by 2 * 2 pixels as shown in Figure 8, can be confirmed as main locations of pixels with first row and the first corresponding positions of row (that is, 803a, 804a and 805a).As another example, can be confirmed as main locations of pixels with the row of first in each pixel groups and the corresponding position of secondary series (that is, 803b, 804b and 805b).
If produce first intermediate image 802 in the above described manner, then intermediate image generation module 302b is mapped to the Pixel Information of the pixel on the same position in each original color separated image on the main pixel with the corresponding pixel groups of described same position.
For example, intermediate image generation module 302b is mapped to first row of each original color separated image and the Pixel Information of first pixel that lists on the main pixel 803a of first row that is positioned at first intermediate image 802 and first pixel groups 803 that is listed as.
Equally, intermediate image generation module 302b is mapped to first row of each color separated image and the Pixel Information of first pixel that lists on the main pixel 804a of pixel groups 804 of the first capable and secondary series that is positioned at first intermediate image 802.
In addition, intermediate image generation module 302b obtains monochrome information based on the colouring information in the Pixel Information of the pixel on the same position in each original color separated image, and the monochrome information that obtains is mapped on the main pixel 803a to 805a of each pixel groups 803 to 805.
In addition, intermediate image generation module 302b is mapped to the monochrome information that obtains on the main pixel of first row that is positioned at first intermediate image 802 and first pixel groups that is listed as.
With reference to Fig. 8, as can be seen: ash (Gr) the look information mapping that green (G) look information, red (R) look information and indigo plant (B) look information that subimage transducer 402a to 402c is provided and subimage transducer 402d provide is on the main pixel of each pixel groups.Although in Fig. 8, do not illustrate, also the monochrome information (Y) that detects from three colouring informations can be mapped on the main pixel 803a to 805a of each pixel groups 803 to 805.
For example, as can be seen: be positioned at first row of green image and first row pixel green (G) look information, first row that is positioned at red image and first row pixel red (R) look information, first row that is positioned at blue image and first row pixel indigo plant (B) look information, first row that is positioned at gray image and first row pixel monochrome information and be mapped on the main pixel 803a of first pixel groups 803 based on the monochrome information of three colouring informations detections.
Equally, as can be seen: be positioned at first row of green image and green (G) look information of the pixel of secondary series, first row that is positioned at red image and secondary series pixel red (R) look information, first row that is positioned at blue image and secondary series pixel indigo plant (B) look information, first row that is positioned at gray image and secondary series pixel monochrome information and be mapped on the main pixel 804a of second pixel groups 804 based on the monochrome information of already provided three colouring informations detection.
Therefore, intermediate image generation module 302b produces second intermediate image 806, and wherein, three colouring informations and two monochrome informations are mapped on main pixel 803a, the 804a and 805a of each pixel groups 803,804 and 805.
Thereafter, intermediate image generation module 302b uses interpolation method that second intermediate image 806 is carried out interpolation.
That is to say that intermediate image generation module 302b obtains to be recorded in Pixel Information among each sub-pixel 803b, 804b and the 805b based on the information of main pixel 803a, 804a shown in Figure 8 and 805a.
Intermediate image generation module 302b can carry out interpolation to second intermediate image 806 according to various algorithms.
For example, can calculate Pixel Information each sub-pixel that is recorded in second intermediate image 806 from the information of grasping with the contiguous main pixel of sub-pixel.
More particularly, in Fig. 8, the Pixel Information among the sub-pixel 803b between the main pixel 803a that is recorded in first pixel groups 803 and the main pixel 804a of second pixel groups 804 can be confirmed as the mean value 807 of the Pixel Information of two main pixel 803a and 804a.
Equally, the Pixel Information among the sub-pixel 804b between the main pixel 805a of main pixel 804a that is recorded in second pixel groups 804 and the 3rd pixel groups 805 can be confirmed as the mean value 808 of the Pixel Information of two main pixel 804a and 805a.
If carry out the interpolation of second intermediate image 806 by this way, then final image generation module 302c carries out the sharpening of second intermediate image 806 that makes interpolation.Consequently, produce final image 809 from the color separated image that obtains by each subimage transducer with high-resolution (that is the resolution of subimage transducer * 4) with low resolution (that is the resolution of subimage transducer).
For the sake of simplicity, the image generation module of describing with reference to Fig. 8 302 will be called as the second image generation module.Fig. 9 is the diagrammatic sketch that the digital camera module of the second embodiment of the present invention is shown.Except following characteristic, digital camera module according to a second embodiment of the present invention has and the identical structure of structure according to the digital camera module of the first embodiment of the present invention shown in Fig. 4 A.That is to say, comprise a plurality of camera lens 901a to 901d with different colours according to the camera model of second embodiment.Here, can a plurality of camera lens 901a value 901d be divided into first group and second group according to transmissivity.Be included in camera lens in second group and can have the color of the transmissivity higher than the transmissivity that is included in the camera lens in first group.
Specifically, for example, first group can comprise having the first green camera lens 901a in four camera lenses, have the second red camera lens 901b and have blue three-lens 901c, be included in four-barrel 901d in second group and can have the color of the transmissivity higher than green, red and blue transmissivity, that is grey.
Therefore, when a plurality of camera lens 901a to 901d have different colors, in a plurality of subimage transducer 902a to 902d, do not form the dichroic filter layer.
In addition, imageing sensor 902 is divided into respectively and the corresponding a plurality of subimage transducer 902a to 902d of a plurality of camera lens 901a to 901d, and by using a plurality of subimage transducer 902a to 902d to obtain color separated image.As shown in Figure 3, provide color separated image from camera model 301, and produce final image, then on display module 303, show based on color separated image by image generation module 302.The image generation module 302 that produces final image will not provide its detailed explanation with top described identical with reference to Fig. 3 to Fig. 8.
Next, with reference to Figure 10 and Figure 11 the method for recovering high pixel image processing is according to an embodiment of the invention described.
Figure 10 illustrates with the camera model with the structure shown in Fig. 4 A to use the first image generation module shown in Fig. 7 B to recover the flow chart of the method for high pixel image processing.
For convenience of explanation, suppose to be shown among a plurality of subimage transducer 402a to 402d that form imageing sensor 402 and form red filter, green filters, blue filter and grey filter respectively as Fig. 7 B.In addition, suppose to form imageing sensor 402, be formed for forming each of a plurality of subimage transducer 402a to 402d of imageing sensor 402 with 4 * 4 pixels (wide * height) with 8 * 8 pixels (wide * height).
At first, at operation S1001, concentrate from the light of predetermine one 705 reflections by four camera lens 401a to 401d.
At operation S1002,, send the light of concentrating by camera lens 401a to 401d by being included in the color filter among each the number of sub images transducer 402a to 402d that mates with camera lens 401a to 401d.
Consequently, at operation S1003, obtain a plurality of color separated image by each number of sub images transducer 402a to 402d.Here, the image of each subimage transducer 402a to 402d acquisition has the resolution of 1/4th (1/4) resolution of imageing sensor 402.That is to say that because the resolution of imageing sensor 402 is 8 * 8, therefore each in a plurality of color separated image that obtain by each number of sub images transducer 402a to 402d all has 4 * 4 resolution.
After operation S1003, at operation S1004, image generation module 302 checks whether the original color separated image 706 that obtains at operation S1003 is positioned at their assigned address.
As check result, depart from their assigned address if determine the original color separated image 706 that obtains, then at operation S1005, image generation module 302 is proofreaied and correct the position of original color separated image 706, thereby original color separated image 706 is by the assigned address that normally navigates to them.
Be positioned at their assigned address if determine the original color separated image 706 that obtains, then at operation S1006, image generation module 302 checks whether original color separated image 706 susceptibilitys that obtain at operation S1003 are consistent.
As check result, if determine that original color separated image 706 susceptibilitys that obtain are inconsistent, then at operation S1007, image generation module 302 is proofreaied and correct the inconsistent of susceptibility based on the susceptibility of the original color separated image with minimum sensitivity grade.
If determine the original color separated image 706 susceptibility unanimities of acquisition, then at operation S1008, image generation module 302 produces a plurality of original images based on the original color separated image 706 that obtains, rearrange the Pixel Information of the pixel on the same position of each original image, thereby can produce intermediate image 707 with resolution higher than the resolution of the original image of each color.
, at operation S1009, middle image removed mosaic, and, make the intermediate image sharpening of mosaic, to produce final image at operation S1010 thereafter.
Next, at operation S1011, by the final image of display module 303 display image generation modules 302 generations.
Figure 11 illustrates with the camera model with the structure shown in Fig. 4 A to use the second image generation module shown in Figure 8 to recover the flow chart of the method for high pixel image processing.
At first, the camera model 301 with the structure shown in Fig. 4 A provides the operation of a plurality of color separated image identical with operation S1001 to S1007 shown in Figure 10.At operation S1101, as shown in Figure 8, original image generation module 302a is divided into a plurality of pixel groups 303,304 and 305 with a plurality of original color separated image, and each pixel groups is formed by 2 * 2 virtual pixels (wide * height).
Then, at operation S1102, intermediate image generation module 302b produces first intermediate image 802 with resolution identical with as shown in Figure 8 imageing sensor 402.
Next, at operation S1103, intermediate image generation module 302b is mapped to the Pixel Information of the pixel on the same position in each original color separated image on the main pixel with the corresponding pixel groups of same position.
At operation S1104, intermediate image generation module 302b produces second intermediate image 806, and wherein, three colouring informations and two monochrome informations are mapped on main pixel 803a, the 804a and 805a of each pixel groups 803,804 and 805.
At operation S1105, intermediate image generation module 302b uses interpolation method that second intermediate image 806 is carried out interpolation.
After second intermediate image 806 was carried out interpolation, at operation S1106, final image generation module 302c carried out the sharpening of second intermediate image 806 that makes interpolation.
Consequently, at operation S1107, has high-resolution (promptly from color separated image generation with low resolution (that is the resolution of subimage transducer), the resolution of subimage transducer * 4) final image 809 then shows this final image 809 by display module 303.
Operating S1001 to operating S1007, except concentrating light by the color camera lens rather than by color filter, and obtain outside a plurality of original color separated image by the subimage transducer, use the high pixel image processing restoration methods shown in Figure 10 of camera model identical in essence with the high pixel image processing restoration methods of using the first image generation module shown in Fig. 7 B with structure shown in Figure 9.Similarly, as mentioned above, provide the operation of a plurality of color separated image identical because have the camera model 301 of the structure shown in Fig. 4 A, so use the high pixel image processing restoration methods shown in Figure 11 of camera model identical in essence with the high pixel image processing restoration methods of using the second image generation module shown in Figure 8 with structure shown in Figure 9 with operation S1001 to S1007 shown in Figure 10.
Generally speaking, use digital camera according to an embodiment of the invention, because the F number is identical can be realized and the identical image brightness of imageing sensor shown in Fig. 2 A, described digital camera comprises four camera lenses and four imageing sensors, in four camera lenses each all has camera lens size and the relative focal length of lacking relatively littler than the camera lens shown in Fig. 2 A, and each in four imageing sensors all has the pixel count that the unit picture element number than four mega-pixel image sensors shown in Fig. 2 A lacks, that is a mega pixel.In addition, shown in Fig. 2 A, use can both be carried out 4 width of cloth images that the subimage transducer of a mega pixel resolution imaging is taken by each shown in Fig. 4 A, can recover to have image with the image equal resolution of taking with the camera that can carry out four mega pixel resolution imagings by prior defined procedure.
Because digital camera has relatively little camera lens and relative short focal length according to an embodiment of the invention, therefore when keeping equal resolution, can realize miniaturization, the lightweight design of digital camera.
According to the method and apparatus of above-mentioned recovery high pixel image processing, can obtain following one or more effects.
Described method and apparatus provides such advantage: can use the miniaturization camera model to obtain high pixel image processing.
Since reduced to be installed in the size of the digital camera on the small-sized digital device, therefore can also the miniaturization digital device.
In addition, the present invention can recover high pixel image processing when reducing to be installed in the size of the digital camera on the small-sized digital device.
In addition, according to the equipment of recovery high pixel image processing of the present invention and method can be in the performance high pixel image processing calibrating optical deviation and susceptibility inconsistent easily.
The color filter that has different transmissivities by use is even the structure that does not change image sensor cell according to the equipment and the method for recovery high pixel image processing of the present invention also can realize high sensitive image sensing and low sensitivity image sensing simultaneously.
In addition, when design and installation during the small-sized digital device of digital camera, allow according to the equipment of recovery high pixel image processing of the present invention and method that more the multiselect item is available.
Although shown and described some embodiments of the present invention, but it should be appreciated by those skilled in the art, without departing from the principles and spirit of the present invention, can change these embodiments, scope of the present invention is limited by claim and equivalent thereof.

Claims (44)

1, a kind of equipment that recovers high pixel image processing comprises:
Camera model, comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of described a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of subimage transducers;
The original image generation module receives a plurality of original color separated image;
The intermediate image generation module rearranges the Pixel Information of the pixel on the same position of a plurality of original color separated image that the original image generation module provides, and produces the intermediate image with the resolution that is higher than each original color separated image; And
The final image generation module removes mosaic to middle image execution, carries out the intermediate image sharpening that makes mosaic, and produces final image.
2, equipment as claimed in claim 1, wherein, described a plurality of camera lens will concentrate on the assigned address of camera lens from the light of predetermine one reflection or concentrate on position based on the camera lens except predetermined camera lens a plurality of camera lenses of the pixel of the offset predetermined quantity of predetermined camera lens.
3, equipment as claimed in claim 1, wherein, when the original color separated image that obtains was not arranged in the assigned address of corresponding subimage transducer, the original image generation module was proofreaied and correct the position of original image.
4, equipment as claimed in claim 1, wherein, when the original color separated image susceptibility that obtains was inconsistent, the original image generation module was proofreaied and correct the sensitivity grade of the original image except the original image with minimum sensitivity grade based on the susceptibility of the original image with minimum sensitivity grade.
5, equipment as claimed in claim 1 wherein, is divided into first filtration zone and second filtration zone according to the transmissivity that scribbles color on it with color filter.
6, a kind of equipment that recovers high pixel image processing comprises:
Camera model, comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of described a plurality of camera lenses, each all comprises color filter described a plurality of subimage transducers, described color filter is divided into a plurality of color regions with different colours;
The original image generation module is divided into a plurality of pixel groups with a plurality of original color separated image;
The intermediate image generation module, the Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image with the resolution that is higher than each original color separated image; And
The final image generation module uses predetermined interpolation algorithm to recover intermediate image, carries out the intermediate image sharpening that makes reception, and produces final image.
7, equipment as claimed in claim 6, wherein, when the original color separated image that obtains was not arranged in the assigned address of corresponding subimage transducer, the original image generation module was proofreaied and correct the position of original image.
8, equipment as claimed in claim 6, wherein, when the original color separated image susceptibility that obtains was inconsistent, the original image generation module was proofreaied and correct the sensitivity grade of the original image except the original image with minimum sensitivity grade based on the susceptibility of the original image with minimum sensitivity grade.
9, equipment as claimed in claim 6, wherein, according to the transmissivity that scribbles color on it color filter is divided into first filtration zone and second filtration zone, and the transmissivity that is coated in the color on second filtration zone is higher than the transmissivity that is coated in the color on first filtration zone.
10, equipment as claimed in claim 6, wherein, each in a plurality of pixel groups all comprises and the corresponding a plurality of pixels of pattern of rows and columns of color filter.
11, equipment as claimed in claim 6, wherein, described Pixel Information comprises: monochrome information that provides with the corresponding subimage transducer of first filtration zone and the colouring information that provides with the corresponding subimage transducer of second filtration zone.
12, a kind of equipment that recovers high pixel image processing comprises:
Camera model, comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of described a plurality of color camera lenses, wherein, each in described a plurality of color camera lenses has monochrome, and obtains a plurality of original color separated image by described a plurality of subimage transducers;
The original image generation module receives a plurality of original color separated image;
The intermediate image generation module rearranges the Pixel Information of the pixel on the same position of a plurality of original color separated image that the original image generation module provides, and produces the intermediate image with the resolution that is higher than each original color separated image; And
The final image generation module removes mosaic to middle image execution, carries out the intermediate image sharpening that makes mosaic, and produces final image.
13, equipment as claimed in claim 12, wherein, described a plurality of camera lens will concentrate on the assigned address of camera lens from the light of predetermine one reflection or concentrate on position based on the camera lens except predetermined camera lens a plurality of camera lenses of the pixel of the offset predetermined quantity of predetermined camera lens.
14, equipment as claimed in claim 12, wherein, when the original color separated image that obtains was not arranged in the assigned address of corresponding subimage transducer, the original image generation module was proofreaied and correct the position of original image.
15, equipment as claimed in claim 12, wherein, when the original color separated image susceptibility that obtains was inconsistent, the original image generation module was proofreaied and correct the sensitivity grade of the original image except the original image with minimum sensitivity grade based on the susceptibility of the original image with minimum sensitivity grade.
16, equipment as claimed in claim 12, wherein, according to the transmissivity that scribbles color on it in a plurality of color camera lenses each is divided into first camera lens zone and the second camera lens zone, and the transmissivity that is coated in the color on the second camera lens zone is higher than the transmissivity that is coated in the color on the first camera lens zone.
17, a kind of equipment that recovers high pixel image processing comprises:
Camera model, comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of described a plurality of color camera lenses, wherein, each in described a plurality of color camera lenses has monochrome, and obtains a plurality of original color separated image by described a plurality of subimage transducers;
The original image generation module is divided into a plurality of pixel groups with a plurality of original color separated image;
The intermediate image generation module is mapped to each pixel with the corresponding pixel groups of described same position with the Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups, and produces intermediate image; And
The final image generation module uses predetermined interpolation algorithm to recover intermediate image, carries out the intermediate image sharpening that makes reception, and produces final image.
18, equipment as claimed in claim 17, wherein, when the original color separated image that obtains was not arranged in the assigned address of corresponding subimage transducer, the original image generation module was proofreaied and correct the position of original image.
19, equipment as claimed in claim 17, wherein, when the original color separated image susceptibility that obtains was inconsistent, the original image generation module was proofreaied and correct the sensitivity grade of the original image except the original image with minimum sensitivity grade based on the susceptibility of the original image with minimum sensitivity grade.
20, equipment as claimed in claim 17, wherein, according to the transmissivity that scribbles color on it in color camera lens each is divided into first camera lens zone and the second camera lens zone, and the transmissivity that is coated in the color on the second camera lens zone is higher than the transmissivity that is coated in the color on the first camera lens zone.
21, equipment as claimed in claim 17, wherein, each in a plurality of pixel groups all comprises and the corresponding a plurality of pixels of pattern of rows and columns of color filter.
22, equipment as claimed in claim 20, wherein, described Pixel Information comprises: monochrome information that provides with the subimage transducer of first camera lens zone coupling and the colouring information that provides with the subimage transducer of second camera lens zone coupling.
23, a kind of method of in camera model, recovering high pixel image processing, described camera model comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of described a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of camera lens, and described method comprises:
Obtain a plurality of original images by each of a plurality of subimage transducers;
Receive a plurality of original images, and produce a plurality of original color separated image;
The Pixel Information of the pixel on the same position of a plurality of original color separated image that provide from the original image generation module of camera model is provided, and produces intermediate image with the resolution that is higher than each original color separated image; And
Mosaic is removed in middle image execution, carry out the intermediate image sharpening that makes mosaic, and produce final image from the image of sharpening.
24, method as claimed in claim 23 also comprises: will concentrate on the assigned address of camera lens from the light of predetermine one reflection or concentrate on position based on the camera lens except predetermined camera lens a plurality of camera lenses of the pixel of the offset predetermined quantity of predetermined camera lens.
25, method as claimed in claim 24 wherein, when the original color separated image that obtains is not arranged in the assigned address of corresponding subimage transducer, produces a plurality of original color separated image and comprises the position of proofreading and correct original image.
26, method as claimed in claim 23, wherein, when the original color separated image susceptibility that obtains is inconsistent, produces a plurality of original color separated image and comprise the sensitivity grade of proofreading and correct the original image except original image based on the susceptibility of original image with minimum sensitivity grade with minimum sensitivity grade.
27, method as claimed in claim 23 also comprises: according to the transmissivity that scribbles color on it color filter is divided into first filtration zone and second filtration zone.
28, a kind of method of in camera model, recovering high pixel image processing, described camera model comprise a plurality of camera lenses and with the corresponding a plurality of subimage transducers of a plurality of camera lenses, each all comprises having monochromatic color filter described a plurality of camera lenses, described method comprises:
Obtain a plurality of original color separated image by each of subimage transducer;
The a plurality of original color separated image that obtains is divided into a plurality of pixel groups;
The Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image; And
Use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes interpolation, and produce final image from the image of sharpening.
29, method as claimed in claim 28 wherein, when the original color separated image that obtains is not arranged in the assigned address of corresponding subimage transducer, produces a plurality of original color separated image and comprises the position of proofreading and correct original image.
30, method as claimed in claim 28, also comprise: produce a plurality of original color separated image from the color separated image that receives, wherein, when the original color separated image susceptibility that obtains is inconsistent, produces a plurality of original color separated image and comprise the sensitivity grade of proofreading and correct the original image except original image based on the susceptibility of original image with minimum sensitivity grade with minimum sensitivity grade.
31, method as claimed in claim 28 also comprises: according to the transmissivity that scribbles color on it color filter is divided into first filtration zone and second filtration zone.
32, method as claimed in claim 28, wherein, each in a plurality of pixel groups all comprises and the corresponding a plurality of pixels of pattern of rows and columns of color filter.
33, method as claimed in claim 31, wherein, described Pixel Information comprises: monochrome information that provides with the subimage transducer of first filtration zone coupling and the colouring information that provides with the subimage transducer of second filtration zone coupling.
34, a kind of method of in camera model, recovering high pixel image processing, described camera model comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, each all comprises having monochromatic color filter described a plurality of color camera lens, and described method comprises:
Obtain a plurality of original color separated image by each of a plurality of subimage transducers;
Receive a plurality of original color separated image;
The Pixel Information of the pixel on the same position of a plurality of original color separated image that provide from the original image generation module of camera model is provided, and produces intermediate image with the resolution that is higher than each original color separated image; And
Mosaic is removed in middle image execution, carry out the intermediate image sharpening that makes mosaic, and produce final image from the image of sharpening.
35, method as claimed in claim 34 also comprises: will concentrate on the assigned address of camera lens from the light of predetermine one reflection or concentrate on position based on the camera lens except predetermined camera lens a plurality of camera lenses of the pixel of the offset predetermined quantity of predetermined camera lens with a plurality of camera lenses.
36, method as claimed in claim 34 wherein, when the original color separated image that obtains is not arranged in the assigned address of corresponding subimage transducer, produces a plurality of original color separated image and comprises the position of proofreading and correct original image.
37, method as claimed in claim 34, also comprise: produce a plurality of original color separated image from the color separated image that receives, wherein, when the original color separated image susceptibility that obtains is inconsistent, produces a plurality of original color separated image and comprise the sensitivity grade of proofreading and correct the original image except original image based on the susceptibility of original image with minimum sensitivity grade with minimum sensitivity grade.
38, method as claimed in claim 34, also comprise: according to the transmissivity that scribbles color on it in a plurality of color camera lenses each is divided into first camera lens zone and the second camera lens zone, and the transmissivity that is coated in the color on the second camera lens zone is higher than the transmissivity that is coated in the color on the first camera lens zone.
39, a kind of method of in camera model, recovering high pixel image processing, described camera model comprise a plurality of color camera lenses and with the corresponding a plurality of subimage transducers of a plurality of color camera lenses, each all has monochrome described a plurality of color camera lenses, described method comprises:
Obtain a plurality of original color separated image by a plurality of subimage transducers;
A plurality of original color separated image are divided into a plurality of pixel groups;
The Pixel Information that is divided into the pixel on the same position in a plurality of original color separated image of a plurality of pixel groups is mapped to each pixel with the corresponding pixel groups of described same position, and produces intermediate image; And
Use predetermined interpolation algorithm to recover intermediate image, carry out the intermediate image sharpening that makes interpolation, and produce final image.
40, method as claimed in claim 39, also comprise: produce a plurality of original color separated image from the color separated image that receives, wherein, when the original color separated image that obtains is not arranged in the assigned address of corresponding subimage transducer, produces a plurality of original color separated image and comprise the position of proofreading and correct original image.
41, method as claimed in claim 39, wherein, when the original color separated image susceptibility that obtains is inconsistent, produces a plurality of original color separated image and comprise the sensitivity grade of proofreading and correct the original image except original image based on the susceptibility of original image with minimum sensitivity grade with minimum sensitivity grade.
42, method as claimed in claim 39, also comprise: according to the transmissivity that scribbles color on it in a plurality of color camera lenses each is divided into first camera lens zone and the second camera lens zone, and the transmissivity that is coated in the color on the second camera lens zone is higher than the transmissivity that is coated in the color on the first camera lens zone.
43, method as claimed in claim 39, wherein, each in a plurality of pixel groups all comprises the corresponding a plurality of pixels of pattern of rows and columns with the color filter of camera model.
44, method as claimed in claim 42, wherein, described Pixel Information comprises: monochrome information that provides with the subimage transducer of first camera lens zone coupling and the colouring information that provides with the subimage transducer of second camera lens zone coupling.
CNA2007101126736A 2006-06-26 2007-06-26 Apparatus and method for recovering high pixel image processing Pending CN101098399A (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
KR1020060057660 2006-06-26
KR20060057660 2006-06-26
KR1020060105348 2006-10-27

Publications (1)

Publication Number Publication Date
CN101098399A true CN101098399A (en) 2008-01-02

Family

ID=39011894

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA2007101126736A Pending CN101098399A (en) 2006-06-26 2007-06-26 Apparatus and method for recovering high pixel image processing

Country Status (2)

Country Link
KR (1) KR100781552B1 (en)
CN (1) CN101098399A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102457683A (en) * 2010-11-03 2012-05-16 索尼公司 Lens and colour filter arrangement, super-resolution camera system and method
CN102547080A (en) * 2010-12-31 2012-07-04 联想(北京)有限公司 Image pick-up module and information processing equipment comprising same
CN102778201A (en) * 2011-05-13 2012-11-14 索尼公司 Image processing device, image processing method, and program
US8326069B2 (en) 2008-06-30 2012-12-04 Intel Corporation Computing higher resolution images from multiple lower resolution images
CN104350743A (en) * 2012-05-11 2015-02-11 英特尔公司 Systems, methods, and computer program products for compound image demosaicing and warping
CN105847770A (en) * 2015-01-15 2016-08-10 展讯通信(上海)有限公司 Image processing chip and caching method for image data in chip
CN107256533A (en) * 2013-04-25 2017-10-17 联发科技股份有限公司 The method for handling mosaic image
CN108769635A (en) * 2018-05-30 2018-11-06 Oppo(重庆)智能科技有限公司 Filming apparatus, electronic equipment and image acquiring method

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100935392B1 (en) * 2008-01-29 2010-01-06 중앙대학교 산학협력단 Aperture, image pick-up device, and apparatus and method for digital autofocusing based on color channel shift
KR101086501B1 (en) * 2008-11-26 2011-11-25 주식회사 하이닉스반도체 Image data processing method, image sensor, and integrated circuit
KR101216668B1 (en) 2011-03-17 2012-12-28 중앙대학교 산학협력단 Apparatus and method for estimating object information of image taken by multiple color filter aperture camera
KR20140125984A (en) * 2013-04-19 2014-10-30 삼성전자주식회사 Image Processing Method, Electronic Device and System
JP6467787B2 (en) 2014-05-27 2019-02-13 株式会社リコー Image processing system, imaging apparatus, image processing method, and program
KR102653106B1 (en) 2018-10-18 2024-04-01 삼성전자주식회사 Apparatus and method for processing image
KR102632474B1 (en) * 2019-02-11 2024-02-01 삼성전자주식회사 Pixel array included in image sensor and image sensor including the same
KR20230111379A (en) 2022-01-18 2023-07-25 삼성전자주식회사 Image sensor and imaging device
KR20240047294A (en) 2022-10-04 2024-04-12 오영식 Method, apparatus, and system for improving the resolution of pictures taken multiple times by a moving camera

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100748325B1 (en) * 2001-06-28 2007-08-09 매그나칩 반도체 유한회사 Image sensor
KR20050103027A (en) * 2004-04-23 2005-10-27 매그나칩 반도체 유한회사 Image sensor having notch filter and method for fabrication of the same
KR100640972B1 (en) * 2004-07-15 2006-11-02 동부일렉트로닉스 주식회사 CMOS Image sensor and Method for fabricating of the same

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8326069B2 (en) 2008-06-30 2012-12-04 Intel Corporation Computing higher resolution images from multiple lower resolution images
US8538202B2 (en) 2008-06-30 2013-09-17 Intel Corporation Computing higher resolution images from multiple lower resolution images
CN102457683B (en) * 2010-11-03 2017-07-04 索尼公司 Lens and color filter arrangement, super-resolution camera system and method
US10158832B2 (en) 2010-11-03 2018-12-18 Sony Corporation Lens and color filter arrangement, super-resolution camera system and method
CN102457683A (en) * 2010-11-03 2012-05-16 索尼公司 Lens and colour filter arrangement, super-resolution camera system and method
US9813680B2 (en) 2010-11-03 2017-11-07 Sony Corporation Lens and color filter arrangement, super-resolution camera system and method
CN102547080A (en) * 2010-12-31 2012-07-04 联想(北京)有限公司 Image pick-up module and information processing equipment comprising same
CN102547080B (en) * 2010-12-31 2015-07-29 联想(北京)有限公司 Camera module and comprise the messaging device of this camera module
CN102778201A (en) * 2011-05-13 2012-11-14 索尼公司 Image processing device, image processing method, and program
CN104350743A (en) * 2012-05-11 2015-02-11 英特尔公司 Systems, methods, and computer program products for compound image demosaicing and warping
CN107256533B (en) * 2013-04-25 2021-01-22 联发科技股份有限公司 Method for processing mosaic image
CN107256533A (en) * 2013-04-25 2017-10-17 联发科技股份有限公司 The method for handling mosaic image
CN105847770B (en) * 2015-01-15 2017-08-08 展讯通信(上海)有限公司 The caching method of view data in picture processing chip and the chip
CN105847770A (en) * 2015-01-15 2016-08-10 展讯通信(上海)有限公司 Image processing chip and caching method for image data in chip
CN108769635A (en) * 2018-05-30 2018-11-06 Oppo(重庆)智能科技有限公司 Filming apparatus, electronic equipment and image acquiring method
CN108769635B (en) * 2018-05-30 2020-01-21 Oppo(重庆)智能科技有限公司 Shooting device, electronic equipment and image acquisition method

Also Published As

Publication number Publication date
KR100781552B1 (en) 2007-12-05

Similar Documents

Publication Publication Date Title
CN101098399A (en) Apparatus and method for recovering high pixel image processing
EP1871091A2 (en) Camera Module
JP3884617B2 (en) Optoelectronic camera
US7453510B2 (en) Imaging device
CN204697179U (en) There is the imageing sensor of pel array
CN101854488B (en) Solid-state imaging device, signal processing method thereof and image capturing apparatus
CN102037717B (en) Capturing and processing of images using monolithic camera array with hetergeneous imagers
CN101098400A (en) Digital camera module
EP1874034A2 (en) Apparatus and method of recovering high pixel image
CN106341628A (en) Image Sensor Having Wide Dynamic Range, Pixel Circuit Of The Image Sensor, And Operating Method Of The Image Sensor
US6275206B1 (en) Block mapping based up-sampling method and apparatus for converting color images
US20130278802A1 (en) Exposure timing manipulation in a multi-lens camera
CN101106641B (en) Method and apparatus for image processing
JP5853166B2 (en) Image processing apparatus, image processing method, and digital camera
JP2006033493A (en) Imaging apparatus
EP3461306B1 (en) Image sensor method and apparatus equipped with multiple contiguous infrared filter elements
CN103597811B (en) Take the image-capturing element of Three-dimensional movable image and planar moving image and be equipped with its image capturing device
CN103999458A (en) Image pickup device
US8928785B2 (en) Imaging device and storage medium storing an imaging program
CN105100558A (en) Wafer level camera having movable color filter grouping
TW200417251A (en) Solid photographing element and digital camera
US20150116562A1 (en) Compact spacer in multi-lens array module
WO2023098552A1 (en) Image sensor, signal processing method and apparatus, camera module, and electronic device
JP2005064828A (en) Solid-state image pickup device, drive method therefor, and image pickup device
US11418737B2 (en) Image signal processor and electronic device and electronic system including the same

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20080102