TW201839487A - Image Sensor, Image Processing Method, Imaging Device And Electronic Device - Google Patents
Image Sensor, Image Processing Method, Imaging Device And Electronic Device Download PDFInfo
- Publication number
- TW201839487A TW201839487A TW107100949A TW107100949A TW201839487A TW 201839487 A TW201839487 A TW 201839487A TW 107100949 A TW107100949 A TW 107100949A TW 107100949 A TW107100949 A TW 107100949A TW 201839487 A TW201839487 A TW 201839487A
- Authority
- TW
- Taiwan
- Prior art keywords
- photosensitive
- focus
- pixel
- photosensitive unit
- output
- Prior art date
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 72
- 238000003672 processing method Methods 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 claims description 55
- 238000001514 detection method Methods 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 20
- 230000000694 effects Effects 0.000 description 15
- 230000006870 function Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000003086 colorant Substances 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14625—Optical elements or arrangements associated with the device
- H01L27/14627—Microlenses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/71—Circuitry for evaluating the brightness variation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/702—SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14603—Special geometry or disposition of pixel-elements, address-lines or gate-electrodes
- H01L27/14607—Geometry of the photosensitive area
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/1462—Coatings
- H01L27/14621—Colour filter arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/53—Control of the integration time
- H04N25/533—Control of the integration time by using differing integration times for different sensor regions
- H04N25/534—Control of the integration time by using differing integration times for different sensor regions depending on the spectral component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
- H04N25/611—Correction of chromatic aberration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
- H04N25/704—Pixels specially adapted for focusing, e.g. phase difference pixel sets
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Power Engineering (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- Electromagnetism (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Focusing (AREA)
- Automatic Focus Adjustment (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
Description
本申請涉及電子技術領域,尤其涉及一種影像感測器、影像處理方法、成像裝置和行動終端。The present application relates to the field of electronic technology, and in particular, to an image sensor, an image processing method, an imaging device, and a mobile terminal.
隨著技術的不斷更新,越來越多的廠家開始使用16M-4M結構的影像感測器。該影像感測器在暗光條件下,通過4M模式輸出影像,可以提高訊噪比,改善雜訊表現。但是,在暗光條件下,4M模式對焦速度慢。With the continuous updating of technology, more and more manufacturers have begun to use 16M-4M structure image sensors. The image sensor outputs images in 4M mode under low light conditions, which can improve the signal-to-noise ratio and improve noise performance. However, in low-light conditions, the 4M mode focuses slowly.
本申請的目的旨在至少在一定程度上解決上述的技術問題之一。The purpose of this application is to solve at least one of the above-mentioned technical problems.
為了解決上述問題,本申請第一方面提出一種影像感測器的影像處理方法,其中,影像感測器包括:感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元,影像感測器具有第一輸出模式和第二輸出模式,該影像處理方法包括以下步驟:檢測當前環境光強度;當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式;在第一輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值;根據第一輸出值和第二輸出值進行對焦控制;在第一輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。In order to solve the above problems, the first aspect of the present application proposes an image processing method of an image sensor, wherein the image sensor includes: a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a filter unit array. The above microlens array, wherein the photosensitive unit array includes a focus sensitive unit and a non-focus sensitive unit, the image sensor has a first output mode and a second output mode, and the image processing method includes the following steps: detecting the current ambient light intensity ; When the current ambient light intensity is less than the preset intensity, control the image sensor to enter the first output mode; in the first output mode, control the photosensitive unit array to enter the focus mode; read the output value of a part of the photosensitive pixels in the focus photosensitive unit As the first output value; read the output value of another part of the photosensitive pixels in the focus sensing unit as the second output value; perform focus control according to the first output value and the second output value; in the first output mode, control the photosensitive The cell array enters the imaging mode; the exposure of the photosensitive cell array is controlled to generate the first The video output mode.
本申請的影像感測器的影像處理方法,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on the image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit The array includes a structure of a focus sensitive unit and a non-focus sensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first The output mode uses the output values of some photosensitive pixels and the output values of other photosensitive pixels in the focus sensing unit to perform focus control, which can effectively improve the focusing speed.
為了解決上述問題,本申請第二方面提出一種影像感測器,該影像感測器包括感光單元陣列;設置在感光單元陣列上的濾光單元陣列;位於濾光單元陣列之上的微透鏡陣列;其中,感光單元陣列包括對焦感光單元和非對焦感光單元,影像感測器具有第一輸出模式和第二輸出模式。In order to solve the above problems, the second aspect of the present application proposes an image sensor, which includes a photosensitive unit array; a filter unit array disposed on the photosensitive unit array; and a microlens array located above the filter unit array. Wherein, the photosensitive unit array includes a focus sensitive unit and a non-focus sensitive unit, and the image sensor has a first output mode and a second output mode.
本申請的影像感測器,基於包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,為提高對焦速度提供硬體基礎。The image sensor of the present application is based on a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focusing photosensitive unit and a non-focusing unit. The structure of the photosensitive unit provides a hardware basis for increasing the focusing speed.
本申請的協力廠商面實施例提出一種成像裝置,該成像裝置包括:上述實施例提出的影像感測器;檢測模組,用於檢測當前環境光強度;控制模組,用於當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式;在第一輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值;根據第一輸出值和第二輸出值進行對焦控制;在第一輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。The third-party embodiment of the present application proposes an imaging device. The imaging device includes: the image sensor provided in the above embodiment; a detection module for detecting the current ambient light intensity; and a control module for the current ambient light When the intensity is less than the preset intensity, the image sensor is controlled to enter the first output mode; in the first output mode, the photosensitive unit array is controlled to enter the focus mode; the output value of a part of the photosensitive pixels in the focus photosensitive unit is read as the first output Value; read the output value of another photosensitive pixel in the focus photosensitive unit as the second output value; perform focus control according to the first output value and the second output value; in the first output mode, control the photosensitive unit array to enter the imaging mode ; Controlling the photosensitive unit array to perform exposure to generate an image in a first output mode.
本申請的成像裝置,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The imaging device of the present application is based on an image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focus photosensitive unit and The structure of the non-focusing photosensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first output mode. Focus control is performed on the output value of some photosensitive pixels and the output value of another photosensitive pixel in the unit, which can effectively improve the focusing speed.
本申請第四方面還提出一種行動終端,該行動終端包括殼體、處理器、記憶體、電路板和電源電路,其中,電路板安置在殼體圍成的空間內部,處理器和記憶體設置在電路板上;電源電路,用於為行動終端的各個電路或裝置供電;記憶體用於儲存可執行程式碼;處理器通過讀取記憶體中儲存的可執行程式碼來運行與可執行程式碼對應的程式,以用於執行上述的影像感測器的影像處理方法。The fourth aspect of the present application also proposes a mobile terminal, which includes a casing, a processor, a memory, a circuit board, and a power circuit, wherein the circuit board is disposed in a space surrounded by the casing, and the processor and the memory are provided. On the circuit board; power circuit for powering various circuits or devices of the mobile terminal; memory for storing executable program code; the processor runs and executes the program by reading the executable program code stored in the memory Code corresponding program for executing the image processing method of the image sensor.
本申請實施例的行動終端,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The mobile terminal according to the embodiment of the present application is based on an image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focus sensor. The structure of the unit and the non-focus photosensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first output mode. Focus control is performed on the output value of some photosensitive pixels and the output value of another photosensitive pixel in the focusing photosensitive unit, which can effectively improve the focusing speed.
本申請第五方面提出一種影像感測器的影像處理方法,其中,影像感測器包括:感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,微透鏡陣列包括第一微透鏡和第二微透鏡,一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元,其中,N為正整數,影像感測器具有第一輸出模式和第二輸出模式,該影像處理方法包括以下步驟:檢測當前環境光強度;當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式;在第一輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值;根據第一輸出值和第二輸出值進行對焦控制;在第一輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。The fifth aspect of the present application provides an image processing method of an image sensor, wherein the image sensor includes: a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens on the filter unit array. An array, wherein the microlens array includes a first microlens and a second microlens, a first microlens covers a focus sensitive unit, and N * N second microlenses cover a non-focus sensitive unit, where N is a positive integer The image sensor has a first output mode and a second output mode. The image processing method includes the following steps: detecting the current ambient light intensity; and controlling the image sensor to enter the first output when the current ambient light intensity is less than a preset intensity. Mode; in the first output mode, control the photosensitive unit array to enter the focus mode; read the output value of a part of the photosensitive pixels in the focus photosensitive unit as the first output value; read the output value of another part of the photosensitive pixels in the focus photosensitive unit and As the second output value; focus control according to the first output value and the second output value; in the first output mode, the control sense The light unit array enters an imaging mode; the photosensitive unit array is controlled to perform exposure to generate an image in a first output mode.
本申請的影像感測器的影像處理方法,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on a structure in which a first micro lens of the image sensor covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit. The image sensor There are two output modes, the first output mode and the second output mode. When the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output value of some photosensitive pixels in the focus photosensitive unit and the other The output value of the light-sensitive pixel is used to focus control, which can effectively increase the focusing speed.
為了解決上述問題,本申請第六方面提出一種影像感測器,該影像感測器包括感光單元陣列;設置在感光單元陣列上的濾光單元陣列;位於濾光單元陣列之上的微透鏡陣列;其中,微透鏡陣列包括第一微透鏡和第二微透鏡,一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元,其中,N為正整數,影像感測器具有第一輸出模式和第二輸出模式。In order to solve the above problem, a sixth aspect of the present application proposes an image sensor, the image sensor includes a photosensitive unit array; a filter unit array disposed on the photosensitive unit array; and a microlens array on the filter unit array. ; Wherein the microlens array includes a first microlens and a second microlens, a first microlens covers a focusing photosensitive unit, and N * N second microlenses cover a non-focus photosensitive unit, where N is a positive integer, The image sensor has a first output mode and a second output mode.
本申請的影像感測器,基於一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,為提高畫面品質和提高對焦速度提供硬體基礎。The image sensor of the present application is based on a structure in which a first micro lens covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit, and provides a hardware foundation for improving picture quality and focusing speed.
本申請的第七方面實施例提出一種成像裝置,該成像裝置包括:影像感測器;檢測模組,用於檢測當前環境光強度;控制模組,用於當當前環境光強度小於預設強度時,控制該影像感測器進入第一輸出模式;在第一輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值;根據第一輸出值和第二輸出值進行對焦控制;在第一輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。An embodiment of the seventh aspect of the present application provides an imaging device, the imaging device includes: an image sensor; a detection module for detecting the current ambient light intensity; a control module for when the current ambient light intensity is less than a preset intensity Control the image sensor to enter the first output mode; in the first output mode, control the photosensitive unit array to enter the focus mode; read the output value of a part of the photosensitive pixels in the focus photosensitive unit as the first output value; read Focus the output value of the other photosensitive pixels in the photosensitive unit as the second output value; perform focus control according to the first output value and the second output value; in the first output mode, control the photosensitive unit array to enter the imaging mode; control the photosensitive unit The array is exposed to produce an image in a first output mode.
本申請的成像裝置,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The imaging device of the present application is based on a structure in which a first micro lens of an image sensor covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit. The image sensor has two output modes. In the first output mode and the second output mode, when the ambient light intensity is less than a preset intensity, the image sensor enters the first output mode, and uses the output value of a part of the photosensitive pixels in the focus photosensitive unit and the output value of another part of the photosensitive pixels. Focus control can effectively increase the focus speed.
本申請第八方面還提出一種行動終端,該行動終端包括殼體、處理器、記憶體、電路板和電源電路,其中,電路板安置在殼體圍成的空間內部,處理器和記憶體設置在電路板上;電源電路,用於為行動終端的各個電路或裝置供電;記憶體用於儲存可執行程式碼;處理器通過讀取記憶體中儲存的可執行程式碼來運行與可執行程式碼對應的程式,以用於執行上述影像感測器的影像處理方法。The eighth aspect of the present application also proposes a mobile terminal, which includes a housing, a processor, a memory, a circuit board, and a power circuit, wherein the circuit board is disposed in a space surrounded by the housing, and the processor and the memory are provided. On the circuit board; power circuit for powering various circuits or devices of the mobile terminal; memory for storing executable program code; the processor runs and executes the program by reading the executable program code stored in the memory Code corresponding program for executing the image processing method of the image sensor.
本申請實施例的行動終端,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The mobile terminal according to the embodiment of the present application is based on a structure in which a first micro lens of an image sensor covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit. The image sensor has two outputs. Mode, the first output mode and the second output mode. When the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output value of some photosensitive pixels in the focus photosensitive unit and the output of another photosensitive pixel. Value, focus control can effectively increase the focus speed.
本申請附加的方面和優點將在下面的描述中部分給出,部分將從下面的描述中變得明顯,或通過本申請的實踐瞭解到。Additional aspects and advantages of the present application will be given in part in the following description, part of which will become apparent from the following description, or be learned through practice of the present application.
下面詳細描述本申請的實施例,該實施例的示例在附圖中示出,其中自始至終相同或類似的標號表示相同或類似的元件或具有相同或類似功能的元件。下面通過參考附圖描述的實施例是示例性的,旨在用於解釋本申請,而不能理解為對本申請的限制。An embodiment of the present application is described in detail below. An example of the embodiment is shown in the accompanying drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are intended to explain the present application, and should not be construed as limiting the present application.
下面參考附圖描述本申請實施例的影像感測器、影像處理方法、成像裝置和行動終端。The following describes an image sensor, an image processing method, an imaging device, and a mobile terminal according to embodiments of the present application with reference to the drawings.
第1圖是根據本申請的一實施例的影像感測器的立體結構圖。FIG. 1 is a three-dimensional structure diagram of an image sensor according to an embodiment of the present application.
如第1圖所示,該影像感測器100包括:感光單元陣列10、濾光單元陣列20和微透鏡陣列30。As shown in FIG. 1, the image sensor 100 includes a photosensitive unit array 10, a filter unit array 20 and a microlens array 30.
其中,濾光單元陣列20設置在感光單元陣列10上,微透鏡陣列30位於濾光單元陣列20之上。感光單元陣列10包括對焦感光單元和非對焦感光單元。對焦感光單元和非對焦感光單元均為感光單元。The filter unit array 20 is disposed on the photosensitive unit array 10, and the microlens array 30 is located above the filter unit array 20. The photosensitive unit array 10 includes a focus photosensitive unit and a non-focus photosensitive unit. The focus photosensitive unit and the non-focus photosensitive unit are both photosensitive units.
本申請實施例中的影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式。The image sensor in the embodiment of the present application has two output modes, a first output mode and a second output mode.
基於第1圖所示的影像感測器的結構,下面對本申請實施例提出的影像感測器的影像處理方法進行說明。 如第2圖所示,該影像感測器的影像處理方法包括:Based on the structure of the image sensor shown in FIG. 1, the image processing method of the image sensor provided in the embodiment of the present application will be described below. As shown in FIG. 2, the image processing method of the image sensor includes:
S21,檢測當前環境光強度。S21: Detect the current ambient light intensity.
通過手機對物體進行拍照時,攝像頭對準拍攝的物體後,檢測當前拍照環境的光強度。When taking a picture of an object with a mobile phone, after the camera is pointed at the photographed object, the light intensity of the current photographing environment is detected.
S22,當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式。S22. When the current ambient light intensity is less than a preset intensity, control the image sensor to enter a first output mode.
將檢測的當前拍照環境的光強度與預設強度進行比較,如果當前環境光強度小於預設強度如300勒克斯,即被拍攝物體處於弱光環境,則控制影像感測器進入第一輸出模式。其中,第一輸出模式可以為4M輸出格式,該格式下輸出的照片同一感光單元內感光像素的顏色相同,解析度相對較低,但訊噪比高。The detected light intensity of the current photographing environment is compared with a preset intensity. If the current ambient light intensity is less than a preset intensity, such as 300 lux, that is, the subject is in a low-light environment, the image sensor is controlled to enter a first output mode. The first output mode may be a 4M output format. The photos output in this format have the same color of the photosensitive pixels in the same photosensitive unit, the resolution is relatively low, but the signal-to-noise ratio is high.
S23,在第一輸出模式下,控制感光單元陣列進入對焦模式。S23. In the first output mode, control the photosensitive unit array to enter a focus mode.
對準要拍攝的物體後,點擊螢幕進行對焦,這時感光單元陣列進入第一輸出模式下的對焦模式。After aligning the object to be shot, tap the screen to focus. At this time, the photosensitive unit array enters the focus mode in the first output mode.
S24,讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值。S24. Read the output value of a part of the photosensitive pixels in the focus photosensitive unit and use it as the first output value.
S25,讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值。S25. Read the output value of another photosensitive pixel in the focus photosensitive unit and use it as the second output value.
S26,根據第一輸出值和第二輸出值進行對焦控制。S26. Perform focus control according to the first output value and the second output value.
在相關技術中,一般地,為了實現PDAF( Phase Detection Auto Focus,相位檢測自動對焦),通常利用影像感測器內相鄰且成對設置的感光像素結構設計(又稱遮蔽像素,masked pixels,遮蔽像素結構相較于普通感光像素結構更加複雜,通常需要改變普通感光像素本身結構或者在感光像素結構上單獨增加一光線遮擋部,以使得射向遮蔽像素上的複數方向光線中特定方向上的光線不能到達遮蔽像素的感光部分,而除了特定方向之外的光線則可以到達遮蔽像素的感光部分,換言之,遮蔽像素通常成對、鄰近且對稱的設置,成對設置的遮蔽像素用於對複數方向的光線進行分離),將射向成對設置的遮蔽像素上的複數方向上的成像光束分離成比如左和右兩部分,通過對比左、右兩部分光線成像後的相位差(即通過採集成對設置的遮蔽像素的輸出)來計算鏡頭需要行動的距離。In related technologies, in general, in order to implement PDAF (Phase Detection Auto Focus), adjacent pixel pairs in the image sensor are usually used to design the photosensitive pixel structure (also known as masked pixels, The masking pixel structure is more complicated than the ordinary photosensitive pixel structure. Usually, the structure of the ordinary photosensitive pixel itself needs to be changed or a light shielding portion is separately added to the photosensitive pixel structure, so that a specific direction of the complex direction light rays on the masking pixel is emitted. The light cannot reach the light-sensitive portion of the obscuring pixel, and light other than a specific direction can reach the light-sensitive portion of the obscuring pixel. In other words, the obscuring pixels are usually arranged in pairs, adjacent and symmetrical, and the obscuring pixels arranged in pairs are used for complex numbers. To separate the imaging beams in the complex direction on the masked pixels that are set in pairs into, for example, the left and right parts. By comparing the phase difference between the left and right parts of the light (ie, by collecting The output of occluded pixels set in pairs) to calculate the lens needs action Distance.
而在本申請的實施例中,通過不同方向的光線訊號對比可以獲取成像影像的相位差資訊,進一步地根據相位差資訊獲得拍攝物體的距離資訊,為相位對焦和景深資訊測試提供資料基礎。顯然,本申請實施例中,只需要利用微透鏡單元、濾光單元和對焦感光單元的配合設計,就可以實現相位對焦的檢測,而無需改變普通感光像素本身結構或者在感光像素結構上單獨增加一光線遮擋部,相位對焦檢測的實現方式也更加簡單。In the embodiment of the present application, phase difference information of the imaging image can be obtained through comparison of light signals in different directions, and further distance information of the photographed object is obtained based on the phase difference information, which provides a data basis for the phase focus and depth of field information testing. Obviously, in the embodiment of the present application, only the coordinated design of the microlens unit, the filter unit and the focus sensitive unit can be used to realize the phase focus detection without changing the structure of the ordinary photosensitive pixel or adding a separate structure on the photosensitive pixel. A light shielding part, the implementation of phase focus detection is also simpler.
下面以對焦感光單元包括2*2個感光像素為例,說明第一輸出模式下的對焦控制方法。The following describes the focus control method in the first output mode with the focus photosensitive unit including 2 * 2 photosensitive pixels as an example.
如第3圖所示,讀取對焦感光單元中左側的兩個感光像素的輸出值Gp30 和Gp32 作為第一輸出值,讀取另外一部分感光像素即右側兩個感光像素的輸出值Gp31 和Gp33 作為第二輸出值。As shown in FIG. 3, read the output values Gp 30 and Gp 32 of the two photosensitive pixels on the left side of the focus photosensitive unit as the first output values, and read the output values Gp 31 of the other photosensitive pixels, that is, the two photosensitive pixels on the right side. And Gp 33 as the second output value.
在獲取第一輸出值和第二輸出值之後,求出對焦感光單元左側兩個感光像素的輸出值Gp30 與Gp32 之間的和,即Gp1 =Gp30 +Gp32 ,產生第一相位值Gp1 。同樣,求出右側兩個感光像素的輸出值Gp31 與Gp33 之間的和,即Gp2 =Gp31 +Gp33 ,產生第二相位值Gp2 。從而,可以獲取Gp1 和Gp2 之間的相位差資訊,進而可以將相位差資訊轉換為對焦距離資訊,根據對焦距離資訊調節鏡頭的位置實現相位對焦,相位對焦檢測的實現方式也更加簡單。After obtaining the first output value and the second output value, find the sum between the output values Gp 30 and Gp 32 of the two photosensitive pixels on the left side of the focus photosensitive unit, that is, Gp 1 = Gp 30 + Gp 32 to generate the first phase Value Gp 1 . Similarly, a sum between the output values Gp 31 and Gp 33 of the two right-side photosensitive pixels is calculated, that is, Gp 2 = Gp 31 + Gp 33 to generate a second phase value Gp 2 . Therefore, the phase difference information between Gp 1 and Gp 2 can be obtained, and then the phase difference information can be converted into focus distance information, and the position of the lens can be adjusted according to the focus distance information to achieve phase focus. The implementation of phase focus detection is also simpler.
在本申請的另一實施例中,如第4圖所示,將對焦感光單元中2*2個感光像素分為上側和下側兩個部分,可讀取對焦感光單元中上側的兩個感光像素的輸出值作為第一輸出值,讀取下側的兩個感光像素的輸出值作為第二輸出值。在獲取第一輸出值和第二輸出值之後,根據第一輸出值產生第一相位值,根據第二輸出值產生第二相位值,再根據第一相位值和第二相位值之間的相位差資訊進行對焦控制。In another embodiment of the present application, as shown in FIG. 4, the 2 * 2 photosensitive pixels in the focus photosensitive unit are divided into two parts, an upper side and a lower side, and the two upper photosensitive units in the focus photosensitive unit can be read. The output value of the pixel is taken as the first output value, and the output values of the two photosensitive pixels on the lower side are read as the second output value. After obtaining the first output value and the second output value, a first phase value is generated according to the first output value, a second phase value is generated according to the second output value, and then a phase between the first phase value and the second phase value is generated. Poor information for focus control.
在本申請的又一實施例中,如第5圖所示,以對焦感光單元的兩條對角線將2*2個感光像素分為兩部分,可讀取左上角的感光像素與右下角的感光像素作為第一輸出值,讀取左下角的感光像素與右上角的感光像素作為第二輸出值。在獲取第一輸出值和第二輸出值之後,根據第一輸出值產生第一相位值,根據第二輸出值產生第二相位值,再根據第一相位值和第二相位值之間的相位差資訊進行對焦控制。In another embodiment of the present application, as shown in FIG. 5, 2 * 2 photosensitive pixels are divided into two parts by two diagonal lines of the focusing photosensitive unit, and the upper left corner and the lower right corner can be read. The light-sensitive pixels are used as the first output value, and the light-sensitive pixels in the lower left corner and the light-sensitive pixels in the upper right corner are read as the second output value. After obtaining the first output value and the second output value, a first phase value is generated according to the first output value, a second phase value is generated according to the second output value, and then a phase between the first phase value and the second phase value is generated. Poor information for focus control.
在本申請實施例中,將對焦感光單元2*2個感光像素左右兩側感光像素的輸出值分別作為第一輸出值和第二輸出值,可以檢測左右方向的相位差資訊;將對焦感光單元2*2個感光像素上下兩側感光像素的輸出值分別作為第一輸出值和第二輸出值,可以檢測上下方向的相位差資訊;將對焦感光單元兩條對角線上的感光像素的輸出值分別作為第一輸出值和第二輸出值,可檢測斜向的相位差資訊。In the embodiment of the present application, the output values of the left and right photosensitive pixels of the 2 * 2 photosensitive pixels of the focusing photosensitive unit are respectively used as the first output value and the second output value, and the phase difference information in the left and right directions can be detected; The output values of the upper and lower photosensitive pixels of 2 * 2 photosensitive pixels are respectively used as the first output value and the second output value, which can detect the phase difference information in the vertical direction; the output values of the photosensitive pixels on the two diagonal lines of the focusing photosensitive unit As the first output value and the second output value, phase difference information in the oblique direction can be detected.
本申請實施例提出的影像感測器處於第一輸出模式時的對焦控制方法,通過讀取對焦感光單元中不同部分的感光像素的輸出值,獲取不同角度入射光線的相位資訊,進行不同方向相位資訊檢測,提高了暗光下的對焦速度,對焦更準確。In the focus control method provided by the embodiment of the present application when the image sensor is in the first output mode, the phase information of incident light at different angles is obtained by reading the output values of the photosensitive pixels in different parts of the focusing photosensitive unit, and the phases in different directions are performed. Information detection improves the focusing speed in dark light and makes focusing more accurate.
S27,在第一輸出模式下,控制感光單元陣列進入成像模式; 在第一輸出模式下,根據對焦感光單元中兩部分感光像素輸出值完成對焦控制後,控制感光單元陣列進入成像模式。S27. In the first output mode, control the photosensitive unit array to enter the imaging mode. In the first output mode, after completing the focus control according to the output values of the two photosensitive pixels in the focusing photosensitive unit, control the photosensitive unit array to enter the imaging mode.
S28,控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。S28. Control the photosensitive unit array to perform exposure to generate an image in a first output mode.
如第6圖所示,藍色B0 、綠色G1 、綠色G3 和紅色R4 組成了一拜耳RGB陣列。對對焦感光單元和非對焦感光單元進行曝光,讀取對焦感光單元的輸出值Gp30 、Gp31 、Gp32 和Gp33 ,非對焦感光單元的輸出值B00 、B01 、B02 、B03 、Gb10 、Gb11 、Gb12 、Gb13 等等。As shown in Figure 6, blue B 0 , green G 1 , green G 3 and red R 4 form a Bayer RGB array. Expose the focus sensor unit and the non-focus sensor unit, read the output values of the focus sensor unit Gp 30 , Gp 31 , Gp 32, and Gp 33 , and the output values of the non-focus sensor unit B 00 , B 01 , B 02 , B 03 , Gb 10 , Gb 11 , Gb 12 , Gb 13, and so on.
讀取對焦感光單元和非對焦感光單元的輸出值之後,將同一對焦感光單元的2*2個感光像素的輸出值Gp30 、Gp31 、Gp32 和Gp33 相加,即Gp30 +Gp31 + Gp32 +Gp33 =G3 ,得到對焦感光單元的像素值G3 。將同一非對焦感光單元的2*2個感光像素輸出值B00 、B01 、B02 、B03 相加,即B00 +B01 +B02 +B03 =B0 ,得到該非對焦感光單元的像素值B0 。同理,可得到非對焦感光單元的像素值,綠色G1 =Gb10 +Gb11 +Gb12 +Gb13 ,紅色R4 =R40 +R41 +R42 +R43 等等。根據對焦感光單元和非對焦感光單元的像素值產生第一輸出模式下的影像。After reading the output values of the focus sensitive unit and the non-focus sensitive unit, the output values of 2 * 2 photosensitive pixels of the same focus sensitive unit are added Gp 30 , Gp 31 , Gp 32, and Gp 33 , that is, Gp 30 + Gp 31 + Gp 32 + Gp 33 = G 3 to obtain the pixel value G 3 of the focus photosensitive unit. Add 2 * 2 photosensitive pixel output values B 00 , B 01 , B 02 , and B 03 of the same non-focus photosensitive unit, that is, B 00 + B 01 + B 02 + B 03 = B 0 to obtain the non-focus photosensitive unit Pixel value B 0 . Similarly, the pixel values of the out-of-focus photosensitive unit can be obtained, green G 1 = Gb 10 + Gb 11 + Gb 12 + Gb 13 , red R 4 = R 40 + R 41 + R 42 + R 43 and so on. An image in the first output mode is generated according to the pixel values of the focus sensitive unit and the non-focus sensitive unit.
本申請實施例的成像方法,將感光單元內的M*M個感光像素的輸出值之和作為該感光單元的像素值,根據對焦感光單元和非對焦感光單元的像素值產生影像,可有效地提升影像的成像靈敏度和訊噪比。The imaging method in the embodiment of the present application uses the sum of the output values of M * M photosensitive pixels in the photosensitive unit as the pixel value of the photosensitive unit, and generates an image according to the pixel values of the focused photosensitive unit and the non-focused photosensitive unit, which can effectively Improve image sensitivity and signal-to-noise ratio.
本申請的影像感測器的影像處理方法,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on the image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit The array includes a structure of a focus sensitive unit and a non-focus sensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first The output mode uses the output values of some photosensitive pixels and the output values of other photosensitive pixels in the focus sensing unit to perform focus control, which can effectively improve the focusing speed.
上述實施例是在拍照環境光強度較低的情況下,影像感測器進入第一輸出模式時對焦方法和成像方法。下面詳細介紹在強光環境下影像感測器的影像處理方法。The above embodiment is a focusing method and an imaging method when the image sensor enters the first output mode in a case where the ambient light intensity of the photograph is low. The image processing method of the image sensor in a strong light environment is described in detail below.
通過手機對物體進行拍照時,攝像頭對準拍攝的物體後,檢測當前拍照環境的光強度。將檢測的拍照環境的光強度與預設強度進行比較,如果拍照環境的光強度大於預設強度如300勒克斯,即拍照環境為強光環境時,則控制影像感測器進入第二輸出模式。When taking a picture of an object with a mobile phone, after the camera is pointed at the photographed object, the light intensity of the current photographing environment is detected. The detected light intensity of the photographing environment is compared with a preset intensity. If the light intensity of the photographing environment is greater than a preset intensity such as 300 lux, that is, when the photographing environment is a strong light environment, the image sensor is controlled to enter a second output mode.
其中,第二輸出模式可以為16M輸出格式,該格式下輸出的照片同一感光單元內的感光像素的顏色不同,相比第一輸出模式輸出的照片具有更多的細節,解析度較高。The second output mode may be an 16M output format. The photos output in this format have different colors of the photosensitive pixels in the same photosensitive unit. Compared with the photos output in the first output mode, the photos have more details and higher resolution.
對準要拍攝的物體後,點擊螢幕進行對焦,這時感光單元陣列進入對焦模式。After aiming at the object to be shot, tap the screen to focus. At this time, the sensor unit array enters the focus mode.
在本申請的實施例中,第二輸出模式下的對焦控制方法與第一輸出模式下的對焦控制方法類似,在此不再贅述。 如第7圖所示,第二輸出模式下成像方法包括:In the embodiment of the present application, the focus control method in the second output mode is similar to the focus control method in the first output mode, and details are not described herein again. As shown in Figure 7, the imaging method in the second output mode includes:
S71,在第二輸出模式下,控制感光單元陣列進入成像模式。S71. In the second output mode, control the photosensitive cell array to enter an imaging mode.
在第二輸出模式下,通過對焦感光單元中兩部分感光像素輸出值獲得相位差資訊,根據相位差資訊完成對焦控制後,控制感光單元陣列進入成像模式。In the second output mode, phase difference information is obtained by focusing on the output values of two photosensitive pixels in the photosensitive unit. After the focus control is completed according to the phase difference information, the photosensitive unit array is controlled to enter the imaging mode.
S72,控制感光單元陣列進行曝光,以產生第二輸出模式下的影像。S72. Control the photosensitive unit array to perform exposure to generate an image in the second output mode.
在本申請的實施例中,控制感光單元進入成像模式後,對感光單元陣列進行曝光,同時讀取感光單元陣列的輸出值。In the embodiment of the present application, after the photosensitive unit is controlled to enter the imaging mode, the photosensitive unit array is exposed, and the output value of the photosensitive unit array is read at the same time.
在讀取感光單元陣列的輸出值後,可通過插值還原演算法獲得感光單元陣列的像素值,再根據像素值產生影像。After reading the output value of the photosensitive cell array, the pixel values of the photosensitive cell array can be obtained by interpolation reduction algorithm, and then an image is generated based on the pixel values.
在本申請的實施例中,插值還原演算法包括:判斷當前像素的顏色與關聯像素的顏色是否相同,如第8圖所示,右邊圖形中R22 為當前像素,關聯像素為左邊影像中的R22 。In the embodiment of the present application, the interpolation restoration algorithm includes: determining whether the color of the current pixel and the color of the associated pixel are the same. As shown in FIG. 8, R 22 in the right figure is the current pixel and the associated pixel is the left image. R 22 .
在當前像素的顏色與關聯像素的顏色相同時,將關聯像素的像素值作為當前像素的像素值。舉例而言,如第8圖所示,由於當前像素R22 顏色為紅色與關聯像素的顏色相同,因此,當前像素的值與關聯像素的值相同即R22 = R22 。When the color of the current pixel is the same as the color of the associated pixel, the pixel value of the associated pixel is used as the pixel value of the current pixel. For example, as shown in FIG. 8, since the color of the current pixel R 22 is red and the color of the associated pixel is the same, the value of the current pixel is the same as the value of the associated pixel, that is, R 22 = R 22 .
在當前像素的顏色與關聯像素的顏色不同時,根據關聯像素單元的像素值通過插值方式計算當前像素的像素值。其中,影像像素單元的陣列包括關聯像素單元,關聯像素單元的顏色與當前像素顏色相同且與當前像素相鄰。When the color of the current pixel is different from the color of the associated pixel, the pixel value of the current pixel is calculated by interpolation according to the pixel value of the associated pixel unit. The array of image pixel units includes associated pixel units, and the color of the associated pixel units is the same as the current pixel color and is adjacent to the current pixel.
舉例而言,如第8圖所示,當前像素Gr23 的顏色為綠色,關聯像素R23 的顏色為紅色,當前像素的顏色與關聯像素的顏色不相同,可根據關聯像素單元的像素值通過插值方式計算當前像素的像素值Gr23 的值,其中,當前像素Gr23 的關聯像素單元為G1 、G3 、G5 、G7 。For example, as shown in FIG. 8, the color of the current pixel Gr 23 is green and the color of the associated pixel R 23 is red. The color of the current pixel and the color of the associated pixel are different, and can be passed according to the pixel value of the associated pixel unit. The interpolation method calculates the value of the pixel value Gr 23 of the current pixel, where the associated pixel units of the current pixel Gr 23 are G 1 , G 3 , G 5 , G 7 .
在本申請的一實施例中,根據感光單元接收的光的顏色不同,插值還原演算法可分為兩種情況,具體示例如下。In an embodiment of the present application, the interpolation restoration algorithm can be divided into two cases according to different colors of light received by the photosensitive unit, and specific examples are as follows.
示例一:以接收到紅光的感光單元為例。如第8圖所示,當前像素R22 的顏色為紅色,與關聯像素R22 顏色也為紅色,因此R22 =R22 。當前像素Gr23 的顏色為綠色與關聯像素R23 的顏色不相同,因此可根據關聯像素單元G1 、G3 、G5 和 G7 的像素值通過插值方式得到當前像素Gr23 的值,如公式一所示。 公式一: Example 1: Take a photosensitive unit that receives red light as an example. As shown in FIG. 8, the color of the current pixel R 22 is red, and the color of the associated pixel R 22 is also red, so R 22 = R 22 . The color of the current pixel Gr 23 is green and the color of the associated pixel R 23 is different. Therefore, the value of the current pixel Gr 23 can be obtained by interpolation according to the pixel values of the associated pixel units G 1 , G 3 , G 5, and G 7 , such as Equation one is shown. Formula one:
當前像素Gb32 的顏色為綠色與關聯像素R32 的顏色不相同,可根據關聯像素單元G1 、G3 、G5 和 G7 的像素值,通過公式二的所示的插值方式得到當前像素Gb32 的像素值。 公式二: The color of the current pixel Gb 32 is green and the color of the associated pixel R 32 is different. According to the pixel values of the associated pixel units G 1 , G 3 , G 5, and G 7 , the current pixel can be obtained by the interpolation method shown in Equation 2. Gb 32 pixel value. Formula two:
當前像素B33 的顏色為藍色,與關聯像素R33 的顏色不相同,可根據關聯像素單元B0 、B2 、B6 和B8 的像素值通過插值演算法得到當前像素B33 的值,如公式三所示。 公式三: The color of the current pixel B 33 is blue, which is different from the color of the associated pixel R 33. The value of the current pixel B 33 can be obtained by interpolation algorithm based on the pixel values of the associated pixel units B 0 , B 2 , B 6, and B 8 . As shown in Equation 3. Formula three:
需要說明的是,接收到藍光的感光單元的插值還原演算法與接收到紅光的感光單元的插值還原演算法相同,此處不再贅述。It should be noted that the interpolation restoration algorithm of the photosensitive unit that receives the blue light is the same as the interpolation restoration algorithm of the photosensitive unit that receives the red light, and will not be repeated here.
示例二:以接收到綠光的感光單元為例。如第9圖所示,當前像素R22 顏色為紅色,與關聯像素Gr22 的顏色不同,可根據關聯像素單元R3 和R5 的像素值通過插值還原得到當前像素R22 的值,如公式四所示。 公式四: Example two: Take a photosensitive unit that receives green light as an example. As shown in Figure 9, the current pixel R 22 is red, which is different from the color of the associated pixel Gr 22. The current pixel R 22 value can be obtained by interpolation based on the pixel values of the associated pixel units R 3 and R 5 as shown in the formula Shown in four. Formula four:
當前像素Gr23 的顏色為綠色與關聯像素Gr23 的顏色相同,則當前像素的值與關聯像素的值相同,即Gr23 =Gr23 。當前像素Gb32 的顏色為綠色與關聯像素的顏色相同,當前像素的值與關聯像素的值相同,即Gr32 =Gr32 。當前像素B33 的顏色為藍色與關聯像素Gr33 的顏色不同,可根據關聯像素單元B1 、B7 的像素值通過插值還原得到當前像素B33 的值,如公式五所示。 公式五: The color of the current pixel Gr 23 is green and the color of the associated pixel Gr 23 is the same, so the value of the current pixel is the same as the value of the associated pixel, that is, Gr 23 = Gr 23 . The color of the current pixel Gb 32 is green and the color of the associated pixel is the same, and the value of the current pixel is the same as the value of the associated pixel, that is, Gr 32 = Gr 32 . The color of the current pixel B 33 is blue and the color of the associated pixel Gr 33 is different. The value of the current pixel B 33 can be obtained by interpolation based on the pixel values of the associated pixel units B 1 and B 7 , as shown in Formula 5. Formula five:
通過上述插值還原演算法獲得感光單元陣列的像素值之後,根據感光單元陣列的像素值產生第二輸出模式下的影像。After obtaining the pixel values of the photosensitive unit array through the interpolation restoration algorithm, an image in the second output mode is generated according to the pixel values of the photosensitive unit array.
本申請實施例提出的影像處理方法,影像感測器可根據拍照環境的光強度選擇對應的資料輸出模式,在環境光強度小於預設強度時,以第一輸出模式產生影像,在環境光強度大於預設強度時,以第二輸出模式產生影像。According to the image processing method provided in the embodiment of the present application, the image sensor can select a corresponding data output mode according to the light intensity of the photographing environment. When the ambient light intensity is less than a preset intensity, an image is generated in the first output mode. When the intensity is greater than the preset intensity, an image is generated in the second output mode.
本申請的影像感測器的影像處理方法,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on the image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit The array includes a structure of a focus sensitive unit and a non-focus sensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first The output mode uses the output values of some photosensitive pixels and the output values of other photosensitive pixels in the focus sensing unit to perform focus control, which can effectively improve the focusing speed.
為了增加對焦感光單元的進光量,提高對焦速度,微透鏡陣列30中的微透鏡大小可以不相同,以獲取大微透鏡對應的感光像素的相位資訊,實現快速對焦。In order to increase the amount of light entering the focusing photosensitive unit and improve the focusing speed, the size of the microlenses in the microlens array 30 may be different, so as to obtain the phase information of the photosensitive pixels corresponding to the large microlenses, and achieve fast focusing.
如第10圖所示,微透鏡陣列30包括兩種大小不同的微透鏡,第一微透鏡31和第二微透鏡32。其中,一第一微透鏡31覆蓋一對焦感光單元11,N*N個第二微透鏡32覆蓋一非對焦感光單元12,其中,N為正整數。對焦感光單元11和非對焦感光單元12均包括M*M個感光像素110,其中,M為正整數。As shown in FIG. 10, the microlens array 30 includes two kinds of microlenses with different sizes, a first microlens 31 and a second microlens 32. Among them, a first microlens 31 covers a focusing photosensitive unit 11, and N * N second microlenses 32 cover a non-focusing photosensitive unit 12, where N is a positive integer. The focus sensitive unit 11 and the non-focus sensitive unit 12 each include M * M photosensitive pixels 110, where M is a positive integer.
在本申請的一實施例中,N和M的取值可以相同。以N和M的取值為2為例,如第11圖所示,影像感測器100中,2*2個第二微透鏡覆蓋一非對焦感光單元,對焦感光單元和非對焦感光單元均包括2*2個感光像素。In an embodiment of the present application, the values of N and M may be the same. Taking the values of N and M as 2 as an example, as shown in FIG. 11, in the image sensor 100, 2 * 2 second microlenses cover a non-focusing photosensitive unit, and both the focusing and non-focusing photosensitive units are Includes 2 * 2 photosensitive pixels.
當N和M的取值相同時,影像感測器的對焦方法和成像方法與上述實施例中的對焦方法和成像方法類似,在此不再贅述。When the values of N and M are the same, the focusing method and imaging method of the image sensor are similar to the focusing method and imaging method in the above embodiment, and details are not described herein again.
當然,N與M的取值也可以不相同。N與M的取值不同時,影像感測器的對焦方法和成像方法與上述實施例中的對焦方法和成像方法類似,在此不再贅述。Of course, the values of N and M can also be different. When the values of N and M are different, the focusing method and imaging method of the image sensor are similar to the focusing method and imaging method in the foregoing embodiment, and details are not described herein again.
需要說明的是,在本申請的實施例中,微透鏡的形狀可以是如第11圖所示的圓形,也可以是矩形,或者其他形狀。It should be noted that, in the embodiment of the present application, the shape of the microlens may be a circle as shown in FIG. 11, a rectangle, or other shapes.
為了使對焦感光單元上聚集更多的光線,在本申請的再一實施例中,微透鏡陣列中一第一微透鏡覆蓋N*N個矩形微透鏡,其中,N為正整數。具體示例,如第12圖所示,該影像感測器一第一微透鏡覆蓋2*2個矩形微透鏡33。In order to focus more light on the focusing photosensitive unit, in another embodiment of the present application, a first microlens in the microlens array covers N * N rectangular microlenses, where N is a positive integer. As a specific example, as shown in FIG. 12, a first microlens of the image sensor covers 2 * 2 rectangular microlenses 33.
也就是說,該實施例中的影像感測器,其微透鏡陣列30包括第一微透鏡31、第二微透鏡32和矩形微透鏡33。通過雙層形狀大小不同的兩種微透鏡覆蓋對焦感光單元,可以使更多的光線聚集到對焦感光單元上,提高暗光環境下的拍攝效果和對焦速度。That is, the microlens array 30 of the image sensor in this embodiment includes a first microlens 31, a second microlens 32, and a rectangular microlens 33. By covering the focus sensitive unit with two kinds of microlenses with different shapes and sizes, more light can be focused on the focus sensitive unit, which improves the shooting effect and focusing speed in a dark light environment.
在本申請的一實施例中,如第13圖所示,微透鏡陣列30可包括水平中心線和垂直中心線,微透鏡陣列包括複數第一微透鏡。複數第一微透鏡包括設置在水平中心線的第一組第一微透鏡和設置在垂直中心線的第二組第一微透鏡。In an embodiment of the present application, as shown in FIG. 13, the micro lens array 30 may include a horizontal center line and a vertical center line, and the micro lens array includes a plurality of first micro lenses. The plurality of first microlenses includes a first group of first microlenses disposed on a horizontal centerline and a second group of first microlenses disposed on a vertical centerline.
為了提高對焦速度,在本申請的另一實施例中,如第14圖所示,複數第一微透鏡還可包括設置在微透鏡陣列四個邊線上的第三組第一微透鏡。In order to improve the focusing speed, in another embodiment of the present application, as shown in FIG. 14, the plurality of first microlenses may further include a third group of first microlenses disposed on the four edges of the microlens array.
從第14圖可以看出,由第一微透鏡覆蓋的對焦感光單元,即圖中的Gp,在整個影像感測器中零散分佈,占總像素個數的3%~5%,影像感測器中心區域Gp分佈更密集,邊緣區域分佈較為稀疏,優先獲取畫面中心的相位資訊,在不影響畫質的情況下,有效提高對焦速度。It can be seen from Figure 14 that the focusing photosensitive unit covered by the first microlens, that is, Gp in the figure, is scattered throughout the image sensor, accounting for 3% to 5% of the total number of pixels. The Gp distribution in the center area of the camera is more dense, and the distribution in the edge area is relatively sparse. The phase information in the center of the screen is preferentially obtained, and the focusing speed is effectively improved without affecting the image quality.
由於透鏡密度越大,透鏡的折射率越大,聚光能力越強,為了使中心區域的對焦感光單元聚集較多的光線,以提升對焦速度和拍攝效果。在本申請的一實施例中,可使第一組第一微透鏡和第二組第一微透鏡的透鏡密度大於第三組第一微透鏡的透鏡密度,從而使中心區域的對焦感光單元的進光量相對邊緣較大,進而提升對焦速度和拍攝效果。As the density of the lens is greater, the refractive index of the lens is greater, and the light-gathering ability is stronger. In order to make the focusing photosensitive unit in the central area gather more light, the focusing speed and the shooting effect are improved. In an embodiment of the present application, the lens density of the first group of the first microlenses and the second group of the first microlenses may be greater than the lens density of the third group of the first microlenses, so that the focus of the central area of the photosensitive unit The amount of incoming light is relatively large, which improves the focus speed and shooting effect.
在本申請的一實施例中,在基於第14圖所示的對焦感光單元的分佈基礎上,當影像感測器處於第二輸出模式下,在對感光單元陣列進行曝光,讀取對焦感光單元和非對焦感光單元的輸出值時,需要對對焦感光單元Gp的感光像素的值進行處理。In an embodiment of the present application, based on the distribution of the focus sensitive units shown in FIG. 14, when the image sensor is in the second output mode, the photosensitive unit array is exposed to read the focus sensitive units. When the output values of the out-of-focus photosensitive unit and the out-of-focus photosensitive unit are processed, the values of the photosensitive pixels of the in-focus photosensitive unit Gp need to be processed.
由於影像感測器中間部分光線強,四周光線較弱,因此影像感測器中間部分的對焦感光單元Gp內感光像素的值是可以直接使用的,離影像感測器中心點越遠,對焦感光單元Gp內感光像素的值的可信度越低,可逐漸被周邊感光像素的值取代。Because the light in the middle part of the image sensor is strong and the surrounding light is weak, the value of the photosensitive pixel in the focus sensor unit Gp in the middle part of the image sensor can be used directly. The farther away from the center point of the image sensor, the focus sensor is. The lower the reliability of the value of the photosensitive pixel in the unit Gp, the more gradually it can be replaced by the value of the surrounding photosensitive pixel.
在本申請的一實施例中,根據對焦感光單元Gp離影像感測器的中心點的距離對對焦感光單元內感光像素的值進行處理。在一具體實施例中,如第15圖所示,當對焦感光單元Gp與影像感測器中心點的距離小於r1時,也就是說,對焦感光單元處於影像感測器的中間部分時,可認為此時對焦感光單元Gp內感光像素的值與非對焦感光單元Gr的值是等價的。如第16圖所示,Gr30 =Gp30 ,Gr31 =Gp31 ,Gr32 =Gp32 ,Gr33 =Gp33 。In an embodiment of the present application, the value of the photosensitive pixel in the focus photosensitive unit is processed according to the distance of the focus photosensitive unit Gp from the center point of the image sensor. In a specific embodiment, as shown in FIG. 15, when the distance between the focus sensing unit Gp and the center point of the image sensor is less than r1, that is, when the focus sensing unit is located in the middle portion of the image sensor, It is considered that the value of the photosensitive pixel in the focus photosensitive unit Gp and the value of the non-focus photosensitive unit Gr are equivalent at this time. As shown in Figure 16, Gr 30 = Gp 30 , Gr 31 = Gp 31 , Gr 32 = Gp 32 , and Gr 33 = Gp 33 .
當對焦感光單元Gp與影像感測器中心點的距離大於r2時,對焦感光單元Gp內感光像素的像素值可通過周邊感光像素插值計算得到。When the distance between the focus sensing unit Gp and the center point of the image sensor is greater than r2, the pixel values of the photosensitive pixels in the focus sensing unit Gp can be calculated by the surrounding photosensitive pixel interpolation.
當對焦感光單元Gp與影像感測器中心點的距離大於r1小於r2時,可以距離為加權權重,通過將距離小於r1的對焦感光單元的感光像素的像素值與距離大於r2的通過插值計算得到的感光像素的像素值進行加權平均,計算得到該對焦感光單元內感光像素的像素值。When the distance between the focus sensing unit Gp and the center point of the image sensor is greater than r1 and less than r2, the distance can be a weighted weight. The pixel value of the photosensitive pixel of the focus sensing unit that is less than r1 and the distance greater than r2 are calculated by interpolation The weighted average of the pixel values of the light-sensitive pixels is calculated to obtain the pixel values of the light-sensitive pixels in the focusing light-sensing unit.
通過上述方法對對焦感光單元Gp的相位資訊進行處理後,可獲得感光單元陣列的輸出值。根據獲得的感光單元陣列的輸出值,通過插值還原演算法可獲得感光單元陣列的像素值。其中,插值還原演算法與上述實施例中的插值還原演算法相同,在此不再贅述。After the phase information of the focus photosensitive unit Gp is processed by the above method, the output value of the photosensitive unit array can be obtained. According to the obtained output value of the photosensitive unit array, the pixel value of the photosensitive unit array can be obtained through an interpolation reduction algorithm. The interpolation restoration algorithm is the same as the interpolation restoration algorithm in the foregoing embodiment, and details are not described herein again.
通過上述插值還原演算法獲得感光單元陣列的像素值之後,根據感光單元陣列的像素值可產生第二輸出模式下的影像。After the pixel values of the photosensitive unit array are obtained through the interpolation restoration algorithm, an image in the second output mode can be generated according to the pixel values of the photosensitive unit array.
本申請實施例提出的方案,在進行插值還原之前,根據第一微透鏡的空間位置,對對焦感光單元內感光像素的值進行處理,與簡單的丟棄對焦感光單元處的相位資訊,通過周邊像素補償的方案相比,提高了影像的清晰度和拍攝效果。In the solution proposed in the embodiment of the present application, before performing interpolation and restoration, based on the spatial position of the first microlens, the value of the photosensitive pixel in the focus sensitive unit is processed, and the phase information at the focus sensitive unit is simply discarded and the surrounding pixels are passed. Compared with the compensation scheme, the image clarity and shooting effect are improved.
為達上述目的,本申請實施例還提出一種感測器。In order to achieve the above object, an embodiment of the present application further provides a sensor.
如第1圖所示,該影像感測器100包括:感光單元陣列10、濾光單元陣列20和微透鏡陣列30。As shown in FIG. 1, the image sensor 100 includes a photosensitive unit array 10, a filter unit array 20 and a microlens array 30.
其中,濾光單元陣列20設置在感光單元陣列10上,微透鏡陣列30位於濾光單元陣列20之上。感光單元陣列10包括對焦感光單元和非對焦感光單元。對焦感光單元和非對焦感光單元均為感光單元。The filter unit array 20 is disposed on the photosensitive unit array 10, and the microlens array 30 is located above the filter unit array 20. The photosensitive unit array 10 includes a focus photosensitive unit and a non-focus photosensitive unit. The focus photosensitive unit and the non-focus photosensitive unit are both photosensitive units.
本申請實施例的影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式。The image sensor in the embodiment of the present application has two output modes, a first output mode and a second output mode.
為了增加對焦感光單元的進光量,提高對焦速度,微透鏡陣列30中的微透鏡大小可以不相同,以獲取大微透鏡對應的感光像素的相位資訊,實現快速對焦。In order to increase the amount of light entering the focusing photosensitive unit and improve the focusing speed, the size of the microlenses in the microlens array 30 may be different, so as to obtain the phase information of the photosensitive pixels corresponding to the large microlenses, and achieve fast focusing.
如第10圖所示,微透鏡陣列30包括兩種大小不同的微透鏡,第一微透鏡31和第二微透鏡32。其中,一第一微透鏡31覆蓋一對焦感光單元11,N*N個第二微透鏡32覆蓋一非對焦感光單元12,其中,N為正整數。對焦感光單元11和非對焦感光單元12均包括M*M個感光像素110,其中,M為正整數。As shown in FIG. 10, the microlens array 30 includes two kinds of microlenses with different sizes, a first microlens 31 and a second microlens 32. Among them, a first microlens 31 covers a focusing photosensitive unit 11, and N * N second microlenses 32 cover a non-focusing photosensitive unit 12, where N is a positive integer. The focus sensitive unit 11 and the non-focus sensitive unit 12 each include M * M photosensitive pixels 110, where M is a positive integer.
在本申請的一實施例中,N和M的取值可以相同。以N和M的取值為2為例,如第11圖所示,影像感測器100中,2*2個第二微透鏡覆蓋一非對焦感光單元,對焦感光單元和非對焦感光單元均包括2*2個感光像素。當然,N與M的取值也可以不相同。In an embodiment of the present application, the values of N and M may be the same. Taking the values of N and M as 2 as an example, as shown in FIG. 11, in the image sensor 100, 2 * 2 second microlenses cover a non-focusing photosensitive unit, and both the focusing and non-focusing photosensitive units are Includes 2 * 2 photosensitive pixels. Of course, the values of N and M can also be different.
需要說明的是,在本申請的實施例中,微透鏡的形狀可以是如第11圖所示的圓形,也可以是矩形,或者其他形狀。It should be noted that, in the embodiment of the present application, the shape of the microlens may be a circle as shown in FIG. 11, a rectangle, or other shapes.
為了使對焦感光單元上聚集更多的光線,在本申請的再一實施例中,微透鏡陣列中一第一微透鏡覆蓋N*N個矩形微透鏡,其中,N為正整數。具體示例,如第12圖所示,該影像感測器一第一微透鏡覆蓋2*2個矩形微透鏡33。In order to focus more light on the focusing photosensitive unit, in another embodiment of the present application, a first microlens in the microlens array covers N * N rectangular microlenses, where N is a positive integer. As a specific example, as shown in FIG. 12, a first microlens of the image sensor covers 2 * 2 rectangular microlenses 33.
也就是說,該實施例中的影像感測器,其微透鏡陣列30包括第一微透鏡31、第二微透鏡32和矩形微透鏡33。通過雙層形狀大小不同的兩種微透鏡覆蓋對焦感光單元,可以使更多的光線聚集到對焦感光單元上,提高暗光環境下的拍攝效果和對焦速度。That is, the microlens array 30 of the image sensor in this embodiment includes a first microlens 31, a second microlens 32, and a rectangular microlens 33. By covering the focus sensitive unit with two kinds of microlenses with different shapes and sizes, more light can be focused on the focus sensitive unit, which improves the shooting effect and focusing speed in a dark light environment.
在本申請的一實施例中,如第13圖所示,微透鏡陣列30可包括水平中心線和垂直中心線,微透鏡陣列包括複數第一微透鏡。複數第一微透鏡包括設置在水平中心線的第一組第一微透鏡和設置在垂直中心線的第二組第一微透鏡。In an embodiment of the present application, as shown in FIG. 13, the micro lens array 30 may include a horizontal center line and a vertical center line, and the micro lens array includes a plurality of first micro lenses. The plurality of first microlenses includes a first group of first microlenses disposed on a horizontal centerline and a second group of first microlenses disposed on a vertical centerline.
為了提高對焦速度,在本申請的另一實施例中,如第14圖所示,複數第一微透鏡還可包括設置在微透鏡陣列四個邊線上的第三組第一微透鏡。In order to improve the focusing speed, in another embodiment of the present application, as shown in FIG. 14, the plurality of first microlenses may further include a third group of first microlenses disposed on the four edges of the microlens array.
從第14圖可以看出,由第一微透鏡覆蓋的對焦感光單元,即圖中的Gp,在整個影像感測器中零散分佈,占總像素個數的3%~5%,影像感測器中心區域Gp分佈更密集,邊緣區域分佈較為稀疏,優先獲取畫面中心的相位資訊,在不影響畫質的情況下,有效提高對焦速度。It can be seen from Figure 14 that the focusing photosensitive unit covered by the first microlens, that is, Gp in the figure, is scattered throughout the image sensor, accounting for 3% to 5% of the total number of pixels. The Gp distribution in the center area of the camera is more dense, and the distribution in the edge area is relatively sparse. The phase information in the center of the screen is preferentially obtained, and the focusing speed is effectively improved without affecting the image quality.
由於透鏡密度越大,透鏡的折射率越大,聚光能力越強,為了使中心區域的對焦感光單元聚集較多的光線,以提升對焦速度和拍攝效果。在本申請的一實施例中,可使第一組第一微透鏡和第二組第一微透鏡的透鏡密度大於第三組第一微透鏡的透鏡密度,從而使中心區域的對焦感光單元的進光量相對邊緣較大,進而提升對焦速度和拍攝效果。As the density of the lens is greater, the refractive index of the lens is greater, and the light-gathering ability is stronger. In order to make the focusing photosensitive unit in the central area gather more light, the focusing speed and the shooting effect are improved. In an embodiment of the present application, the lens density of the first group of the first microlenses and the second group of the first microlenses may be greater than the lens density of the third group of the first microlenses, so that the focus of the central area of the photosensitive unit The amount of incoming light is relatively large, which improves the focus speed and shooting effect.
本申請的影像感測器,基於包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,為提高對焦速度提供硬體基礎。The image sensor of the present application is based on a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focusing photosensitive unit and a non-focusing unit. The structure of the photosensitive unit provides a hardware basis for increasing the focusing speed.
為達上述目的,下面對本申請再一方面實施例的成像裝置進行說明。To achieve the above purpose, an imaging device according to another embodiment of the present application is described below.
第17圖是根據本申請的一實施例的成像裝置的框圖,如第17圖所示,該成像裝置1700包括上述方面的影像感測器1710、檢測模組1720和控制模組1730。FIG. 17 is a block diagram of an imaging device according to an embodiment of the present application. As shown in FIG. 17, the imaging device 1700 includes an image sensor 1710, a detection module 1720, and a control module 1730 in the above aspects.
控制模組1730用於當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式;在第一輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值;根據第一輸出值和第二輸出值進行對焦控制;在第一輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。The control module 1730 is used to control the image sensor to enter the first output mode when the current ambient light intensity is less than the preset intensity; in the first output mode, control the photosensitive unit array to enter the focus mode; read a part of the focus photosensitive unit The output value of the photosensitive pixel is used as the first output value; the output value of another photosensitive pixel in the focus sensing unit is read as the second output value; focus control is performed according to the first output value and the second output value; at the first output In the mode, the photosensitive unit array is controlled to enter an imaging mode; the photosensitive unit array is controlled to perform exposure to generate an image in a first output mode.
控制模組1730具體用於:根據第一輸出值產生第一相位值;根據第二輸出值產生第二相位值;根據第一相位值和第二相位值進行對焦控制。The control module 1730 is specifically configured to generate a first phase value according to a first output value; generate a second phase value according to a second output value; and perform focus control according to the first phase value and the second phase value.
控制模組1730具體用於控制對焦感光單元和非對焦感光單元進行曝光,並讀取對焦感光單元和非對焦感光單元的輸出值;將同一對焦感光單元的M*M個感光像素或同一非對焦感光單元的M*M個感光像素的輸出值相加以得到對焦感光單元和非對焦感光單元的像素值從而產生第一輸出模式下的影像。The control module 1730 is specifically configured to control the focus sensitive unit and the non-focus sensitive unit for exposure, and read the output values of the focus sensitive unit and the non-focus sensitive unit; and M * M photosensitive pixels of the same focus sensitive unit or the same non-focus The output values of the M * M photosensitive pixels of the photosensitive unit are added to obtain the pixel values of the focused photosensitive unit and the non-focused photosensitive unit, thereby generating an image in the first output mode.
控制模組1730還用於當當前環境光強度大於預設強度時,控制影像感測器進入第二輸出模式;在第二輸出模式下,控制感光單元陣列進入對焦模式;讀取對焦感光單元中一部分感光像素的輸出值並作為第三輸出值;讀取對焦感光單元中另一部分感光像素的輸出值並作為第四輸出值;根據第三輸出值和第四輸出值進行對焦控制;在第二輸出模式下,控制感光單元陣列進入成像模式;控制感光單元陣列進行曝光,以產生第二輸出模式下的影像。The control module 1730 is also used to control the image sensor to enter the second output mode when the current ambient light intensity is greater than the preset intensity; in the second output mode, control the photosensitive unit array to enter the focus mode; read the focus photosensitive unit The output value of a part of the photosensitive pixels is used as the third output value; the output value of another part of the photosensitive pixels in the focus photosensitive unit is read as the fourth output value; focus control is performed according to the third output value and the fourth output value; In the output mode, the photosensitive unit array is controlled to enter the imaging mode; the photosensitive unit array is controlled to perform exposure to generate an image in the second output mode.
控制模組1730具體用於:控制感光單元陣列進行曝光,並讀取感光單元陣列的輸出值,以得到感光單元陣列的像素值從而產生影像,其中,感光單元陣列的像素值通過插值還原演算法獲得。The control module 1730 is specifically configured to control the photosensitive unit array for exposure and read the output value of the photosensitive unit array to obtain the pixel values of the photosensitive unit array to generate an image. The pixel values of the photosensitive unit array are restored by interpolation. obtain.
控制模組1730具體用於:判斷當前像素的顏色與關聯像素的顏色是否相同;在當前像素的顏色與關聯像素的顏色相同時,將關聯像素的像素值作為當前像素的像素值;和在當前像素的顏色與關聯像素的顏色不同時,根據關聯像素單元的像素值通過插值方式計算當前像素的像素值,影像像素單元的陣列包括關聯像素單元,關聯像素單元的顏色與當前像素顏色相同且與當前像素相鄰。The control module 1730 is specifically configured to: determine whether the color of the current pixel is the same as the color of the associated pixel; when the color of the current pixel is the same as the color of the associated pixel, use the pixel value of the associated pixel as the pixel value of the current pixel; and When the color of the pixel is different from the color of the associated pixel, the pixel value of the current pixel is calculated by interpolation based on the pixel value of the associated pixel unit. The array of image pixel units includes the associated pixel unit. The color of the associated pixel unit is the same as the current pixel color and The current pixel is adjacent.
本申請的成像裝置,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The imaging device of the present application is based on an image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focus photosensitive unit and The structure of the non-focusing photosensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first output mode. Focus control is performed on the output value of some photosensitive pixels and the output value of another photosensitive pixel in the unit, which can effectively improve the focusing speed.
本申請再一方面實施例還提出一種行動終端。Another embodiment of the present application further provides a mobile terminal.
如第18圖所示,該行動終端包括殼體1810、處理器1820、記憶體1830、電路板1840和電源電路1850,其中,電路板1840安置在殼體1810圍成的空間內部,處理器1820和記憶體1830設置在電路板1840上;電源電路1850,用於為行動終端的各個電路或裝置供電;記憶體1830用於儲存可執行程式碼;處理器1820通過讀取記憶體1830中儲存的可執行程式碼來運行與可執行程式碼對應的程式,以用於執行上述方面的影像感測器的影像處理方法。As shown in FIG. 18, the mobile terminal includes a housing 1810, a processor 1820, a memory 1830, a circuit board 1840, and a power circuit 1850. The circuit board 1840 is disposed in a space surrounded by the housing 1810, and the processor 1820 And memory 1830 are disposed on the circuit board 1840; the power circuit 1850 is used to supply power to various circuits or devices of the mobile terminal; the memory 1830 is used to store executable code; the processor 1820 reads the data stored in the memory 1830 The executable code runs a program corresponding to the executable code, and is used to execute the image processing method of the image sensor of the above aspect.
本申請實施例的行動終端,基於影像感測器包括感光單元陣列、設置在感光單元陣列上的濾光單元陣列和位於濾光單元陣列之上的微透鏡陣列,其中,感光單元陣列包括對焦感光單元和非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The mobile terminal according to the embodiment of the present application is based on an image sensor including a photosensitive unit array, a filter unit array disposed on the photosensitive unit array, and a microlens array located above the filter unit array, wherein the photosensitive unit array includes a focus sensor. The structure of the unit and the non-focus photosensitive unit. The image sensor has two output modes, a first output mode and a second output mode. When the ambient light intensity is less than a preset intensity, the image sensor enters the first output mode. Focus control is performed on the output value of some photosensitive pixels and the output value of another photosensitive pixel in the focusing photosensitive unit, which can effectively improve the focusing speed.
為達上述目的,本申請實施例還提出了一種影像感測器的影像處理方法。To achieve the above object, an embodiment of the present application further provides an image processing method of an image sensor.
如第10圖所示,該影像感測器100包括感光單元陣列10、濾光單元陣列20和微透鏡陣列30。As shown in FIG. 10, the image sensor 100 includes a photosensitive unit array 10, a filter unit array 20 and a microlens array 30.
其中,濾光單元陣列20設置在感光單元陣列10上,微透鏡陣列30位於濾光單元陣列20之上。感光單元陣列10包括複數對焦感光單元11和複數非對焦感光單元12。The filter unit array 20 is disposed on the photosensitive unit array 10, and the microlens array 30 is located above the filter unit array 20. The photosensitive unit array 10 includes a plurality of focusing photosensitive units 11 and a plurality of non-focusing photosensitive units 12.
其中,微透鏡陣列30包括第一微透鏡31和第二微透鏡32。第一微透鏡31覆蓋一對焦感光單元11。N*N個第二微透鏡32覆蓋一非對焦感光單元12。對焦感光單元11和非對焦感光單元12均包括M*M個感光像素110,其中,M為正整數。The microlens array 30 includes a first microlens 31 and a second microlens 32. The first microlens 31 covers a focusing photosensitive unit 11. The N * N second microlenses 32 cover a non-focus photosensitive unit 12. The focus sensitive unit 11 and the non-focus sensitive unit 12 each include M * M photosensitive pixels 110, where M is a positive integer.
基於第10圖所示的影像感測器的結構,下麵對本申請實施例的影像感測器的影像處理方法進行說明。 如第2圖所示,該影像感測器的影像處理方法包括:Based on the structure of the image sensor shown in FIG. 10, the image processing method of the image sensor of the embodiment of the present application will be described below. As shown in FIG. 2, the image processing method of the image sensor includes:
S21,檢測當前環境光強度。S21: Detect the current ambient light intensity.
通過手機對物體進行拍照時,攝像頭對準拍攝的物體後,檢測當前拍照環境的光強度。When taking a picture of an object with a mobile phone, after the camera is pointed at the photographed object, the light intensity of the current photographing environment is detected.
S22,當當前環境光強度小於預設強度時,控制影像感測器進入第一輸出模式。S22. When the current ambient light intensity is less than a preset intensity, control the image sensor to enter a first output mode.
將檢測的當前拍照環境的光強度與預設強度進行比較,如果當前環境光強度小於預設強度如300勒克斯,即被拍攝物體處於弱光環境,則控制影像感測器進入第一輸出模式。其中,第一輸出模式可以為4M輸出格式,該格式下輸出的照片同一感光單元內感光像素的顏色相同,解析度相對較低,但訊噪比高。The detected light intensity of the current photographing environment is compared with a preset intensity. If the current ambient light intensity is less than a preset intensity, such as 300 lux, that is, the subject is in a low-light environment, the image sensor is controlled to enter a first output mode. The first output mode may be a 4M output format. The photos output in this format have the same color of the photosensitive pixels in the same photosensitive unit, the resolution is relatively low, but the signal-to-noise ratio is high.
S23,在第一輸出模式下,控制感光單元陣列進入對焦模式。S23. In the first output mode, control the photosensitive unit array to enter a focus mode.
對準要拍攝的物體後,點擊螢幕進行對焦,這時感光單元陣列進入第一輸出模式下的對焦模式。After aligning the object to be shot, tap the screen to focus. At this time, the photosensitive unit array enters the focus mode in the first output mode.
S24,讀取對焦感光單元中一部分感光像素的輸出值並作為第一輸出值。S24. Read the output value of a part of the photosensitive pixels in the focus photosensitive unit and use it as the first output value.
S25,讀取對焦感光單元中另一部分感光像素的輸出值並作為第二輸出值。S25. Read the output value of another photosensitive pixel in the focus photosensitive unit and use it as the second output value.
S26,根據第一輸出值和第二輸出值進行對焦控制。S26. Perform focus control according to the first output value and the second output value.
下面以對焦感光單元包括2*2個感光像素為例,說明第一輸出模式下的對焦控制方法。The following describes the focus control method in the first output mode with the focus photosensitive unit including 2 * 2 photosensitive pixels as an example.
如第19圖所示,讀取對焦感光單元中左側的兩個感光像素的輸出值Gp30 和Gp32 作為第一輸出值,讀取另外一部分感光像素即右側兩個感光像素的輸出值Gp31 和Gp33 作為第二輸出值。As shown in FIG. 19, read the output values Gp 30 and Gp 32 of the two left photosensitive pixels in the focus photosensitive unit as the first output values, and read the output values Gp 31 of the other two photosensitive pixels, that is, the two right photosensitive pixels. And Gp 33 as the second output value.
在獲取第一輸出值和第二輸出值之後,求出對焦感光單元左側兩個感光像素的輸出值Gp30 與Gp32 之間的和,即Gp1 =Gp30 +Gp32 ,產生第一相位值Gp1 。同樣,求出右側兩個感光像素的輸出值Gp31 與Gp33 之間的和,即Gp2 =Gp31 +Gp33 ,產生第二相位值Gp2 。從而,可以獲取Gp1 和Gp2 之間的相位差資訊,進而可以將相位差資訊轉換為對焦距離資訊,根據對焦距離資訊調節鏡頭的位置實現相位對焦,相位對焦檢測的實現方式也更加簡單。After obtaining the first output value and the second output value, find the sum between the output values Gp 30 and Gp 32 of the two photosensitive pixels on the left side of the focus photosensitive unit, that is, Gp 1 = Gp 30 + Gp 32 to generate the first phase Value Gp 1 . Similarly, a sum between the output values Gp 31 and Gp 33 of the two right-side photosensitive pixels is calculated, that is, Gp 2 = Gp 31 + Gp 33 to generate a second phase value Gp 2 . Therefore, the phase difference information between Gp 1 and Gp 2 can be obtained, and then the phase difference information can be converted into focus distance information, and the position of the lens can be adjusted according to the focus distance information to achieve phase focus. The implementation of phase focus detection is also simpler.
在本申請的另一實施例中,如第20圖所示,將對焦感光單元中2*2個感光像素分為上側和下側兩個部分,可讀取對焦感光單元中上側的兩個感光像素的輸出值作為第一輸出值,讀取下側的兩個感光像素的輸出值作為第二輸出值。在獲取第一輸出值和第二輸出值之後,根據第一輸出值產生第一相位值,根據第二輸出值產生第二相位值,再根據第一相位值和第二相位值之間的相位差資訊進行對焦控制。In another embodiment of the present application, as shown in FIG. 20, the 2 * 2 photosensitive pixels in the focus sensitive unit are divided into two parts, the upper side and the lower side, and the two upper photosensitive parts in the focus photosensitive unit can be read. The output value of the pixel is taken as the first output value, and the output values of the two photosensitive pixels on the lower side are read as the second output value. After obtaining the first output value and the second output value, a first phase value is generated according to the first output value, a second phase value is generated according to the second output value, and then a phase between the first phase value and the second phase value is generated. Poor information for focus control.
在本申請的又一實施例中,如第21圖所示,以對焦感光單元的兩條對角線將2*2個感光像素分為兩部分,可讀取左上角的感光像素與右下角的感光像素作為第一輸出值,讀取左下角的感光像素與右上角的感光像素作為第二輸出值。在獲取第一輸出值和第二輸出值之後,根據第一輸出值產生第一相位值,根據第二輸出值產生第二相位值,再根據第一相位值和第二相位值之間的相位差資訊進行對焦控制。In another embodiment of the present application, as shown in FIG. 21, 2 * 2 photosensitive pixels are divided into two parts by two diagonal lines of the focusing photosensitive unit, and the upper left corner and the lower right corner can be read. The light-sensitive pixels are used as the first output value, and the light-sensitive pixels in the lower left corner and the light-sensitive pixels in the upper right corner are read as the second output value. After obtaining the first output value and the second output value, a first phase value is generated according to the first output value, a second phase value is generated according to the second output value, and then a phase between the first phase value and the second phase value is generated. Poor information for focus control.
在本申請實施例中,將對焦感光單元2*2個感光像素左右兩側感光像素的輸出值分別作為第一輸出值和第二輸出值,可以檢測左右方向的相位差資訊;將對焦感光單元2*2個感光像素上下兩側感光像素的輸出值分別作為第一輸出值和第二輸出值,可以檢測上下方向的相位差資訊;將對焦感光單元兩條對角線上的感光像素的輸出值分別作為第一輸出值和第二輸出值,可檢測斜向的相位差資訊。In the embodiment of the present application, the output values of the left and right photosensitive pixels of the 2 * 2 photosensitive pixels of the focusing photosensitive unit are respectively used as the first output value and the second output value, and the phase difference information in the left and right directions can be detected; The output values of the upper and lower photosensitive pixels of 2 * 2 photosensitive pixels are respectively used as the first output value and the second output value, which can detect the phase difference information in the vertical direction; the output values of the photosensitive pixels on the two diagonal lines of the focusing photosensitive unit As the first output value and the second output value, phase difference information in the oblique direction can be detected.
本申請實施例提出的影像感測器處於第一輸出模式時的對焦控制方法,通過讀取對焦感光單元中不同部分的感光像素的輸出值,獲取不同角度入射光線的相位資訊,進行不同方向相位資訊檢測,提高了暗光下的對焦速度,對焦更準確。In the focus control method provided by the embodiment of the present application when the image sensor is in the first output mode, the phase information of incident light at different angles is obtained by reading the output values of the photosensitive pixels in different parts of the focusing photosensitive unit, and the phases in different directions are performed. Information detection improves the focusing speed in dark light and makes focusing more accurate.
S27,在第一輸出模式下,控制感光單元陣列進入成像模式。S27. In the first output mode, control the photosensitive cell array to enter the imaging mode.
在第一輸出模式下,通過對焦感光單元中兩部分感光像素輸出值獲得相位差資訊,根據相位差資訊完成對焦控制後,控制感光單元陣列進入成像模式。In the first output mode, phase difference information is obtained by focusing on the output values of two photosensitive pixels in the photosensitive unit. After the focus control is completed according to the phase difference information, the photosensitive unit array is controlled to enter the imaging mode.
S28,控制感光單元陣列進行曝光,以產生第一輸出模式下的影像。S28. Control the photosensitive unit array to perform exposure to generate an image in a first output mode.
本申請實施例以對焦感光單元和非對焦感光單元包括2*2個感光像素為例。如第22圖所示,藍色B0 、綠色G1 、綠色G3 和紅色R4 組成了一拜耳RGB陣列。對對焦感光單元和非對焦感光單元進行曝光,讀取對焦感光單元的輸出值Gp30 、Gp31 、Gp32 和Gp33 ,非對焦感光單元的輸出值B00 、B01 、B02 、B03 、Gb10 、Gb11 、Gb12 、Gb13 等等。In the embodiment of the present application, the focusing photosensitive unit and the non-focus photosensitive unit include 2 * 2 photosensitive pixels as an example. As shown in FIG. 22, blue B 0 , green G 1 , green G 3 and red R 4 form a Bayer RGB array. Expose the focus sensor unit and the non-focus sensor unit, read the output values of the focus sensor unit Gp 30 , Gp 31 , Gp 32, and Gp 33 , and the output values of the non-focus sensor unit B 00 , B 01 , B 02 , B 03 , Gb 10 , Gb 11 , Gb 12 , Gb 13, and so on.
讀取對焦感光單元和非對焦感光單元的輸出值之後,將同一對焦感光單元的2*2個感光像素的輸出值Gp30 、Gp31 、Gp32 和Gp33 相加,即Gp30 +Gp31 + Gp32 +Gp33 =G3 ,得到對焦感光單元的像素值G3 。將同一非對焦感光單元的2*2個感光像素輸出值B00 、B01 、B02 、B03 相加,即B00 +B01 +B02 +B03 =B0 ,得到該非對焦感光單元的像素值B0 。同理,可得到非對焦感光單元的像素值,綠色G1 =Gb10 +Gb11 +Gb12 +Gb13 ,紅色R4 =R40 +R41 +R42 +R43 等等。根據對焦感光單元和非對焦感光單元的像素值產生第一輸出模式下的影像。After reading the output values of the focus sensitive unit and the non-focus sensitive unit, the output values of 2 * 2 photosensitive pixels of the same focus sensitive unit are added Gp 30 , Gp 31 , Gp 32, and Gp 33 , that is, Gp 30 + Gp 31 + Gp 32 + Gp 33 = G 3 to obtain the pixel value G 3 of the focus photosensitive unit. Add 2 * 2 photosensitive pixel output values B 00 , B 01 , B 02 , and B 03 of the same non-focus photosensitive unit, that is, B 00 + B 01 + B 02 + B 03 = B 0 to obtain the non-focus photosensitive unit Pixel value B 0 . Similarly, the pixel values of the out-of-focus photosensitive unit can be obtained, green G 1 = Gb 10 + Gb 11 + Gb 12 + Gb 13 , red R 4 = R 40 + R 41 + R 42 + R 43 and so on. An image in the first output mode is generated according to the pixel values of the focus sensitive unit and the non-focus sensitive unit.
本申請實施例的成像方法,將感光單元內的M*M個感光像素的輸出值之和作為該感光單元的像素值,根據對焦感光單元和非對焦感光單元的像素值產生影像,可有效地提升影像的成像靈敏度和訊噪比。The imaging method in the embodiment of the present application uses the sum of the output values of M * M photosensitive pixels in the photosensitive unit as the pixel value of the photosensitive unit, and generates an image according to the pixel values of the focused photosensitive unit and the non-focused photosensitive unit, which can effectively Improve image sensitivity and signal-to-noise ratio.
本申請的影像感測器的影像處理方法,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有兩種輸出模式,第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on a structure in which a first micro lens of the image sensor covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit. The image sensor There are two output modes, the first output mode and the second output mode. When the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output value of some photosensitive pixels in the focus photosensitive unit and the other The output value of the light-sensitive pixel is used to focus control, which can effectively increase the focusing speed.
上述實施例是在拍照環境光強度較低的情況下,影像感測器進入第一輸出模式時對焦方法和成像方法。下面詳細介紹在強光環境下影像感測器的影像處理方法。The above embodiment is a focusing method and an imaging method when the image sensor enters the first output mode in a case where the ambient light intensity of the photograph is low. The image processing method of the image sensor in a strong light environment is described in detail below.
通過手機對物體進行拍照時,攝像頭對準拍攝的物體後,檢測當前拍照環境的光強度。將檢測的拍照環境的光強度與預設強度進行比較,如果拍照環境的光強度大於預設強度如300勒克斯,即拍照環境為強光環境時,則控制影像感測器進入第二輸出模式。When taking a picture of an object with a mobile phone, after the camera is pointed at the photographed object, the light intensity of the current photographing environment is detected. The detected light intensity of the photographing environment is compared with a preset intensity. If the light intensity of the photographing environment is greater than a preset intensity such as 300 lux, that is, when the photographing environment is a strong light environment, the image sensor is controlled to enter a second output mode.
其中,第二輸出模式可以為16M輸出格式,該格式下輸出的照片同一感光單元內的感光像素的顏色不同,相比第一輸出模式輸出的照片具有更多的細節,解析度較高。The second output mode may be an 16M output format. The photos output in this format have different colors of the photosensitive pixels in the same photosensitive unit. Compared with the photos output in the first output mode, the photos have more details and higher resolution.
對準要拍攝的物體後,點擊螢幕進行對焦,這時感光單元陣列進入對焦模式。After aiming at the object to be shot, tap the screen to focus. At this time, the sensor unit array enters the focus mode.
在本申請的實施例中,第二輸出模式下的對焦控制方法與第一輸出模式下的對焦控制方法類似,成像方法與上述實施例中第二輸出模式下成像方法類似,在此不再贅述。In the embodiment of the present application, the focus control method in the second output mode is similar to the focus control method in the first output mode, and the imaging method is similar to the imaging method in the second output mode in the above embodiment, and details are not described herein. .
在本申請的一實施例中,N和M的取值可以相同。以N和M的取值為2為例,如第11圖所示,影像感測器100中,2*2個第二微透鏡覆蓋一對焦感光單元,對焦感光單元和非對焦感光單元均包括2*2個感光像素。In an embodiment of the present application, the values of N and M may be the same. Taking the value of N and M as an example, as shown in FIG. 11, in the image sensor 100, 2 * 2 second microlenses cover a focus sensitive unit, and both the focus sensitive unit and the non-focus sensitive unit include 2 * 2 photosensitive pixels.
當N和M的取值相同時,影像感測器的對焦方法和成像方法與上述實施例中的對焦方法和成像方法類似,在此不再贅述。When the values of N and M are the same, the focusing method and imaging method of the image sensor are similar to the focusing method and imaging method in the above embodiment, and details are not described herein again.
當然,N與M的取值也可以不相同。N與M的取值不同時,影像感測器的對焦方法和成像方法與上述實施例中的對焦方法和成像方法類似,在此不再贅述。Of course, the values of N and M can also be different. When the values of N and M are different, the focusing method and imaging method of the image sensor are similar to the focusing method and imaging method in the foregoing embodiment, and details are not described herein again.
在本申請的一實施例中,如第13圖所示,微透鏡陣列30可包括水平中心線和垂直中心線,微透鏡陣列包括複數第一微透鏡。複數第一微透鏡包括設置在水平中心線的第一組第一微透鏡和設置在垂直中心線的第二組第一微透鏡。In an embodiment of the present application, as shown in FIG. 13, the micro lens array 30 may include a horizontal center line and a vertical center line, and the micro lens array includes a plurality of first micro lenses. The plurality of first microlenses includes a first group of first microlenses disposed on a horizontal centerline and a second group of first microlenses disposed on a vertical centerline.
為了提高對焦速度,在本申請的另一實施例中,如第14圖所示,複數第一微透鏡還可包括設置在微透鏡陣列四個邊線上的第三組第一微透鏡。In order to improve the focusing speed, in another embodiment of the present application, as shown in FIG. 14, the plurality of first microlenses may further include a third group of first microlenses disposed on the four edges of the microlens array.
從第14圖可以看出,由第一微透鏡覆蓋的對焦感光單元,即圖中的Gp,在整個影像感測器中零散分佈,占總像素個數的3%~5%,影像感測器中心區域Gp分佈更密集,邊緣區域分佈較為稀疏,優先獲取畫面中心的相位資訊,在不影響畫質的情況下,有效提高對焦速度。It can be seen from Figure 14 that the focusing photosensitive unit covered by the first microlens, that is, Gp in the figure, is scattered throughout the image sensor, accounting for 3% to 5% of the total number of pixels. The Gp distribution in the center area of the camera is more dense, and the distribution in the edge area is relatively sparse. The phase information in the center of the screen is preferentially obtained, and the focusing speed is effectively improved without affecting the image quality.
由於透鏡密度越大,透鏡的折射率越大,聚光能力越強,為了使中心區域的對焦感光單元聚集較多的光線,以提升對焦速度和拍攝效果。在本申請的一實施例中,可使第一組第一微透鏡和第二組第一微透鏡的透鏡密度大於第三組第一微透鏡的透鏡密度,從而使中心區域的對焦感光單元的進光量相對邊緣較大,進而提升對焦速度和拍攝效果。As the density of the lens is greater, the refractive index of the lens is greater, and the light-gathering ability is stronger. In order to make the focusing photosensitive unit in the central area gather more light, the focusing speed and the shooting effect are improved. In an embodiment of the present application, the lens density of the first group of the first microlenses and the second group of the first microlenses may be greater than the lens density of the third group of the first microlenses, so that the focus of the central area of the photosensitive unit The amount of incoming light is relatively large, which improves the focus speed and shooting effect.
在本申請的一實施例中,在基於第14圖所示的對焦感光單元的分佈基礎上,當影像感測器處於第二輸出模式下,在對感光單元陣列進行曝光,讀取對焦感光單元和非對焦感光單元的輸出值時,需要對對焦感光單元Gp的感光像素的值進行處理。In an embodiment of the present application, based on the distribution of the focus sensitive units shown in FIG. 14, when the image sensor is in the second output mode, the photosensitive unit array is exposed to read the focus sensitive units. When the output values of the out-of-focus photosensitive unit and the out-of-focus photosensitive unit are processed, the values of the photosensitive pixels of the in-focus photosensitive unit Gp need to be processed.
由於影像感測器中間部分光線強,四周光線較弱,因此影像感測器中間部分的對焦感光單元Gp內感光像素的值是可以直接使用的,離影像感測器中心點越遠,對焦感光單元Gp內感光像素的值的可信度越低,可逐漸被周邊感光像素的值取代。Because the light in the middle part of the image sensor is strong and the surrounding light is weak, the value of the photosensitive pixel in the focus sensor unit Gp in the middle part of the image sensor can be used directly. The farther away from the center point of the image sensor, the focus sensor is. The lower the reliability of the value of the photosensitive pixel in the unit Gp, the more gradually it can be replaced by the value of the surrounding photosensitive pixel.
在本申請的一實施例中,根據對焦感光單元Gp離影像感測器的中心點的距離對對焦感光單元內感光像素的值進行處理。在一具體實施例中,如第15圖所示,當對焦感光單元Gp與影像感測器中心點的距離小於r1時,也就是說,對焦感光單元處於影像感測器的中間部分時,可認為此時對焦感光單元Gp內感光像素的值與非對焦感光單元Gr的值是等價的。如第16圖所示,Gr30 =Gp30 ,Gr31 =Gp31 ,Gr32 =Gp32 ,Gr33 =Gp33 。In an embodiment of the present application, the value of the photosensitive pixel in the focus photosensitive unit is processed according to the distance of the focus photosensitive unit Gp from the center point of the image sensor. In a specific embodiment, as shown in FIG. 15, when the distance between the focus sensing unit Gp and the center point of the image sensor is less than r1, that is, when the focus sensing unit is located in the middle portion of the image sensor, It is considered that the value of the photosensitive pixel in the focus photosensitive unit Gp and the value of the non-focus photosensitive unit Gr are equivalent at this time. As shown in Figure 16, Gr 30 = Gp 30 , Gr 31 = Gp 31 , Gr 32 = Gp 32 , and Gr 33 = Gp 33 .
當對焦感光單元Gp與影像感測器中心點的距離大於r2時,對焦感光單元Gp內感光像素的像素值可通過周邊感光像素插值計算得到。When the distance between the focus sensing unit Gp and the center point of the image sensor is greater than r2, the pixel values of the photosensitive pixels in the focus sensing unit Gp can be calculated by the surrounding photosensitive pixel interpolation.
當對焦感光單元Gp與影像感測器中心點的距離大於r1小於r2時,可以距離為加權權重,通過將距離小於r1的對焦感光單元的感光像素的像素值與距離大於r2的通過插值計算得到的感光像素的像素值進行加權平均,計算得到該對焦感光單元內感光像素的像素值。When the distance between the focus sensing unit Gp and the center point of the image sensor is greater than r1 and less than r2, the distance can be a weighted weight. The pixel value of the photosensitive pixel of the focus sensing unit that is less than r1 and the distance greater than r2 are calculated by interpolation The weighted average of the pixel values of the light-sensitive pixels is calculated to obtain the pixel values of the light-sensitive pixels in the focusing light-sensing unit.
通過上述方法對對焦感光單元Gp的相位資訊進行處理後,可獲得感光單元陣列的輸出值。根據獲得的感光單元陣列的輸出值,通過插值還原演算法可獲得感光單元陣列的像素值。其中,插值還原演算法與上述實施例中的插值還原演算法相同,在此不再贅述。After the phase information of the focus photosensitive unit Gp is processed by the above method, the output value of the photosensitive unit array can be obtained. According to the obtained output value of the photosensitive unit array, the pixel value of the photosensitive unit array can be obtained through an interpolation reduction algorithm. The interpolation restoration algorithm is the same as the interpolation restoration algorithm in the foregoing embodiment, and details are not described herein again.
通過上述插值還原演算法獲得感光單元陣列的像素值之後,根據感光單元陣列的像素值可產生第二輸出模式下的影像。本申請實施例提出的方案,在進行插值還原之前,根據第一微透鏡的空置位置,對對焦感光單元內感光像素的值進行處理,與簡單的丟棄對焦感光單元處的相位資訊,通過周邊像素補償的方案相比,提高了影像的清晰度和拍攝效果。After the pixel values of the photosensitive unit array are obtained through the interpolation restoration algorithm, an image in the second output mode can be generated according to the pixel values of the photosensitive unit array. In the solution proposed by the embodiment of the present application, before performing interpolation and restoration, the value of the photosensitive pixel in the focus sensitive unit is processed according to the empty position of the first microlens, and the phase information at the focus sensitive unit is simply discarded, and the surrounding pixels are passed. Compared with the compensation scheme, the image clarity and shooting effect are improved.
本申請的影像感測器的影像處理方法,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The image processing method of the image sensor of the present application is based on a structure in which a first micro lens of the image sensor covers a focusing photosensitive unit, and N * N second micro lenses cover a non-focus photosensitive unit. The image sensor It has a first output mode and a second output mode. When the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output values of some photosensitive pixels and the output values of other photosensitive pixels in the focus photosensitive unit. , Focus control can effectively increase the focus speed.
為達上述目的,本申請實施例又提出一種影像感測器。To achieve the above object, an embodiment of the present application further provides an image sensor.
如第10圖所示,該影像感測器100包括感光單元陣列10、濾光單元陣列20和微透鏡陣列30。As shown in FIG. 10, the image sensor 100 includes a photosensitive unit array 10, a filter unit array 20 and a microlens array 30.
其中,濾光單元陣列20設置在感光單元陣列10上,微透鏡陣列30位於濾光單元陣列20之上。感光單元陣列10包括複數對焦感光單元11和複數非對焦感光單元12。The filter unit array 20 is disposed on the photosensitive unit array 10, and the microlens array 30 is located above the filter unit array 20. The photosensitive unit array 10 includes a plurality of focusing photosensitive units 11 and a plurality of non-focusing photosensitive units 12.
其中,微透鏡陣列30包括第一微透鏡31和第二微透鏡32。第一微透鏡31覆蓋一對焦感光單元11。N*N個第二微透鏡32覆蓋一非對焦感光單元12。對焦感光單元11和非對焦感光單元12包括M*M個感光像素110,其中,M為正整數。The microlens array 30 includes a first microlens 31 and a second microlens 32. The first microlens 31 covers a focusing photosensitive unit 11. The N * N second microlenses 32 cover a non-focus photosensitive unit 12. The focus photosensitive unit 11 and the non-focus photosensitive unit 12 include M * M photosensitive pixels 110, where M is a positive integer.
在本申請的一實施例中,N和M的取值可以相同。以N和M的取值為2為例,如第11圖所示,影像感測器100中,2*2個第二微透鏡覆蓋一對焦感光單元,對焦感光單元和非對焦感光單元均包括2*2個感光像素。當然,N與M的取值也可以不相同。In an embodiment of the present application, the values of N and M may be the same. Taking the value of N and M as an example, as shown in FIG. 11, in the image sensor 100, 2 * 2 second microlenses cover a focus sensitive unit, and both the focus sensitive unit and the non-focus sensitive unit include 2 * 2 photosensitive pixels. Of course, the values of N and M can also be different.
在本申請的一實施例中,如第13圖所示,微透鏡陣列30可包括水平中心線和垂直中心線,微透鏡陣列包括複數第一微透鏡。複數第一微透鏡包括設置在水平中心線的第一組第一微透鏡和設置在垂直中心線的第二組第一微透鏡。In an embodiment of the present application, as shown in FIG. 13, the micro lens array 30 may include a horizontal center line and a vertical center line, and the micro lens array includes a plurality of first micro lenses. The plurality of first microlenses includes a first group of first microlenses disposed on a horizontal centerline and a second group of first microlenses disposed on a vertical centerline.
為了提高對焦速度,在本申請的另一實施例中,如第14圖所示,複數第一微透鏡還可包括設置在微透鏡陣列四個邊線上的第三組第一微透鏡。In order to improve the focusing speed, in another embodiment of the present application, as shown in FIG. 14, the plurality of first microlenses may further include a third group of first microlenses disposed on the four edges of the microlens array.
從第14圖可以看出,由第一微透鏡覆蓋的對焦感光單元,即圖中的Gp,在整個影像感測器中零散分佈,占總像素個數的3%~5%,影像感測器中心區域Gp分佈更密集,邊緣區域分佈較為稀疏,優先獲取畫面中心的相位資訊,在不影響畫質的情況下,有效提高對焦速度。It can be seen from Figure 14 that the focusing photosensitive unit covered by the first microlens, that is, Gp in the figure, is scattered throughout the image sensor, accounting for 3% to 5% of the total number of pixels. The Gp distribution in the center area of the camera is more dense, and the distribution in the edge area is relatively sparse. The phase information in the center of the screen is preferentially obtained, and the focusing speed is effectively improved without affecting the image quality.
由於透鏡密度越大,透鏡的折射率越大,聚光能力越強,為了使中心區域的對焦感光單元聚集較多的光線,以提升對焦速度和拍攝效果。在本申請的一實施例中,可使第一組第一微透鏡和第二組第一微透鏡的透鏡密度大於第三組第一微透鏡的透鏡密度,從而使中心區域的對焦感光單元的進光量相對邊緣較大,進而提升對焦速度和拍攝效果。As the density of the lens is greater, the refractive index of the lens is greater, and the light-gathering ability is stronger. In order to make the focusing photosensitive unit in the central area gather more light, the focusing speed and the shooting effect are improved. In an embodiment of the present application, the lens density of the first group of the first microlenses and the second group of the first microlenses may be greater than the lens density of the third group of the first microlenses, so that the focus of the central area of the photosensitive unit The amount of incoming light is relatively large, which improves the focus speed and shooting effect.
為達上述目的,本申請實施例還提出一種成像裝置。本申請實施例提出的成像裝置,與上述實施例中的成像裝置的結構和功能類似,在此不再贅述。To achieve the above object, an embodiment of the present application further provides an imaging device. The imaging device provided in the embodiment of the present application has a structure and function similar to those of the imaging device in the foregoing embodiment, and details are not described herein again.
本申請實施例的成像裝置,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The imaging device of the embodiment of the present application is based on a structure in which a first microlens of an image sensor covers a focusing photosensitive unit, and N * N second microlenses cover a non-focusing photosensitive unit. The image sensor has a first output. Mode and second output mode, when the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output values of some photosensitive pixels in the focus photosensitive unit and the output values of other photosensitive pixels for focus control , Can effectively increase the focusing speed.
為達上述目的,本申請實施例還提出了一種行動終端,其結構與第18圖所示的行動終端結構類似,處理器運行與可執行程式碼對應的程式,可實現上述實施例提出的影像感測器的影像處理方法。In order to achieve the above purpose, the embodiment of the present application also proposes a mobile terminal with a structure similar to that of the mobile terminal shown in FIG. 18, and the processor runs a program corresponding to the executable code to implement the image provided by the above embodiment. Image processing method of sensor.
本申請實施例的行動終端,基於影像感測器的一第一微透鏡覆蓋一對焦感光單元,N*N個第二微透鏡覆蓋一非對焦感光單元的結構,影像感測器具有第一輸出模式和第二輸出模式,當環境光強度小於預設強度時,影像感測器進入第一輸出模式,利用對焦感光單元中一部分感光像素的輸出值與另外一部分感光像素的輸出值,進行對焦控制,能有效提高對焦速度。The mobile terminal according to the embodiment of the present application is based on a structure in which a first microlens of an image sensor covers a focusing photosensitive unit, and N * N second microlenses cover a non-focusing photosensitive unit. The image sensor has a first output. Mode and second output mode, when the ambient light intensity is less than the preset intensity, the image sensor enters the first output mode, and uses the output values of some photosensitive pixels in the focus photosensitive unit and the output values of other photosensitive pixels for focus control , Can effectively increase the focusing speed.
需要說明的是,在本文中,諸如第一和第二等之類的關係術語僅僅用來將一實體或者操作與另一實體或操作區分開來,而不一定要求或者暗示這些實體或操作之間存在任何這種實際的關係或者順序。而且,術語“包括”、“包含”或者其任何其他變體意在涵蓋非排他性的包含,從而使得包括一系列要素的過程、方法、物品或者裝置不僅包括那些要素,而且還包括沒有明確列出的其他要素,或者是還包括為這種過程、方法、物品或者裝置所固有的要素。在沒有更多限制的情況下,由語句“包括一……”限定的要素,並不排除在包括該要素的過程、方法、物品或者裝置中還存在另外的相同要素。It should be noted that in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order among them. Moreover, the terms "including", "comprising", or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also those that are not explicitly listed Or other elements that are inherent to this process, method, article, or device. Without more restrictions, the elements defined by the sentence "including one ..." do not exclude the existence of other identical elements in the process, method, article, or device that includes the elements.
在流程圖中表示或在此以其他方式描述的邏輯和/或步驟,例如,可以被認為是用於實現邏輯功能的可執行指令的定序列表,可以具體實現在任何電腦可讀媒體中,以供指令執行系統、裝置或設備(如基於電腦的系統、包括處理器的系統或其他可以從指令執行系統、裝置或設備取指令並執行指令的系統)使用,或結合這些指令執行系統、裝置或設備而使用。就本說明書而言,"電腦可讀媒體"可以是任何可以包含、儲存、通訊、傳播或傳輸程式以供指令執行系統、裝置或設備或結合這些指令執行系統、裝置或設備而使用的裝置。電腦可讀媒體的更具體的示例(非窮盡性列表)包括以下:具有一或複數佈線的電連接部(電子裝置)、可攜式電腦碟盒(磁裝置)、隨機存取記憶體(RAM)、唯讀記憶體(ROM)、可抹除可程式化唯讀記憶體(EPROM或快閃記憶體)、光纖裝置,以及可攜式光碟唯讀記憶體(CDROM)。另外,電腦可讀媒體甚至可以是可在其上列印該程式的紙或其他合適的媒體,因為可以例如通過對紙或其他媒體進行光學掃描,接著進行編輯、解譯或必要時以其他合適方式進行處理來以電子方式獲得該程式,然後將其儲存在電腦記憶體中。The logic and / or steps represented in the flowchart or otherwise described herein, for example, a sequenced list of executable instructions that can be considered to implement a logical function, can be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (such as a computer-based system, a system that includes a processor, or another system that can fetch and execute instructions from an instruction execution system, device, or device) Or equipment. For the purpose of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, device, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM) ), Read-only memory (ROM), programmable erasable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program can be printed, because, for example, by optically scanning the paper or other media, and then editing, interpreting, or other suitable Process to obtain the program electronically and then store it in computer memory.
應當理解,本申請的各部分可以用硬體、軟體、韌體或它們的組合來實現。在上述實施方式中,複數步驟或方法可以用儲存在記憶體中且由合適的指令執行系統執行的軟體或韌體來實現。例如,如果用硬體來實現,和在另一實施方式中一樣,可用本領域公知的下列技術中的任一項或他們的組合來實現:具有用於對資料訊號實現邏輯功能的邏輯閘電路的離散邏輯電路,具有合適的組合邏輯閘電路的專用積體電路,可程式閘陣列(PGA)、現場可程式閘陣列(FPGA)等。It should be understood that each part of the application may be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the plurality of steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it may be implemented using any one or a combination of the following techniques known in the art: a logic gate circuit for implementing a logic function on the data signal Discrete logic circuits, dedicated integrated circuits with suitable combinational logic gate circuits, programmable gate array (PGA), field programmable gate array (FPGA), etc.
需要說明的是,在本說明書的描述中,參考術語“一實施例”、“一些實施例”、 “示例”、“具體示例”,或“一些示例”等的描述意指結合該實施例或示例描述的具體特徵、結構、材料或者特點包含於本申請的至少一實施例或示例中。在本說明書中,對上述術語的示意性表述不必須針對的是相同的實施例或示例。而且,描述的具體特徵、結構、材料或者特點可以在任一或複數實施例或示例中以合適的方式結合。此外,在不相互矛盾的情況下,本領域的技術人員可以將本說明書中描述的不同實施例或示例以及不同實施例或示例的特徵進行結合和組合。It should be noted that, in the description of the present specification, the description with reference to the terms “an embodiment”, “some embodiments”, “examples”, “specific examples”, or “some examples” means in combination with this embodiment or The specific features, structures, materials, or characteristics described in the examples are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any or multiple embodiments or examples. In addition, without any contradiction, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.
在本說明書的描述中,參考術語“一實施例”、“一些實施例”、 “示例”、“具體示例”,或“一些示例”等的描述意指結合該實施例或示例描述的具體特徵、結構、材料或者特點包含於本申請的至少一實施例或示例中。在本說明書中,對上述術語的示意性表述不必須針對的是相同的實施例或示例。而且,描述的具體特徵、結構、材料或者特點可以在任一或複數實施例或示例中以合適的方式結合。此外,在不相互矛盾的情況下,本領域的技術人員可以將本說明書中描述的不同實施例或示例以及不同實施例或示例的特徵進行結合和組合。In the description of this specification, the description with reference to the terms “an embodiment”, “some embodiments”, “examples”, “specific examples”, or “some examples” and the like means specific features described in conjunction with the embodiments or examples , Structure, materials or features are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any or multiple embodiments or examples. In addition, without any contradiction, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.
儘管上面已經示出和描述了本申請的實施例,可以理解的是,上述實施例是示例性的,不能理解為對本申請的限制,本領域的普通技術人員在本申請的範圍內可以對上述實施例進行變化、修改、替換和變型。Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limitations on the present application. Those skilled in the art may, within the scope of the present application, understand the above. Embodiments are subject to change, modification, substitution, and modification.
10‧‧‧感光單元陣列10‧‧‧ Photosensitive unit array
11‧‧‧對焦感光單元11‧‧‧Focus sensor
12‧‧‧非對焦感光單元12‧‧‧ Non-focusing photosensitive unit
20‧‧‧濾光單元陣列20‧‧‧ filter unit array
30‧‧‧微透鏡陣列30‧‧‧Micro lens array
31‧‧‧第一微透鏡31‧‧‧The first micro lens
32‧‧‧第二微透鏡32‧‧‧Second micro lens
33‧‧‧矩形微透鏡33‧‧‧ rectangular micro lens
100‧‧‧影像感測器100‧‧‧Image Sensor
110‧‧‧感光像素110‧‧‧sensitive pixels
1700‧‧‧成像裝置1700‧‧‧imaging device
1710‧‧‧影像感測器1710‧‧‧Image Sensor
1720‧‧‧檢測模組1720‧‧‧Detection Module
1730‧‧‧控制模組1730‧‧‧Control Module
1810‧‧‧殼體1810‧‧‧shell
1820‧‧‧處理器1820‧‧‧Processor
1830‧‧‧記憶體1830‧‧‧Memory
1840‧‧‧電路板1840‧‧‧Circuit Board
1850‧‧‧電源電路1850‧‧‧Power Circuit
B00、B01、B02、B03、Gb10、Gb11、Gb12、Gb13、Gp30、Gp31、Gp32、Gp33‧‧‧輸出值B 00 , B 01 , B 02 , B 03 , Gb 10 , Gb 11 , Gb 12 , Gb 13 , Gp 30 , Gp 31 , Gp 32 , Gp 33 ‧‧‧ output value
G1、G3、G5、G7‧‧‧關聯像素單元G 1 , G 3 , G 5 , G 7 ‧‧‧ associated pixel units
Gp‧‧‧對焦感光單元Gp‧‧‧Focus sensor
Gp1‧‧‧第一相位值Gp 1 ‧‧‧ first phase value
Gp2‧‧‧第二相位值Gp 2 ‧‧‧ second phase value
Gr23‧‧‧像素Gr 23 ‧‧‧ pixels
R23‧‧‧關聯像素R 23 ‧‧‧ associated pixels
本申請上述的和/或附加的方面和優點從下面結合附圖對實施例的描述中將變得明顯和容易理解,其中: 第1圖是根據本申請的一實施例的影像感測器的立體結構圖; 第2圖是根據本申請的一實施例的影像感測器的影像處理方法的流程圖; 第3圖是根據本申請的一實施例的將對焦感光單元2*2個感光像素分為左側和右側兩部分進行對焦的效果示意圖; 第4圖是根據本申請的一實施例的將對焦感光單元2*2個感光像素分為上側和下側兩部分的示意圖; 第5圖是根據本申請的一實施例的以對焦感光單元的兩條對角線將2*2個感光像素分為兩部分的示意圖; 第6圖是根據本申請的一實施例的第一輸出模式下成像的效果示意圖; 第7圖是根據本申請的一實施例的第二輸出模式下成像模式的影像處理方法的流程圖; 第8圖是根據本申請的一實施例的對接收到紅光的感光單元進行插值還原的示意圖; 第9圖是根據本申請的一實施例的對接收到綠光的感光單元進行插值還原的示意圖; 第10圖是根據本申請的一實施例的影像感測器的剖面圖; 第11圖是根據本申請的一實施例的N和M的取值為2時影像感測器的俯視圖; 第12圖是根據本申請的另一實施例的影像感測器的剖面圖; 第13圖是根據本申請的一實施例的影像感測器中對焦感光單元的分佈示意圖; 第14圖是根據本申請的另一實施例的影像感測器中對焦感光單元的分佈示意圖; 第15圖是根據本申請的一實施例的對焦感光單元Gp與影像感測器中心點的距離的示意圖; 第16圖是根據本申請的一實施例的處理對焦感光單元Gp的示意圖; 第17圖是根據本申請的一實施例的成像裝置的框圖; 第18圖是根據本申請的一實施例的行動終端的結構示意圖; 第19圖是根據本申請的另一實施例的將對焦感光單元2*2個感光像素分為左側和右側兩部分進行對焦的效果示意圖; 第20圖是根據本申請的另一實施例的將對焦感光單元2*2個感光像素分為上側和下側兩部分的示意圖; 第21圖是根據本申請的另一實施例的以對焦感光單元的兩條對角線將2*2個感光像素分為兩部分的示意圖; 第22圖是根據本申請的另一實施例的第一輸出模式下成像的效果示意圖。The above and / or additional aspects and advantages of the present application will become apparent and easy to understand from the following description of the embodiments with reference to the accompanying drawings, wherein: FIG. 1 is a diagram of an image sensor according to an embodiment of the present application; 3D structure diagram; FIG. 2 is a flowchart of an image processing method of an image sensor according to an embodiment of the present application; and FIG. 3 is a diagram of focusing a photosensitive unit 2 * 2 photosensitive pixels according to an embodiment of the present application The effect of focusing is divided into two parts on the left and right; FIG. 4 is a schematic diagram of dividing the 2 * 2 photosensitive pixels of the focusing photosensitive unit into upper and lower parts according to an embodiment of the present application; FIG. 5 is A schematic diagram of dividing 2 * 2 photosensitive pixels into two parts by using two diagonal lines of a focusing photosensitive unit according to an embodiment of the present application; FIG. 6 is an image in a first output mode according to an embodiment of the present application FIG. 7 is a flowchart of an image processing method for an imaging mode in a second output mode according to an embodiment of the present application; and FIG. 8 is a photoreceptor for receiving red light according to an embodiment of the present application single Schematic diagram of interpolation restoration; FIG. 9 is a schematic diagram of interpolation restoration of a photosensitive unit receiving green light according to an embodiment of the application; and FIG. 10 is a cross-section of an image sensor according to an embodiment of the application 11 is a top view of an image sensor when N and M have a value of 2 according to an embodiment of the present application; FIG. 12 is a cross-sectional view of an image sensor according to another embodiment of the present application 13 is a schematic diagram of the distribution of focus sensitive units in an image sensor according to an embodiment of the present application; FIG. 14 is a schematic diagram of the distribution of focus sensitive units in an image sensor according to another embodiment of the present application; FIG. 15 is a schematic diagram of the distance between the focus sensitive unit Gp and the center point of the image sensor according to an embodiment of the present application; FIG. 16 is a schematic diagram of the focus sensitive unit Gp according to an embodiment of the present application; FIG. Is a block diagram of an imaging device according to an embodiment of the present application; FIG. 18 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application; FIG. 19 is a schematic view of a mobile terminal according to another embodiment of the present application; Schematic diagram of the effect of focusing the 2 * 2 photosensitive pixels of the focusing photosensitive unit into two parts on the left and the right; FIG. 20 is an illustration of dividing the 2 * 2 photosensitive pixels of the focusing photosensitive unit into an upper side and a lower side according to another embodiment of the present application A schematic diagram of two parts on the side; FIG. 21 is a schematic diagram of dividing 2 * 2 photosensitive pixels into two parts by two diagonal lines of a focusing photosensitive unit according to another embodiment of the present application; FIG. 22 is a diagram according to the present application A schematic diagram of an imaging effect in a first output mode of another embodiment of the present invention.
Claims (44)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710297653.4 | 2017-04-28 | ||
??201710297653.4 | 2017-04-28 | ||
CN201710297653.4A CN107105141B (en) | 2017-04-28 | 2017-04-28 | Imaging sensor, image processing method, imaging device and mobile terminal |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201839487A true TW201839487A (en) | 2018-11-01 |
TWI651582B TWI651582B (en) | 2019-02-21 |
Family
ID=59657465
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW107100949A TWI651582B (en) | 2017-04-28 | 2018-01-10 | Image sensor, image processing method, imaging device, and mobile terminal |
Country Status (7)
Country | Link |
---|---|
US (1) | US10484627B2 (en) |
EP (1) | EP3396942B1 (en) |
JP (1) | JP6878604B2 (en) |
KR (1) | KR102161557B1 (en) |
CN (1) | CN107105141B (en) |
TW (1) | TWI651582B (en) |
WO (1) | WO2018196568A1 (en) |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107105141B (en) | 2017-04-28 | 2019-06-28 | Oppo广东移动通信有限公司 | Imaging sensor, image processing method, imaging device and mobile terminal |
US11019294B2 (en) | 2018-07-18 | 2021-05-25 | Apple Inc. | Seamless readout mode transitions in image sensors |
US10848693B2 (en) * | 2018-07-18 | 2020-11-24 | Apple Inc. | Image flare detection using asymmetric pixels |
CN108900772A (en) * | 2018-07-19 | 2018-11-27 | 维沃移动通信有限公司 | A kind of mobile terminal and image capturing method |
CN109167941B (en) * | 2018-11-09 | 2021-02-09 | 德淮半导体有限公司 | Image sensor and method for manufacturing the same |
KR102624107B1 (en) * | 2018-12-06 | 2024-01-12 | 삼성전자주식회사 | Image Sensor for generating Depth Data by a path difference of light generated through Micro Lens covering a plurality of sub-pixels and Electronic Device including the image sensor |
US11245878B2 (en) * | 2019-02-28 | 2022-02-08 | Qualcomm Incorporated | Quad color filter array image sensor with aperture simulation and phase detection |
CN110062144A (en) * | 2019-05-14 | 2019-07-26 | 德淮半导体有限公司 | Phase focus image sensor and forming method thereof, working method |
KR102662032B1 (en) * | 2019-06-19 | 2024-05-03 | 삼성전자주식회사 | Image sensor and electrical device including image sensor |
CN112444793B (en) * | 2019-08-30 | 2024-07-23 | Oppo广东移动通信有限公司 | Time-of-flight TOF system and TOF transmitter |
KR20210080875A (en) | 2019-12-23 | 2021-07-01 | 삼성전자주식회사 | Electronic device comprising image sensor and method of operation thereof |
TWI738322B (en) * | 2020-01-20 | 2021-09-01 | 勝薪科技股份有限公司 | Electronic device |
US11373431B2 (en) | 2020-01-20 | 2022-06-28 | Visual Sensing Technology Co., Ltd. | Electronic device |
KR20210099379A (en) * | 2020-02-04 | 2021-08-12 | 삼성전자주식회사 | The electronic device and the method for performing auto focus |
KR20210156493A (en) | 2020-06-18 | 2021-12-27 | 에스케이하이닉스 주식회사 | Image sensing device |
WO2022000485A1 (en) * | 2020-07-03 | 2022-01-06 | 深圳市汇顶科技股份有限公司 | Photoelectric conversion unit, image sensor, and focusing method |
CN114467176A (en) * | 2020-08-03 | 2022-05-10 | 深圳市汇顶科技股份有限公司 | Image sensor, image processing method and device and imaging system |
US11563910B2 (en) | 2020-08-04 | 2023-01-24 | Apple Inc. | Image capture devices having phase detection auto-focus pixels |
KR20220051881A (en) | 2020-10-19 | 2022-04-27 | 삼성전자주식회사 | Image sensor configured to improve artifact in binning mode |
KR20220077735A (en) | 2020-12-02 | 2022-06-09 | 삼성전자주식회사 | Image sensor and pixel array of image sensor |
US11546532B1 (en) | 2021-03-16 | 2023-01-03 | Apple Inc. | Dynamic correlated double sampling for noise rejection in image sensors |
KR20230014490A (en) * | 2021-07-21 | 2023-01-30 | 삼성전자주식회사 | Method for providing auto focus function and electronic device thereof |
Family Cites Families (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6638786B2 (en) | 2002-10-25 | 2003-10-28 | Hua Wei Semiconductor (Shanghai ) Co., Ltd. | Image sensor having large micro-lenses at the peripheral regions |
US7505206B2 (en) * | 2006-07-10 | 2009-03-17 | Taiwan Semiconductor Manufacturing Company | Microlens structure for improved CMOS image sensor sensitivity |
JP4961993B2 (en) * | 2006-12-18 | 2012-06-27 | 株式会社ニコン | Imaging device, focus detection device, and imaging device |
JP5672989B2 (en) | 2010-11-05 | 2015-02-18 | ソニー株式会社 | Imaging device |
DE102013102667A1 (en) * | 2013-03-15 | 2014-10-02 | Osram Opto Semiconductors Gmbh | display device |
US9372286B2 (en) * | 2013-04-11 | 2016-06-21 | Omnivision Technologies, Inc. | Method of forming dual size microlenses for image sensors |
KR102039464B1 (en) * | 2013-05-21 | 2019-11-01 | 삼성전자주식회사 | Electronic sensor and control method of the same |
CN103531603B (en) * | 2013-10-30 | 2018-10-16 | 上海集成电路研发中心有限公司 | A kind of cmos image sensor |
WO2015126778A1 (en) * | 2014-02-24 | 2015-08-27 | 3M Innovative Properties Company | Optical device including remote downconverter |
EP3171587A4 (en) * | 2014-07-16 | 2018-03-14 | Sony Corporation | Compound-eye imaging device |
CN104064577A (en) * | 2014-07-16 | 2014-09-24 | 上海集成电路研发中心有限公司 | Image sensor achieving automatic focusing |
KR20160010986A (en) * | 2014-07-21 | 2016-01-29 | 에스케이하이닉스 주식회사 | Image sensor |
KR102294316B1 (en) * | 2014-08-04 | 2021-08-26 | 엘지이노텍 주식회사 | Image sensor and image pick-up apparatus including the same |
CN104241310B (en) * | 2014-09-23 | 2017-11-07 | 上海集成电路研发中心有限公司 | A kind of CMOS image pixel array with double micro lens layer |
KR102195407B1 (en) | 2015-03-16 | 2020-12-29 | 삼성전자주식회사 | Image signal processor and devices having the same |
US9853076B2 (en) * | 2015-04-16 | 2017-12-26 | Taiwan Semiconductor Manufacturing Co., Ltd. | Stacked grid for more uniform optical input |
WO2016181626A1 (en) * | 2015-05-14 | 2016-11-17 | パナソニックIpマネジメント株式会社 | Imaging device |
US10002899B2 (en) * | 2015-09-16 | 2018-06-19 | Taiwan Semiconductor Manufacturing Co., Ltd. | Microlens for a phase detection auto focus (PDAF) pixel of a composite grid structure |
US10044959B2 (en) | 2015-09-24 | 2018-08-07 | Qualcomm Incorporated | Mask-less phase detection autofocus |
KR102390836B1 (en) | 2015-10-05 | 2022-04-26 | 삼성전자주식회사 | Electronic device and method for generating an image data |
CN105611122B (en) * | 2015-12-18 | 2019-03-01 | Oppo广东移动通信有限公司 | Imaging sensor and output method, phase focusing method, imaging device and terminal |
CN107040724B (en) | 2017-04-28 | 2020-05-15 | Oppo广东移动通信有限公司 | Dual-core focusing image sensor, focusing control method thereof and imaging device |
CN106982329B (en) | 2017-04-28 | 2020-08-07 | Oppo广东移动通信有限公司 | Image sensor, focusing control method, imaging device and mobile terminal |
CN107105141B (en) | 2017-04-28 | 2019-06-28 | Oppo广东移动通信有限公司 | Imaging sensor, image processing method, imaging device and mobile terminal |
CN107135340A (en) | 2017-04-28 | 2017-09-05 | 广东欧珀移动通信有限公司 | Imaging sensor, focusing control method, imaging device and mobile terminal |
CN107040702B (en) | 2017-04-28 | 2020-06-05 | Oppo广东移动通信有限公司 | Image sensor, focusing control method, imaging device and mobile terminal |
-
2017
- 2017-04-28 CN CN201710297653.4A patent/CN107105141B/en active Active
-
2018
- 2018-01-10 TW TW107100949A patent/TWI651582B/en active
- 2018-02-09 US US15/893,475 patent/US10484627B2/en active Active
- 2018-03-21 EP EP18163006.2A patent/EP3396942B1/en active Active
- 2018-04-03 KR KR1020197018149A patent/KR102161557B1/en active IP Right Grant
- 2018-04-03 JP JP2019540603A patent/JP6878604B2/en active Active
- 2018-04-03 WO PCT/CN2018/081781 patent/WO2018196568A1/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
JP2020505863A (en) | 2020-02-20 |
KR102161557B1 (en) | 2020-10-06 |
US20180316878A1 (en) | 2018-11-01 |
EP3396942A1 (en) | 2018-10-31 |
TWI651582B (en) | 2019-02-21 |
US10484627B2 (en) | 2019-11-19 |
WO2018196568A1 (en) | 2018-11-01 |
EP3396942B1 (en) | 2020-05-13 |
KR20190089017A (en) | 2019-07-29 |
CN107105141B (en) | 2019-06-28 |
JP6878604B2 (en) | 2021-05-26 |
CN107105141A (en) | 2017-08-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI651582B (en) | Image sensor, image processing method, imaging device, and mobile terminal | |
TWI636314B (en) | Dual-core focus image sensor and focus control method, imaging device and mobile terminal | |
US10764522B2 (en) | Image sensor, output method, phase focusing method, imaging device, and terminal | |
CN106982329B (en) | Image sensor, focusing control method, imaging device and mobile terminal | |
KR20200031689A (en) | Image processing methods, devices and devices | |
WO2018196703A1 (en) | Image sensor, focusing control method, imaging device and mobile terminal | |
WO2018099011A1 (en) | Image processing method, image processing device, imaging device, and electronic device | |
KR20120017006A (en) | Image capture using luminance and chrominance sensors | |
WO2018196704A1 (en) | Dual-core focusing image sensor, focusing control method thereof, and imaging device | |
CN107040702B (en) | Image sensor, focusing control method, imaging device and mobile terminal | |
WO2018099030A1 (en) | Control method and electronic device | |
JP6800650B2 (en) | Reliability generator, reliability generator, and imaging device | |
KR20190051371A (en) | Camera module including filter array of complementary colors and electronic device including the camera module | |
JP5183441B2 (en) | Imaging device | |
JP6678505B2 (en) | Imaging device, control method therefor, program, and storage medium | |
JP2015203756A (en) | Parallax amount calculation device, distance calculation device, imaging apparatus, and parallax amount calculation method | |
US20230217126A1 (en) | Imaging apparatus, imaging method | |
JP2015204579A (en) | Imaging apparatus | |
JP2019191681A (en) | Image processing apparatus, control method thereof, and program | |
CN117356102A (en) | Lens positioning for auxiliary camera in multi-camera system |