WO2020015627A1 - 图像传感器、移动终端及图像拍摄方法 - Google Patents
图像传感器、移动终端及图像拍摄方法 Download PDFInfo
- Publication number
- WO2020015627A1 WO2020015627A1 PCT/CN2019/096128 CN2019096128W WO2020015627A1 WO 2020015627 A1 WO2020015627 A1 WO 2020015627A1 CN 2019096128 W CN2019096128 W CN 2019096128W WO 2020015627 A1 WO2020015627 A1 WO 2020015627A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- sub
- infrared
- pixels
- green
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/131—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements including elements passing infrared wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/53—Constructional details of electronic viewfinders, e.g. rotatable or detachable
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
- H04N25/704—Pixels specially adapted for focusing, e.g. phase difference pixel sets
Definitions
- the present disclosure relates to the field of image processing technologies, and in particular, to an image sensor, a mobile terminal, and an image capturing method.
- CMOS complementary metal oxide semiconductor
- R red
- G green
- B blue
- Bayer pixel array arrangement pattern As shown in Figures 1a and 1b, this arrangement cannot detect the distance of objects, and can only be used to receive natural light and take pictures to record images under normal lighting.
- the pixel array arrangement pattern of full pixel dual core focus (2PD) technology is shown in Figure 1c and Figure 1d.
- This arrangement method can only be used to receive natural light and used to take pictures and record images, but relatively Phase detection autofocus (PDAF) technology solution can increase the distance of the detected object and complete the focusing action more quickly.
- PDAF Phase detection autofocus
- the pixel array arrangement of CMOS image sensors cannot detect the distance of objects and can only accept natural light.
- the pixel array arrangement of 2PD technology can detect the distance of objects, it can only accept natural light, so the images in related technologies
- the sensor's pixel array arrangement mode has the problems of limited shooting scenes, slow focusing, and affecting the user's shooting experience.
- Some embodiments of the present disclosure provide an image sensor, a mobile terminal, and an image shooting method to solve the problems of limited shooting scenes, slow focusing, and affecting the user's shooting experience when shooting in related technologies.
- an image sensor including:
- a pixel array including a preset number of pixel units arranged in a predetermined manner, the pixel unit including a first pixel and a second pixel adjacent to the first pixel position, the first pixel including a red sub-pixel, a green sub-pixel, and A blue sub-pixel, a second pixel including at least one of a red sub-pixel and a blue sub-pixel, and a green sub-pixel and an infrared sub-pixel, and the first pixel and the second pixel are all full-pixel dual-core focus pixels, the first pixel And the second pixel each include four full-pixel dual-core focus sub-pixels;
- the position of the infrared sub-pixel in the second pixel is the same as the position of the red sub-pixel, the green sub-pixel, the blue sub-pixel, the first combined sub-pixel, or the second combined sub-pixel in the first pixel.
- a pixel is a combination of 1/2 red subpixels and 1/2 green subpixels adjacent to each other, and a second combined subpixel is a combination of 1/2 green subpixels and 1/2 blue subpixels adjacent to each other; or
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel, 1/2 green sub-pixel, or 1/2 blue sub-pixel in the first pixel, and two adjacent The 1/2 infrared sub-pixels in the second pixel are combined to form a complete infrared sub-pixel.
- some embodiments of the present disclosure provide a mobile terminal including an imaging system and an infrared transmitting module.
- the imaging system includes the foregoing image sensor, and further includes:
- a driving module for driving the lens module to move
- a filtering module provided between the lens module and the image sensor
- An image data processing module connected to the image sensor
- a display module connected to the image data processing module
- the infrared emitting module is disposed on the periphery of the lens module.
- some embodiments of the present disclosure provide an image shooting method, which is applied to a mobile terminal.
- the mobile terminal includes the image sensor described above, and further includes an infrared transmitting module.
- the method includes:
- Emitting infrared light through an infrared transmitting module Emitting infrared light through an infrared transmitting module
- stereoscopic information acquisition is performed on the object to be photographed.
- an image sensor is formed by using a pixel array combining 2PD RGB sub-pixels and infrared sub-pixels, and shooting is performed according to the image sensor, and the distance between the object to be photographed and the mobile terminal can be detected based on the recorded image.
- FIG. 1a is a schematic diagram of a conventional RGB arrangement in the related art
- Figure 1b shows a cross-sectional view of a conventional pixel
- FIG. 1c shows a pixel array layout of 2PD
- Figure 1d shows a cross-sectional view of a 2PD pixel
- FIG. 2a shows one of the schematic diagrams of a pixel unit according to some embodiments of the present disclosure
- FIG. 2b shows a second schematic diagram of a pixel unit in some embodiments of the present disclosure
- 2c shows a third schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 3a shows a fourth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- 3b shows a fifth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 4a shows the sixth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- 4b shows a seventh schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 5a shows the eighth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 5b shows a ninth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 6a shows the tenth schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 6b shows the eleventh schematic diagram of a pixel unit according to some embodiments of the present disclosure
- FIG. 7 shows a pixel cross-sectional view of some embodiments of the present disclosure
- FIG. 8a shows a schematic diagram of a mobile terminal according to some embodiments of the present disclosure
- FIG. 8b shows a schematic diagram of an imaging system according to some embodiments of the present disclosure
- FIG. 9 is a schematic diagram of an image capturing method according to some embodiments of the present disclosure.
- FIG. 10 is a schematic diagram of a hardware structure of a mobile terminal according to some embodiments of the present disclosure.
- Some embodiments of the present disclosure provide an image sensor including a pixel array including a preset number of pixel units arranged in a predetermined manner, as shown in FIGS. 2a to 2c, FIGS. 3a to 3b, and FIGS. 4a to 4b.
- the pixel unit includes a first pixel and a second pixel adjacent to the first pixel position.
- the first pixel includes a red sub-pixel, a green sub-pixel, and a blue sub-pixel.
- the second pixel includes a red sub-pixel and a blue sub-pixel.
- At least one of the pixels, the green sub-pixel and the infrared sub-pixel, and the first pixel and the second pixel are all full-pixel dual-core focus pixels, and the first pixel and the second pixel each include four full-pixel dual-core focus sub-pixels;
- the position of the infrared sub-pixel in the second pixel is the same as the position of the red sub-pixel, the green sub-pixel, the blue sub-pixel, the first combined sub-pixel, or the second combined sub-pixel in the first pixel.
- a pixel is a combination of 1/2 red subpixels and 1/2 green subpixels adjacent to each other, and a second combined subpixel is a combination of 1/2 green subpixels and 1/2 blue subpixels adjacent to each other; or
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel, 1/2 green sub-pixel, or 1/2 blue sub-pixel in the first pixel, and two adjacent The 1/2 infrared sub-pixels in the second pixel are combined to form a complete infrared sub-pixel.
- the pixel array included in the image sensor includes a preset number of pixel units, where the preset number of pixel units are arranged in a predetermined manner.
- the preset number of pixel units each include a first pixel and a second pixel.
- the first pixel is different from the second pixel.
- the first pixel includes a red sub-pixel (R), a green sub-pixel (G), and a blue sub-pixel.
- Pixel (B) the second pixel includes at least one of a red sub-pixel and a blue sub-pixel, and further includes a green sub-pixel and an infrared sub-pixel (IR).
- the first pixel and the second pixel in some embodiments of the present disclosure are all-pixel dual-core focus (2PD) pixels.
- 2PD pixels By using the 2PD pixels, the object distance can be detected and the focusing action can be completed more quickly.
- the first pixel and The second pixel is a 2PD pixel, that is, the sub-pixels in the first pixel and the second pixel are 2PD sub-pixels.
- the red, green, and blue sub-pixels in the first pixel are arranged in a certain manner, and the first pixel includes four full-pixel dual-core focus sub-pixels, which are a red sub-pixel and a blue sub-pixel, respectively.
- the color sub-pixel and the two green sub-pixels are referred to herein as a first green sub-pixel and a second green sub-pixel, respectively, in order to facilitate distinction, wherein the first green sub-pixel is the same as the second green sub-pixel.
- the red sub-pixel is adjacent to the first green sub-pixel, the second green sub-pixel is located below the red sub-pixel, the blue sub-pixel is located below the first green sub-pixel, and the second green sub-pixel is adjacent to the blue sub-pixel .
- the second pixel includes four full-pixel dual-core focus sub-pixels, specifically including at least one of a red sub-pixel and a blue sub-pixel, and further includes a green sub-pixel and an infrared sub-pixel, that is, the second pixel may include a red sub-pixel,
- the green sub-pixel and the infrared sub-pixel may include a green sub-pixel, a blue sub-pixel, and an infrared sub-pixel, and may further include a green sub-pixel, a red sub-pixel, a blue sub-pixel, and an infrared sub-pixel.
- the position of the infrared sub-pixel in the second pixel may be the same as the position of a certain sub-pixel in the first pixel, or may be the same as the position of two 1/2 different sub-pixels adjacent to each other in the first pixel.
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of any 1/2 sub-pixel in the first pixel.
- the 1/2 infrared of two adjacent second pixels can be used.
- the sub-pixels are combined to form a complete infrared sub-pixel.
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel in the first pixel, and the position of the 1/2 infrared sub-pixel in the second pixel is the same as the 1/2 green sub-pixel.
- the pixels are in the same position in the first pixel, so far, a complete infrared sub-pixel can be formed by combining two second pixels.
- mobile terminals can receive infrared light. Under the circumstances, image shooting is performed to realize dark state imaging to ensure the user shooting experience.
- the image sensor of some embodiments of the present disclosure can detect the distance between the object to be photographed and the mobile terminal, realize fast focusing and background blurring.
- the imaging effect of the image can be improved, and stereo photography can be realized.
- the position of the infrared sub-pixel in the second pixel is different from that of the red sub-pixel, green sub-pixel, blue sub-pixel, first
- the position of the combined sub-pixel or the second combined sub-pixel in the first pixel is the same.
- the first combined sub-pixel is a combination of 1/2 red sub-pixels and 1/2 green sub-pixels adjacent to each other; the second combined sub-pixel is A combination of 1/2 green subpixels and 1/2 blue subpixels adjacent to each other.
- the infrared sub-pixel replaces the red sub-pixel in the first pixel.
- the position of the infrared sub-pixel in the second pixel is the same as that of the red sub-pixel.
- the pixels are in the same position in the first pixel.
- the infrared sub-pixel replaces the blue sub-pixel in the first pixel.
- the position of the infrared sub-pixel in the second pixel and the blue The positions of the sub-pixels in the first pixel are the same.
- the infrared sub-pixel may replace a green sub-pixel in the first pixel.
- the infrared sub-pixel is in the second pixel.
- the position in is the same as the position of the green sub-pixel in the first pixel.
- the second pixel may include a red subpixel, a green subpixel, a 1/2 blue subpixel, 1/2 green sub-pixel and one infrared sub-pixel, the infrared sub-pixel replaces the 1/2 green sub-pixel and 1/2 blue sub-pixel in the first pixel, that is, the position of the infrared sub-pixel in the second pixel
- the positions of the 1/2 green sub-pixel and the 1/2 blue sub-pixel adjacent to the position are the same in the first pixel.
- the second pixel may also include a blue sub-pixel, a green sub-pixel, a 1/2 red sub-pixel, 1/2 green sub-pixel and one infrared sub-pixel.
- the infrared sub-pixel replaces the 1/2 green sub-pixel and 1/2 red sub-pixel in the first pixel, that is, the position of the infrared sub-pixel in the second pixel is the same
- the positions of the 1 ⁇ 2 green sub-pixel and the 1 ⁇ 2 red sub-pixel adjacent to each other in the first pixel are the same.
- the pixel unit when the position of the infrared sub-pixel in the second pixel is in the first pixel with the red sub-pixel, green sub-pixel, blue sub-pixel, first combined sub-pixel, or second combined sub-pixel, the pixel unit includes a second pixel and at least one first pixel.
- the pixel unit includes a second pixel and a first pixel, the point density of the infrared sub-pixel in the pixel unit is 1/8. For example, as shown in FIG.
- the pixel unit includes a first pixel and a second pixel, where the second pixel includes a red sub-pixel, two green sub-pixels, and an infrared sub-pixel, and the first pixel includes one The red sub-pixel, two green sub-pixels, and one blue sub-pixel.
- the ratio of the infrared sub-pixels in the pixel unit is 1/8, that is, the point density of the infrared sub-pixels is 1/8.
- the density of the infrared sub-pixels in the pixel unit is 1/12.
- the pixel unit includes two first pixels and one second pixel.
- the second pixel includes a blue sub-pixel, two green sub-pixels, and an infrared sub-pixel. It includes one red sub-pixel, two green sub-pixels, and one blue sub-pixel.
- the ratio of the infrared sub-pixels in the pixel unit is 1/12, that is, the point density of the infrared sub-pixels is 1/12.
- the density of the infrared sub-pixels in the pixel unit is 1/16.
- the pixel unit includes three first pixels and one second pixel, where the second pixel includes a blue subpixel, a green subpixel, a 1/2 red subpixel, and a 1/2
- the green sub-pixel and an infrared sub-pixel can be based on the first pixel.
- the 1/2 red sub-pixel and the 1/2 green sub-pixel of the 2PD sub-pixel can be taken as the infrared sub-pixel.
- the first pixel includes a red sub-pixel, Two green sub-pixels and one blue sub-pixel. At this time, the ratio of the infrared sub-pixels in the pixel unit is 1/16, that is, the point density of the infrared sub-pixels is 1/16.
- the above corresponding access points are only used for illustration, and may be other access points, which will not be introduced one by one here.
- the density of the infrared sub-pixels in the pixel unit is 1 / 4n, and n is an integer greater than or equal to 2, and the size of the pixel array applicable to the infrared sub-pixels is not limited.
- the blue sub-pixels have the same position in the first pixel, and the 1/2 infrared sub-pixels in two adjacent second pixels are combined to form a complete infrared sub-pixel.
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel, 1/2 green sub-pixel, or 1/2 blue sub-pixel in the first pixel, and two phases
- the 1/2 infrared sub-pixels in the adjacent second pixel are combined to form a complete infrared sub-pixel, in the pixel unit, the number of the second pixels is two, and the number of the first pixels is greater than or equal to zero.
- the second pixel may include only 1/2 infrared sub-pixels. At this time, a complete infrared sub-pixel may be obtained by combining two adjacent second pixels.
- the position of the 1/2 infrared sub-pixel in the second pixel may be the same as the position of the 1/2 red sub-pixel in the first pixel, or may be The position of the green sub-pixel in the first pixel may be the same, or may be the same as the position of the 1/2 blue sub-pixel in the first pixel.
- the position of the 1/2 infrared sub-pixel in a second pixel is the same as the position of the 1/2 red sub-pixel in the first pixel, then the position of the 1/2 infrared sub-pixel in another second pixel is equal to 1
- the position of the / 2 green sub-pixel in the first pixel is the same.
- the position of the 1/2 infrared sub-pixel in a second pixel is the same as the position of the 1/2 green sub-pixel in the first pixel, then the position of the 1/2 infrared sub-pixel in another second pixel is 1
- the position of the / 2 blue sub-pixel or the 1/2 red sub-pixel in the first pixel is the same.
- the number of the second pixels is two, and the number of the first pixels is greater than or equal to zero.
- the pixel unit includes two second pixels and first pixels whose number is greater than or equal to zero.
- the density of the infrared sub-pixels in the pixel unit is 1/8.
- the pixel unit includes two second pixels, where a second pixel includes a red subpixel, a green subpixel, a blue subpixel, a 1/2 green subpixel, and a 1
- the / 2 infrared sub-pixel includes one red sub-pixel, two green sub-pixels, a 1/2 blue sub-pixel, and a 1/2 infrared sub-pixel in another second pixel.
- the position of the 1/2 infrared sub-pixel in a second pixel is the same as the position of the 1/2 green sub-pixel in the first pixel, and the position of the 1/2 infrared sub-pixel in another second pixel is 1 / 2
- the position of the blue sub-pixel in the first pixel is the same.
- the ratio of the infrared sub-pixels in the pixel unit is 1/8, that is, the point density of the infrared sub-pixels is 1/8.
- the point density of the infrared sub-pixels in the pixel unit is 1/12.
- the pixel unit includes two second pixels and a first pixel, where a second pixel includes a red sub-pixel, a green sub-pixel, a blue sub-pixel, 1/2 The green sub-pixel and the 1/2 infrared sub-pixel, and the other second pixel includes a 1/2 red sub-pixel, two green sub-pixels, a blue sub-pixel, and a 1/2 infrared sub-pixel.
- the position of the infrared sub-pixel in a second pixel is the same as the position of the 1/2 green sub-pixel in the first pixel, and the position of the 1/2 infrared sub-pixel in another second pixel is the same as the position of the 1/2 red sub-pixel.
- the positions in the first pixel are the same.
- the ratio of the infrared sub-pixels in the pixel unit is 1/12, that is, the point density of the infrared sub-pixels is 1/12.
- the point density of the infrared sub-pixels in the pixel unit is 1/16.
- the pixel unit includes two second pixels and two first pixels, where a second pixel includes a red sub-pixel, a green sub-pixel, a blue sub-pixel, 1 / 2 green sub-pixel and 1/2 infrared sub-pixel, including a blue sub-pixel, two green sub-pixels, 1/2 red sub-pixel, and 1/2 infrared sub-pixel in another second pixel, at this time 1 / 2
- the position of the infrared sub-pixel in a second pixel is the same as the position of the 1/2 green sub-pixel in the first pixel, and the position of the 1/2 infrared sub-pixel in the other second pixel is 1/2 of the red sub-pixel
- the position in the first pixel is the same.
- the ratio of the infrared sub-pixels in the pixel unit is 1/16
- the pixel array can be a 1/8 density RGB + IR pixel unit, a 1/12 density RGB + IR pixel unit or a 1/16 density RGB + IR pixel unit as a pixel unit array, and the pixel unit array is periodically The array is composed.
- the pixel array can also be in other forms, which will not be enumerated here.
- infrared sub-pixel picking which can also be modified on this basis, or adopt other point picking methods, that is, the infrared sub-pixel picking methods of some embodiments of the present disclosure are not done. Further limitation, the corresponding multiple implementation manners are not described here one by one.
- the position of the infrared sub-pixel in the pixel unit (the position of the second pixel) is not specifically limited in some embodiments of the present disclosure.
- the density of the infrared sub-pixels in the pixel unit is 1 / 4n, and n is an integer greater than or equal to 2, and the size of the pixel array applicable to the infrared sub-pixels is not limited.
- the red, green, or blue sub-pixels of the 2PD sub-pixel can be taken as infrared sub-pixels, which facilitates the diversification of the pixel unit form. It can also take 1/2 red and 1/2 green sub-pixels of the 2PD sub-pixel. Pixels or take 1/2 blue and 1/2 green sub-pixels as infrared sub-pixels, which can reduce the effect of IR dead pixels during RGB processing.
- the red sub-pixel includes a semiconductor layer, a metal layer, a photodiode, a red filter, and a micromirror, which are sequentially stacked
- the green sub-pixel includes a semiconductor layer, a metal layer, and a photodiode, which are sequentially stacked.
- Green filters and micromirrors; blue subpixels include semiconductor layers, metal layers, photodiodes, blue filters and micromirrors stacked in sequence; infrared subpixels include semiconductor layers, metal layers, Photodiodes, infrared filters and micromirrors.
- the semiconductor layer, metal layer, photodiode, red filter, and micromirror included in the red sub-pixel are arranged in order from bottom to top.
- the semiconductor layer, metal layer, photodiode, green filter, and micromirror included in the corresponding green sub-pixel are arranged in order from bottom to top.
- the semiconductor layer, the metal layer, the photodiode, the blue filter, and the micromirror included in the blue sub-pixel are arranged in order from bottom to top.
- the semiconductor layer, metal layer, photodiode, infrared filter, and micromirror included in the infrared sub-pixel are arranged in order from bottom to top.
- the semiconductor layer here may be a silicon substrate, but is not limited thereto.
- the structure of the red, green, blue, and infrared sub-pixels can be seen in FIG. 7. Although only the blue and infrared sub-pixels are shown in FIG. 7, the structure of the red-green sub-pixels can be obtained based on this.
- the structure of the red sub-pixel or the green sub-pixel can be obtained by replacing the blue filter with a red or green filter.
- the red, green, and blue subpixels are used to obtain the color information of the pixels of the composite image, which block the entry of infrared rays; for example, only visible light with a wavelength of 380 to 700nm is allowed to enter, which can directly generate a complete and realistic image under high illumination .
- the infrared sub-pixel has a photosensitive bandwidth of 750 to 1100 nm.
- the infrared filter area can be used to pass the infrared band, which can improve the imaging effect in the dark state and realize the infrared ranging function.
- the RGB sub-pixel point is a light receiving element corresponding to wavelength light of each RGB color
- the IR sub-pixel point is a light receiving element corresponding to infrared light.
- the image sensor is a complementary metal oxide semiconductor CMOS image sensor, a charge-coupled element CCD image sensor, or a quantum thin film image sensor.
- the RGB-IR pixel array arrangement of the present disclosure is not limited to the applicable image sensor type. It can be a CMOS-based image sensor, a charge-coupled device (CCD) -based image sensor, or a CCD-based image sensor. Quantum film image sensors can of course also be other types of image sensors. And the image sensor of some embodiments of the present disclosure can be applied to any electronic product including a camera module.
- the mobile terminal 1 includes an imaging system 2 and an infrared transmitting module 3.
- the imaging system 2 includes the image sensor 21 described above, and further includes:
- the display module 26 is connected to the data processing module 25.
- the infrared emitting module 3 is disposed on the periphery of the lens module 22.
- the mobile terminal 1 includes an imaging system 2 and further includes an infrared emitting module 3, wherein the imaging system 2 includes the above-mentioned image sensor 21, and the imaging system 2 further includes a lens module 22 for focusing light.
- the module 22 is connected to the driving module 23, and the driving module 23 is configured to adjust the position of the lens module 22 according to the distance of the object to be photographed.
- a filter module 24 is provided between the lens module 22 and the image sensor 21, where light is focused by the lens module 22, and after passing through the filter module 24, it can be focused on the pixel array of the image sensor 21.
- the image sensor 21 is connected to an image data processing module 25, and the image data processing module 25 is connected to a display module 26. After the light is focused on the pixel array of the image sensor 21, after the image sensor 21 performs photoelectric conversion, the data is transmitted to the image data processing module 25. After the image data processing module 25 processes the data, it is displayed in the form of a picture on the display module 26 Render.
- the 2PD pixels in the image sensor 21 can be used to obtain the phase difference, so as to obtain the distance between the object and the imaging surface, thereby achieving fast focusing.
- the RGB + IR pixel array arrangement based on the 2PD image sensor in the present disclosure can cooperate with the infrared transmitting module 3 to realize stereo related functions, such as: face recognition unlocking, secure payment, stereo imaging and other terminal applications. On the basis, the functionality of the mobile terminal is improved.
- the filtering module 24 in some embodiments of the present disclosure may pass a light wavelength of 380 nm to 1100 nm. At this time, after the light is focused by the lens module 22, it can be filtered by the filtering module 24, wherein the filtering module 24 can be used to pass natural light and infrared light, and can be used to ensure the imaging effect of the imaging system 2.
- the infrared emitting module 3 on the mobile terminal is disposed on the periphery of the lens module 22.
- the infrared transmitting module 3 emits infrared rays, and the infrared rays will be reflected after encountering obstacles; when the imaging system 2 captures the reflected infrared rays, it performs photoelectric conversion through infrared sub-pixels, and can obtain the time difference between the infrared rays being emitted and being received. Because the speed of light propagation is fixed, the distance between the obstacle and the mobile terminal can be calculated, and finally the distance from each minimal unit on the obstacle to the mobile terminal can be obtained to realize the stereo imaging recording function. Of course, the distance between each infrared light reflection point on the obstacle and the mobile terminal can also be obtained by acquiring the phase difference of the infrared light.
- the mobile terminal of some embodiments of the present disclosure improves the arrangement of the pixel array of the image sensor, and improves the arrangement of the RGB pixel array to the arrangement of the RGB-IR pixel array of 2PD. Based on the recorded images, the distance between the object to be photographed and the mobile terminal is detected to achieve fast focusing and background blurring, improve the imaging effect of the image, and realize the related application functions of stereo photography and dark imaging to ensure the user's shooting experience. At the same time, the function of the mobile terminal is guaranteed to be diversified.
- Some embodiments of the present disclosure also provide an image shooting method, which is applied to a mobile terminal.
- the mobile terminal includes the image sensor described above, and further includes an infrared transmitting module. As shown in FIG. 9, the method includes:
- Step 901 Transmit infrared light through an infrared transmitting module.
- the infrared transmitting module on the mobile terminal can emit infrared rays.
- the infrared rays will be reflected after encountering the object to be photographed, and the reflected infrared light will be received by the imaging system of the mobile terminal.
- the image sensor of the mobile terminal forms an RGB-IR pixel array, so photoelectric conversion can be performed through infrared sub-pixels.
- Step 902 Obtain the distance between each infrared light reflection point on the to-be-photographed object and the mobile terminal according to the infrared light reflected by the to-be-photographed object.
- the process of actually acquiring the distance between the object to be photographed and the imaging surface, and acquiring the distance between each infrared light reflection point on the object to be photographed and the mobile terminal is: A pixel array including a second pixel receives infrared light reflected by each infrared light reflection point on the object to be photographed; according to the time difference between sending and receiving infrared light and the propagation speed of the infrared light or obtaining the phase difference of the infrared light, the object to be photographed is obtained The distance between each infrared light reflection point on the mobile terminal and the mobile terminal.
- photoelectric conversion is performed through the infrared sub-pixels to obtain the time difference between the infrared emission and the infrared reception. Since the light propagation speed is fixed, it can be calculated based on 1/2 of the product of the time difference and the propagation speed.
- the distance of the obstacle from the mobile terminal The time taken by the mobile terminal to receive the infrared light reflected by each infrared light reflection point is different. Therefore, a distance can be calculated for each infrared light reflection point, and the distance between each infrared light reflection point and the mobile terminal can be obtained.
- the distance between each infrared light reflection point and the mobile terminal can also be obtained by acquiring the phase difference of the infrared light.
- TOF Time of Flight
- the pixel array of the image sensor includes a preset number of pixel units arranged in a predetermined manner.
- the pixel unit includes a first pixel and a second pixel.
- the first pixel includes a red sub-pixel and a green sub-pixel.
- blue sub-pixel the second pixel includes at least one of a red sub-pixel and a blue sub-pixel, and a green sub-pixel and an infrared sub-pixel, and the first pixel and the second pixel are all full-pixel dual-core focus pixels.
- Each of the pixel and the second pixel includes four full-pixel dual-core focus sub-pixels, where the position of the infrared sub-pixel in the second pixel is the same as that of the red, green, blue, first combined sub-pixel, or first sub-pixel.
- the two combined subpixels have the same position in the first pixel.
- the first combined subpixel is a combination of 1/2 red subpixels and 1/2 green subpixels adjacent to each other, and the second combined subpixel is 1 adjacent to each other.
- Step 903 Obtain stereo information of the object to be photographed according to the distance between each infrared light reflection point on the object to be photographed and the mobile terminal.
- the shooting method of some embodiments of the present disclosure can improve the pixel array arrangement of an image sensor, and improve the RGB pixel array arrangement to a 2PD RGB-IR pixel array arrangement, which can receive infrared light. Based on the recorded image, the distance between the object to be photographed and the mobile terminal is detected to achieve fast focusing and background blurring, improve the imaging effect of the image, and realize the related application functions of stereo photography and dark imaging, ensuring the user's shooting At the same time, the functions of the mobile terminal are guaranteed to be diversified.
- FIG. 10 is a schematic diagram of a hardware structure of a mobile terminal that implements various embodiments of the present disclosure.
- the mobile terminal 1000 includes, but is not limited to, a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, and a display unit. 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010, and a power supply 1011.
- the mobile terminal 1000 further includes an imaging system and an infrared emitting module.
- the imaging system includes an image sensor and a lens module; a driving module for driving the movement of the lens module; a filtering module provided between the lens module and the image sensor; and an image A sensor-connected image data processing module; and a display module connected to the image data processing module.
- the filter module can pass the light wavelength of 380nm to 1100nm, and the infrared emission module is arranged on the periphery of the lens module.
- the image sensor includes:
- a pixel array including a preset number of pixel units arranged in a predetermined manner, the pixel unit including a first pixel and a second pixel adjacent to the first pixel position, the first pixel including a red sub-pixel, a green sub-pixel, and A blue sub-pixel, a second pixel including at least one of a red sub-pixel and a blue sub-pixel, and a green sub-pixel and an infrared sub-pixel, and the first pixel and the second pixel are all full-pixel dual-core focus pixels, the first pixel And the second pixel each include four full-pixel dual-core focus sub-pixels;
- the position of the infrared sub-pixel in the second pixel is the same as the position of the red sub-pixel, the green sub-pixel, the blue sub-pixel, the first combined sub-pixel, or the second combined sub-pixel in the first pixel.
- a pixel is a combination of 1/2 red subpixels and 1/2 green subpixels adjacent to each other, and a second combined subpixel is a combination of 1/2 green subpixels and 1/2 blue subpixels adjacent to each other; or
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel, 1/2 green sub-pixel, or 1/2 blue sub-pixel in the first pixel, and two adjacent The 1/2 infrared sub-pixels in the second pixel are combined to form a complete infrared sub-pixel.
- the unit includes a second pixel and at least one first pixel.
- the position of the 1/2 infrared sub-pixel in the second pixel is the same as the position of the 1/2 red sub-pixel, 1/2 green sub-pixel, or 1/2 blue sub-pixel in the first pixel, and two When 1/2 of the infrared sub-pixels of two adjacent second pixels are combined to form a complete infrared sub-pixel, in the pixel unit, the number of the second pixels is two, and the number of the first pixels is greater than or equal to zero.
- the red sub-pixel includes a semiconductor layer, a metal layer, a photodiode, a red filter, and a micromirror, which are sequentially stacked.
- the green sub-pixel includes a semiconductor layer, a metal layer, a photodiode, a green filter, and micro-mirror, which are sequentially stacked.
- Mirrors; blue sub-pixels include semiconductor layers, metal layers, photodiodes, blue filters, and micromirrors that are sequentially stacked; infrared sub-pixels include semiconductor layers, metal layers, photodiodes, and infrared filters that are sequentially stacked And micromirrors.
- the image sensor is a complementary metal oxide semiconductor CMOS image sensor, a charge-coupled element CCD image sensor, or a quantum thin film image sensor.
- the structure of the mobile terminal shown in FIG. 10 does not constitute a limitation on the mobile terminal, and the mobile terminal may include more or fewer components than shown in the figure, or combine some components, or different components. Layout.
- the mobile terminal includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a car terminal, a wearable device, a pedometer, and the like.
- the processor 1010 is configured to emit infrared light through an infrared transmitting module; obtain the distance between each infrared light reflection point on the object to be photographed and the mobile terminal according to the infrared light reflected by the object to be photographed; The distance between the light reflection point and the mobile terminal controls the imaging system to obtain stereoscopic information about the subject to be photographed.
- the radio frequency unit 1001 may be used to receive and send signals during the process of transmitting and receiving information or during a call. Specifically, the downlink data from the base station is received and processed by the processor 1010; To send uplink data to the base station.
- the radio frequency unit 1001 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.
- the radio frequency unit 1001 can also communicate with a network and other devices through a wireless communication system.
- the mobile terminal provides users with wireless broadband Internet access through the network module 1002, such as helping users to send and receive email, browse web pages, and access streaming media.
- the audio output unit 1003 may convert audio data received by the radio frequency unit 1001 or the network module 1002 or stored in the memory 1009 into audio signals and output them as sound. Also, the audio output unit 1003 may also provide audio output (for example, a call signal receiving sound, a message receiving sound, etc.) related to a specific function performed by the mobile terminal 1000.
- the audio output unit 1003 includes a speaker, a buzzer, a receiver, and the like.
- the input unit 1004 is used to receive audio or video signals.
- the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042.
- the graphics processor 10041 pairs images of still pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode Data is processed.
- the processed image frame can be displayed on the display unit 1006, and the display unit here is the above-mentioned display module.
- the image frames processed by the graphics processor 10041 may be stored in the memory 1009 (or other storage medium) or transmitted via the radio frequency unit 1001 or the network module 1002.
- the graphics processor 10041 is the above-mentioned image data processing module.
- the microphone 10042 can receive sound, and can process such sound into audio data.
- the processed audio data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 1001 in the case of a telephone call mode and output.
- the mobile terminal 1000 further includes at least one sensor 1005, such as a light sensor, a motion sensor, and other sensors.
- the light sensor includes an ambient light sensor and a proximity sensor.
- the ambient light sensor can adjust the brightness of the display panel 10061 according to the brightness of the ambient light.
- the proximity sensor can close the display panel 10061 and the mobile terminal 1000 when the mobile terminal 1000 moves to the ear. / Or backlight.
- the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is stationary, which can be used to identify mobile terminal attitudes (such as horizontal and vertical screen switching, related games , Magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc .; sensor 1005 can also include fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, Infrared sensors, etc. are not repeated here.
- the display unit 1006 is configured to display information input by the user or information provided to the user.
- the display unit 1006 may include a display panel 10061.
- the display panel 10061 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
- the user input unit 1007 may be used to receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of the mobile terminal.
- the user input unit 1007 includes a touch panel 10071 and other input devices 10072.
- the touch panel 10071 also known as a touch screen, can collect user's touch operations on or near it (for example, the user uses a finger, a stylus or any suitable object or accessory on the touch panel 10071 or near the touch panel 10071 operating).
- the touch panel 10071 may include two parts, a touch detection device and a touch controller.
- the touch detection device detects the user's touch position, and detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into contact coordinates, and sends it
- the processor 1010 receives a command sent from the processor 1010 and executes the command.
- the touch panel 10071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave.
- the user input unit 1007 may also include other input devices 10072.
- the other input device 10072 may include, but is not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operation lever, and details are not described herein again.
- the touch panel 10071 may be overlaid on the display panel 10061.
- the touch panel 10071 detects a touch operation on or near the touch panel 10071, the touch panel 10071 transmits the touch operation to the processor 1010 to determine the type of the touch event.
- the type of event provides a corresponding visual output on the display panel 10061.
- the touch panel 10071 and the display panel 10061 are implemented as two independent components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 10071 and the display panel 10061 may be integrated. The implementation of the input and output functions of the mobile terminal is not specifically limited here.
- the interface unit 1008 is an interface through which an external device is connected to the mobile terminal 1000.
- the external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, and audio input / output (I / O) port, video I / O port, headphone port, and more.
- the interface unit 1008 may be used to receive an input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements in the mobile terminal 1000 or may be used in the mobile terminal 1000 and externally Transfer data between devices.
- the memory 1009 can be used to store software programs and various data.
- the memory 1009 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function; the storage data area may store data according to Data (such as audio data, phone book, etc.) created by the use of mobile phones.
- the memory 1009 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
- the processor 1010 is a control center of the mobile terminal, and connects various parts of the entire mobile terminal by using various interfaces and lines, and runs or executes software programs and / or modules stored in the memory 1009, and calls data stored in the memory 1009. , Perform various functions of the mobile terminal and process data, so as to monitor the mobile terminal as a whole.
- the processor 1010 may include one or more processing units; optionally, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and an application program, etc.
- the tuning processor mainly handles wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 1010.
- the mobile terminal 1000 may further include a power source 1011 (such as a battery) for supplying power to various components.
- a power source 1011 such as a battery
- the power source 1011 may be logically connected to the processor 1010 through a power management system, thereby implementing management of charging, discharging, and power consumption through the power management system Management and other functions.
- the mobile terminal 1000 includes some functional modules that are not shown, and details are not described herein again.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Color Television Image Signal Generators (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
本公开提供了一种图像传感器、移动终端及图像拍摄方法,图像传感器的像素阵列包括预设数目个像素单元,像素单元包括全像素双核对焦的第一和第二像素,第一像素包括红绿蓝子像素,第二像素包括红蓝子像素中的至少一种以及绿和红外子像素;第二像素中的红外子像素与红、绿、蓝、第一组合或第二组合子像素在第一像素中的位置相同,第一组合子像素是相邻的1/2红绿子像素;第二组合子像素是相邻的1/2绿蓝子像素;或者1/2红外子像素在第二像素中的位置,与1/2红1/2绿或1/2蓝子像素在第一像素中的位置相同,两个相邻第二像素中的1/2红外子像素形成红外子像素。
Description
相关申请的交叉引用
本申请主张在2018年7月19日在中国提交的中国专利申请号No.201810797355.6的优先权,其全部内容通过引用包含于此。
本公开涉及图像处理技术领域,尤其涉及一种图像传感器、移动终端及图像拍摄方法。
目前,智能电子产品已经逐渐成为人们生活中的必需品,拍照功能作为电子产品的一重要配置也在逐渐发展。但随着拍照功能的推广和普及,人们已经不在满足当前的智能电子产品中摄像头仅有的拍照功能,更加期望实现拍照效果多样化、玩法多样化以及功能多样化。
目前市场上,基于互补金属氧化物半导体(Complementary Metal Oxide Semiconductor,CMOS)的图像传感器像素阵列排布中,最常用的是R(红色)G(绿色)B(蓝色)拜耳像素阵列排布模式,如图1a和图1b所示,但此种排布方式不能检测物体距离,且只能用于接受自然光,在正常光照时拍照记录图像。
全像素双核对焦(full pixel dual core focus,2PD)技术的像素阵列排布模式如图1c和图1d所示,此种排布方式也只能用于接受自然光,用于拍照记录图像,但相对相位检测自动对焦(Phase Detection Auto Focus,PDAF)技术方案,可以增加检测物体距离,更加快速的完成对焦动作。
其中2PD相位检测技术原理说明如下:由图1c和图1d可见,像素阵列中部分R,G和B子像素被一分为二,根据不同入射方向获取的光能量不一样,从而左边子像素点和右边子像素点即构成一对相位检测对;当左边子像素点和右边子像素点亮度值均达到相对最大峰值时,此刻图像相对最清晰,即为合焦,然后通过算法计算获得物距,从而实现快速对焦。
综上所述,CMOS的图像传感器像素阵列排布中无法检测物体距离且只能接受自然光,2PD技术的像素阵列排布中虽然可以检测物体距离,但是仅可接受自然光,因此相关技术中的图像传感器的像素阵列排布模式,存在拍摄场景受限、对焦缓慢,影响用户拍摄体验的问题。
发明内容
本公开的一些实施例提供一种图像传感器、移动终端及图像拍摄方法,以解决相关技术中拍摄时存在拍摄场景受限、对焦缓慢,影响用户拍摄体验的问题。
为了解决上述问题,本公开的一些实施例是这样实现的:
第一方面,本公开的一些实施例提供一种图像传感器,包括:
像素阵列,像素阵列包括按照预定方式排布的预设数目个像素单元,像素单元包括第一像素和与第一像素位置相邻的第二像素,第一像素包括红色子像素、绿色子像素和蓝色子像素,第二像素包括红色子像素和蓝色子像素中的至少一种以及绿色子像素和红外子像素,且第一像素和第二像素均为全像素双核对焦像素,第一像素和第二像素中各包括四个全像素双核对焦子像素;
其中,红外子像素在第二像素中的位置与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同,第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合,第二组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合;或者
1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素。
第二方面,本公开的一些实施例提供一种移动终端,包括成像***以及红外发射模块,成像***包括上述的图像传感器,还包括:
透镜模组;
用于驱动透镜模组移动的驱动模块;
设置于透镜模组与图像传感器之间的滤波模块;
与图像传感器连接的图像数据处理模块;以及
与图像数据处理模块连接的显示模块;
其中红外发射模块设置于透镜模组的周缘。
第三方面,本公开的一些实施例提供一种图像拍摄方法,应用于移动终端,移动终端包括上述的图像传感器,还包括红外发射模块,该方法包括:
通过红外发射模块发射红外光;
根据待拍摄对象所反射的红外光获取待拍摄对象上各个红外光反射点与移动终端之间的距离;
根据待拍摄对象上各个红外光反射点与移动终端之间的距离,对待拍摄对象进行立体信息获取。
本公开技术方案,通过采用2PD的RGB子像素与红外子像素结合的像素阵列形成图像传感器,根据图像传感器执行拍摄,可以在拍摄记录图像的基础上,检测待拍摄对象与移动终端之间的距离,实现快速对焦和背景虚化,同时基于红外光可提升图像的暗态成像效果,实现立体拍照的相关应用功能,在保证用户拍摄体验的同时,保证移动终端的功能多样化。
为了更清楚地说明本公开的一些实施例的技术方案,下面将对本公开的一些实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本公开的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1a表示相关技术中的常规RGB排布示意图;
图1b表示常规像素点的切面图;
图1c表示2PD的像素阵列排布图;
图1d表示2PD像素切面图;
图2a表示本公开的一些实施例像素单元的示意图之一;
图2b表示本公开的一些实施例像素单元的示意图之二;
图2c表示本公开的一些实施例像素单元的示意图之三;
图3a表示本公开的一些实施例像素单元的示意图之四;
图3b表示本公开的一些实施例像素单元的示意图之五;
图4a表示本公开的一些实施例像素单元的示意图之六;
图4b表示本公开的一些实施例像素单元的示意图之七;
图5a表示本公开的一些实施例像素单元的示意图之八;
图5b表示本公开的一些实施例像素单元的示意图之九;
图6a表示本公开的一些实施例像素单元的示意图之十;
图6b表示本公开的一些实施例像素单元的示意图之十一;
图7表示本公开的一些实施例像素切面图;
图8a表示本公开的一些实施例移动终端示意图;
图8b表示本公开的一些实施例成像***示意图;
图9表示本公开的一些实施例图像拍摄方法示意图;
图10表示本公开的一些实施例移动终端硬件结构示意图。
下面将结合本公开的一些实施例中的附图,对本公开的一些实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本公开一部分实施例,而不是全部的实施例。基于本公开中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本公开保护的范围。
本公开的一些实施例提供一种图像传感器,包括:像素阵列,像素阵列包括按照预定方式排布的预设数目个像素单元,如图2a至2c、图3a至图3b以及图4a至图4b所示,像素单元包括第一像素和与第一像素位置相邻的第二像素,第一像素包括红色子像素、绿色子像素和蓝色子像素,第二像素包括红色子像素和蓝色子像素中的至少一种以及绿色子像素和红外子像素,且第一像素和第二像素均为全像素双核对焦像素,第一像素和第二像素中各包括四个全像素双核对焦子像素;
其中,红外子像素在第二像素中的位置与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同,第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合,第二 组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合;或者
1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素。
本公开的一些实施例提供的图像传感器所包含的像素阵列,包含预设数目个像素单元,其中预设数目个像素单元按照预定方式进行排布。预设数目个像素单元均包括第一像素以及第二像素,其中第一像素与第二像素有所区别,第一像素中包括红色子像素(R)、绿色子像素(G)和蓝色子像素(B),第二像素中包括红色子像素和蓝色子像素中的至少一种,还包括绿色子像素和红外子像素(IR),通过在第二像素中设置红外子像素,可以在接收红外光的情况下进行图像拍摄,实现暗态成像,保证用户拍摄体验。
其中,本公开的一些实施例中的第一像素以及第二像素均为全像素双核对焦(2PD)像素,通过采用2PD像素可以检测物体距离,更加快速的完成对焦动作,这里的第一像素以及第二像素均为2PD像素,也就是第一像素和第二像素中的子像素均为2PD子像素。
其中,第一像素中的红色子像素、绿色子像素和蓝色子像素按照一定的方式进行排列,且第一像素中包含四个全像素双核对焦子像素,分别为一个红色子像素、一个蓝色子像素以及两个绿色子像素,在这里为了便于区别将两个绿色子像素分别称为第一绿色子像素和第二绿色子像素,其中第一绿色子像素与第二绿色子像素相同。红色子像素与第一绿色子像素相邻,第二绿色子像素位于红色子像素的下方,蓝色子像素位于第一绿色子像素的下方,且第二绿色子像素与蓝色子像素相邻。
第二像素包含四个全像素双核对焦子像素,具体包括红色子像素和蓝色子像素中的至少一种,还包括绿色子像素以及红外子像素,即第二像素中可以包括红色子像素、绿色子像素和红外子像素,可以包括绿色子像素、蓝色子像素以及红外子像素,还可以包括绿色子像素、红色子像素、蓝色子像素以及红外子像素。
其中红外子像素在第二像素中的位置可以与第一像素中某一子像素的位置相同,还可以与第一像素中位置相邻的两个1/2不同子像素的位置相同。 当然还可以是1/2红外子像素在第二像素中的位置与第一像素中任一1/2子像素的位置相同,此时可由两个相邻的第二像素中的1/2红外子像素组合形成一完整的红外子像素。例如1/2红外子像素在第二像素中的位置与1/2红色子像素在第一像素中的位置相同,另外1/2红外子像素在第二像素中的位置与1/2绿色子像素在第一像素中的位置相同,至此可以通过两个第二像素组合形成完整的红外子像素。
本公开的一些实施例,通过在RGB像素阵列排布方式上进行改进,将RGB像素阵列排布方式改为RGB-IR(红外)的像素阵列排布方式,可以使得移动终端在接收红外光的情况下进行图像拍摄,实现暗态成像,保证用户拍摄体验。
同时本公开的一些实施例的图像传感器可以实现检测待拍摄对象与移动终端之间的距离,实现快速对焦和背景虚化,通过与红外发射模块配合,可提升图像的成像效果,并实现立体拍照的相关应用功能,在保证用户拍摄体验的同时,增强移动终端的功能性。
在本公开的一些实施例中,如图2a至图2c、图3a以及图4a所示,红外子像素在第二像素中的位置与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同,第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合;第二组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合。
当第二像素中包括蓝色子像素、绿色子像素和红外子像素时,红外子像素替代了第一像素中的红色子像素,此时红外子像素在第二像素中的位置,与红色子像素在第一像素中的位置相同。
当第二像素中包括红色子像素、绿色子像素和红外子像素时,红外子像素替代了第一像素中的蓝色子像素,此时红外子像素在第二像素中的位置,与蓝色子像素在第一像素中的位置相同。
当第二像素中包括红色子像素、绿色子像素、蓝色子像素和红外子像素时,可以是红外子像素替代了第一像素中的一绿色子像素,此时红外子像素在第二像素中的位置,与绿色子像素在第一像素中的位置相同。
当第二像素中包括红色子像素、绿色子像素、蓝色子像素和红外子像素 时,还可以是第二像素中包括一个红色子像素、一个绿色子像素、1/2蓝色子像素、1/2绿色子像素以及一个红外子像素,此时红外子像素代替了第一像素中的1/2绿色子像素和1/2蓝色子像素,即红外子像素在第二像素中的位置与位置相邻的1/2绿色子像素和1/2蓝色子像素在第一像素中的位置相同。
当第二像素中包括红色子像素、绿色子像素、蓝色子像素和红外子像素时,也可以是第二像素中包括一个蓝色子像素、一个绿色子像素、1/2红色子像素、1/2绿色子像素以及一个红外子像素,此时红外子像素代替了第一像素中的1/2绿色子像素和1/2红色子像素,即红外子像素在第二像素中的位置与位置相邻的1/2绿色子像素和1/2红色子像素在第一像素中的位置相同。
在本公开的一些实施例中,当红外子像素在第二像素中的位置,与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同时,像素单元中包括一个第二像素以及至少一个第一像素。
若红外子像素在第二像素中的位置,与第一像素中的红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素的位置相同时,此时像素单元中第二像素的数量为一个,第一像素的数量可以大于或者等于一个。当像素单元中包括一个第二像素以及一个第一像素时,此时红外子像素在像素单元中的取点密度为1/8。例如,如图5a所示,像素单元中包括一个第一像素以及一个第二像素,其中第二像素中包括一个红色子像素、两个绿色子像素以及一个红外子像素,第一像素中包括一个红色子像素、两个绿色子像素以及一个蓝色子像素,此时红外子像素在像素单元中的占比为1/8,也就是说红外子像素的取点密度为1/8。
当像素单元中包括两个第一像素以及一个第二像素,此时红外子像素在像素单元中的密度为1/12。例如,如图5b所示,像素单元中包括两个第一像素以及一个第二像素,其中第二像素中包括一个蓝色子像素、两个绿色子像素以及一个红外子像素,第一像素中包括一个红色子像素、两个绿色子像素以及一个蓝色子像素,此时红外子像素在像素单元中的占比为1/12,也就是说红外子像素的取点密度为1/12。
当像素单元中包括三个第一像素以及一个第二像素,此时红外子像素在 像素单元中的密度为1/16。例如,如图4a所示,像素单元中包括三个第一像素以及一个第二像素,其中第二像素中包括一个蓝色子像素、一个绿色子像素、1/2红色子像素、1/2绿色子像素以及一个红外子像素,可以在第一像素的基础上取2PD子像素的1/2红色子像素和1/2绿色子像素为红外子像素,第一像素中包括一个红色子像素、两个绿色子像素以及一个蓝色子像素,此时红外子像素在像素单元中的占比为1/16,也就是说红外子像素的取点密度为1/16。
上述对应的几种取点方式仅用于举例说明,还可以是其他的取点方式,这里不再一一介绍。其中红外子像素在像素单元中的密度为1/4n,且n为大于或者等于2的整数,且红外子像素点所适用的像素阵列大小不限。
在本公开的一些实施例中,如图3b和图4b所示,1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素。
当1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素时,在像素单元中,第二像素的数量为两个,第一像素的数量大于或者等于零。
在第二像素中可以仅包含1/2红外子像素,此时通过两个相邻的第二像素进行组合可以得到一个完整的红外子像素。
当第二像素中包含1/2红外子像素时,1/2红外子像素在第二像素中的位置可以与1/2红色子像素在第一像素中的位置相同,还可以与1/2绿色子像素在第一像素中的位置相同,也可以与1/2蓝色子像素在第一像素中的位置相同。
当1/2红外子像素在一第二像素中的位置与1/2红色子像素在第一像素中的位置相同时,则在另一第二像素中1/2红外子像素的位置与1/2绿色子像素在第一像素中的位置相同。当1/2红外子像素在一第二像素中的位置与1/2绿色子像素在第一像素中的位置相同时,则在另一第二像素中1/2红外子像素的位置与1/2蓝色子像素或1/2红色子像素在第一像素中的位置相同。
相应的,此时在像素单元中,第二像素的数量为两个,第一像素的数量大于或者等于零。
像素单元中像素的数量至少为两个,则像素单元中包括两个第二像素以及数量大于或者等于零的第一像素。当像素单元中像素的数量为两个时,包括两个第二像素,此时红外子像素在像素单元中的取点密度为1/8。例如,如图6a所示,像素单元中包括两个第二像素,其中在一第二像素中包括一个红色子像素、一个绿色子像素、一个蓝色子像素、1/2绿色子像素以及1/2红外子像素,在另一第二像素中包括一个红色子像素、两个绿色子像素、1/2蓝色子像素以及1/2红外子像素。此时1/2红外子像素在一第二像素中的位置与1/2绿色子像素在第一像素中的位置相同,在另一第二像素中1/2红外子像素的位置与1/2蓝色子像素在第一像素中的位置相同。红外子像素在像素单元中的占比为1/8,也就是说红外子像素的取点密度为1/8。
当像素单元中像素的数量为三个时,包括两个第二像素以及一个第一像素,此时红外子像素在像素单元中的取点密度为1/12。例如,如图6b所示,像素单元中包括两个第二像素以及一个第一像素,其中在一第二像素中包括一个红色子像素、一个绿色子像素、一个蓝色子像素、1/2绿色子像素以及1/2红外子像素,在另一第二像素中包括1/2红色子像素、两个绿色子像素、一个蓝色子像素以及1/2红外子像素,此时1/2红外子像素在一第二像素中的位置与1/2绿色子像素在第一像素中的位置相同,在另一第二像素中1/2红外子像素的位置与1/2红色子像素在第一像素中的位置相同。红外子像素在像素单元中的占比为1/12,也就是说红外子像素的取点密度为1/12。
当像素单元中像素的数量为四个时,包括两个第二像素以及两个第一像素,此时红外子像素在像素单元中的取点密度为1/16。例如,如图4b所示,像素单元中包括两个第二像素以及两个第一像素,其中在一第二像素中包括一个红色子像素、一个绿色子像素、一个蓝色子像素、1/2绿色子像素以及1/2红外子像素,在另一第二像素中包括一个蓝色子像素、两个绿色子像素、1/2红色子像素以及1/2红外子像素,此时1/2红外子像素在一第二像素中的位置与1/2绿色子像素在第一像素中的位置相同,在另一第二像素中1/2红外子像素的位置与1/2红色子像素在第一像素中的位置相同。红外子像素在像 素单元中的占比为1/16,也就是说红外子像素的取点密度为1/16。
像素阵列可以是由1/8密度的RGB+IR像素单元、1/12密度的RGB+IR像素单元或1/16密度的RGB+IR像素单元作为一个像素单位阵列,像素单位阵列再进行周期性阵列排布组成。当然像素阵列还可以是其他形式,这里不再列举。
上述仅仅为几种对应的红外子像素的取点实施方式,还可以在此基础上进行变形,或者采用其他的取点方式,即本公开的一些实施例的红外子像素的取点方式不做进一步限制,其对应的多种实现方式这里不再一一阐述。红外子像素在像素单元中的取点位置(第二像素的位置)在本公开的一些实施例中不做具体限制。其中红外子像素在像素单元中的密度为1/4n,且n为大于或者等于2的整数,且红外子像素点所适用的像素阵列大小不限。
上述的像素单元形式,可以取2PD子像素的红色、绿色或者蓝色子像素为红外子像素,便于像素单元的形式多样化,还可以取2PD子像素的1/2红色和1/2绿色子像素或者取1/2蓝色和1/2绿色子像素为红外子像素,可以减少RGB处理时IR坏点影响。
在本公开的一些实施例中,红色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红色滤光片以及微镜;绿色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、绿色滤光片以及微镜;蓝色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、蓝色滤光片以及微镜;红外子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红外滤光片以及微镜。
红色子像素所包含的半导体层、金属层、光电二极管、红色滤光片以及微镜,由下至上依次排列。相应的绿色子像素所包含的半导体层、金属层、光电二极管、绿色滤光片以及微镜,由下至上依次排列。蓝色子像素所包含的半导体层、金属层、光电二极管、蓝色滤光片以及微镜,由下至上依次排列。红外子像素所包含的半导体层、金属层、光电二极管、红外滤光片以及微镜,由下至上依次排列。这里的半导体层可以为硅基板,但并不局限于此。红色、绿色、蓝色以及红外子像素的结构可参见图7所示,图7中虽然只示出了蓝色以及红外子像素,但在此基础上可以将获知红绿子像素的结构。可 将蓝色滤光片替换为红色或者绿色滤光片,即可获得红色子像素或绿子像素的结构。
红色、绿色以及蓝色子像素用于获取合成图像的像素的色彩信息,其阻挡红外线的进入;例如,仅使波长在380~700nm的可见光进入,可在高照度下直接生成色彩完整逼真的图像。红外子像素感光带宽为750~1100nm,红外滤光区可用于使红外波段通过,可提升暗态成像效果,实现红外测距功能。
由以上说明可见,RGB子像素点是对应于每种RGB颜色的波长光的光接收元件,IR子像素点是对应红外光的光接收元件。
在本公开的一些实施例中,图像传感器为互补金属氧化物半导体CMOS图像传感器、电荷耦合元件CCD图像传感器或量子薄膜图像传感器。
本公开RGB-IR的像素阵列排布方式,适用的图像传感器类型不限,可以是基于CMOS的图像传感器,可以是基于电荷耦合元件(Charge-coupled Device,CCD)的图像传感器,也可以是基于量子薄膜的图像传感器,当然还可以是其他类型的图像传感器。且本公开的一些实施例的图像传感器可适用于任何包含摄像头模组的电子产品中。
本公开的一些实施例还提供一种移动终端,如图8a和图8b所示,移动终端1包括成像***2以及红外发射模块3,成像***2包括上述的图像传感器21,还包括:
透镜模组22;用于驱动透镜模组22移动的驱动模块23;设置于透镜模组22与图像传感器21之间的滤波模块24;与图像传感器21连接的图像数据处理模块25;以及与图像数据处理模块25连接的显示模块26;其中红外发射模块3设置于透镜模组22的周缘。
本公开的一些实施例的移动终端1包括成像***2,还包括红外发射模块3,其中成像***2包括上述的图像传感器21,成像***2还包括用于对光线聚焦的透镜模组22,透镜模组22与驱动模块23连接,驱动模块23用于随着待拍摄对象的远近,从而进行调整透镜模组22的位置。
在透镜模组22与图像传感器21之间设置有滤波模块24,其中在光线通过透镜模组22聚焦,经过滤波模块24后,可以聚焦在图像传感器21的像素阵列上。图像传感器21与图像数据处理模块25连接,图像数据处理模块25 与显示模块26连接。在光线聚焦在图像传感器21的像素阵列上之后,图像传感器21进行光电转换后,将数据传输给图像数据处理模块25,图像数据处理模块25对数据进行处理后在显示模块26上以图片的形式呈现。
其中在驱动模块23调整透镜模组22的位置之后,可以利用图像传感器21中的2PD像素获取相位差,从而获取物体与成像面距离,进而实现快速对焦。
另外,本公开中基于2PD图像传感器的RGB+IR像素阵列排布方式,可以配合红外发射模块3实现立体相关功能,例如:人脸识别解锁,安全支付,立体成像等终端应用,在保证成像的基础上,提升移动终端的功能性。
本公开的一些实施例中的滤波模块24可通过380nm至1100nm的光波长。此时在光线通过透镜模组22聚焦后,可通过滤波模块24进行滤波,其中滤波模块24可用于自然光和红外光的通过,可用于保证成像***2的成像效果。
其中,移动终端上的红外发射模块3设置于透镜模组22的周缘。红外发射模块3发出红外线,红外线在遇到障碍物后会发生反射;当成像***2捕捉到反射回来的红外光线后,经过红外子像素进行光电转换,可获取红外线从发射到接收到红外线的时间差,由于光的传播速度固定,从而可以计算出障碍物距离移动终端的距离,最终可获取障碍物上每个极小单位到移动终端的距离,实现立体成像记录功能。当然还可以通过获取红外光相位差的方式,获取障碍物上各个红外光反射点与移动终端之间的距离。
本公开的一些实施例的移动终端,通过对图像传感器的像素阵列排布方式进行改进,将RGB像素阵列排布方式改进为2PD的RGB-IR的像素阵列排布方式,可以在接收红外光拍摄记录图像的基础上,检测待拍摄对象与移动终端之间的距离,实现快速对焦和背景虚化,提升图像的成像效果,并实现立体拍照的相关应用功能以及暗态成像,在保证用户拍摄体验的同时,保证移动终端的功能多样化。
本公开的一些实施例还提供一种图像拍摄方法,应用于移动终端,移动终端包括上述的图像传感器,还包括红外发射模块,如图9所示,该方法包括:
步骤901、通过红外发射模块发射红外光。
移动终端上的红外发射模块可以发出红外线,红外线在遇到待拍摄对象后会发生反射,其反射的红外光会被移动终端的成像***所接收。其中移动终端的图像传感器形成RGB-IR的像素阵列,因此可通过红外子像素进行光电转换。
步骤902、根据待拍摄对象所反射的红外光获取待拍摄对象上各个红外光反射点与移动终端之间的距离。
在获取待拍摄对象与移动终端之间的距离时,实际为获取待拍摄对象与成像面之间的距离,获取待拍摄对象上各个红外光反射点与移动终端之间的距离的过程为:通过包含第二像素的像素阵列接收待拍摄对象上各个红外光反射点所反射的红外光;根据发送与接收红外光的时间差以及红外光的传播速度或者通过获取红外光的相位差,获取待拍摄对象上各个红外光反射点与移动终端之间的距离。当捕捉到反射回来的红外光线后,经过红外子像素进行光电转换,可获取红外线从发射到接收到红外线的时间差,由于光的传播速度固定,从而可以根据时间差与传播速度乘积的1/2计算出障碍物距离移动终端的距离。其中移动终端接收到各个红外光反射点反射红外光的时间有所区别,因此针对每一个红外光反射点可以对应计算一距离,进而可以获得各个红外光反射点与移动终端之间的距离。还可以通过获取红外光的相位差获取各个红外光反射点与移动终端之间的距离,具体参见飞行时间(Time of Flight,TOF)技术,在此不再详细赘述。
其中,本公开的一些实施例的图像传感器的像素阵列包括按照预定方式排布的预设数目个像素单元,像素单元包括第一像素和第二像素,第一像素包括红色子像素、绿色子像素和蓝色子像素,第二像素包括红色子像素和蓝色子像素中的至少一种以及绿色子像素和红外子像素,且第一像素和第二像素均为全像素双核对焦像素,第一像素和第二像素中各包括四个全像素双核对焦子像素;其中,红外子像素在第二像素中的位置与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同,第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合,第二组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合;或者1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像 素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素。
步骤903、根据待拍摄对象上各个红外光反射点与移动终端之间的距离,对待拍摄对象进行立体信息获取。
在获取待拍摄对象上各个红外光反射点与移动终端之间的距离时,具体为获取待拍摄对象上每个极小单位到移动终端的距离,然后执行对待拍摄对象拍摄的流程,实现立体成像记录功能。
本公开的一些实施例的拍摄方法,可以通过对图像传感器的像素阵列排布方式进行改进,将RGB像素阵列排布方式改进为2PD的RGB-IR的像素阵列排布方式,可以在接收红外光拍摄记录图像的基础上,检测待拍摄对象与移动终端之间的距离,实现快速对焦和背景虚化,提升图像的成像效果,并实现立体拍照的相关应用功能以及暗态成像,在保证用户拍摄体验的同时,保证移动终端的功能多样化。
图10为实现本公开各个实施例的一种移动终端的硬件结构示意图,该移动终端1000包括但不限于:射频单元1001、网络模块1002、音频输出单元1003、输入单元1004、传感器1005、显示单元1006、用户输入单元1007、接口单元1008、存储器1009、处理器1010、以及电源1011等部件。
移动终端1000,还包括成像***以及红外发射模块,成像***包括图像传感器,透镜模组;用于驱动透镜模组移动的驱动模块;设置于透镜模组与图像传感器之间的滤波模块;与图像传感器连接的图像数据处理模块;以及与图像数据处理模块连接的显示模块。
其中,滤波模块可通过380nm至1100nm的光波长,红外发射模块设置于透镜模组的周缘。
其中,图像传感器包括:
像素阵列,像素阵列包括按照预定方式排布的预设数目个像素单元,像素单元包括第一像素和与第一像素位置相邻的第二像素,第一像素包括红色子像素、绿色子像素和蓝色子像素,第二像素包括红色子像素和蓝色子像素中的至少一种以及绿色子像素和红外子像素,且第一像素和第二像素均为全像素双核对焦像素,第一像素和第二像素中各包括四个全像素双核对焦子像 素;
其中,红外子像素在第二像素中的位置与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同,第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合,第二组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合;或者
1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素。
其中,当红外子像素在第二像素中的位置,与红色子像素、绿色子像素、蓝色子像素、第一组合子像素或者第二组合子像素在第一像素中的位置相同时,像素单元中包括一个第二像素以及至少一个第一像素。
其中,当1/2红外子像素在第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在第一像素中的位置相同,且两个相邻第二像素中的1/2红外子像素组合形成一完整的红外子像素时,在像素单元中,第二像素的数量为两个,第一像素的数量大于或者等于零。
其中,红色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红色滤光片以及微镜;绿色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、绿色滤光片以及微镜;蓝色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、蓝色滤光片以及微镜;红外子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红外滤光片以及微镜。
其中,图像传感器为互补金属氧化物半导体CMOS图像传感器、电荷耦合元件CCD图像传感器或量子薄膜图像传感器。
本领域技术人员可以理解,图10中示出的移动终端结构并不构成对移动终端的限定,移动终端可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。在本公开的一些实施例中,移动终端包括但不限于手机、平板电脑、笔记本电脑、掌上电脑、车载终端、可穿戴设备、以及计步器等。
其中,处理器1010用于:通过红外发射模块发射红外光;根据待拍摄对象所反射的红外光获取待拍摄对象上各个红外光反射点与移动终端之间的距 离;根据待拍摄对象上各个红外光反射点与移动终端之间的距离,控制成像***对待拍摄对象进行立体信息获取。
这样,通过对图像传感器的像素阵列排布方式进行改进,将RGB像素阵列排布方式改进为2PD的RGB-IR的像素阵列排布方式,可以在接收红外光拍摄记录图像的基础上,检测待拍摄对象与移动终端之间的距离,实现快速对焦和背景虚化,提升图像的成像效果,并实现立体拍照的相关应用功能以及暗态成像,在保证用户拍摄体验的同时,保证移动终端的功能多样化。
应理解的是,本公开的一些实施例中,射频单元1001可用于收发信息或通话过程中,信号的接收和发送,具体的,将来自基站的下行数据接收后,给处理器1010处理;另外,将上行的数据发送给基站。通常,射频单元1001包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等。此外,射频单元1001还可以通过无线通信***与网络和其他设备通信。
移动终端通过网络模块1002为用户提供了无线的宽带互联网访问,如帮助用户收发电子邮件、浏览网页和访问流式媒体等。
音频输出单元1003可以将射频单元1001或网络模块1002接收的或者在存储器1009中存储的音频数据转换成音频信号并且输出为声音。而且,音频输出单元1003还可以提供与移动终端1000执行的特定功能相关的音频输出(例如,呼叫信号接收声音、消息接收声音等等)。音频输出单元1003包括扬声器、蜂鸣器以及受话器等。
输入单元1004用于接收音频或视频信号。输入单元1004可以包括图形处理器(Graphics Processing Unit,GPU)10041和麦克风10042,图形处理器10041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。处理后的图像帧可以显示在显示单元1006上,这里的显示单元即为上述的显示模块。经图形处理器10041处理后的图像帧可以存储在存储器1009(或其它存储介质)中或者经由射频单元1001或网络模块1002进行发送。其中图形处理器10041即为上述的图像数据处理模块。麦克风10042可以接收声音,并且能够将这样的声音处理为音频数据。处理后的音频数据可以在电话通话模式的情况下转换为可经由 射频单元1001发送到移动通信基站的格式输出。
移动终端1000还包括至少一种传感器1005,比如光传感器、运动传感器以及其他传感器。具体地,光传感器包括环境光传感器及接近传感器,其中,环境光传感器可根据环境光线的明暗来调节显示面板10061的亮度,接近传感器可在移动终端1000移动到耳边时,关闭显示面板10061和/或背光。作为运动传感器的一种,加速计传感器可检测各个方向上(一般为三轴)加速度的大小,静止时可检测出重力的大小及方向,可用于识别移动终端姿态(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等;传感器1005还可以包括指纹传感器、压力传感器、虹膜传感器、分子传感器、陀螺仪、气压计、湿度计、温度计、红外线传感器等,在此不再赘述。
显示单元1006用于显示由用户输入的信息或提供给用户的信息。显示单元1006可包括显示面板10061,可以采用液晶显示器(Liquid Crystal Display,LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板10061。
用户输入单元1007可用于接收输入的数字或字符信息,以及产生与移动终端的用户设置以及功能控制有关的键信号输入。具体地,用户输入单元1007包括触控面板10071以及其他输入设备10072。触控面板10071,也称为触摸屏,可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板10071上或在触控面板10071附近的操作)。触控面板10071可包括触摸检测装置和触摸控制器两个部分。其中,触摸检测装置检测用户的触摸方位,并检测触摸操作带来的信号,将信号传送给触摸控制器;触摸控制器从触摸检测装置上接收触摸信息,并将它转换成触点坐标,再送给处理器1010,接收处理器1010发来的命令并加以执行。此外,可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板10071。除了触控面板10071,用户输入单元1007还可以包括其他输入设备10072。具体地,其他输入设备10072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆,在此不再赘述。
进一步的,触控面板10071可覆盖在显示面板10061上,当触控面板10071 检测到在其上或附近的触摸操作后,传送给处理器1010以确定触摸事件的类型,随后处理器1010根据触摸事件的类型在显示面板10061上提供相应的视觉输出。虽然在图10中,触控面板10071与显示面板10061是作为两个独立的部件来实现移动终端的输入和输出功能,但是在某些实施例中,可以将触控面板10071与显示面板10061集成而实现移动终端的输入和输出功能,具体此处不做限定。
接口单元1008为外部装置与移动终端1000连接的接口。例如,外部装置可以包括有线或无线头戴式耳机端口、外部电源(或电池充电器)端口、有线或无线数据端口、存储卡端口、用于连接具有识别模块的装置的端口、音频输入/输出(I/O)端口、视频I/O端口、耳机端口等等。接口单元1008可以用于接收来自外部装置的输入(例如,数据信息、电力等等)并且将接收到的输入传输到移动终端1000内的一个或多个元件或者可以用于在移动终端1000和外部装置之间传输数据。
存储器1009可用于存储软件程序以及各种数据。存储器1009可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等;存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外,存储器1009可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
处理器1010是移动终端的控制中心,利用各种接口和线路连接整个移动终端的各个部分,通过运行或执行存储在存储器1009内的软件程序和/或模块,以及调用存储在存储器1009内的数据,执行移动终端的各种功能和处理数据,从而对移动终端进行整体监控。处理器1010可包括一个或多个处理单元;可选的,处理器1010可集成应用处理器和调制解调处理器,其中,应用处理器主要处理操作***、用户界面和应用程序等,调制解调处理器主要处理无线通信。可以理解的是,上述调制解调处理器也可以不集成到处理器1010中。
移动终端1000还可以包括给各个部件供电的电源1011(比如电池),可选的,电源1011可以通过电源管理***与处理器1010逻辑相连,从而通过 电源管理***实现管理充电、放电、以及功耗管理等功能。
另外,移动终端1000包括一些未示出的功能模块,在此不再赘述。
需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本公开的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端(可以是手机,计算机,服务器,空调器,或者网络设备等)执行本公开各个实施例所述的方法。
上面结合附图对本公开的实施例进行了描述,但是本公开并不局限于上述的具体实施方式,上述的具体实施方式仅仅是示意性的,而不是限制性的,本领域的普通技术人员在本公开的启示下,在不脱离本公开宗旨和权利要求所保护的范围情况下,还可做出很多形式,均属于本公开的保护之内。
Claims (10)
- 一种图像传感器,包括:像素阵列,所述像素阵列包括按照预定方式排布的预设数目个像素单元,所述像素单元包括第一像素和与所述第一像素位置相邻的第二像素,所述第一像素包括红色子像素、绿色子像素和蓝色子像素,所述第二像素包括所述红色子像素和所述蓝色子像素中的至少一种以及所述绿色子像素和红外子像素,且所述第一像素和所述第二像素均为全像素双核对焦像素,所述第一像素和所述第二像素中各包括四个全像素双核对焦子像素;其中,所述红外子像素在所述第二像素中的位置与所述红色子像素、所述绿色子像素、所述蓝色子像素、第一组合子像素或者第二组合子像素在所述第一像素中的位置相同,所述第一组合子像素是位置相邻的1/2红色子像素和1/2绿色子像素的组合,所述第二组合子像素是位置相邻的1/2绿色子像素和1/2蓝色子像素的组合;或者1/2红外子像素在所述第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在所述第一像素中的位置相同,且两个相邻所述第二像素中的1/2红外子像素组合形成一完整的所述红外子像素。
- 根据权利要求1所述的图像传感器,其中,当所述红外子像素在所述第二像素中的位置,与所述红色子像素、所述绿色子像素、所述蓝色子像素、所述第一组合子像素或者所述第二组合子像素在所述第一像素中的位置相同时,所述像素单元中包括一个所述第二像素以及至少一个所述第一像素。
- 根据权利要求1所述的图像传感器,其中,当1/2红外子像素在所述第二像素中的位置,与1/2红色子像素、1/2绿色子像素或者1/2蓝色子像素在所述第一像素中的位置相同,且两个相邻所述第二像素中的1/2红外子像素组合形成一完整的所述红外子像素时,在所述像素单元中,所述第二像素的数量为两个,所述第一像素的数量大于或者等于零。
- 根据权利要求1所述的图像传感器,其中,所述红色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红色滤光片以及微镜;所述绿色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、绿色滤光片以及微镜;所述蓝色子像素包括依次堆叠设置的半导体层、金属层、光电二极管、蓝色滤光片以及微镜;所述红外子像素包括依次堆叠设置的半导体层、金属层、光电二极管、红外滤光片以及微镜。
- 根据权利要求1所述的图像传感器,其中,所述图像传感器为互补金属氧化物半导体CMOS图像传感器、电荷耦合元件CCD图像传感器或量子薄膜图像传感器。
- 一种移动终端,包括成像***以及红外发射模块,所述成像***包括如权利要求1至5任一项所述的图像传感器,还包括:透镜模组;用于驱动所述透镜模组移动的驱动模块;设置于所述透镜模组与所述图像传感器之间的滤波模块;与所述图像传感器连接的图像数据处理模块;以及与所述图像数据处理模块连接的显示模块;其中所述红外发射模块设置于所述透镜模组的周缘。
- 根据权利要求6所述的移动终端,其中,所述滤波模块可通过380nm至1100nm的光波长。
- 一种图像拍摄方法,应用于移动终端,所述移动终端包括如权利要求1所述的图像传感器,还包括红外发射模块,所述方法包括:通过所述红外发射模块发射红外光;根据待拍摄对象所反射的红外光获取所述待拍摄对象上各个红外光反射点与所述移动终端之间的距离;根据所述待拍摄对象上各个红外光反射点与所述移动终端之间的距离,对所述待拍摄对象进行立体信息获取。
- 一种移动终端,包括存储器、处理器及存储在存储器上并可在处理器 上运行的计算机程序,其中,所述处理器执行所述计算机程序时实现如权利要求8所述图像拍摄方法中的步骤。
- 一种计算机可读存储介质,其上存储有计算机程序,其中,该计算机程序被处理器执行时实现如权利要求8所述图像拍摄方法中的步骤。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP23187255.7A EP4250750A3 (en) | 2018-07-19 | 2019-07-16 | Image sensor, mobile terminal, and image photographing method |
EP19837519.8A EP3809686A4 (en) | 2018-07-19 | 2019-07-16 | IMAGE SENSOR, MOBILE TERMINAL, AND IMAGE PHOTOGRAPHING PROCESS |
US17/151,927 US11743605B2 (en) | 2018-07-19 | 2021-01-19 | Image sensor, mobile terminal, and image photographing method |
US18/317,216 US11962918B2 (en) | 2018-07-19 | 2023-05-15 | Image sensor, mobile terminal, and image photographing method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810797355.6 | 2018-07-19 | ||
CN201810797355.6A CN108600712B (zh) | 2018-07-19 | 2018-07-19 | 一种图像传感器、移动终端及图像拍摄方法 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/151,927 Continuation US11743605B2 (en) | 2018-07-19 | 2021-01-19 | Image sensor, mobile terminal, and image photographing method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020015627A1 true WO2020015627A1 (zh) | 2020-01-23 |
Family
ID=63618698
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/096128 WO2020015627A1 (zh) | 2018-07-19 | 2019-07-16 | 图像传感器、移动终端及图像拍摄方法 |
Country Status (4)
Country | Link |
---|---|
US (2) | US11743605B2 (zh) |
EP (2) | EP3809686A4 (zh) |
CN (1) | CN108600712B (zh) |
WO (1) | WO2020015627A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112616009A (zh) * | 2020-12-31 | 2021-04-06 | 维沃移动通信有限公司 | 电子设备及其摄像模组 |
WO2022143682A1 (zh) * | 2020-12-31 | 2022-07-07 | 维沃移动通信有限公司 | 电子设备及其摄像模组 |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108600712B (zh) | 2018-07-19 | 2020-03-31 | 维沃移动通信有限公司 | 一种图像传感器、移动终端及图像拍摄方法 |
CN108965665B (zh) * | 2018-07-19 | 2020-01-31 | 维沃移动通信有限公司 | 一种图像传感器及移动终端 |
CN109194862A (zh) * | 2018-11-12 | 2019-01-11 | 德淮半导体有限公司 | 图像传感器及其形成方法 |
CN110225247B (zh) * | 2019-05-29 | 2021-06-15 | 联想(北京)有限公司 | 一种图像处理方法及电子设备 |
CN111933069A (zh) * | 2020-07-14 | 2020-11-13 | 福建华佳彩有限公司 | 一种显示屏驱动方法及显示屏 |
CN113824934B (zh) * | 2021-09-23 | 2024-02-09 | 合肥埃科光电科技股份有限公司 | 线扫描相机色彩插值方法、装置、设备及存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107040724A (zh) * | 2017-04-28 | 2017-08-11 | 广东欧珀移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
WO2017171412A2 (ko) * | 2016-03-30 | 2017-10-05 | 엘지전자 주식회사 | 이미지 처리 장치 및 이동 단말기 |
CN207354459U (zh) * | 2017-10-19 | 2018-05-11 | 维沃移动通信有限公司 | 一种摄像头模组和移动终端 |
CN108271012A (zh) * | 2017-12-29 | 2018-07-10 | 维沃移动通信有限公司 | 一种深度信息的获取方法、装置以及移动终端 |
CN108600712A (zh) * | 2018-07-19 | 2018-09-28 | 维沃移动通信有限公司 | 一种图像传感器、移动终端及图像拍摄方法 |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4621292A (en) * | 1983-10-19 | 1986-11-04 | Matsushita Electric Industrial Co., Ltd. | Automatic focusing device for a video camera |
US9313376B1 (en) * | 2009-04-01 | 2016-04-12 | Microsoft Technology Licensing, Llc | Dynamic depth power equalization |
CN101621697B (zh) * | 2009-07-28 | 2010-11-10 | 黄松涛 | 一种用多色图像传感器的背景消除装置及使用方法 |
TW201128486A (en) * | 2010-02-01 | 2011-08-16 | Acer Inc | Optical touch control display apparatus and method thereof |
KR102070778B1 (ko) * | 2012-11-23 | 2020-03-02 | 엘지전자 주식회사 | Rgb-ir 센서 및 이를 이용한 3차원 영상 획득 방법 및 장치 |
US9667933B2 (en) * | 2013-07-01 | 2017-05-30 | Omnivision Technologies, Inc. | Color and infrared filter array patterns to reduce color aliasing |
US20160161332A1 (en) * | 2014-12-09 | 2016-06-09 | Stmicroelectronics (Research & Development) Limited | Image sensor using pixels with combined rgb and ir sensing |
JP6608148B2 (ja) * | 2015-02-24 | 2019-11-20 | キヤノン株式会社 | 距離情報生成装置、撮像装置、距離情報生成方法および距離情報生成プログラム |
KR102195407B1 (ko) * | 2015-03-16 | 2020-12-29 | 삼성전자주식회사 | 이미지 신호 프로세서와 이를 포함하는 장치들 |
CN105049829B (zh) * | 2015-07-10 | 2018-12-25 | 上海图漾信息科技有限公司 | 滤光片、图像传感器、成像装置以及三维成像*** |
KR102348760B1 (ko) * | 2015-07-24 | 2022-01-07 | 삼성전자주식회사 | 이미지 센서 및 그에 따른 신호 처리 방법 |
CN106878690A (zh) * | 2015-12-14 | 2017-06-20 | 比亚迪股份有限公司 | 图像传感器的成像方法、成像装置和电子设备 |
SG11201800816VA (en) * | 2015-12-18 | 2018-02-27 | Guangdong Oppo Mobile Telecommunications Corp Ltd | Image sensor, control method, and electronic device |
CN105611136B (zh) * | 2016-02-26 | 2019-04-23 | 联想(北京)有限公司 | 一种图像传感器以及电子设备 |
CN106973233A (zh) * | 2017-04-27 | 2017-07-21 | 维沃移动通信有限公司 | 一种摄像头、移动终端及拍照处理方法 |
CN107105140B (zh) | 2017-04-28 | 2020-01-24 | Oppo广东移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
CN106973206B (zh) | 2017-04-28 | 2020-06-05 | Oppo广东移动通信有限公司 | 摄像模组摄像处理方法、装置和终端设备 |
CN107146797B (zh) | 2017-04-28 | 2020-03-27 | Oppo广东移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
CN106921823B (zh) | 2017-04-28 | 2019-09-17 | Oppo广东移动通信有限公司 | 图像传感器、摄像模组和终端设备 |
CN106982329B (zh) * | 2017-04-28 | 2020-08-07 | Oppo广东移动通信有限公司 | 图像传感器、对焦控制方法、成像装置和移动终端 |
CN106982328B (zh) * | 2017-04-28 | 2020-01-10 | Oppo广东移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
CN107124536B (zh) | 2017-04-28 | 2020-05-08 | Oppo广东移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
-
2018
- 2018-07-19 CN CN201810797355.6A patent/CN108600712B/zh active Active
-
2019
- 2019-07-16 EP EP19837519.8A patent/EP3809686A4/en active Pending
- 2019-07-16 WO PCT/CN2019/096128 patent/WO2020015627A1/zh unknown
- 2019-07-16 EP EP23187255.7A patent/EP4250750A3/en active Pending
-
2021
- 2021-01-19 US US17/151,927 patent/US11743605B2/en active Active
-
2023
- 2023-05-15 US US18/317,216 patent/US11962918B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017171412A2 (ko) * | 2016-03-30 | 2017-10-05 | 엘지전자 주식회사 | 이미지 처리 장치 및 이동 단말기 |
CN107040724A (zh) * | 2017-04-28 | 2017-08-11 | 广东欧珀移动通信有限公司 | 双核对焦图像传感器及其对焦控制方法和成像装置 |
CN207354459U (zh) * | 2017-10-19 | 2018-05-11 | 维沃移动通信有限公司 | 一种摄像头模组和移动终端 |
CN108271012A (zh) * | 2017-12-29 | 2018-07-10 | 维沃移动通信有限公司 | 一种深度信息的获取方法、装置以及移动终端 |
CN108600712A (zh) * | 2018-07-19 | 2018-09-28 | 维沃移动通信有限公司 | 一种图像传感器、移动终端及图像拍摄方法 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112616009A (zh) * | 2020-12-31 | 2021-04-06 | 维沃移动通信有限公司 | 电子设备及其摄像模组 |
WO2022143682A1 (zh) * | 2020-12-31 | 2022-07-07 | 维沃移动通信有限公司 | 电子设备及其摄像模组 |
Also Published As
Publication number | Publication date |
---|---|
CN108600712A (zh) | 2018-09-28 |
EP3809686A4 (en) | 2021-06-09 |
EP4250750A3 (en) | 2023-11-15 |
US11743605B2 (en) | 2023-08-29 |
EP3809686A1 (en) | 2021-04-21 |
EP4250750A2 (en) | 2023-09-27 |
US20230336880A1 (en) | 2023-10-19 |
CN108600712B (zh) | 2020-03-31 |
US11962918B2 (en) | 2024-04-16 |
US20220368866A1 (en) | 2022-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020015627A1 (zh) | 图像传感器、移动终端及图像拍摄方法 | |
WO2020015532A1 (zh) | 图像传感器、移动终端及拍摄方法 | |
CN108900750B (zh) | 一种图像传感器及移动终端 | |
WO2022143280A1 (zh) | 图像传感器、摄像模组和电子设备 | |
WO2020015626A1 (zh) | 移动终端及图像拍摄方法 | |
WO2020015560A1 (zh) | 图像传感器及移动终端 | |
CN108965666B (zh) | 一种移动终端及图像拍摄方法 | |
WO2019091426A1 (zh) | 摄像头组件、图像获取方法及移动终端 | |
WO2019144956A1 (zh) | 图像传感器、镜头模组、移动终端、人脸识别方法及装置 | |
US11996421B2 (en) | Image sensor, mobile terminal, and image capturing method | |
CN113676651B (zh) | 图像传感器、控制方法、控制装置、电子设备和存储介质 | |
CN110913144B (zh) | 图像处理方法及摄像装置 | |
CN108965703A (zh) | 一种图像传感器、移动终端及图像拍摄方法 | |
WO2019170121A1 (zh) | 摄像头模组及移动终端 | |
CN115278000A (zh) | 图像传感器、图像生成方法、摄像头模组及电子设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19837519 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2019837519 Country of ref document: EP Effective date: 20210118 |