WO2021184353A1

WO2021184353A1 - Camera device

Info

Publication number: WO2021184353A1
Application number: PCT/CN2020/080408
Authority: WO
Inventors: 黄进新; 汪鹏程; 刘军
Original assignee: 华为技术有限公司
Priority date: 2020-03-20
Filing date: 2020-03-20
Publication date: 2021-09-23
Also published as: CN113728618A

Abstract

The embodiments of the present application disclose a camera device using asymmetric image sensors. The camera device comprises two sensors having different resolutions, wherein the image sensor having a low resolution generates a color image, and the image sensor having a high resolution generates a color image or a grayscale image. Color information and luminance information of the images generated by the two image sensors are acquired and fused to generate a target image, improving an imaging effect of a camera in a low illumination scene.

Description

一种摄像装置A camera device

技术领域Technical field

本申请实施例涉及安防监控领域，尤其涉及一种摄像装置。The embodiments of the present application relate to the field of security monitoring, and in particular, to a camera device.

背景技术Background technique

低照度场景(low illumination scene/low light scene)，指光线不足的场景，例如夜晚的户外或者没有充足照明的室内。在安防监控领域中，为了在低照度场景下获取清晰度高且色彩丰富的图像，摄像机常与可见光补光灯或红外光补光灯配套使用。但是，使用可见光补光灯易造成光污染且不利于隐蔽监控；使用红外补光灯虽然成像清晰，但无法记录色彩。近年来，业界开始广泛采用双光融合架构，在这种架构下，摄像机使用2个传感器对红外光和可见光分别成像，再对红外图像与可见光图像进行融合的方式，以提升摄像机在低照度下的成像能力。Low illumination scene (low illumination scene/low light scene) refers to scenes with insufficient light, such as outdoors at night or indoors without sufficient lighting. In the field of security surveillance, in order to obtain high-definition and colorful images in low-illuminance scenes, cameras are often used in conjunction with visible light supplementary lights or infrared light supplementary lights. However, the use of visible light supplementary light is easy to cause light pollution and is not conducive to covert monitoring; although the use of infrared supplementary light has a clear image, it cannot record colors. In recent years, the industry has begun to widely adopt a dual-light fusion architecture. Under this architecture, the camera uses two sensors to image infrared light and visible light separately, and then integrates the infrared image and the visible light image to improve the camera’s low illumination Imaging capabilities.

具体地，在摄像机中设置分光棱镜，该分光棱镜根据频谱将入射光线分为可见光和红外光。然后，该摄像机将前述可见光和红外光分别输入两个相同的图像传感器。其中，输入可见光的图像传感器输出彩色图像，输入红外光的图像传感器输出灰度图像，并且，前述彩色图像和灰度图像的尺寸大小和分辨率均相同。该摄像机再将前述彩色图像和灰度图像进行融合得到目标图像，该目标图像的细节和纹理主要来自于该灰度图像，而色彩信息来自于该彩色图像。Specifically, a light splitting prism is set in the camera, and the light splitting prism divides the incident light into visible light and infrared light according to the frequency spectrum. Then, the camera inputs the aforementioned visible light and infrared light into two identical image sensors respectively. Among them, the image sensor inputting visible light outputs a color image, and the image sensor inputting infrared light outputs a grayscale image, and the size and resolution of the foregoing color image and grayscale image are the same. The camera then fuses the aforementioned color image and grayscale image to obtain a target image. The details and texture of the target image mainly come from the grayscale image, and the color information comes from the color image.

在前述方案中，当环境光照度低于可见光图像传感器的成像信噪比下限时，彩色图像的色彩信息被噪声淹没，导致最终输出的融合图像色彩饱和度很低，甚至退化为灰度图像。因此，在当前双光融合架构下，进一步降低摄像机的工作照度下限显得十分重要。In the foregoing solution, when the ambient illuminance is lower than the lower limit of the imaging signal-to-noise ratio of the visible light image sensor, the color information of the color image is overwhelmed by noise, resulting in low color saturation of the final output fused image, or even a grayscale image. Therefore, under the current dual-light fusion architecture, it is very important to further reduce the lower limit of the camera's operating illuminance.

发明内容Summary of the invention

本申请实施例提供了一种摄像装置，用于在低照度环境下输出较高质量的图像，降低摄像装置的工作照度下限。The embodiments of the present application provide a camera device, which is used to output a higher-quality image in a low-illuminance environment and reduce the lower limit of the operating illuminance of the camera device.

第一方面，本申请实施例提供了一种摄像装置，该摄像装置包括：光学组件、第一图像传感器、第二图像传感器和图像处理器。该第一图像传感器的分辨率小于该第二图像传感器的分辨率。其中，该光学组件为对入射光信号进行预处理的组件，该光学组件用于接收入射光信号并将入射光信号处理为第一光信号和第二光信号。该第一图像传感器，用于感应该第一光信号生成第一图像；同时，该第二图像传感器，用于感应该第二光信号生成第二图像。其中，该第一图像的图像信息包括第一色彩信息和第一亮度信息；该第二图像的图像信息包括第二亮度信息。此外，该第一图像的分辨率小于该第二图像的分辨率。该图像处理器，用于基于该第一图像和该第二图像生成目标图像。该目标图像的色彩和亮度由该第一图像的图像信息和该第二图像的图像信息确定。In the first aspect, an embodiment of the present application provides a camera device, which includes an optical component, a first image sensor, a second image sensor, and an image processor. The resolution of the first image sensor is smaller than the resolution of the second image sensor. Wherein, the optical component is a component for preprocessing the incident optical signal, and the optical component is used to receive the incident optical signal and process the incident optical signal into a first optical signal and a second optical signal. The first image sensor is used to sense the first light signal to generate a first image; at the same time, the second image sensor is used to sense the second light signal to generate a second image. Wherein, the image information of the first image includes first color information and first brightness information; the image information of the second image includes second brightness information. In addition, the resolution of the first image is smaller than the resolution of the second image. The image processor is configured to generate a target image based on the first image and the second image. The color and brightness of the target image are determined by the image information of the first image and the image information of the second image.

本申请实施例中，摄像装置采用两个分辨率不同的图像传感器。由于，图像分辨率与色彩灵敏度呈反相关，与图像清晰度呈正相关。因此，低分辨率的第一图像传感器输出的第一图像具有较高的色彩灵敏度，可以保证第一图像的色彩真实。而高分辨率的第二图像传感器由于像素较多，因此输出的第二图像具有较高的清晰度，可以呈现丰富的细节和纹理。因此，基于前述两幅图像生成的目标图像可以保留第一图像和第二图像的优点，因此，可以使摄像装置工作于更低光照强度的环境。In the embodiment of the present application, the camera device uses two image sensors with different resolutions. Because image resolution is inversely related to color sensitivity, and positively related to image sharpness. Therefore, the first image output by the low-resolution first image sensor has higher color sensitivity, which can ensure the true color of the first image. Since the second image sensor with high resolution has more pixels, the output second image has higher definition and can present rich details and textures. Therefore, the target image generated based on the foregoing two images can retain the advantages of the first image and the second image, and therefore, the camera device can be made to work in an environment with lower light intensity.

根据第一方面，本申请实施例第一方面的第一种实施方式中，该图像处理器，具体用于：将该第一图像的分辨率调整至与该第二图像的分辨率相同，得到第三图像；然后，将该第三图像与该第二图像融合，得到目标图像。其中，该第三图像携带该第一色彩信息和该第一亮度信息。According to the first aspect, in the first implementation manner of the first aspect of the embodiments of the present application, the image processor is specifically configured to: adjust the resolution of the first image to be the same as the resolution of the second image to obtain The third image; then, the third image and the second image are fused to obtain the target image. Wherein, the third image carries the first color information and the first brightness information.

本实施方式中，由于，第一图像的分辨率小于第二图像的分辨率，因此，将第一图像的分辨率调整至与第二图像的分辨率相同，可以理解为，将第一图像的分辨率调高至第二图像的分辨率得到第三图像。由于，该第一图像具有较高的色彩灵敏度、真实的色彩且有较高的亮度，因此，第三图像也具有真实的色彩和较高的亮度。因此，基于前述第三图像和第二图像确定的目标图像可以保留第三图像和第二图像的优点，因此，可以使摄像装置工作于更低光照强度的环境。In this embodiment, since the resolution of the first image is smaller than the resolution of the second image, adjusting the resolution of the first image to be the same as the resolution of the second image can be understood as changing the resolution of the first image The resolution is increased to the resolution of the second image to obtain a third image. Since the first image has higher color sensitivity, true colors, and higher brightness, the third image also has true colors and higher brightness. Therefore, the target image determined based on the foregoing third image and the second image can retain the advantages of the third image and the second image, and therefore, the camera device can be made to work in an environment with lower light intensity.

根据第一方面或第一方面的第一种实施方式，本申请实施例第一方面的第二种实施方式中，该第一光信号包括可见光，该第二光信号包括可见光和红外光，该第一光信号中的可见光的能量大于该第二光信号中的可见光的能量，该第一光信号中的可见光的频段与该第二光信号中的可见光的频段相同。According to the first aspect or the first implementation manner of the first aspect, in a second implementation manner of the first aspect of the embodiments of the present application, the first optical signal includes visible light, and the second optical signal includes visible light and infrared light. The energy of the visible light in the first optical signal is greater than the energy of the visible light in the second optical signal, and the frequency band of the visible light in the first optical signal is the same as the frequency band of the visible light in the second optical signal.

其中，前述第一光信号中的可见光的能量与第二光信号中的可见光的能量之间的比例可以根据实际应用需求灵活控制。Wherein, the ratio between the energy of the visible light in the first optical signal and the energy of the visible light in the second optical signal can be flexibly controlled according to actual application requirements.

本实施方式中，提出前述光学组件对入射光信号的处理不仅仅是分频，还包括分能量。其中，分能量指对入射光信号中的可见光进行划分，也可以理解为，通过处理使得第一光信号中的可见光的强度与第二光信号中的可见光的强度不同。由于，第一光信号中的可见光的能量与第二光信号中的可见光的能量不同，有利于两个图像传感器在光照强度高于预设值时(即白天)得到的两幅图像的亮度不同。基于前述两幅亮度不同亮度的图像确定目标图像有利于提升动态范围。In this embodiment, it is proposed that the processing of the aforementioned optical component on the incident light signal is not only frequency division, but also energy division. Wherein, energy division refers to dividing the visible light in the incident light signal, which can also be understood as processing to make the intensity of the visible light in the first light signal different from the intensity of the visible light in the second light signal. Since the energy of the visible light in the first light signal is different from the energy of the visible light in the second light signal, it is beneficial for the two image sensors to have different brightness of the two images obtained when the light intensity is higher than the preset value (that is, during the day). . Determining the target image based on the aforementioned two images with different brightness is beneficial to improve the dynamic range.

根据第一方面的第二种实施方式，本申请实施例第一方面的第三种实施方式中，该第一图像传感器为彩色图像传感器，该第二图像传感器为黑白图像传感器。According to the second implementation manner of the first aspect, in the third implementation manner of the first aspect of the embodiments of the present application, the first image sensor is a color image sensor, and the second image sensor is a black and white image sensor.

本实施方式中，提出第二图像传感器采用黑白图像传感器。由于，黑白图像传感器的色彩滤镜矩阵比相同规格的彩色图像传感器的色彩滤镜矩阵的透光率更高，光电转换效率更高。因此，可以提升第二图像传感器输出的第二图像的亮度，进而有利于提升目标图像的质量。因此，可以使摄像装置工作于更低光照强度的环境。In this embodiment, it is proposed that the second image sensor is a black-and-white image sensor. Because the color filter matrix of the black-and-white image sensor has higher light transmittance than the color filter matrix of the color image sensor of the same specification, the photoelectric conversion efficiency is higher. Therefore, the brightness of the second image output by the second image sensor can be improved, which is beneficial to improve the quality of the target image. Therefore, the camera device can be operated in a lower light intensity environment.

根据第一方面的第三种实施方式，本申请实施例第一方面的第四种实施方式中，该摄像装置还包括红外截止滤光片。该摄像装置，还用于当光照强度高于预设值时启动该红外截止滤光片，该红外截止滤光片位于该光学组件和该第二图像传感器之间，该红外截止滤光片用于滤除该第二光信号中的红外光。该第二图像传感器，具体用于感应该第二光信号中的可见光，生成该第二图像。该图像处理器，具体用于将该第一图像的第一色彩信息与该第二图像的第二亮度信息组合，得到该目标图像，该目标图像的色彩由该第一色彩信息确定，该目标图像的亮度由该第二亮度信息确定。According to the third implementation manner of the first aspect, in the fourth implementation manner of the first aspect of the embodiments of the present application, the imaging device further includes an infrared cut-off filter. The camera device is also used to activate the infrared cut-off filter when the light intensity is higher than a preset value, the infrared cut-off filter is located between the optical component and the second image sensor, and the infrared cut-off filter is used for To filter out the infrared light in the second optical signal. The second image sensor is specifically used to sense visible light in the second light signal to generate the second image. The image processor is specifically configured to combine the first color information of the first image with the second brightness information of the second image to obtain the target image, the color of the target image is determined by the first color information, and the target The brightness of the image is determined by the second brightness information.

本实施方式中，提出在光照强度较高的情况下采用红外截止滤光片滤除第二光信号中的红外光，以截止第二图像传感器在感应可见光时受红外光的影响。此外，该第二图像传感器为黑白图像传感器，由该第二图像传感器仅感应第二光信号中的可见光确定的第二图像仅呈现亮度而无色彩。虽然，第一图像与第二图像的分辨率不同，但是，第一图像提供了丰富的色彩(即第一色彩信息)，第二图像提供了高亮度的纹理细节(即第二亮度信息)。因此，在一些情况下，可以采用第一图像中的部分或全部的第一色彩信息和第二图像中的部分或全部的第二亮度信息组合以获得目标图像。例如，当前述第一亮度信息指示的亮度与第二亮度信息指示的亮度相当时，可以将前述第一图像中的部分或全部的第一色彩信息和第二图像中的部分或全部的第二亮度信息组合以获得目标图像。In this embodiment, it is proposed to use an infrared cut filter to filter the infrared light in the second light signal when the light intensity is high, so as to cut off the second image sensor from being affected by infrared light when sensing visible light. In addition, the second image sensor is a black and white image sensor, and the second image determined by the second image sensor only sensing the visible light in the second light signal presents only brightness and no color. Although the resolution of the first image and the second image are different, the first image provides rich colors (ie, first color information), and the second image provides high-brightness texture details (ie, second brightness information). Therefore, in some cases, part or all of the first color information in the first image and part or all of the second brightness information in the second image may be combined to obtain the target image. For example, when the brightness indicated by the first brightness information is equivalent to the brightness indicated by the second brightness information, part or all of the first color information in the first image and part or all of the second color information in the second image may be combined. The brightness information is combined to obtain the target image.

根据第一方面的第三种实施方式，本申请实施例第一方面的第五种实施方式中，当光照强度低于预设值时，该第二图像传感器，具体用于感应该第二光信号中的可见光和红外光，生成该第二图像。该图像处理器，具体用于将该第三图像与该第二图像融合，得到该目标图像。According to the third implementation manner of the first aspect, in the fifth implementation manner of the first aspect of the embodiments of the present application, when the light intensity is lower than a preset value, the second image sensor is specifically used to sense the second light The visible light and infrared light in the signal generate the second image. The image processor is specifically configured to fuse the third image with the second image to obtain the target image.

本实施方式中，提出该第二图像传感器在光照强度较低时同时感应红外光和一部分可见光。由于，该第二图像传感器为黑白图像传感器，该第二图像传感器输出的第二图像仅有亮度。但是，由于该第二图像传感器除了感应红外光以外还感应可见光，因此，该第二图像传感器感应红外光和可见光输出的第二图像的亮度大于该第二图像传感器仅感应红外光输出的第二图像的亮度。因此，可以提升第二图像的质量，进而提升目标图像的质量。In this embodiment, it is proposed that the second image sensor simultaneously senses infrared light and a part of visible light when the light intensity is low. Since the second image sensor is a black and white image sensor, the second image output by the second image sensor has only brightness. However, because the second image sensor senses visible light in addition to infrared light, the brightness of the second image output by the second image sensor that senses infrared light and visible light is greater than that of the second image sensor that only senses infrared light output. The brightness of the image. Therefore, the quality of the second image can be improved, thereby improving the quality of the target image.

根据第一方面的第二种实施方式，本申请实施例第一方面的第六种实施方式中，该第一图像传感器与该第二图像传感器均为彩色图像传感器。According to the second implementation manner of the first aspect, in the sixth implementation manner of the first aspect of the embodiments of the present application, the first image sensor and the second image sensor are both color image sensors.

根据第一方面的第六种实施方式，本申请实施例第一方面的第七种实施方式中，该摄像装置还包括红外截止滤光片。该摄像装置，还用于当光照强度高于预设值时启动该红外截止滤光片，该红外截止滤光片位于该光学组件和该第二图像传感器之间，该红外截止滤光片用于滤除该第二光信号中的红外光。该第二图像传感器，具体用于感应该第二光信号中的可见光，生成该第二图像，该第二图像的图像信息还包括第二色彩信息。该图像处理器，具体用于将该第三图像与该第二图像融合，得到该目标图像。According to the sixth implementation manner of the first aspect, in the seventh implementation manner of the first aspect of the embodiments of the present application, the imaging device further includes an infrared cut filter. The camera device is also used to activate the infrared cut-off filter when the light intensity is higher than a preset value, the infrared cut-off filter is located between the optical component and the second image sensor, and the infrared cut-off filter is used for To filter out the infrared light in the second optical signal. The second image sensor is specifically configured to sense visible light in the second light signal to generate the second image, and the image information of the second image further includes second color information. The image processor is specifically configured to fuse the third image with the second image to obtain the target image.

本实施方式中，提出在光照强度较高的情况下采用红外截止滤光片滤除第二光信号中的红外光，以截止第二图像传感器在感应可见光时受红外光的影响。由于第二图像传感器为彩色图像传感器，因此该第二图像传感器感应可见光确定的第二图像不仅包括第二亮度信息，还包括第二色彩信息。又由于，第一图像传感器和第二图像传感器感应的可见光的能量不同，因此，前述第三图像和前述第二图像为亮度不同的两幅图像。基于前述两幅亮度不同的图像确定目标图像，可以提升目标图像的质量还可以提升动态范围。In this embodiment, it is proposed to use an infrared cut filter to filter the infrared light in the second light signal when the light intensity is high, so as to cut off the second image sensor from being affected by infrared light when sensing visible light. Since the second image sensor is a color image sensor, the second image determined by the second image sensor sensing visible light includes not only the second brightness information, but also the second color information. In addition, since the energy of the visible light sensed by the first image sensor and the second image sensor are different, the aforementioned third image and the aforementioned second image are two images with different brightness. Determining the target image based on the aforementioned two images with different brightness can improve the quality of the target image and also increase the dynamic range.

根据第一方面的第六种实施方式，本申请实施例第一方面的第八种实施方式中，该摄像装置还包括可见光截止滤光片。该摄像装置，还用于当光照强度低于预设值时启动该可见光截止滤光片，该可见光截止滤光片位于该光学组件和该第二图像传感器之间，该可见光截止滤光片用于滤除该第二光信号中的可见光。该第二图像传感器，具体用于感应该第二光信号中的红外光，生成该第二图像。该图像处理器，具体用于将该第三图像与该第二图像融合，得到该目标图像。According to the sixth implementation manner of the first aspect, in the eighth implementation manner of the first aspect of the embodiments of the present application, the imaging device further includes a visible light cutoff filter. The camera device is also used to activate the visible light cut-off filter when the light intensity is lower than a preset value, the visible light cut-off filter is located between the optical component and the second image sensor, and the visible light cut-off filter is used for To filter out the visible light in the second optical signal. The second image sensor is specifically used to sense infrared light in the second light signal to generate the second image. The image processor is specifically configured to fuse the third image with the second image to obtain the target image.

本实施方式中，提出在光照强度较低时采用可见光滤光片滤除第二光信号中的可见光，以降低第二图像传感器在感应红外光时受可见光的影响。虽然，第二图像传感器为彩色图像传感器，但该第二光信号中仅包含红外光。因此，该第二图像仅包含第二亮度信息而无色彩信息。In this embodiment, it is proposed to use a visible light filter to filter out visible light in the second light signal when the light intensity is low, so as to reduce the influence of visible light on the second image sensor when sensing infrared light. Although the second image sensor is a color image sensor, the second light signal only contains infrared light. Therefore, the second image only contains the second brightness information and no color information.

根据第一方面、第一方面的第一种实施方式至第一方面的第八种实施方式中的任意一种实施方式，本申请实施例第一方面的第九种实施方式中，该光学组件包括镜头与分光棱镜，该分光棱镜位于所述镜头与图像传感器之间。该镜头，用于接收该入射光信号。该分光棱镜，用于将该镜头接收的入射光信号分为该第一光信号和该第二光信号。According to any one of the first aspect, the first implementation manner of the first aspect to the eighth implementation manner of the first aspect, in the ninth implementation manner of the first aspect of the embodiments of the present application, the optical assembly It includes a lens and a dichroic prism, and the dichroic prism is located between the lens and the image sensor. The lens is used to receive the incident light signal. The light splitting prism is used for dividing the incident light signal received by the lens into the first light signal and the second light signal.

根据第一方面的第九种实施方式，本申请实施例第一方面的第十种实施方式中，该镜头为红外共焦镜头。According to the ninth implementation manner of the first aspect, in the tenth implementation manner of the first aspect of the embodiments of the present application, the lens is an infrared confocal lens.

根据第一方面、第一方面的第一种实施方式至第一方面的第八种实施方式中的任意一种实施方式，本申请实施例第一方面的第十一种实施方式中，该摄像装置还包括红外截止滤光片。该光学组件包括第一镜头和第二镜头，该第一镜头与该第二镜头用于共同接收该入射光信号，该第一镜头的焦距与该第二镜头的焦距相同，该第一镜头的光圈大于该第二镜头的光圈，该第一镜头与该第一图像传感器之间设有红外截止滤光片，该第二镜头为红外共焦镜头。该第一镜头，用于接收该入射光信号中的一部分，并将收到的光信号传输至该红外截止滤光片。该红外截止滤光片，用于滤除来自该第一镜头的光信号中的红外光，得到该第一光信号，并将该第一光信号传输至该第一图像传感器。该第二镜头，用于接收该入射光信号中的余下部分，并将收到的光信号作为第二光信号传输至该第二图像传感器。According to any one of the first aspect, the first implementation manner of the first aspect to the eighth implementation manner of the first aspect, in the eleventh implementation manner of the first aspect of the embodiments of the present application, the camera The device also includes an infrared cut filter. The optical assembly includes a first lens and a second lens. The first lens and the second lens are used to jointly receive the incident light signal. The focal length of the first lens is the same as the focal length of the second lens. The aperture is larger than the aperture of the second lens, an infrared cut filter is arranged between the first lens and the first image sensor, and the second lens is an infrared confocal lens. The first lens is used to receive a part of the incident light signal and transmit the received light signal to the infrared cut filter. The infrared cut-off filter is used to filter infrared light in the optical signal from the first lens to obtain the first optical signal, and transmit the first optical signal to the first image sensor. The second lens is used to receive the remaining part of the incident light signal, and transmit the received light signal to the second image sensor as a second light signal.

本实施方式中，提出可以采用不同光圈的双目镜头使照向不同图像传感器的光信号的能量不同。由于，光圈越大光通量越多，因此，前述第一镜头输出的第一光信号中的可见光的能量大于第二镜头输出的第二光信号中的可见光的能量。此外，该第一镜头与第一图像传感器之间还设置有红外截止滤光片，可以使该第一光信号中仅有可见光而不含红外光。In this embodiment, it is proposed that binocular lenses with different apertures can be used to make the energy of the light signals illuminating different image sensors different. Since the larger the aperture, the greater the luminous flux. Therefore, the energy of the visible light in the first light signal output by the first lens is greater than the energy of the visible light in the second light signal output by the second lens. In addition, an infrared cut filter is also arranged between the first lens and the first image sensor, so that only visible light but no infrared light is included in the first light signal.

根据前述任意一种实施方式，本申请实施例第一方面的第十二种实施方式中，该第二图像的分辨率等于该目标图像的分辨率。According to any one of the foregoing implementation manners, in the twelfth implementation manner of the first aspect of the embodiments of the present application, the resolution of the second image is equal to the resolution of the target image.

第二方面，本申请实施例提供了一种图像处理器，该图像处理器与摄像装置中的存储器连接。该存储器，用于存储该处理器处理的数据或程序，例如前述第一图像、第二图像以及第三图像；该图像处理器，用于调用该存储器中的程序对前述第一图像、第二图像以及第三图像进行图像处理。In the second aspect, an embodiment of the present application provides an image processor, which is connected to a memory in a camera device. The memory is used to store data or programs processed by the processor, such as the aforementioned first image, second image, and third image; The image and the third image are subjected to image processing.

从以上技术方案可以看出，本申请实施例具有以下优点：It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

本申请实施例中，摄像装置采用两个分辨率不同的图像传感器。由于，图像分辨率与色彩灵敏度呈反相关，与图像清晰度呈正相关。因此，低分辨率的第一图像传感器输出的第一图像具有较高的色彩灵敏度，可以保证第一图像的色彩真实。而高分辨率的第二图像传感器由于像素较多，因此输出的第二图像具有较高的清晰度，可以呈现丰富的细节和纹理。因此，基于前述两幅图像确定的目标图像可以保留第一图像和第二图像的优点，因此，可以使摄像装置工作于更低光照强度的环境。In the embodiment of the present application, the camera device uses two image sensors with different resolutions. Because image resolution is inversely related to color sensitivity, and positively related to image sharpness. Therefore, the first image output by the low-resolution first image sensor has higher color sensitivity, which can ensure the true color of the first image. The high-resolution second image sensor has more pixels, so the output second image has higher definition and can present rich details and textures. Therefore, the target image determined based on the foregoing two images can retain the advantages of the first image and the second image, and therefore, the camera device can be made to work in an environment with lower light intensity.

附图说明Description of the drawings

为了更清楚地说明本申请实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例。In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application.

图1A为本申请实施例中摄像装置的一个实施例的示意图；FIG. 1A is a schematic diagram of an embodiment of a camera device in an embodiment of the application;

图1B为本申请实施例中摄像装置的另一个实施例的示意图；FIG. 1B is a schematic diagram of another embodiment of the camera device in the embodiment of the application;

图1C为本申请实施例中摄像装置的另一个实施例的示意图；FIG. 1C is a schematic diagram of another embodiment of the imaging device in the embodiment of the application;

图2A为本申请实施例中摄像装置的另一个实施例的示意图；2A is a schematic diagram of another embodiment of the camera device in the embodiment of the application;

图2B为本申请实施例中图像处理流程的一个实施例的示意图；2B is a schematic diagram of an embodiment of an image processing flow in an embodiment of the application;

图2C为本申请实施例中图像处理流程的另一个实施例的示意图；2C is a schematic diagram of another embodiment of the image processing flow in the embodiment of the application;

图3A为本申请实施例中摄像装置的另一个实施例的示意图；3A is a schematic diagram of another embodiment of the camera device in the embodiment of the application;

图3B为本申请实施例中图像处理流程的另一个实施例的示意图；3B is a schematic diagram of another embodiment of the image processing flow in the embodiment of the application;

图3C为本申请实施例中图像处理流程的另一个实施例的示意图。FIG. 3C is a schematic diagram of another embodiment of the image processing flow in the embodiment of the application.

具体实施方式Detailed ways

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, without having to use To describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Those steps or units may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

为便于理解，下面先对本申请实施例所提出的摄像装置的应用场景进行介绍：For ease of understanding, the following first introduces the application scenarios of the camera device proposed in the embodiments of the present application:

本申请实施例所提出的摄像装置可以应用于低照度(low illumination/low light)环境下的摄像。具体地，该低照度环境指光照强度低于一定的值的环境，一般由摄像装置中图像传感器的单位面积上接受可见光的能量来衡量，单位为勒克斯(Lux，也简称为Lx)。一般地，可以将大于0Lux且小于1Lux的光照环境称为低照度环境。具体地，该低照度环境可以为室外昏暗的街道，例如，夜晚时的街道、或阴雨天的街道；也可以为仅有微弱灯光的室内，例如，仅有微弱灯光的商店、仓库等，具体此处不做限定。The camera device proposed in the embodiment of the present application can be applied to camera in a low illumination/low light environment. Specifically, the low-illuminance environment refers to an environment where the light intensity is lower than a certain value, which is generally measured by the energy received by the visible light per unit area of the image sensor in the imaging device, and the unit is Lux (Lx). Generally, an illumination environment greater than 0 Lux and less than 1 Lux can be referred to as a low-illuminance environment. Specifically, the low-illuminance environment can be a dim outdoor street, for example, a street at night, or a street on a rainy day; it can also be an interior with only weak light, for example, a store, warehouse, etc., with only weak light. There is no limitation here.

在前述低照度场景下，由于，当图像传感器的感光面积固定时，输出的图像的色彩灵敏度和图像的分辨率之间呈反相关。考虑到图像传感器感应红外光生成的灰度图像能够提供较高的空间分辨率，并且，人眼视觉***对亮度信息更为敏感而对色彩空间分辨率不太敏感。因此，可以牺牲部分彩色图像的空间分辨率以换取色彩灵敏度的提升，进而达到降低摄像装置的工作照度下限的效果。例如，原本摄像机的工作照度要求下限是1Lux，低于1Lux难以获得可以被人眼接受的彩色图像。使用本发明实施例提供的方案后，可以将摄像装置的工作照度下限降低至0.1Lux，甚至0.01Lux，具体此处不做限定。In the aforementioned low-illuminance scene, because when the photosensitive area of the image sensor is fixed, there is an inverse correlation between the color sensitivity of the output image and the resolution of the image. Taking into account that the grayscale image generated by the image sensor sensing infrared light can provide higher spatial resolution, and the human visual system is more sensitive to brightness information and less sensitive to color spatial resolution. Therefore, the spatial resolution of some color images can be sacrificed in exchange for an increase in color sensitivity, thereby achieving the effect of reducing the lower limit of the operating illuminance of the camera device. For example, the lower limit of the original camera's working illumination requirement is 1Lux, and it is difficult to obtain a color image that can be accepted by the human eye if it is lower than 1Lux. After using the solution provided by the embodiment of the present invention, the lower limit of the operating illuminance of the camera device can be reduced to 0.1 Lux, or even 0.01 Lux, which is not specifically limited here.

下面将对本申请实施例提出的摄像装置进行介绍，如图1A所示，该摄像装置10(例如摄像机)包括：第一图像传感器101、第二图像传感器102、光学组件103和图像处理器104。其中，该光学组件103用于接收入射光信号，并将入射光信号处理为第一光信号和第二光信号，其中入射光信号由被摄像装置10拍摄的对象发出。该光学组件103还用于控制该第一光信号射向第一图像传感器101，控制该第二光信号射向第二图像传感器102。其中，该第一图像传感器101，用于感应该第一光信号生成第一图像，该第一图像是彩色图像，该第一图像的图像信息包括第一色彩信息和第一亮度信息。该第二图像传感器102，用于感应该第二光信号生成第二图像，该第二图像是彩色图像或者灰度图像，该第二图像的图像信息包括第二亮度信息(当第二图像是彩色图像时，该第二图像还包括第二色彩信息)。此外，该第一图像传感器101的分辨率小于该第二图像传感器102的分辨率。该图像处理器104，用于基于该第一图像和该第二图像生成目标图像，该目标图像的色彩和亮度由该第一图像的图像信息和该第二图像的图像信息确定。例如，该图像处理器104可以是片上***(system on chip，SoC)。The camera device proposed in the embodiment of the present application will be introduced below. As shown in FIG. Wherein, the optical component 103 is used to receive the incident light signal and process the incident light signal into a first light signal and a second light signal, wherein the incident light signal is emitted by an object photographed by the imaging device 10. The optical component 103 is also used to control the first light signal to be directed to the first image sensor 101 and control the second light signal to be directed to the second image sensor 102. The first image sensor 101 is used to sense the first light signal to generate a first image, the first image is a color image, and the image information of the first image includes first color information and first brightness information. The second image sensor 102 is used to sense the second light signal to generate a second image, the second image is a color image or a grayscale image, and the image information of the second image includes second brightness information (when the second image is In the case of a color image, the second image also includes second color information). In addition, the resolution of the first image sensor 101 is smaller than the resolution of the second image sensor 102. The image processor 104 is configured to generate a target image based on the first image and the second image, and the color and brightness of the target image are determined by the image information of the first image and the image information of the second image. For example, the image processor 104 may be a system on chip (system on chip, SoC).

应当理解的是，由于该第一图像传感器101的分辨率小于该第二图像传感器102的分辨率，因此，该第一图像的分辨率小于该第二图像的分辨率。还应理解的是，由于该第一图像包含第一色彩信息，即第一图像能够呈现出色彩，因此，该第一图像传感器101为彩色图像传感器。而该第二图像传感器102可以为彩色图像传感器，也可以为黑白图像传感器，具体请参阅后文图2A和图3A对应实施例的相关介绍，此处不再赘述。It should be understood that since the resolution of the first image sensor 101 is smaller than the resolution of the second image sensor 102, the resolution of the first image is smaller than the resolution of the second image. It should also be understood that since the first image contains first color information, that is, the first image can present colors, the first image sensor 101 is a color image sensor. The second image sensor 102 may be a color image sensor or a black-and-white image sensor. For details, please refer to the related description of the corresponding embodiments in FIG. 2A and FIG. 3A later, and details are not repeated here.

本实施例中，该摄像装置10采用两个分辨率不同的图像传感器。由于，图像分辨率与色彩灵敏度呈反相关，与图像清晰度呈正相关。因此，低分辨率的第一图像传感器101输出的第一图像具有较高的色彩灵敏度，可以保证第一图像的色彩真实。而高分辨率的第二图像传感器102由于像素较多，因此输出的第二图像具有较高的清晰度，可以呈现丰富的细节和纹理。因此，基于前述两幅图像确定的目标图像可以保留第一图像和第二图像的优点，因此，可以使摄像装置工作于更低光照强度的环境。In this embodiment, the imaging device 10 uses two image sensors with different resolutions. Because image resolution is inversely related to color sensitivity, and positively related to image sharpness. Therefore, the first image output by the low-resolution first image sensor 101 has high color sensitivity, which can ensure the true color of the first image. Since the high-resolution second image sensor 102 has more pixels, the output second image has higher definition and can present rich details and textures. Therefore, the target image determined based on the foregoing two images can retain the advantages of the first image and the second image, and therefore, the camera device can be made to work in an environment with lower light intensity.

基于前述实施例，该图像处理器104确定目标图像的过程可以包括如下：Based on the foregoing embodiment, the process of determining the target image by the image processor 104 may include the following:

首先，该图像处理器104将第一图像的分辨率调整至与第二图像的分辨率相同，得到第三图像。然后，该图像处理器104基于该第三图像和该第二图像确定该目标图像。可选的，该目标图像的分辨率等于该第二图像的分辨率。在前述过程中，虽然该第三图像的分辨率与该第一图像的分辨率不同，但是，该第三图像呈现的色彩和亮度来自于该第一图像。因此，将前述低分辨率的第一图像调整为高分辨率的第三图像的过程，不仅可以保留第一图像呈现的色彩和亮度，还有利于基于第三图像和第二图像确定目标图像。First, the image processor 104 adjusts the resolution of the first image to be the same as the resolution of the second image to obtain a third image. Then, the image processor 104 determines the target image based on the third image and the second image. Optionally, the resolution of the target image is equal to the resolution of the second image. In the foregoing process, although the resolution of the third image is different from the resolution of the first image, the color and brightness presented by the third image are derived from the first image. Therefore, the process of adjusting the aforementioned low-resolution first image to a high-resolution third image can not only preserve the color and brightness presented by the first image, but also facilitate the determination of the target image based on the third image and the second image.

应当理解的是，本实施例中前述两幅图像的分辨率相同，可以理解为，两幅图像的分辨率完全相同；也可以理解为，两幅图像的分辨率之间存在一定差异，但是，差异不足以影响两幅图像的后续处理，具体此处不做限定。It should be understood that in this embodiment, the resolutions of the aforementioned two images are the same. It can be understood that the resolutions of the two images are exactly the same; it can also be understood that there is a certain difference between the resolutions of the two images. However, The difference is not enough to affect the subsequent processing of the two images, and the specifics are not limited here.

应当理解的是，在该图像处理器104将第一图像的分辨率调整至与第二图像的分辨率相同的过程中，该图像处理器104可以采用上采样算法或超分辨率算法，具体此处不做限定。此外，该图像处理器104基于前述第三图像和第二图像确定目标图像的过程，主要指图像融合(image fusing)过程。图像融合(image fusion)指一种图像处理技术，可以使用特定的算法将两幅或多幅图像综合成一幅新的图像，合成后的图像具备原始图像(即合成前的两幅或多幅图像)的优良特性。例如，亮度、清晰度以及色彩等特性。It should be understood that when the image processor 104 adjusts the resolution of the first image to be the same as the resolution of the second image, the image processor 104 may adopt an up-sampling algorithm or a super-resolution algorithm. There is no limit. In addition, the process of determining the target image by the image processor 104 based on the aforementioned third image and second image mainly refers to an image fusing process. Image fusion (image fusion) refers to an image processing technology that can use a specific algorithm to combine two or more images into a new image. The combined image has the original image (that is, the two or more images before synthesis). ) Excellent characteristics. For example, characteristics such as brightness, clarity, and color.

可选的，该摄像装置10还包括图像信号处理器(image signal processor，ISP)芯片(图未示)，该ISP芯片位于前述图像传感器和图像处理器104之间。可选的，前述ISP芯片可以为两路处理进程，分别对第一图像传感器101输出的第一图像和第二图像传感器102输出的第二图像进行处理，并将处理后的第一图像和第二图像发送至前述图像处理器104进行后续处理。可选的，该ISP芯片可以对前述第一图像和第二图像进行多项ISP处理，例如，3D降噪、去马赛克、亮度矫正以及色彩矫正等处理。前述基本的图像处理可以根据实际应用需求进行调整，具体此处不限定前述基本的图像处理的内容。Optionally, the camera device 10 further includes an image signal processor (image signal processor, ISP) chip (not shown), and the ISP chip is located between the aforementioned image sensor and the image processor 104. Optionally, the aforementioned ISP chip may be a two-way processing process, respectively processing the first image output by the first image sensor 101 and the second image output by the second image sensor 102, and combining the processed first image and the second image. The two images are sent to the aforementioned image processor 104 for subsequent processing. Optionally, the ISP chip can perform multiple ISP processing on the aforementioned first image and second image, for example, 3D noise reduction, demosaicing, brightness correction, and color correction. The aforementioned basic image processing can be adjusted according to actual application requirements. Specifically, the content of the aforementioned basic image processing is not limited here.

需要说明的是，本实施例以及后续实施例中，将从第一图像传感器101输出的图像到调整分辨率之前的图像均称为第一图像。类似的，将从第二图像传感器102输出的图像到进行图像融合或组合之前的图像均称为第二图像。也就是说，在实际应用中，从图像传感器输出的原始图像可以经过多种处理流程，而本申请实施例并没有对前述处理流程进行限定。换句话说，在后续如果直接对图像传感器输出的RAW图像进行融合，那么所述第一图像是RAW格式；如果对ISP芯片输出的的RGB图像进行融合，那么所述第一图像是RGB格式；如果对ISP芯片输出的YUV图像进行融合，那么所述第一图像是YUV格式(为减少冗余，后续实施例仅以第一、第二图像均为YUV格式的图像为例进行介绍)。此外，第一图像与第二图像均未经过压缩编码，融合后的图像可以通过编码生成易于被人眼识别并且占用存储空间更小的图像格式，例如jpeg格式(简称为jpg格式)、bmp格式、tga格式png格式以及gif格式。It should be noted that in this embodiment and subsequent embodiments, the image output from the first image sensor 101 to the image before the resolution is adjusted are all referred to as the first image. Similarly, the image output from the second image sensor 102 to the image before the image fusion or combination is called the second image. That is to say, in practical applications, the original image output from the image sensor may go through a variety of processing procedures, but the embodiment of the present application does not limit the foregoing processing procedures. In other words, if the RAW image output by the image sensor is directly fused later, then the first image is in the RAW format; if the RGB image output by the ISP chip is fused, then the first image is in the RGB format; If the YUV image output by the ISP chip is fused, the first image is in the YUV format (in order to reduce redundancy, the subsequent embodiments only use images in the YUV format as an example for the introduction of the first and second images). In addition, the first image and the second image are not compressed and encoded, and the fused image can be encoded into an image format that is easy to be recognized by the human eye and occupies a smaller storage space, such as jpeg format (referred to as jpg format), bmp format , Tga format, png format and gif format.

可选的，在将前述第三图像和第二图像进行融合之前，该ISP芯片还可以对前述第三图像和第二图像进行图像校正。具体地，该ISP芯片可以将第三图像的坐标系矫正至第二图像的坐标系中，使得前述两幅图像中的场景对齐，也可以理解为，使前述两幅图像的呈现的纹理细节对齐。可选的，该ISP芯片可以采用预设的矫正参数对前述两幅图像进行调整，也可以根据当前温度的变化自适应配置矫正参数以完成图像矫正。Optionally, before fusing the foregoing third image and the second image, the ISP chip may also perform image correction on the foregoing third image and the second image. Specifically, the ISP chip can correct the coordinate system of the third image to the coordinate system of the second image, so that the scenes in the foregoing two images are aligned, which can also be understood as aligning the texture details of the foregoing two images. . Optionally, the ISP chip may use preset correction parameters to adjust the aforementioned two images, or may adaptively configure the correction parameters according to changes in the current temperature to complete image correction.

可选的，前述第一图像传感器101或第二图像传感器102可以为由电荷耦合器件(charged coupled device，CCD)构成的CCD图像传感器，也可以为由互补金属氧化物半导体(complementary metal oxide semiconductor，CMOS)构成的CMOS图像传感器，还可以为其他类型的图像传感器，具体此处不做限定。Optionally, the aforementioned first image sensor 101 or the second image sensor 102 may be a CCD image sensor composed of a charged coupled device (CCD), or a complementary metal oxide semiconductor (complementary metal oxide semiconductor, The CMOS image sensor composed of CMOS) may also be other types of image sensors, which is not specifically limited here.

在前述实施例中，该摄像装置10采用非对称图像传感器架构(即摄像装置10中的两个图像传感器的分辨率不同)，可以通过降低感应可见光的第一图像传感器的空间分辨率换取色彩敏度(即低照灵敏度)。因此，可以使该摄像装置10更好地工作于低照度环境。在此基础之上，该光学组件103具体用于通过对入射光信号处理，使该第一光信号中的可见光的能量大于该第二光信号中的可见光的能量。其中，第一光信号包括可见光，第二光信号包括可见光和红外光。也可以理解为，该光学组件103对入射光信号进行分频分能量。其中，分频指将入射光信号按照不同频率进行划分，例如，将入射光信号分为可见光和红外光。而前述能量与光波的振幅的平方成正比，前述可见光的能量也可以理解为可见光的强度。因此，分能量可以理解为通过使用透镜的镀膜等物理构造，把入射光信号中的可见光分离成第一光信号与第二光信号，第一光信号中的可见光的强度与第二光信号中的可见光的强度不同，并且前述第一光信号中的可见光与第二光信号中的可见光之间的强度比例固定。应当注意的是，该第一光信号中的可见光的频段与该第二光信号中的可见光的频段相同；或者，第一光信号中的可见光的频段与该第二光信号中的可见光的拥有相同频段的光，例如第一光信号和第二光信号都拥有绿光。In the foregoing embodiment, the camera device 10 adopts an asymmetric image sensor architecture (that is, the resolutions of the two image sensors in the camera device 10 are different), and the color sensitivity can be obtained by reducing the spatial resolution of the first image sensor that senses visible light. Degree (ie, low sensitivity). Therefore, the camera device 10 can be made to work better in a low-illuminance environment. On this basis, the optical component 103 is specifically used to process the incident light signal so that the energy of the visible light in the first light signal is greater than the energy of the visible light in the second light signal. Wherein, the first light signal includes visible light, and the second light signal includes visible light and infrared light. It can also be understood that the optical component 103 performs frequency division and energy division on the incident light signal. Among them, frequency division refers to dividing the incident light signal according to different frequencies, for example, dividing the incident light signal into visible light and infrared light. The aforementioned energy is proportional to the square of the amplitude of the light wave, and the aforementioned visible light energy can also be understood as the intensity of visible light. Therefore, energy division can be understood as separating the visible light in the incident light signal into the first light signal and the second light signal by using physical structures such as coating of the lens. The intensity of the visible light in the first light signal and the second light signal are The intensity of the visible light is different, and the intensity ratio between the visible light in the first light signal and the visible light in the second light signal is fixed. It should be noted that the frequency band of visible light in the first optical signal is the same as the frequency band of visible light in the second optical signal; or, the frequency band of visible light in the first optical signal is the same as the possession of visible light in the second optical signal. Light of the same frequency band, for example, both the first optical signal and the second optical signal have green light.

在这样的实施方式中，由于，第一光信号中的可见光的能量与第二光信号中的可见光的能量不同，有利于两个图像传感器在光照强度高于预设值时(即白天)得到的两幅图像的亮度不同。基于前述两幅亮度不同亮度的图像确定目标图像有利于提升动态范围。此外，前述第一光信号中的可见光的能量与第二光信号中的可见光的能量之间的比例可以根据实际应用需求灵活控制。例如，第一光信号与第二光信号包含的可见光频段相同，并且第一光信号中的可见光的能量与第二光信号中的可见光的能量之比可以保持为9：1，或者保持为8:2，或者保持为7:3，或者保持为6:4，或者保持为6.5:3.5等，具体此处不做限定。In such an embodiment, since the energy of the visible light in the first light signal is different from the energy of the visible light in the second light signal, it is beneficial for the two image sensors to obtain the result when the light intensity is higher than the preset value (that is, during the day). The brightness of the two images is different. Determining the target image based on the aforementioned two images with different brightness is beneficial to improve the dynamic range. In addition, the ratio between the energy of the visible light in the first optical signal and the energy of the visible light in the second optical signal can be flexibly controlled according to actual application requirements. For example, the first optical signal and the second optical signal contain the same visible light frequency band, and the ratio of the energy of the visible light in the first optical signal to the energy of the visible light in the second optical signal can be kept at 9:1, or kept at 8. :2, or remain as 7:3, or remain as 6:4, or remain as 6.5:3.5, etc. The specifics are not limited here.

进一步地，前述光学组件103可以采用如下任意一种实现方式：Further, the foregoing optical component 103 may adopt any one of the following implementation modes:

如图1B所示，为前述光学组件103的一种实现方式。该光学组件103包括分光棱镜1031和镜头1032，该分光棱镜1031也可以被称为分束棱镜(beam splitter)，是一种光学器件，通过在光学玻璃表面镀上一层或多层薄膜，利用光线的折射和反射将入射光信号分成两束。本实施例中，该分光棱镜1031用于将该入射光信号分为第一光信号和第二光信号。其中，第一光信号包括可见光，第二光信号包括可见光和红外光。具体地，入射光信号中的一部分可见光通过镀膜层射向该第一图像传感器101，该入射光信号中的另一部分可见光和全部红外光在镀膜层处反射，并照射向该第二图像传感器102。应当理解的是，如果不同光学组件选择使用具有不同厚度、组成成分镀膜层的光学玻璃，那么，不同的光学组件各自对应第一光信号中的可见光的能量与第二光信号中的可见光能量之间的比例亦会不同。可选的，该镜头1032为红外共焦镜头，该红外共焦镜头用于实现红外共焦。As shown in FIG. 1B, it is an implementation manner of the aforementioned optical assembly 103. The optical component 103 includes a dichroic prism 1031 and a lens 1032. The dichroic prism 1031 can also be called a beam splitter. It is an optical device that is coated with one or more films on the surface of the optical glass. The refraction and reflection of light split the incident light signal into two beams. In this embodiment, the dichroic prism 1031 is used to divide the incident optical signal into a first optical signal and a second optical signal. Wherein, the first light signal includes visible light, and the second light signal includes visible light and infrared light. Specifically, part of the visible light in the incident light signal is directed to the first image sensor 101 through the coating layer, and the other part of the visible light and all infrared light in the incident light signal is reflected at the coating layer and irradiated to the second image sensor 102 . It should be understood that if different optical components choose to use optical glass with different thicknesses and composition coating layers, then different optical components correspond to the energy of the visible light in the first optical signal and the energy of the visible light in the second optical signal. The ratio between the two will also be different. Optionally, the lens 1032 is an infrared confocal lens, and the infrared confocal lens is used to realize infrared confocal.

此外，该光学组件103常与滤光片配套使用。具体地，可以在分光棱镜1031与第一图像传感器101之间设置滤光片(例如，滤光片1051)，也可以在分光棱镜1032与第二图像传感器102之间设置滤光片(例如，滤光片1052)。可选的，当前述滤光片1051为红外截止滤光片时，可以防止进入第一图像传感器101中的第一光信号混入红外光。可选的，当前述滤光片1052为红外截止滤光片时，可以滤除第二光信号中的红外光；当前述滤光片1052为可见光截止滤光片时，可以滤除第二光信号中的可见光；当前述滤光片1052为白玻片(该白玻片的材料是无色透明玻璃，不滤除光线)时，可以供该第二光信号中的可见光和红外光通过，即允许全频段的光信号通过该白玻片。In addition, the optical component 103 is often used in conjunction with a filter. Specifically, a filter (e.g., filter 1051) may be provided between the dichroic prism 1031 and the first image sensor 101, or a filter (e.g., Filter 1052). Optionally, when the aforementioned filter 1051 is an infrared cut filter, the first light signal entering the first image sensor 101 can be prevented from being mixed with infrared light. Optionally, when the aforementioned filter 1052 is an infrared cut filter, the infrared light in the second optical signal can be filtered; when the aforementioned filter 1052 is a visible light cut filter, the second light can be filtered Visible light in the signal; when the aforementioned filter 1052 is a white glass (the material of the white glass is colorless and transparent glass, which does not filter light), it can pass the visible light and infrared light in the second optical signal, That is, the optical signal of the full frequency band is allowed to pass through the white glass slide.

本实施方式中，通过重新设计分光棱镜并结合滤光片以实现分频(在第一、第二光信号中，通过使用红外滤截止光片，避免了第一光信号中出现红外光，实现了红外光频段与可见光频段的切分)分能量(在第一、第二光信号中均包括相同频段的可见光，而且这两路可见光的能量不同，实现了能量的切分)，可以将可见光和红外光分频到两个图像传感器，同时可以控制入射光信号中的可见光分别进入到两个图像传感器的比例。因此，可以使最终输出的融合图像，在光照强度较高时(例如，白天)可以提升动态范围(dynamic range)，在光照强度较低时(例如，黑夜)在前述提升低照度的基础上可以进一步使目标图像的色彩更自然。In this embodiment, the splitting prism is redesigned and the filter is combined to achieve frequency division (in the first and second optical signals, the infrared filter is used to cut off the light, avoiding infrared light in the first optical signal, and achieving Infrared light frequency band and visible light frequency band are divided) energy (the first and second light signals both include visible light of the same frequency band, and the energy of the two visible lights is different, which realizes the energy division), which can divide the visible light And infrared light is divided into two image sensors, and the ratio of visible light in the incident light signal into the two image sensors can be controlled at the same time. Therefore, the final output of the fused image can be increased when the light intensity is high (for example, during the day), and the dynamic range can be increased when the light intensity is low (for example, at night). Further make the color of the target image more natural.

如图1C所示，为前述光学组件103的另一种实现方式。As shown in FIG. 1C, it is another implementation of the aforementioned optical assembly 103.

该光学组件103包括第一镜头1033和第二镜头1034，该第一镜头1033用于汇聚入射光信号中的一部分以使得输出的光信号照向第一图像传感器101，该第二镜头1034用于汇聚入射光信号中的余下部分以使得输出的光信号照向第二图像传感器102。其中，该第一镜头1033的焦距与该第二镜头1034的焦距相同。可选的，该第一镜头1033的光圈大于该第二镜头1034的光圈。因此，该第一镜头1033的光通量大于该第二镜头1034的光通量。因此，第一镜头1033输出的可见光的能量大于第二镜头1034输出的可见光的能量。也可以理解为，前述第一镜头1033与第二镜头1034实际通过的光信号不同。此外，关于可见光的能量的解释可以参阅前述图1B对应实施例的相关介绍，此处不再赘述。The optical assembly 103 includes a first lens 1033 and a second lens 1034. The first lens 1033 is used to converge a part of the incident light signal so that the output light signal illuminates the first image sensor 101, and the second lens 1034 is used for The remaining part of the incident light signal is converged to make the output light signal illuminate the second image sensor 102. Wherein, the focal length of the first lens 1033 is the same as the focal length of the second lens 1034. Optionally, the aperture of the first lens 1033 is larger than the aperture of the second lens 1034. Therefore, the luminous flux of the first lens 1033 is greater than the luminous flux of the second lens 1034. Therefore, the energy of the visible light output by the first lens 1033 is greater than the energy of the visible light output by the second lens 1034. It can also be understood that the light signals actually passed by the first lens 1033 and the second lens 1034 are different. In addition, for the explanation of the energy of the visible light, please refer to the relevant introduction of the corresponding embodiment in FIG. 1B, which will not be repeated here.

此外，该第一镜头1033与该第一图像传感器101之间设有滤光片(例如，滤光片1051)，该第二镜头1034与该第二图像传感器102之间也可以设置滤光片(例如，滤光片1052)。具体地，前述滤光片1051为红外截止滤光片，用于过滤来自该第一镜头1033的光信号中的红外光，以使得该第一图像传感器101感应的第一光信号中仅有可见光而不含红外光。可选的，当前述滤光片1052为红外截止滤光片时，可以滤除第二光信号中的红外光；当前述滤光片1052为可见光截止滤光片时，可以滤除第二光信号中的可见光；当前述滤光片1052为白玻片(该白玻片的材料是无色透明玻璃，不滤除光线)时，可以供该第二光信号中的可见光和红外光通过，即允许全波段的光信号通过该白玻片。此外，该第二镜头1034为红外共焦镜头。In addition, a filter (for example, a filter 1051) is provided between the first lens 1033 and the first image sensor 101, and a filter may also be provided between the second lens 1034 and the second image sensor 102 (For example, filter 1052). Specifically, the aforementioned filter 1051 is an infrared cut filter for filtering infrared light in the optical signal from the first lens 1033, so that only visible light is included in the first optical signal sensed by the first image sensor 101 It does not contain infrared light. Optionally, when the aforementioned filter 1052 is an infrared cut filter, the infrared light in the second optical signal can be filtered; when the aforementioned filter 1052 is a visible light cut filter, the second light can be filtered Visible light in the signal; when the aforementioned filter 1052 is a white glass (the material of the white glass is colorless and transparent glass, which does not filter light), it can pass the visible light and infrared light in the second optical signal, That is, the full-wavelength optical signal is allowed to pass through the white glass slide. In addition, the second lens 1034 is an infrared confocal lens.

本实施方式中，采用焦距相同但光圈大小不同的两个镜头组成双目镜头，以实现分能量(不同传感器分到的光能量不同)。由于光圈的尺寸大小决定了能够通过镜头的光线强弱，因此采用不同大小的光圈，可以控制进入两个图像传感器的可见光的能量不同。再结合前述非对称图像传感器架构，并且在不同的光照强度下采用不同的滤光片，可以使最终输出的融合图像，在光照强度较高时(例如，白天)可以提升动态范围，在光照强度较低时(例如，黑夜)进一步使目标图像的色彩更自然。In this embodiment, two lenses with the same focal length but different aperture sizes are used to form a binocular lens to achieve energy distribution (different sensors receive different light energy). Since the size of the aperture determines the intensity of the light that can pass through the lens, the use of apertures of different sizes can control the energy of the visible light entering the two image sensors to be different. Combined with the aforementioned asymmetric image sensor architecture, and using different filters under different light intensities, the final output of the fused image can be increased when the light intensity is high (for example, during the day). When it is lower (for example, dark night), the color of the target image is more natural.

在实际应用中，前述图1A中的光学组件103以及后文涉及的光学组件可以采用图1B所示的实现方式，也可以采用图1C所示的实现方式，具体可以根据应用场景选择，此处不做限定。In practical applications, the aforementioned optical component 103 in FIG. 1A and the optical components involved in the following may adopt the implementation shown in FIG. 1B or the implementation shown in FIG. Not limited.

应当理解的是，前述摄像装置中的两个图像传感器除了分辨率不同以外，感光性能也可以不相同。下面分别进行介绍：It should be understood that, in addition to the different resolutions of the two image sensors in the aforementioned imaging device, the photosensitive performance may also be different. The following are introduced separately:

如图2A所示，为前述摄像装置的一种实施方式。在该实施方式中，前述第一图像传感器为彩色图像传感器，前述第二图像传感器为黑白图像传感器。由于，该第一图像传感器的分辨率小于该第二图像传感器的分辨率，因此，在后文称前述第一图像传感器为低分辨率彩色图像传感器，称前述第二图像传感器为高分辨率黑白图像传感器。可选的，前述低分辨率彩色图像传感器可以为拜耳格式图像传感器(Bayer image sensor)或者其他格式的彩色图像传感器；前述高分辨率黑白图像传感器可以为MONO格式图像传感器(Mono image sensor)或者其他格式的黑白图像传感器，具体此处不做限定。该低分辨率彩色图像传感器与光学组件之间设置有红外截止滤光片，该高分辨率黑白图像传感器和该光学组件之间设置有双光滤光片。该双光滤光片也被称为IR-CUT自动切换滤镜。该IR-CUT自动切换滤镜上设置有感光装置，或该IR-CUT自动切换滤镜与感光装置相连，该感光装置将感应到的光照强度传输至摄像装置。当侦测到光照强度变化时，该摄像装置(具体而言可以是摄像装置中的滤光片控制芯片)可以控制IR-CUT自动切换滤镜自动切换。例如，在光照强度大于预设值时(例如白天)，切换至红外截止滤光片，在光照强度小于预设值时(例如黑夜)，切换至可见光截止外滤光片。可选的，双光滤光片也可以切换为白玻片，允许可见光和红外光均通过。可选的，可以把双光滤光片替换成红外截止滤光片，由摄像装置控制红外截止滤光片的使能与无效。As shown in FIG. 2A, it is an embodiment of the aforementioned imaging device. In this embodiment, the first image sensor is a color image sensor, and the second image sensor is a black and white image sensor. Since the resolution of the first image sensor is smaller than the resolution of the second image sensor, the aforementioned first image sensor is referred to as a low-resolution color image sensor, and the aforementioned second image sensor is referred to as a high-resolution black-and-white image sensor. Image Sensor. Optionally, the aforementioned low-resolution color image sensor can be a Bayer image sensor or other format color image sensors; the aforementioned high-resolution black and white image sensor can be a MONO format image sensor (Mono image sensor) or others The format of the black and white image sensor is not limited here. An infrared cut filter is arranged between the low-resolution color image sensor and the optical component, and a dual-optical filter is arranged between the high-resolution black-and-white image sensor and the optical component. This dual optical filter is also called an IR-CUT automatic switching filter. The IR-CUT automatic switching filter is provided with a photosensitive device, or the IR-CUT automatic switching filter is connected to a photosensitive device, and the photosensitive device transmits the sensed light intensity to the imaging device. When a change in light intensity is detected, the camera device (specifically, a filter control chip in the camera device) can control the IR-CUT to automatically switch the filter automatically. For example, when the light intensity is greater than the preset value (for example, during the day), switch to the infrared cut filter, and when the light intensity is less than the preset value (for example, at night), switch to the visible light cut off filter. Optionally, the dual light filter can also be switched to a white glass plate, allowing both visible light and infrared light to pass. Optionally, the dual-optical filter can be replaced with an infrared cut-off filter, and the infrared cut-off filter is controlled by the camera device to enable and disable the infrared cut-off filter.

当光照强度低于预设值时，结合图2B进行介绍。When the light intensity is lower than the preset value, it will be introduced in conjunction with Figure 2B.

该低分辨率彩色图像传感器感应第一光信号中的可见光，输出低分辨率彩色图像。其中，该低分辨率彩色图像包括第一色彩信息和第一亮度信息。该第一色彩信息用于指示该低分辨率彩色图像的色彩，该第一亮度信息用于指示该低分辨率彩色图像呈现的亮度。本实施例不限定第一亮度信息、第一色彩信息的具体形式。The low-resolution color image sensor senses visible light in the first light signal and outputs a low-resolution color image. Wherein, the low-resolution color image includes first color information and first brightness information. The first color information is used to indicate the color of the low-resolution color image, and the first brightness information is used to indicate the brightness of the low-resolution color image. This embodiment does not limit the specific forms of the first brightness information and the first color information.

该高分辨率黑白图像传感器和该光学组件之间设置的双光滤光片切换为白玻片，该高分辨率黑白图像传感器感应该第二光信号中的可见光和红外光，输出高分辨率灰度图像。其中，该高分辨率灰度图像包括第二亮度信息，该第二亮度信息用于指示高分辨率灰度图像呈现的亮度。可选的，该第二亮度信息可以采用亮度分量Y表示。本申请实施例不限定第二亮度信息的具体形式。前述图像传感器直接生成的图像是RAW格式，根据图像传感器设计的不同，RAW格式也分成多种类型，例如可以是拜耳RGGB，RYYB，RCCC、RCCB、RGBW、CMYW等多种格式。使用ISP芯片，可以把各种格式的RAW图像转换成RGB格式图像。使用ISP芯片，亦可以把RAW格式图像转换成YUV格式图像，或者HSV格式图像、或者Lab格式图像、或者CMY格式图像、或者YCbCr格式图像。例如：ISP芯片先把RAW格式图像转换成RGB格式图像，再把RGB格式图像转换成YUV格式图像。该摄像装置中的ISP芯片还可以对前述低分辨率色彩图像和高分辨率灰度图像进行基本的图像处理，例如，3D降噪、去马赛克、亮度矫正以及色彩矫正等处理。The dual-optical filter set between the high-resolution black-and-white image sensor and the optical assembly is switched to a white glass plate, and the high-resolution black-and-white image sensor senses visible light and infrared light in the second light signal, and outputs high-resolution Grayscale image. Wherein, the high-resolution gray-scale image includes second brightness information, and the second brightness information is used to indicate the brightness of the high-resolution gray-scale image. Optionally, the second brightness information may be represented by a brightness component Y. The embodiment of the present application does not limit the specific form of the second brightness information. The image directly generated by the aforementioned image sensor is in the RAW format. Depending on the design of the image sensor, the RAW format is also divided into multiple types, such as Bayer RGGB, RYYB, RCCC, RCCB, RGBW, CMYW and other formats. Using the ISP chip, you can convert various formats of RAW images into RGB format images. Using the ISP chip, you can also convert RAW format images into YUV format images, or HSV format images, or Lab format images, or CMY format images, or YCbCr format images. For example: the ISP chip first converts RAW format images into RGB format images, and then converts RGB format images into YUV format images. The ISP chip in the camera device can also perform basic image processing on the aforementioned low-resolution color images and high-resolution grayscale images, such as 3D noise reduction, demosaicing, brightness correction, and color correction.

此外，该图像处理器采用上采样算法或超分辨率算法将前述低分辨率彩色图像调整为高分辨率彩色图像，该高分辨率彩色图像与前述高分辨率灰度图像具有相同的分辨率。然后，该图像处理器将前述高分辨率彩色图像和高分辨率灰度图像融合，得到目标图像。In addition, the image processor adopts an up-sampling algorithm or a super-resolution algorithm to adjust the aforementioned low-resolution color image to a high-resolution color image, and the high-resolution color image has the same resolution as the aforementioned high-resolution grayscale image. Then, the image processor fuses the aforementioned high-resolution color image and high-resolution gray-scale image to obtain the target image.

以图2B为例，将低分辨率的RGB格式图像201转换为低分辨率的YUV格式图像202，然后，再对低分辨率的YUV格式图像202进行上采样得到高分辨率的YUV格式图像203。然后，将高分辨率的YUV格式图像203和仅有Y分量的高分辨率灰度图像204进行融合，得到高分辨率的YUV格式图像205(即目标图像)。Taking Figure 2B as an example, the low-resolution RGB format image 201 is converted into a low-resolution YUV format image 202, and then the low-resolution YUV format image 202 is up-sampled to obtain a high-resolution YUV format image 203 . Then, the high-resolution YUV format image 203 and the high-resolution grayscale image 204 with only the Y component are fused to obtain a high-resolution YUV format image 205 (ie, the target image).

应当理解的是，在实际图像处理过程中，除了前述图2B中所列举的图像以外还可以出现其他格式的图像，具体此处不做限定。It should be understood that, in the actual image processing process, in addition to the images listed in FIG. 2B, images in other formats may also appear, which is not specifically limited here.

应当理解的是，前述Y分量即YUV格式中的Y，表示明亮度(luminance/luma)，而U和V表示色度/浓度(chrominance/chroma)。YUV格式主要包括YUV420、YUV422和YUV444。其中，YUV444指每一个Y分量对应一组UV分量；YUV422指每两个Y分量共用一组UV分量；YUV420指每四个Y分量共用一组UV分量。虽然，前述图2B采用YUV420列举示例，但是在实际应用中，可以根据具体需求调整图像的格式，具体此处不做限定。此外，虽然前述YUV格式与RGB格式为不同的颜色编码方式，但编码格式的改变不会影响图像呈现的色彩。It should be understood that the aforementioned Y component, which is Y in the YUV format, represents brightness (luminance/luma), and U and V represent chrominance/chroma. YUV formats mainly include YUV420, YUV422 and YUV444. Among them, YUV444 means that each Y component corresponds to a set of UV components; YUV422 means that every two Y components share a set of UV components; YUV420 means that every four Y components share a set of UV components. Although the foregoing FIG. 2B uses YUV420 as an example, in practical applications, the format of the image can be adjusted according to specific requirements, which is not specifically limited here. In addition, although the aforementioned YUV format and the RGB format are different color coding methods, the change of the coding format will not affect the colors presented by the image.

本实施例中，提出在光照强度较低时，前述高分辨率黑白图像传感器同时感应红外光和一部分可见光，而非仅感应红外光或仅感应可见光。因此，该第二图像传感器感应红外光和可见光输出的第二图像的亮度大于该第二图像传感器仅感应红外光输出的第二图像的亮度。因此，可以提升第二图像的质量，进而提升目标图像的质量。In this embodiment, it is proposed that when the light intensity is low, the aforementioned high-resolution black and white image sensor simultaneously senses infrared light and a part of visible light, instead of only sensing infrared light or only sensing visible light. Therefore, the brightness of the second image output by the second image sensor by sensing infrared light and visible light is greater than the brightness of the second image output by the second image sensor by sensing only infrared light. Therefore, the quality of the second image can be improved, thereby improving the quality of the target image.

当光照强度高于预设值时，结合图2C进行介绍。When the light intensity is higher than the preset value, it will be introduced in conjunction with Figure 2C.

该低分辨率彩色图像传感器感应第一光信号中的可见光，输出低分辨率彩色图像。其中，该低分辨率彩色图像包括第一色彩信息和第一亮度信息。该第一色彩信息用于指示该低分辨率彩色图像的色彩，该第一亮度信息用于指示该低分辨率彩色图像呈现的亮度。本申请实施例不限定第一亮度信息、第一色彩信息的具体形式。可选的，当将该低分辨率彩色图像转换为YUV格式时，该第一色彩信息为U/V分量，该第一亮度信息为Y分量。The low-resolution color image sensor senses visible light in the first light signal and outputs a low-resolution color image. Wherein, the low-resolution color image includes first color information and first brightness information. The first color information is used to indicate the color of the low-resolution color image, and the first brightness information is used to indicate the brightness of the low-resolution color image. The embodiment of the present application does not limit the specific form of the first brightness information and the first color information. Optionally, when the low-resolution color image is converted into the YUV format, the first color information is a U/V component, and the first brightness information is a Y component.

该高分辨率黑白图像传感器和该光学组件之间设置的双光滤光片切换为红外截止滤光片，该红外截止滤光片用于滤除该第二光信号中的红外光。因此，该高分辨率黑白图像传感器感应该第二光信号中的可见光，输出高分辨率灰度图像。其中，该高分辨率灰度图像包括第二亮度信息，该第二亮度信息用于指示高分辨率灰度图像呈现的亮度。可选的，该第二亮度信息可以采用亮度分量Y表示。由于，该高分辨率黑白图像传感器无法记录色彩，因此该第二图像仅呈现亮度而无法呈现色彩。因此，该第二图像仅有亮度分量Y而无色度分量U/V。应当理解的是，当前述图像传感器的格式不同时，输出的图像的格式也将不同，具体前文已作详细介绍，此处不再赘述。The dual-optical filter arranged between the high-resolution black and white image sensor and the optical assembly is switched to an infrared cut-off filter, and the infrared cut-off filter is used to filter the infrared light in the second optical signal. Therefore, the high-resolution black and white image sensor senses the visible light in the second light signal and outputs a high-resolution grayscale image. Wherein, the high-resolution gray-scale image includes second brightness information, and the second brightness information is used to indicate the brightness of the high-resolution gray-scale image. Optionally, the second brightness information may be represented by a brightness component Y. Since the high-resolution black-and-white image sensor cannot record colors, the second image only presents brightness and cannot present colors. Therefore, the second image has only the luminance component Y and no chrominance component U/V. It should be understood that when the format of the aforementioned image sensor is different, the format of the output image will also be different, which has been described in detail in the foregoing, and will not be repeated here.

本实施例中，当通过控制曝光时间和增益使第一亮度信息呈现的亮度和第二亮度信息呈现的亮度相当时，该图像处理器可以将该低分辨率彩色图像的第一色彩信息与该高分辨率灰度图像的第二亮度信息组合，得到该目标图像。其中，该目标图像的色彩由该第一色彩信息确定，该目标图像的亮度由该第二亮度信息确定。具体地，该图像处理器可以将前述低分辨率彩色图像的色彩分量(即U/V分量)与高分辨率的灰度图像的亮度分量(即Y分量)组合，得到目标图像。在这样的实施方式中，无需采用复杂的融合算法即可获得较高质量的目标图像，可以降低图像处理器的数据处理量。In this embodiment, when the brightness presented by the first brightness information is equivalent to the brightness presented by the second brightness information by controlling the exposure time and gain, the image processor can compare the first color information of the low-resolution color image with the brightness of the second brightness information. The second brightness information of the high-resolution grayscale image is combined to obtain the target image. Wherein, the color of the target image is determined by the first color information, and the brightness of the target image is determined by the second brightness information. Specifically, the image processor may combine the color component (ie U/V component) of the aforementioned low-resolution color image with the brightness component (ie Y component) of the high-resolution grayscale image to obtain the target image. In such an implementation manner, a higher-quality target image can be obtained without using a complex fusion algorithm, which can reduce the amount of data processing of the image processor.

可选的，该彩色图像传感器的分辨率与该黑白图像传感器的分辨率之比为1:4，并且，该低分辨率彩色图像(即第一图像)的分辨率与高分辨率灰度图像(即第二图像)的分辨率之比为1:4。可选的，将前述低分辨率彩色图像和高分辨率灰度图像均采用YUV格式表示。可选的，前述低分辨率彩色图像采用YUV444格式，前述高分辨率灰度图像采用YUV420格式。此时，前述高分辨率的灰度图像的Y分量的数量与前述低分辨率彩色图像的U/V分量的数量之比为4:1。因此，该图像处理器可以输出格式为YUV420的目标图像。以图2C为例，将低分辨率的RGB格式图像211转换为低分辨率的YUV444格式图像212。然后，将低分辨率的YUV444格式图像212中的U/V分量和高分辨率的灰度图像213中的Y分量进行组合，得到高分辨率的YUV420格式图像214(即目标图像)。Optionally, the ratio of the resolution of the color image sensor to the resolution of the black-and-white image sensor is 1:4, and the resolution of the low-resolution color image (that is, the first image) and the high-resolution grayscale image (Ie, the second image) has a resolution ratio of 1:4. Optionally, the aforementioned low-resolution color image and high-resolution gray-scale image are both expressed in the YUV format. Optionally, the aforementioned low-resolution color image adopts the YUV444 format, and the aforementioned high-resolution gray-scale image adopts the YUV420 format. At this time, the ratio of the number of Y components of the aforementioned high-resolution grayscale image to the number of U/V components of the aforementioned low-resolution color image is 4:1. Therefore, the image processor can output the target image in the YUV420 format. Taking FIG. 2C as an example, the low-resolution RGB format image 211 is converted into a low-resolution YUV444 format image 212. Then, the U/V component in the low-resolution YUV444 format image 212 and the Y component in the high-resolution grayscale image 213 are combined to obtain a high-resolution YUV420 format image 214 (that is, the target image).

应当理解的是，前述低分辨率彩色图像传感器的分辨率与高分辨率灰度图像传感器的分辨率之比可以为其他值，例如，1:2或1:16等，具体此处不做限定。当前述两个图像传感器的分辨率的比值不同时，图像融合过程采用的图像格式也将适应性调整，以便于在输出较好质量的目标图像的同时，降低输出目标图像的计算量。It should be understood that the ratio of the resolution of the aforementioned low-resolution color image sensor to the resolution of the high-resolution grayscale image sensor can be other values, for example, 1:2 or 1:16, etc., which is not specifically limited here. . When the ratios of the resolutions of the aforementioned two image sensors are different, the image format used in the image fusion process will also be adaptively adjusted, so as to output a better quality target image while reducing the amount of calculation to output the target image.

可选的，本实施方式中的第一光信号中的可见光与第二光信号中的可见光的比例为3:2。也可以理解为，前述光学组件同时按频谱和能量进行分束，其中，入射光信号中60％的可见光照射向低分辨率彩色图像传感器，入射光信号中40％的可见光和100％的红外光照射向高分辨率黑白图像传感器。当然，由于滤光片的作用，可以进一步调整实际进入前述两个图像传感器的光信号。Optionally, the ratio of visible light in the first optical signal to visible light in the second optical signal in this embodiment is 3:2. It can also be understood that the aforementioned optical component splits beams according to frequency spectrum and energy at the same time, wherein 60% of the visible light in the incident light signal is irradiated to the low-resolution color image sensor, and 40% of the visible light and 100% of the infrared light in the incident light signal are irradiated to the low-resolution color image sensor. Illuminate to the high-resolution black-and-white image sensor. Of course, due to the effect of the filter, the light signals actually entering the two image sensors can be further adjusted.

本实施例中，提出第一图像传感器采用低分辨率彩色图像传感器、第二图像传感器采用高分辨率黑白图像传感器。首先，非对称图像传感器架构可以降低摄像装置的工作照度下限。其次，黑白图像传感器的色彩滤镜矩阵比相同规格的彩色图像传感器的色彩滤镜矩阵的透光率更高，光电转换效率更高。因此，可以提升高分辨率黑白图像传感器输出的高分辨率灰度图像的亮度(即第二亮度信息指示的亮度)，进而有利于提升目标图像的质量。因此，可以进一步使摄像装置工作于更低光照强度的环境。In this embodiment, it is proposed that the first image sensor adopts a low-resolution color image sensor, and the second image sensor adopts a high-resolution black-and-white image sensor. First, the asymmetric image sensor architecture can reduce the lower limit of the operating illuminance of the camera device. Secondly, the color filter matrix of the black and white image sensor has higher light transmittance and higher photoelectric conversion efficiency than the color filter matrix of the color image sensor of the same specification. Therefore, the brightness of the high-resolution grayscale image output by the high-resolution black-and-white image sensor (that is, the brightness indicated by the second brightness information) can be improved, thereby helping to improve the quality of the target image. Therefore, it is possible to further make the camera device work in an environment with lower light intensity.

如图3A所示，为前述摄像装置的另一种实施方式。在该实施方式中，前述第一图像传感器和前述第二图像传感器均为彩色图像传感器。可选的，该彩色图像传感器可以为拜耳格式图像传感器(Bayer image sensor)或者其他格式的彩色图像传感器。由于，该第一图像传感器的分辨率小于该第二图像传感器的分辨率，因此，在后文称前述第一图像传感器为低分辨率彩色图像传感器，称前述第二图像传感器为高分辨率彩色图像传感器。该低分辨率彩色图像传感器与光学组件之间设置有红外截止滤光片，该高分辨率彩色图像传感器和该光学组件之间设置有双光滤光片。该双光滤光片已在前文图2A对应的实施例中介绍，此处不再赘述。As shown in FIG. 3A, it is another embodiment of the aforementioned imaging device. In this embodiment, both the aforementioned first image sensor and the aforementioned second image sensor are color image sensors. Optionally, the color image sensor may be a Bayer image sensor (Bayer image sensor) or a color image sensor in other formats. Since the resolution of the first image sensor is smaller than the resolution of the second image sensor, the aforementioned first image sensor is referred to as a low-resolution color image sensor, and the aforementioned second image sensor is referred to as a high-resolution color image sensor. Image Sensor. An infrared cut filter is arranged between the low-resolution color image sensor and the optical component, and a dual-optical filter is arranged between the high-resolution color image sensor and the optical component. The dual optical filter has been introduced in the previous embodiment corresponding to FIG. 2A, and will not be repeated here.

当光照强度低于预设值时，结合图3B进行介绍。When the light intensity is lower than the preset value, it will be introduced in conjunction with Figure 3B.

该低分辨率彩色图像传感器感应第一光信号中的可见光，输出低分辨率彩色图像。其中，该低分辨率彩色图像包括第一色彩信息和第一亮度信息。该第一色彩信息用于指示该低分辨率彩色图像的色彩，该第一亮度信息用于指示该低分辨率彩色图像呈现的亮度。本申请实施例不限定第一亮度信息、第一色彩信息的具体形式。当将该低分辨率彩色图像转换为YUV格式时，该第一色彩信息为U/V分量，该第一亮度信息为Y分量。The low-resolution color image sensor senses visible light in the first light signal and outputs a low-resolution color image. Wherein, the low-resolution color image includes first color information and first brightness information. The first color information is used to indicate the color of the low-resolution color image, and the first brightness information is used to indicate the brightness of the low-resolution color image. The embodiment of the present application does not limit the specific form of the first brightness information and the first color information. When the low-resolution color image is converted into the YUV format, the first color information is the U/V component, and the first brightness information is the Y component.

该高分辨率彩色图像传感器和该光学组件之间设置的双光滤光片切换为可见光截止滤光片，该高分辨率彩色图像传感器感应该第二光信号中的红外光，输出高分辨率灰度图像。其中，该高分辨率灰度图像包括第二亮度信息，该第二亮度信息用于指示高分辨率灰度图像呈现的亮度。该第二亮度信息可以采用亮度分量Y表示。应当理解的是，虽然该高分辨率彩色图像传感器可以记录色彩，但该高分辨率彩色图像传感器仅感应到红外光而无可见光，因此该第二图像仅呈现亮度而无法呈现色彩。因此，该第二图像仅有亮度分量Y而无色度分量U/V。The dual-optical filter provided between the high-resolution color image sensor and the optical component is switched to a visible light cut-off filter, and the high-resolution color image sensor senses the infrared light in the second light signal and outputs high-resolution Grayscale image. Wherein, the high-resolution gray-scale image includes second brightness information, and the second brightness information is used to indicate the brightness of the high-resolution gray-scale image. The second brightness information can be represented by a brightness component Y. It should be understood that, although the high-resolution color image sensor can record colors, the high-resolution color image sensor only senses infrared light and no visible light, so the second image only presents brightness and cannot present colors. Therefore, the second image has only the luminance component Y and no chrominance component U/V.

该摄像装置中的ISP芯片可以对前述低分辨率彩色图像和高分辨率彩色图像分别进行前述ISP处理。例如，3D降噪、去马赛克、亮度矫正以及色彩矫正等处理。可选的，该ISP芯片还可以对前述低分辨率彩色图像和高分辨率彩色图像的格式进行调整，例如，将拜耳格式调整为YUV格式等，具体此处不做限定。The ISP chip in the camera device can perform the aforementioned ISP processing on the aforementioned low-resolution color image and the aforementioned high-resolution color image respectively. For example, 3D noise reduction, demosaicing, brightness correction and color correction and other processing. Optionally, the ISP chip can also adjust the format of the aforementioned low-resolution color image and high-resolution color image, for example, adjusting the Bayer format to the YUV format, etc., which is not specifically limited here.

此外，该图像处理器采用上采样算法或超分辨率算法将前述低分辨率彩色图像调整为高分辨率彩色图像，该高分辨率的彩色图像与前述高分辨率的灰度图像具有相同的分辨率。然后，该图像处理器将前述高分辨率彩色图像和高分辨率的灰度图像融合，得到目标图像。In addition, the image processor adopts an up-sampling algorithm or a super-resolution algorithm to adjust the aforementioned low-resolution color image to a high-resolution color image, and the high-resolution color image has the same resolution as the aforementioned high-resolution grayscale image. Rate. Then, the image processor fuses the aforementioned high-resolution color image and high-resolution grayscale image to obtain the target image.

以图3B为例，将低分辨率的RGB格式图像301转换为低分辨率的YUV格式图像302，然后，再对低分辨率的YUV格式图像302进行上采样得到高分辨率的YUV格式图像303。然后，将高分辨率的YUV格式图像303和仅有Y分量的灰度图像304进行融合，得到高分辨率的YUV格式图像305(即目标图像)。Taking Figure 3B as an example, the low-resolution RGB format image 301 is converted into a low-resolution YUV format image 302, and then the low-resolution YUV format image 302 is up-sampled to obtain a high-resolution YUV format image 303 . Then, the high-resolution YUV format image 303 and the Y-component grayscale image 304 are merged to obtain a high-resolution YUV format image 305 (ie, the target image).

应当理解的是，在实际图像处理过程中，除了前述图3B中所列举的图像以外还可以出现其他格式的图像，具体此处不做限定。It should be understood that, in the actual image processing process, in addition to the images listed in FIG. 3B, images in other formats may also appear, which is not specifically limited here.

本实施例中，提出在光照强度较低时，前述高分辨率彩色图像传感器仅感应第二光信号中的红外光，生成仅有亮度的高分辨率灰度图像。将前述低分辨率彩色图像和高分辨率灰度图像进行融合，可以保留前述两幅图像的优点，提升目标图像的质量。In this embodiment, it is proposed that when the light intensity is low, the aforementioned high-resolution color image sensor only senses infrared light in the second light signal to generate a high-resolution grayscale image with only brightness. Combining the aforementioned low-resolution color image and high-resolution grayscale image can retain the advantages of the aforementioned two images and improve the quality of the target image.

当光照强度高于预设值时，结合图3C进行介绍。When the light intensity is higher than the preset value, it will be introduced in conjunction with Figure 3C.

该低分辨率彩色图像传感器感应第一光信号中的可见光，输出低分辨率彩色图像。其中，该低分辨率彩色图像包括第一色彩信息和第一亮度信息。具体地，与光照强度高于预设值的情况类似，此处不再赘述。The low-resolution color image sensor senses visible light in the first light signal and outputs a low-resolution color image. Wherein, the low-resolution color image includes first color information and first brightness information. Specifically, it is similar to the case where the light intensity is higher than the preset value, and will not be repeated here.

该高分辨率彩色图像传感器和该光学组件之间设置的双光滤光片切换为红外截止滤光片，该红外截止滤光片用于滤除该第二光信号中的红外光。因此，该高分辨率彩色图像传感器感应该第二光信号中的可见光，输出高分辨率彩色图像。此时，该高分辨率彩色图像(即前述第二图像)不仅包括第二亮度信息还包括第二色彩信息。该第二色彩信息用于指示该高分辨率彩色图像的色彩，该第二亮度信息用于指示该高分辨率彩色图像呈现的亮度。本申请实施例不限定第二亮度信息、第二色彩信息的具体形式。可选的，当将该高分辨率彩色图像转换为YUV格式时，该第二色彩信息为U/V分量，该第二亮度信息为Y分量。The dual-optical filter arranged between the high-resolution color image sensor and the optical assembly is switched to an infrared cut-off filter, and the infrared cut-off filter is used to filter out the infrared light in the second optical signal. Therefore, the high-resolution color image sensor senses the visible light in the second light signal and outputs a high-resolution color image. At this time, the high-resolution color image (that is, the aforementioned second image) includes not only the second brightness information but also the second color information. The second color information is used to indicate the color of the high-resolution color image, and the second brightness information is used to indicate the brightness of the high-resolution color image. The embodiment of the present application does not limit the specific form of the second brightness information and the second color information. Optionally, when the high-resolution color image is converted into a YUV format, the second color information is a U/V component, and the second brightness information is a Y component.

该摄像装置中的ISP芯片可以对前述低分辨率彩色图像和高分辨率彩色图像进行前述ISP处理。例如，3D降噪、去马赛克、亮度矫正以及色彩矫正等处理。可选的，该ISP芯片还可以对前述低分辨率彩色图像和高分辨率彩色图像的格式进行调整，例如，将拜耳格式调整为YUV格式等，具体此处不做限定。The ISP chip in the camera device can perform the aforementioned ISP processing on the aforementioned low-resolution color images and high-resolution color images. For example, 3D noise reduction, demosaicing, brightness correction and color correction and other processing. Optionally, the ISP chip can also adjust the format of the aforementioned low-resolution color image and high-resolution color image, for example, adjusting the Bayer format to the YUV format, etc., which is not specifically limited here.

此外，该图像处理器采用上采样算法或超分辨率算法将前述低分辨率彩色图像调整为高分辨率彩色图像，前述两幅高分辨率的彩色图像具有相同的分辨率。然后，该图像处理器将前述两幅高分辨率彩色图像融合，得到目标图像。有利于提升目标图像的动态范围。In addition, the image processor adopts an up-sampling algorithm or a super-resolution algorithm to adjust the aforementioned low-resolution color image to a high-resolution color image, and the aforementioned two high-resolution color images have the same resolution. Then, the image processor fuses the aforementioned two high-resolution color images to obtain the target image. It is helpful to improve the dynamic range of the target image.

以图3C为例，将低分辨率的RGB格式图像311转换为低分辨率的YUV格式图像312，然后，再对低分辨率的YUV格式图像312进行上采样得到高分辨率的YUV格式图像313。与此同时，将高分辨率的RGB格式图像314转换为高分辨率的YUV格式图像315。然后，再将高分辨率的YUV格式图像313和高分辨率的YUV格式图像315进行融合，得到高分辨率的YUV格式图像316(即目标图像)。该目标图像可以具备前述两幅图像的优点，在提升目标图像的质量的同时提升动态范围。虽然，前述图3B和图3C采用YUV420列举示例，但是在实际应用中，可以根据具体需求调整图像的格式，具体此处不做限定。此外，虽然前述YUV格式与RGB格式为不同的颜色编码方式，但编码格式的改变不会影响图像呈现的色彩。Taking FIG. 3C as an example, the low-resolution RGB format image 311 is converted into a low-resolution YUV format image 312, and then the low-resolution YUV format image 312 is up-sampled to obtain a high-resolution YUV format image 313 . At the same time, the high-resolution RGB format image 314 is converted into the high-resolution YUV format image 315. Then, the high-resolution YUV format image 313 and the high-resolution YUV format image 315 are merged to obtain a high-resolution YUV format image 316 (that is, the target image). The target image can have the advantages of the aforementioned two images, and the dynamic range is improved while the quality of the target image is improved. Although the foregoing FIG. 3B and FIG. 3C use YUV420 as examples, in actual applications, the image format can be adjusted according to specific requirements, which is not specifically limited here. In addition, although the aforementioned YUV format and the RGB format are different color coding methods, the change of the coding format will not affect the colors presented by the image.

可选的，本实施方式中的第一光信号中的可见光与第二光信号中的可见光的比例为4:1。也可以理解为，前述光学组件同时按频谱和能量进行分束，其中，入射光信号中80％的可见光照射向低分辨率彩色图像传感器，入射光信号中20％的可见光和100％的红外光照射向高分辨率彩色图像传感器。并且结合滤光片进一步调整实际进入前述两个图像传感器的光信号。Optionally, the ratio of visible light in the first optical signal to visible light in the second optical signal in this embodiment is 4:1. It can also be understood that the aforementioned optical components are split according to frequency spectrum and energy at the same time, where 80% of the visible light in the incident light signal is irradiated to the low-resolution color image sensor, and 20% of the visible light and 100% of the infrared light in the incident light signal are irradiated to the low-resolution color image sensor. Illuminate to the high-resolution color image sensor. And combined with the filter to further adjust the light signal actually entering the two image sensors.

本实施例中，由于，两个图像传感器均为彩色图像传感器，当光照强度大于预设值时，该高分辨率色彩图像传感器相比于前述高分辨率黑白图像传感器来说，可以输出彩色的图像。由于，低分辨率色彩图像传感器和高分辨率色彩图像传感器之间感应的可见光的能量不同，因此，输出的低分辨率彩色图像和高分辨率彩色图像的亮度也不同。将前述两幅图像融合不仅可以提升动态范围，还可以使目标图像更真实。In this embodiment, since the two image sensors are both color image sensors, when the light intensity is greater than the preset value, the high-resolution color image sensor can output color images compared to the aforementioned high-resolution black-and-white image sensor. image. Since the energy of the visible light sensed between the low-resolution color image sensor and the high-resolution color image sensor is different, the brightness of the output low-resolution color image and the high-resolution color image are also different. Fusion of the aforementioned two images can not only increase the dynamic range, but also make the target image more realistic.

本发明还提出一种图像处理方法，用于执行上述实施例中图像处理器的功能，例如：将前述第一图像的分辨率调整至与前述第二图像的分辨率相同；以及基于前述第一图像和前述第二图像生成目标图像。The present invention also provides an image processing method for performing the functions of the image processor in the foregoing embodiment, for example: adjusting the resolution of the foregoing first image to be the same as the resolution of the foregoing second image; and based on the foregoing first image The image and the aforementioned second image generate a target image.

应当理解的是，本申请实施例中，不同的附图中相同的附图标记可视为同一对象。除在有特别说明，前述各个附图之间相同的附图标记的解释可以相互引用。所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。It should be understood that, in the embodiments of the present application, the same reference numerals in different drawings may be regarded as the same object. Unless otherwise specified, the explanations of the same reference numerals between the foregoing various drawings may be mutually cited. Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。For the convenience and conciseness of the description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。The above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, it is still possible to modify the technical solutions recorded in the foregoing embodiments, or to Some of the technical features are equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

一种摄像装置，其特征在于，包括：A camera device, characterized in that it comprises:

光学组件，用于接收入射光信号，将所述入射光信号处理为第一光信号和第二光信号；An optical component for receiving an incident optical signal, and processing the incident optical signal into a first optical signal and a second optical signal;

第一图像传感器，用于感应所述第一光信号生成第一图像，所述第一图像的图像信息包括第一色彩信息和第一亮度信息；A first image sensor, configured to sense the first light signal to generate a first image, and the image information of the first image includes first color information and first brightness information;

第二图像传感器，用于感应所述第二光信号生成第二图像，所述第二图像的图像信息包括第二亮度信息，所述第一图像传感器的分辨率小于所述第二图像传感器的分辨率，所述第二图像的分辨率大于所述第一图像的分辨率；The second image sensor is used to sense the second light signal to generate a second image, the image information of the second image includes second brightness information, and the resolution of the first image sensor is smaller than that of the second image sensor. Resolution, the resolution of the second image is greater than the resolution of the first image;

图像处理器，用于基于所述第一图像和所述第二图像生成目标图像，所述目标图像的色彩和亮度由所述第一图像的图像信息和所述第二图像的图像信息确定。The image processor is configured to generate a target image based on the first image and the second image, and the color and brightness of the target image are determined by the image information of the first image and the image information of the second image.
根据权利要求1所述的摄像装置，其特征在于，所述图像处理器，具体用于：The imaging device according to claim 1, wherein the image processor is specifically configured to:

将所述第一图像的分辨率调整至与所述第二图像的分辨率相同，得到第三图像，所述第三图像携带所述第一色彩信息和所述第一亮度信息；Adjusting the resolution of the first image to be the same as the resolution of the second image to obtain a third image, the third image carrying the first color information and the first brightness information;

将所述第三图像与所述第二图像融合，得到所述目标图像。The third image and the second image are merged to obtain the target image.
根据权利要求1或2所述的摄像装置，其特征在于，所述第一光信号包括可见光，所述第二光信号包括可见光和红外光，所述第一光信号中的可见光的能量大于所述第二光信号中的可见光的能量，所述第一光信号中的可见光的频段与所述第二光信号中的可见光的频段相同。The imaging device according to claim 1 or 2, wherein the first optical signal includes visible light, the second optical signal includes visible light and infrared light, and the energy of the visible light in the first optical signal is greater than that of the visible light. For the energy of the visible light in the second optical signal, the frequency band of the visible light in the first optical signal is the same as the frequency band of the visible light in the second optical signal.
根据权利要求3所述的摄像装置，其特征在于，所述第一图像传感器为彩色图像传感器，所述第二图像传感器为黑白图像传感器。3. The imaging device of claim 3, wherein the first image sensor is a color image sensor, and the second image sensor is a black and white image sensor.
根据权利要求4所述的摄像装置，其特征在于，所述摄像装置还包括红外截止滤光片；4. The imaging device according to claim 4, wherein the imaging device further comprises an infrared cut filter;

所述摄像装置，还用于当光照强度高于预设值时启动所述红外截止滤光片，所述红外截止滤光片位于所述光学组件和所述第二图像传感器之间，所述红外截止滤光片用于滤除所述第二光信号中的红外光；The camera device is further configured to activate the infrared cut-off filter when the light intensity is higher than a preset value, the infrared cut-off filter is located between the optical assembly and the second image sensor, the The infrared cut filter is used to filter the infrared light in the second optical signal;

所述第二图像传感器，具体用于感应所述第二光信号中的可见光，生成所述第二图像；The second image sensor is specifically configured to sense visible light in the second light signal to generate the second image;

所述图像处理器，具体用于将所述第一图像的第一色彩信息与所述第二图像的第二亮度信息组合，得到所述目标图像，所述目标图像的色彩由所述第一色彩信息确定，所述目标图像的亮度由所述第二亮度信息确定。The image processor is specifically configured to combine the first color information of the first image with the second brightness information of the second image to obtain the target image, and the color of the target image is determined by the first image. The color information is determined, and the brightness of the target image is determined by the second brightness information.
根据权利要求4所述的摄像装置，其特征在于，当光照强度低于预设值时，所述第二图像传感器，具体用于感应所述第二光信号中的可见光和红外光，生成所述第二图像；The imaging device according to claim 4, wherein when the light intensity is lower than a preset value, the second image sensor is specifically configured to sense visible light and infrared light in the second light signal to generate the The second image;

所述图像处理器，具体用于将所述第三图像与所述第二图像融合，得到所述目标图像。The image processor is specifically configured to fuse the third image with the second image to obtain the target image.
根据权利要求3所述的摄像装置，其特征在于，所述第一图像传感器与所述第二图像传感器均为彩色图像传感器。4. The imaging device of claim 3, wherein the first image sensor and the second image sensor are both color image sensors.
根据权利要求7所述的摄像装置，其特征在于，所述摄像装置还包括红外截止滤光片；8. The imaging device according to claim 7, wherein the imaging device further comprises an infrared cut filter;

所述摄像装置，还用于当光照强度高于预设值时启动所述红外截止滤光片，所述红外截止滤光片位于所述光学组件和所述第二图像传感器之间，所述红外截止滤光片用于滤除所述第二光信号中的红外光；The camera device is further configured to activate the infrared cut-off filter when the light intensity is higher than a preset value, the infrared cut-off filter is located between the optical assembly and the second image sensor, the The infrared cut filter is used to filter the infrared light in the second optical signal;

所述第二图像传感器，具体用于感应所述第二光信号中的可见光，生成所述第二图像，所述第二图像的图像信息还包括第二色彩信息；The second image sensor is specifically configured to sense visible light in the second light signal to generate the second image, and the image information of the second image further includes second color information;

所述图像处理器，具体用于将所述第三图像与所述第二图像融合，得到所述目标图像。The image processor is specifically configured to fuse the third image with the second image to obtain the target image.
根据权利要求7所述的摄像装置，其特征在于，所述摄像装置还包括可见光截止滤光片；8. The imaging device of claim 7, wherein the imaging device further comprises a visible light cut-off filter;

所述摄像装置，还用于当光照强度低于预设值时启动所述可见光截止滤光片，所述可见光截止滤光片位于所述光学组件和所述第二图像传感器之间，所述可见光截止滤光片用于滤除所述第二光信号中的可见光；The imaging device is further configured to activate the visible light cut-off filter when the light intensity is lower than a preset value, the visible light cut-off filter is located between the optical assembly and the second image sensor, and the The visible light cut-off filter is used to filter out visible light in the second optical signal;

所述第二图像传感器，具体用于感应所述第二光信号中的红外光，生成所述第二图像；The second image sensor is specifically configured to sense infrared light in the second light signal to generate the second image;

所述图像处理器，具体用于将所述第三图像与所述第二图像融合，得到所述目标图像。The image processor is specifically configured to fuse the third image with the second image to obtain the target image.
根据权利要求1至9中任意一项所述的摄像装置，其特征在于，所述光学组件包括镜头与分光棱镜，所述分光棱镜位于所述镜头与图像传感器之间；The imaging device according to any one of claims 1 to 9, wherein the optical component comprises a lens and a dichroic prism, and the dichroic prism is located between the lens and the image sensor;

所述镜头，用于接收所述入射光信号；The lens is used to receive the incident light signal;

所述分光棱镜，用于将所述镜头接收的入射光信号分为所述第一光信号和所述第二光信号。The dichroic prism is used to divide the incident light signal received by the lens into the first light signal and the second light signal.
根据权利要求10所述的摄像装置，其特征在于，所述镜头为红外共焦镜头。The imaging device according to claim 10, wherein the lens is an infrared confocal lens.
根据权利要求1至9中任意一项所述的摄像装置，其特征在于，所述摄像装置还包括红外截止滤光片；The imaging device according to any one of claims 1 to 9, wherein the imaging device further comprises an infrared cut filter;

所述光学组件包括第一镜头和第二镜头，所述第一镜头与所述第二镜头用于共同接收所述入射光信号，所述第一镜头的焦距与所述第二镜头的焦距相同，所述第一镜头的光圈大于所述第二镜头的光圈，所述第一镜头与所述第一图像传感器之间设有所述红外截止滤光片，所述第二镜头为红外共焦镜头；The optical assembly includes a first lens and a second lens, the first lens and the second lens are used to jointly receive the incident light signal, and the focal length of the first lens is the same as the focal length of the second lens , The aperture of the first lens is larger than the aperture of the second lens, the infrared cut filter is provided between the first lens and the first image sensor, and the second lens is infrared confocal Lens

所述第一镜头，用于接收所述入射光信号中的一部分，并将收到的光信号传输至所述红外截止滤光片；The first lens is used to receive a part of the incident light signal and transmit the received light signal to the infrared cut filter;

所述红外截止滤光片，用于滤除来自所述第一镜头的光信号中的红外光，得到所述第一光信号，并将所述第一光信号传输至所述第一图像传感器；The infrared cut filter is used to filter infrared light in the optical signal from the first lens to obtain the first optical signal, and transmit the first optical signal to the first image sensor ；

所述第二镜头，用于接收所述入射光信号中的余下部分，并将收到的光信号作为第二光信号传输至所述第二图像传感器。The second lens is used to receive the remaining part of the incident light signal, and transmit the received light signal as a second light signal to the second image sensor.
根据权利要求1或2所述的摄像装置，其特征在于，The imaging device according to claim 1 or 2, wherein:

所述第一图像的格式是YUV格式；The format of the first image is YUV format;

所述第二图像的格式是YUV格式。The format of the second image is YUV format.