WO2023098284A1

WO2023098284A1 - Image sensor, camera module, electronic device, and image generation method and apparatus

Info

Publication number: WO2023098284A1
Application number: PCT/CN2022/124077
Authority: WO
Inventors: 李小涛
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-12-01
Filing date: 2022-10-09
Publication date: 2023-06-08
Also published as: CN114125242A

Abstract

An image sensor 21, a camera module 20, an electronic device 100, an image generation method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The image sensor 21 comprises a filter array 23 and a pixel array 24; the filter array 23 comprises a minimum repetitive unit 230; the minimum repetitive unit 230 comprises a plurality of filter groups, and each filter group only comprises color filters 234 and panchromatic filters 233 of two colors; the amount of light transmitted by the panchromatic filters 233 is greater than the amount of light transmitted by the color filters 234; the panchromatic filters 233 and the color filters 234 are alternately arranged in each row and each column; each panchromatic filter 233 and each color filter 234 both comprise N rows and N columns of panchromatic sub-filters of the same color, wherein N is a positive integer; each pixel in the pixel array 24 corresponds to a sub-filter of the filter array 23, and the pixel array 24 is configured to receive light passing through the filter array 23 to generate an electrical signal.

Description

图像传感器、摄像模组、电子设备、图像生成方法和装置Image sensor, camera module, electronic equipment, image generation method and device

相关申请的交叉引用Cross References to Related Applications

本申请要求于2021年12月01日提交中国专利局、申请号为202111458638.6、发明名称为“图像传感器、摄像模组、电子设备、图像生成方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202111458638.6 and the title of the invention "image sensor, camera module, electronic equipment, image generation method and device" submitted to the China Patent Office on December 01, 2021, all of which The contents are incorporated by reference in this application.

技术领域technical field

本申请涉及计算机技术领域，特别是涉及一种图像传感器、摄像模组、电子设备、图像生成方法、装置、电子设备、计算机可读存储介质和计算机程序产品。The present application relates to the field of computer technology, in particular to an image sensor, camera module, electronic equipment, image generation method, device, electronic equipment, computer readable storage medium and computer program product.

背景技术Background technique

随着计算机技术的发展，手机等电子设备中大部分都配置有摄像头，以通过摄像头实现拍照功能。摄像头中设置有图像传感器，通过图像传感器采集彩色图像。为了实现彩色图像的采集，图像传感器中通常会设置以拜耳(Bayer)阵列形式排布的滤光片阵列，以使得图像传感器中的多个像素能够接收穿过对应的滤光片的光线，从而生成具有不同色彩通道的像素信号，进而生成图像。With the development of computer technology, most of electronic devices such as mobile phones are equipped with cameras, so as to realize the function of taking pictures through the cameras. An image sensor is arranged in the camera, and a color image is collected by the image sensor. In order to realize the collection of color images, an optical filter array arranged in the form of a Bayer (Bayer) array is usually arranged in the image sensor, so that multiple pixels in the image sensor can receive light passing through the corresponding optical filter, thereby Generate pixel signals with different color channels to generate an image.

然而，传统的图像传感器所生成的图像清晰度较低。However, conventional image sensors produce low-resolution images.

发明内容Contents of the invention

根据本申请的各种实施例提供一种图像传感器、摄像模组、电子设备、图像生成方法、装置、电子设备、计算机可读存储介质和计算机程序产品。Various embodiments according to the present application provide an image sensor, a camera module, an electronic device, an image generating method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

一种图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。An image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each filter group only It includes two colors of color filters and panchromatic filters, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; the panchromatic filter and the The color filters are alternately arranged on each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, and each of the color filters The light sheet includes color sub-filters in N rows and N columns, the color sub-filters in N rows and N columns are the same color as the color filter, and N is a positive integer; each of the pixel arrays Pixels are arranged corresponding to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signals.

一种摄像模组，所述摄像模组包括镜头和上述的图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号。A camera module, the camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.

一种电子设备，包括：An electronic device comprising:

上述的摄像模组；及the aforementioned camera module; and

壳体，所述摄像模组设置在所述壳体上。A casing, the camera module is arranged on the casing.

上述图像传感器、摄像模组和电子设备，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，全色滤光片透过的进光量大于彩色滤光片透过的进光量，可在拍摄时通过全色滤光片获得更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。The above-mentioned image sensor, camera module and electronic equipment, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, and only Including 2 colors of color filter and panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than that of the color filter, and more can be obtained through the panchromatic filter when shooting There is no need to adjust the shooting parameters, and the sharpness of imaging in low light is improved without affecting the stability of shooting. When imaging in dark light, both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.

并且，全色滤光片与彩色滤光片在每一行和每一列上交替排布，每个全色滤光片包括N行N列个全色子滤光片，每个彩色滤光片包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为正整数；而像素阵列中各个像素与滤光片阵列的子滤光片对应设置，也即像素阵列中每一行和每一列上均包括每种颜色的彩色像素，可以提高成像的每一行每一列的彩色分辨率，使得成像的颜色更加丰富。And, the panchromatic filter and the color filter are alternately arranged on each row and each column, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer; and each pixel in the pixel array and the sub-filter of the filter array Corresponding setting of slices, that is, each row and each column in the pixel array includes color pixels of each color, which can improve the color resolution of each row and column of imaging, and make the color of imaging richer.

一种图像生成方法，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；An image generation method, applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each The filter group only includes color filters of two colors and a panchromatic filter, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; The color filters and the color filters are alternately arranged in each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, Each of the color filters includes color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns are the same color as the color filter, and the N is greater than or equal to A positive integer of 2; each pixel in the pixel array is set corresponding to the sub-filter of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical Signal;

所述方法包括：The methods include:

在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；In the full-resolution mode, the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full-resolution panchromatic pixel value, and each of the color filters is The color pixel corresponding to the color sub-filter reads out the full-resolution color pixel value;

基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。A full resolution target image is generated based on each of said full resolution panchromatic pixel values and each of said full resolution color pixel values.

一种图像生成装置，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；An image generating device, applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each The filter group only includes color filters of two colors and a panchromatic filter, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; The color filters and the color filters are alternately arranged in each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, Each of the color filters includes color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns are the same color as the color filter, and the N is greater than or equal to A positive integer of 2; each pixel in the pixel array is set corresponding to the sub-filter of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical Signal;

所述装置包括：The devices include:

读出模块，用于在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素；The readout module is used to read out the full-resolution panchromatic pixels from the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read the full-resolution panchromatic pixels of the color filter The color pixels corresponding to each color sub-filter in the light sheet read out the full resolution color pixels;

图像生成模块，用于基于各个所述全分辨率全色像素和各个所述全分辨率彩色像素，生成全分辨率目标图像。An image generating module, configured to generate a full-resolution target image based on each of the full-resolution panchromatic pixels and each of the full-resolution color pixels.

一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如上述的图像生成方法的操作。An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute the operation of the above-mentioned image generation method.

一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如上述的方法的操作。A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the operation of the above-mentioned method is realized.

一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如上述的方法的操作。A computer program product includes a computer program, and when the computer program is executed by a processor, the operations of the above-mentioned method are realized.

上述图像生成方法、装置、电子设备、计算机可读存储介质和计算机程序产品，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素，而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素和各个全分辨率彩色像素，能够生成信息更多、细节解析更清晰的全分辨率目标图像。In the image generation method, device, electronic equipment, computer readable storage medium and computer program product described above, in the full resolution mode, the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter are read out in full Resolution panchromatic pixels, and read out the color pixels corresponding to each color sub-filter in the color filter as full-resolution color pixels, and the amount of light transmitted by the panchromatic filter is greater than that transmitted by the color filter The amount of incoming light can integrate panchromatic channel information into the image to increase the overall light incoming amount, so that based on each full-resolution panchromatic pixel and each full-resolution color pixel, it can generate full-color images with more information and clearer detail analysis. Resolution target image.

并且，全色滤光片与彩色滤光片在每一行和每一列上交替排布，每个全色滤光片包括N行N列个全色子滤光片，每个彩色滤光片包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为正整数；而像素阵列中各个像素与滤光片阵列的子滤光片对应设置，也即像素阵列中每一行和每一列上均包括每种颜色的彩色像素，可以提高生成的第一目标图像的每一行每一列的彩色分辨率，使得第一目标图像的颜色更加丰富。And, the panchromatic filter and the color filter are alternately arranged on each row and each column, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer; and each pixel in the pixel array and the sub-filter of the filter array Slice corresponding setting, that is, each row and each column in the pixel array includes color pixels of each color, which can improve the color resolution of each row and each column of the generated first target image, so that the color of the first target image is more accurate Rich.

附图说明Description of drawings

为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

图1为一个实施例中电子设备的结构示意图。Fig. 1 is a schematic structural diagram of an electronic device in an embodiment.

图2为一个实施例中图像传感器的分解示意图。FIG. 2 is an exploded schematic diagram of an image sensor in one embodiment.

图3为一个实施例中像素阵列和读出电路的连接示意图。Fig. 3 is a schematic diagram of connection between a pixel array and a readout circuit in one embodiment.

图4为一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。FIG. 4 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in an embodiment.

图5为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。FIG. 5 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.

图6为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。FIG. 6 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.

图7为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。FIG. 7 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.

图8为一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。FIG. 8 is a schematic diagram of an arrangement of the smallest repeating unit in an optical filter array in which N is 2 in an embodiment.

图9为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。FIG. 9 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.

图10为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。FIG. 10 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.

图11为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。FIG. 11 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.

图12为一个实施例中图像生成方法的流程示意图。Fig. 12 is a schematic flowchart of an image generation method in an embodiment.

图13为一个实施例中第一目标图像的示意图。Fig. 13 is a schematic diagram of a first target image in an embodiment.

图14为一个实施例中生成第二目标图像的流程示意图。Fig. 14 is a schematic flowchart of generating a second target image in an embodiment.

图15为一个实施例中第一彩色图像和第一全色图像的示意图。Figure 15 is a schematic diagram of a first color image and a first panchromatic image in one embodiment.

图16为一个实施例中第二目标图像的示意图。Fig. 16 is a schematic diagram of a second target image in one embodiment.

图17为另一个实施例中第二目标图像的示意图。Fig. 17 is a schematic diagram of a second target image in another embodiment.

图18为另一个实施例中第二目标图像的示意图。Fig. 18 is a schematic diagram of a second target image in another embodiment.

图19为另一个实施例中第二目标图像的示意图。Fig. 19 is a schematic diagram of a second target image in another embodiment.

图20为一个实施例中生成第三目标图像的流程示意图。Fig. 20 is a schematic flowchart of generating a third target image in an embodiment.

图21为一个实施例中第一彩色图像和第一全色图像中各像素进行合并读出的示意图。Fig. 21 is a schematic diagram of combined readout of pixels in the first color image and the first panchromatic image in an embodiment.

图22为一个实施例中对角线的第二全色图像、反对角线的第二全色图像、对角线的第二彩色图像和反对角线的第二彩色图像的示意图。Fig. 22 is a schematic diagram of a second full-color image of diagonal lines, a second full-color image of anti-diagonal lines, a second color image of diagonal lines, and a second color image of anti-diagonal lines in an embodiment.

图23为一个实施例中第三目标图像的示意图。Fig. 23 is a schematic diagram of a third target image in one embodiment.

图24为一个实施例中图像生成装置的结构框图。Fig. 24 is a structural block diagram of an image generating device in an embodiment.

图25为一个实施例中电子设备的内部结构示意图。Fig. 25 is a schematic diagram of the internal structure of an electronic device in one embodiment.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

可以理解，本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件，但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说，在不脱离本申请的范围的情况下，可以将第一目标图像称为第二目标图像，且类似地，可将第二目标图像称为第一目标图像。第一目标图像和第二目标图像两者都是目标图像，但其不是同一目标图像。It can be understood that the terms "first", "second" and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first object image could be termed a second object image, and, similarly, a second object image could be termed a first object image, without departing from the scope of the present application. Both the first object image and the second object image are object images, but they are not the same object image.

在一个实施例中，电子设备100包括手机、平板电脑、笔记本电脑、柜员机、闸机、智能手表、头显设备等，可以理解，电子设备100还可以是其他任意图像处理功能的装置。电子设备100包括摄像模组20、处理器30和壳体40。摄像模组20和处理器30均设置在壳体40内，壳体40还可用于安装电子设备100的供电装置、通信装置等功能模块，以使壳体40为功能模块提供防尘、防摔、防水等保护。In one embodiment, the electronic device 100 includes a mobile phone, a tablet computer, a notebook computer, an ATM, a gate machine, a smart watch, a head-mounted display device, etc. It can be understood that the electronic device 100 can also be any other device with image processing functions. The electronic device 100 includes a camera module 20 , a processor 30 and a casing 40 . The camera module 20 and the processor 30 are both arranged in the housing 40, and the housing 40 can also be used to install functional modules such as power supply devices and communication devices of the electronic device 100, so that the housing 40 provides dustproof and drop-proof protection for the functional modules. , waterproof and other protection.

摄像模组20可以是前置摄像模组、后置摄像模组、侧置摄像模组、屏下摄像模组等，在此不做限制。摄像模组20包括镜头及图像传感器21，摄像模组20在拍摄图像时，光线穿过镜头并到达图像传感器21，图像传感器21用于将照射到图像传感器21上的光信号转化为电信号。The camera module 20 may be a front camera module, a rear camera module, a side camera module, an under-screen camera module, etc., which is not limited here. The camera module 20 includes a lens and an image sensor 21. When the camera module 20 captures an image, light passes through the lens and reaches the image sensor 21. The image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.

在一个实施例中，如图2所示，图像传感器21包括微透镜阵列22、滤光片阵列23、像素阵列24。In one embodiment, as shown in FIG. 2 , the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .

微透镜阵列22包括多个微透镜221，微透镜221、滤光片阵列23中的子滤光片和像素阵列24中的像素一一对应设置，微透镜221用于将入射的光线进行聚集，聚集之后的光线会穿过对应的子滤光片，然后投射至像素上，被对应的像素接收，像素再将接收的光线转化成电信号。The microlens array 22 includes a plurality of microlenses 221, the microlenses 221, the sub-filters in the filter array 23, and the pixels in the pixel array 24 are set in one-to-one correspondence, and the microlenses 221 are used to gather the incident light. The collected light will pass through the corresponding sub-filter, and then projected onto the pixel, and be received by the corresponding pixel, and the pixel converts the received light into an electrical signal.

在一个实施例中，滤光片阵列23包括多个最小重复单元230。最小重复单元230包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量。全色滤光片233与彩色滤光片234在每一行和每一列上交替排布。每个全色滤光片233包括N行N列个全色子滤光片，每个彩色滤光片234包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为正整数。In one embodiment, the filter array 23 includes a plurality of minimal repeating units 230 . The minimum repeating unit 230 includes a plurality of filter groups, and each filter group only includes two colors of color filters 234 and panchromatic filters 233, and the amount of light transmitted by the panchromatic filters 233 is greater than The amount of light transmitted by the color filter 234 . Panchromatic filters 233 and color filters 234 are alternately arranged in each row and each column. Each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, each color filter 234 includes N rows and N columns of color sub-filters, and N rows and N columns of color sub-filters It is the same as the color of the color filter, and N is a positive integer.

进一步地，最小重复单元230的每一行和每一列上均包括每种颜色的彩色滤光片234，即将各个颜色的彩色滤光片分散排列，可以提高了彩色分辨能力和亮度变化分辨能力，各个颜色的彩色滤光片混合排列，也降低了伪色的风险。可以理解的是，全色滤光片233与彩色滤光片234在每一行和每一列上交替排布，也即全色滤光片233在最小重复单元、第一滤光片组或第二滤光片组中占50％的数量，可以提高了成像中每个局部区域的进光量。Further, each row and each column of the minimum repeating unit 230 includes a color filter 234 of each color, that is, the color filters of each color are dispersed and arranged, which can improve the color resolution and brightness change resolution. The color filter mix arrangement of colors also reduces the risk of false colors. It can be understood that the panchromatic filter 233 and the color filter 234 are alternately arranged in each row and each column, that is, the panchromatic filter 233 is arranged in the smallest repeating unit, the first filter group or the second filter group. The number of 50% in the filter group can increase the amount of light entering each local area in the imaging.

在另一个实施例中，滤光片阵列23包括最小重复单元230，最小重复单元230包括彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量；全色滤光片233与彩色滤光片234在最小重复单元230的每一行和每一列上交替排布，每一行和每一列上均包括每种颜色的彩色滤光片234；每个全色滤光片233包括N行N列个全色子滤光片，每个彩色滤光片234包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片234的颜色相同，N为正整数；像素阵列24中各个像素与滤光片阵列23的子滤光片对应设置，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。In another embodiment, the filter array 23 includes a minimum repeating unit 230, the minimum repeating unit 230 includes a color filter 234 and a panchromatic filter 233, and the amount of light transmitted by the panchromatic filter 233 is greater than that of the color filter. The amount of light that the light sheet 234 passes through; the panchromatic filter 233 and the color filter 234 are alternately arranged on each row and each column of the minimum repeating unit 230, and each row and each column includes the color of each color Filter 234; each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each color filter 234 includes N rows and N columns of color sub-filters, N rows and N columns The color sub-filter is the same as the color filter 234, and N is a positive integer; each pixel in the pixel array 24 is set correspondingly to the sub-filter of the filter array 23, and the pixel array 24 is configured to receive Light from the light sheet array 23 is filtered to generate electrical signals.

需要说明的是，以下的各个特征或者各个实施例均可与上述的两个实施例中的任意一个进行结合。It should be noted that each of the following features or embodiments can be combined with any one of the above two embodiments.

在图2中，最小重复单元230包括2个第一滤光片组231和2个第二滤光片组232，2个第一滤光片组231在最小重复单元230的对角线上排布，2个第二滤光片组232在最小重复单元230的反对角线上排布。2个第一滤光片组231和2个第二滤光片组232呈矩阵排布。In Fig. 2, the minimum repeating unit 230 includes 2

first filter groups

231 and 2 second filter groups 232, and the 2 first filter groups 231 are arranged on the diagonal of the minimum repeating unit 230 The two second filter groups 232 are arranged on the anti-diagonal lines of the smallest repeating unit 230 . Two first filter sets 231 and two second filter sets 232 are arranged in matrix.

其中，对角线可以是左上角和右下角的连线，也可以是右上角和左下角的连线。对角线和反对角线相互垂直。也就是说，若对角线是左上角和右下角的连线，则反对角线是右上角和左下角的连线；若对角线是右上角和左下角的连线，则反对角线是左上角和右下角的连线。Wherein, the diagonal line may be a connecting line between the upper left corner and the lower right corner, or may be a connecting line between the upper right corner and the lower left corner. Diagonals and anti-diagonals are perpendicular to each other. That is to say, if the diagonal line is the line connecting the upper left corner and the lower right corner, then the anti-diagonal line is the line connecting the upper right corner and the lower left corner; if the diagonal line is the line connecting the upper right corner and the lower left corner, then the anti-diagonal line is is the line connecting the upper left and lower right corners.

每个滤光片组的彩色滤光片234排布在对应滤光片组的对角线及平行于对角线的方向上。对应滤光片组即彩色滤光片234所在的滤光片组。The color filters 234 of each filter set are arranged on the diagonal of the corresponding filter set and in a direction parallel to the diagonal. The corresponding filter group is the filter group where the color filter 234 is located.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片234和全色滤光片233，子单元中的彩色滤光片234排布在子单元的对角线，子单元中的全色滤光片233排布在子单元的反对角线。In one embodiment, each filter set includes a plurality of subunits, each subunit includes a color filter 234 and a panchromatic filter 233, and the color filters 234 in the subunits are arranged on opposite sides of the subunits. Diagonal line, the panchromatic filter 233 in the sub-unit is arranged on the anti-diagonal line of the sub-unit.

彩色滤光片234包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片。最小重复单元230至少包括第一滤光片组231和第二滤光片组232，第一滤光片组231包括第一颜色滤光片和第二颜色滤光片，第二滤光片组232包括第二颜色滤光片和第三颜色滤光片。The color filter 234 includes a first color filter, a second color filter and a third color filter. The minimum repeating unit 230 includes at least a first filter group 231 and a second filter group 232, the first filter group 231 includes a first color filter and a second color filter, and the second filter group 232 includes a second color filter and a third color filter.

第一颜色滤光片、第二颜色滤光片和第三颜色滤光片是三种不同彩色的滤光片。第一颜色滤光片、第二颜色滤光片和第三颜色滤光片的颜色均可以根据需要进行设置。例如，第一颜色滤光片可以为红色滤光片，第二颜色滤光片可以为绿色滤光片，第三颜色滤光片可以为蓝色滤光片。The first color filter, the second color filter and the third color filter are filters of three different colors. The colors of the first color filter, the second color filter and the third color filter can all be set as required. For example, the first color filter may be a red filter, the second color filter may be a green filter, and the third color filter may be a blue filter.

进一步地，第一滤光片组231中第二颜色滤光片设置在第一滤光片组231的对角线上；第二滤光片组232中第二颜色滤光片设置在第二滤光片组232的对角线上。那么，第一滤光片组231中第一颜色滤光片设置在第一滤光片组231中平行于该第一滤光片组的对角线的方向上；第二滤光片组232中第三颜色滤光片设置在第二滤光片组232中平行于该第二滤光片组的对角线上的方向上。Further, the second color filter in the first filter group 231 is arranged on the diagonal of the first filter group 231; the second color filter in the second filter group 232 is arranged on the second On the diagonal of the filter group 232. Then, the first color filter in the first filter group 231 is arranged on the direction parallel to the diagonal of the first filter group in the first filter group 231; the second filter group 232 The third color filter is arranged in the direction parallel to the diagonal of the second filter group 232 in the second filter group.

该滤光片阵列，可以使得在对角方向也即斜45度的第一颜色滤光片和第三颜色滤光片分布更加均衡。例如，设想一些列方向的彩色变化条纹，每条彩色线占一个像素宽度，由于这个图像传感器中滤光片排列在每一列上面都可以得到对应的彩色像素存在，那么就可以估计出这个列方向上所有像素的彩色像素值。The filter array can make the distribution of the first color filter and the third color filter more balanced in the diagonal direction, that is, obliquely at 45 degrees. For example, imagine some color changing stripes in the column direction, each color line occupies a pixel width, since the filter in this image sensor is arranged on each column, the corresponding color pixels can be obtained, then the column direction can be estimated Colored pixel values for all pixels above.

彩色滤光片234的透过的光线的波段的宽度小于全色滤光片233透过的光线的波段的宽度，例如，彩色滤光片234的透过的光线的波段可对应红光的波段、绿光的波段、或蓝光的波段，全色滤光片透过的光线的波段为所有可见光的波段，也即是说，彩色滤光片234仅允许特定颜色光线透光，而全色滤光片233可通过所有颜色的光线。当然，彩色滤光片234的透过的光线的波段还可对应其他色光的波段，如品红色光、紫色光、青色光、黄色光等，在此不作限制。The width of the wavelength band of the light transmitted by the color filter 234 is smaller than the width of the wavelength band of the light transmitted by the panchromatic filter 233, for example, the wavelength band of the transmitted light of the color filter 234 can correspond to the wavelength band of red light , the wavelength band of green light, or the wavelength band of blue light, the wavelength band of the light transmitted by the panchromatic filter is the wavelength band of all visible light, that is to say, the color filter 234 only allows light of a specific color to transmit light, while the panchromatic filter The light sheet 233 can pass light of all colors. Of course, the wavelength band of the light transmitted by the color filter 234 may also correspond to the wavelength band of other colored light, such as magenta light, purple light, cyan light, yellow light, etc., which is not limited here.

像素阵列24包括多个像素，像素阵列24的像素与滤光片阵列23的子滤光片对应设置。像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。The pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23 . The pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals.

其中，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号，是指像素阵列24用于对穿过滤光片阵列23的被摄对象的给定集合的场景的光线进行光电转换，以生成电信号。被摄对象的给定集合的场景的光线用于生成图像数据。例如，被摄对象是建筑物，被摄对象的给定集合的场景是指该建筑物所处的场景，该场景中还可以包含其他对象。Wherein, the pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to detect a scene of a given set of subjects passing through the filter array 23 The light is photoelectrically converted to generate an electrical signal. The light rays of the scene for a given set of subjects are used to generate image data. For example, the subject is a building, and the scene of a given set of subjects refers to the scene where the building is located, which may also contain other objects.

在一个实施例中，像素阵列24包括多个最小重复单元240，最小重复单元240还包括多个全色像素241和多个不同颜色的彩色像素242，全色像素241与彩色像素242在每一行和每一列上交替排布，每一行和每一列上均包括每种颜色的彩色像素；每个全色像素242对应全色滤光片233中的一个子滤光片，全色像素242接收穿过对应的子滤光片的光线以生成电信号。每个彩色像素242对应彩色滤光片234的一个子滤光片，彩色像素242接收穿过对应的子滤光片的光线以生成电信号。In one embodiment, the pixel array 24 includes a plurality of minimum repeating units 240, and the minimum repeating unit 240 also includes a plurality of panchromatic pixels 241 and a plurality of color pixels 242 of different colors, and the panchromatic pixels 241 and the color pixels 242 are arranged in each row. Alternately arranged on each column, each row and each column include color pixels of each color; each panchromatic pixel 242 is corresponding to a sub-filter in the panchromatic filter 233, and the panchromatic pixel 242 receives the pass through Light that passes through the corresponding sub-filters to generate electrical signals. Each color pixel 242 corresponds to a sub-filter of the color filter 234, and the color pixel 242 receives light passing through the corresponding sub-filter to generate an electrical signal.

如图3所示，读出电路25与像素阵列24电连接，用于控制像素阵列24的曝光以及像素的像素值的读取和输出。读出电路25包括垂直驱动单元251、控制单元252、列处理单元253和水平驱动单元254。垂直驱动单元251包括移位寄存器和地址译码器。垂直驱动单元251包括读出扫描和复位扫描功能。控制单元252根据操作模式配置时序信号，利用多种时序信号来控制垂直驱动单元251、列处理单元253和水平驱动单元254协同工作。列处理单元253可以具有用于将模拟像素信号转换为数字格式的模数(A/D)转换功能。水平驱动单元254包括移位寄存器和地址译码器。水平驱动单元254顺序逐列扫描像素阵列24。As shown in FIG. 3 , the readout circuit 25 is electrically connected to the pixel array 24 for controlling the exposure of the pixel array 24 and reading and outputting the pixel values of the pixels. The readout circuit 25 includes a vertical drive unit 251 , a control unit 252 , a column processing unit 253 , and a horizontal drive unit 254 . The vertical driving unit 251 includes a shift register and an address decoder. The vertical driving unit 251 includes readout scanning and reset scanning functions. The control unit 252 configures timing signals according to the operation mode, and uses various timing signals to control the vertical driving unit 251 , the column processing unit 253 and the horizontal driving unit 254 to work together. The column processing unit 253 may have an analog-to-digital (A/D) conversion function for converting an analog pixel signal into a digital format. The horizontal driving unit 254 includes a shift register and an address decoder. The horizontal driving unit 254 sequentially scans the pixel array 24 column by column.

上述图像传感器，图像传感器包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，可在拍摄时通过全色滤光片233获得更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。The above-mentioned image sensor, the image sensor includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, and each filter group only includes 2 colors The color filter 234 and the panchromatic filter 233, the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234, and more can be obtained by the panchromatic filter 233 when shooting. There is more light, so there is no need to adjust shooting parameters, and the clarity of imaging in low light is improved without affecting the stability of shooting. When imaging in dark light, both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.

并且，全色滤光片233与彩色滤光片234在每一行和每一列上交替排布，也即滤光片阵列中包括50％的全色滤光片，可以提高进光量，也可以提高行方向纹理和列方向纹理的彩色分辨能力。Moreover, the panchromatic filters 233 and the color filters 234 are arranged alternately in each row and each column, that is, the filter array includes 50% of the panchromatic filters, which can increase the amount of incoming light, and can also improve Color resolution capability for row-oriented textures and column-oriented textures.

每个全色滤光片233包括N行N列个全色子滤光片，每个彩色滤光片234包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为正整数；而像素阵列24中各个像素与滤光片阵列23的子滤光片对应设置，也即像素阵列24中每一行和每一列上均包括每种颜色的彩色像素242，可以提高成像的每一行每一列的彩色分辨率，使得成像的颜色更加丰富。Each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, each color filter 234 includes N rows and N columns of color sub-filters, and N rows and N columns of color sub-filters The same as the color of the color filter, N is a positive integer; and each pixel in the pixel array 24 corresponds to the sub-filter of the filter array 23, that is, each row and each column in the pixel array 24 includes each The color pixels 242 of different colors can improve the color resolution of each row and each column of the imaging, making the imaging colors richer.

当N为1时，每个全色滤光片233包括1行1列个全色子滤光片，每个彩色滤光片234包括1行1列个彩色子滤光片，也就是说，每个全色子滤光片为全色滤光片233，每个彩色子滤光片为彩色滤光片。When N is 1, each panchromatic filter 233 includes 1 row and 1 column of panchromatic sub-filters, and each color filter 234 includes 1 row and 1 column of color sub-filters, that is to say, Each panchromatic sub-filter is a panchromatic filter 233, and each color sub-filter is a color filter.

在一个实施例中，如图4所示，当N为1，最小重复单元包括8行8列64个滤光片，排布方式为：In one embodiment, as shown in FIG. 4, when N is 1, the minimum repeating unit includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:

其中，w表示全色滤光片，a、b和c均表示彩色滤光片。Among them, w represents a panchromatic filter, and a, b and c all represent color filters.

在另一个实施例中，如图5所示，当N为1，最小重复单元包括8行8列64个滤光片，排布方式为：In another embodiment, as shown in FIG. 5, when N is 1, the minimum repeating unit includes 8 rows and 8 columns of 64 filters, and the arrangement is as follows:

在另一个实施例中，如图6所示，当N为1，最小重复单元包括8行8列64个滤光片，排布方式为：In another embodiment, as shown in FIG. 6, when N is 1, the minimum repeating unit includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:

在另一个实施例中，如图7所示，当N为1，最小重复单元包括8行8列64个滤光片，排布方式为：In another embodiment, as shown in FIG. 7, when N is 1, the minimum repeating unit includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:

其中，w可以是白色滤光片，a为红色滤光片，b为绿色滤光片，c为蓝色滤光片，或者例如a为品红色滤光片，b为青色滤光片，c为黄色滤光片等，在此不做限制。where w can be a white filter, a a red filter, b a green filter, c a blue filter, or for example a a magenta filter, b a cyan filter, c It is a yellow filter or the like, and is not limited here.

在一个实施例中，如图8所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：In one embodiment, as shown in FIG. 8, N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:

其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。Among them, w represents a panchromatic sub-filter, and a, b, and c all represent color sub-filters.

在一个实施例中，如图9所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：In one embodiment, as shown in FIG. 9, N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:

在一个实施例中，如图10所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：In one embodiment, as shown in FIG. 10, N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:

在一个实施例中，如图11所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：In one embodiment, as shown in FIG. 11, N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:

需要说明的是，N还可以为3、4或5等其他正整数，排布方式与N为1或2类似，在此不做赘述。It should be noted that N can also be other positive integers such as 3, 4, or 5, and the arrangement method is similar to that of N being 1 or 2, which will not be repeated here.

其中，w可以是白色子滤光片，a为红色子滤光片，b为绿色子滤光片，c为蓝色子滤光片，或者例如a为品红色子滤光片，b为青色子滤光片，c为黄色子滤光片等，在此不做限制。where w can be a white sub-filter, a a red sub-filter, b a green sub-filter, c a blue sub-filter, or for example a a magenta sub-filter and b a cyan Sub-filters, c is a yellow sub-filter, etc., which is not limited here.

在一个实施例中，还提供了一种摄像模组，摄像模组包括镜头和上述的图像传感器；图像传感器用于接收穿过镜头的光线，像素根据光线生成电信号。In one embodiment, a camera module is also provided. The camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.

在一个实施例中，还提供了一种电子设备，包括上述的摄像模组；及壳体，摄像模组设置在壳体上。In one embodiment, there is also provided an electronic device, comprising the above-mentioned camera module; and a casing, on which the camera module is arranged.

在一个实施例中，一种图像生成方法，应用于图像传感器，图像传感器包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量；全色滤光片233与彩色滤光片234在每一行和每一列上交替排布；每个全色滤光片233包括N行N列个全色子滤光片，每个彩色滤光片234包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为大于或等于2的正整数；像素阵列24中各个像素与滤光片阵列23的子滤光片对应设置，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。In one embodiment, an image generation method is applied to an image sensor, the image sensor includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups , each filter group only includes color filters 234 and panchromatic filters 233 of two colors, and the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234; Panchromatic filters 233 and color filters 234 are alternately arranged in each row and column; each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each color filter 234 includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer greater than or equal to 2; each pixel in the pixel array 24 and the filter The sub-filters of the optical filter array 23 are arranged correspondingly, and the pixel array 24 is configured to receive light passing through the optical filter array 23 to generate electrical signals.

如图12所示，上述方法包括：As shown in Figure 12, the above method includes:

操作1202，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值。 Operation 1202, in the full-resolution mode, read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter, and read out the full-resolution panchromatic pixel value of each color sub-filter in the color filter The color pixel corresponding to the sub-filter reads out the full-resolution color pixel value.

全分辨率模式是将每个子滤光片读出为一个像素的模式。Full resolution mode is a mode in which each sub-filter is read out as one pixel.

彩色滤光片234具有比全色滤光片233的更窄的光谱响应，则全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，即彩色滤光片234透过的光线的波段宽度小于全色滤光片233透过的光线的波段宽度，全色滤光片233透过更多的光线，通过全色滤光片233得到相应的全色像素241具有更高的信噪比，该全色像素241包含有更多的信息，可以解析出更多的纹理细节。其中，信噪比是指正常信号与噪声信号之间的比值。像素的信噪比越高，则该像素包含的正常信号的比例越高，从该像素中解析到的信息也越多。The color filter 234 has a narrower spectral response than that of the panchromatic filter 233, so the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234, that is, the color filter 234 The wavelength band width of the transmitted light is smaller than the wavelength band width of the light transmitted by the panchromatic filter 233, and the panchromatic filter 233 transmits more light, and the corresponding panchromatic pixel 241 has With a higher signal-to-noise ratio, the panchromatic pixel 241 contains more information, and can resolve more texture details. Among them, the signal-to-noise ratio refers to the ratio between the normal signal and the noise signal. The higher the signal-to-noise ratio of a pixel, the higher the proportion of normal signals contained in the pixel, and the more information can be analyzed from the pixel.

彩色像素242可以是G(Green，绿色)像素、R(Red，红色)像素和B(Blue，蓝色)像素等，但不限于此。The color pixels 242 may be G (Green, green) pixels, R (Red, red) pixels, B (Blue, blue) pixels, etc., but are not limited thereto.

在接收到拍摄指令的情况下，检测用户是否选择所需使用的分辨率模式，当检测到用户选择使用全分辨率模式的情况下，或者，在用户未选择所需使用的分辨率模式，未使用预览拍摄、且当前环境非夜景模式的情况下，使用全分辨率模式响应该拍摄指令。In the case of receiving a shooting instruction, detect whether the user selects the resolution mode to be used, when it is detected that the user selects to use the full resolution mode, or, when the user does not select the resolution mode to be used, does not When using preview shooting and the current environment is not in night scene mode, use full resolution mode to respond to the shooting command.

在全分辨率模式下，全色滤光片233中的全色子滤光片透过的光线投射至对应的全色像素241上，全色像素241接收穿过全色子滤光片的光线以生成电信号。彩色滤光片234中的彩色子滤光片透过的光线投射至对应的彩色像素242上，彩色像素242穿过对应的彩色子滤光片的光线以生成电信号。In the full resolution mode, the light transmitted by the panchromatic sub-filter in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter to generate electrical signals. The light transmitted by the color sub-filter in the color filter 234 is projected onto the corresponding color pixel 242 , and the color pixel 242 generates an electrical signal through the light passing through the corresponding color sub-filter.

每个全色滤光片233包括N行N列个全色子滤光片，则每个全色滤光片233对应N行N列个全色像素241。每个彩色滤光片234包括N行N列个同个颜色的彩色子滤光片，则每个彩色滤光片234对应N行N列个彩色像素242。N为大于或等于2的正整数。Each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each panchromatic filter 233 corresponds to N rows and N columns of panchromatic pixels 241 . Each color filter 234 includes N rows and N columns of color sub-filters of the same color, and each color filter 234 corresponds to N rows and N columns of color pixels 242 . N is a positive integer greater than or equal to 2.

在其他实施例中，N也可以为1，即每个全色滤光片233对应1个全色像素241，每个彩色滤光片234对应1个彩色像素242。In other embodiments, N may also be 1, that is, each panchromatic filter 233 corresponds to one panchromatic pixel 241 , and each color filter 234 corresponds to one color pixel 242 .

操作1204，基于各个全分辨率全色像素值和各个全分辨率彩色像素值，生成全分辨率目标图像。At operation 1204, a full resolution target image is generated based on the respective full resolution panchromatic pixel values and the respective full resolution color pixel values.

电子设备可根据预设像素读取方式，从各个全分辨率全色像素值和各个全分辨率彩色像素值中读取像素值以生成全分辨率目标图像。预设像素读取方式是预先设置的像素读取方式。The electronic device can read pixel values from each full-resolution panchromatic pixel value and each full-resolution color pixel value according to a preset pixel reading method to generate a full-resolution target image. The preset pixel reading mode is a preset pixel reading mode.

上述图像生成方法，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素，而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素和各个全分辨率彩色像素，能够生成信息更多、细节解析更清晰的全分辨率目标图像。In the above image generation method, in the full resolution mode, the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full resolution panchromatic pixel, and each panchromatic pixel in the color filter is The color pixels corresponding to the color sub-filters read out full-resolution color pixels, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, which can integrate the information of the panchromatic channel into the image. The overall light input is increased, so that based on each full-resolution panchromatic pixel and each full-resolution color pixel, a full-resolution target image with more information and clearer detail analysis can be generated.

在一个实施例中，上述方法还包括：在第一分辨率模式下，将每个全色滤光片233中各个全色子滤光片对应的全色像素241合并读出第一全色像素值，以及将每个彩色滤光片234中各个彩色子滤光片对应的彩色像素242合并读出第一彩色像素值；基于各个第一全色像素值和各个第一彩色像素值，生成第一目标图像。In one embodiment, the above method further includes: in the first resolution mode, combining the panchromatic pixels 241 corresponding to the panchromatic sub-filters in each panchromatic filter 233 to read out the first panchromatic pixel value, and the color pixels 242 corresponding to each color sub-filter in each color filter 234 are combined to read out the first color pixel value; based on each first panchromatic pixel value and each first color pixel value, generate the first color pixel value a target image.

第一分辨率模式是指分辨率、功耗、信噪比和帧率均比较均衡的一级像素合并读出模式。第一分辨率模式具体可以是图像、视频拍摄的默认模式。The first resolution mode refers to a first-level pixel binning readout mode in which resolution, power consumption, signal-to-noise ratio, and frame rate are relatively balanced. Specifically, the first resolution mode may be a default mode for shooting images and videos.

在接收到拍摄指令的情况下，检测用户是否选择所需使用的分辨率模式，当检测到用户选择使用第一分辨率模式的情况下，或者，在用户未选择所需使用的分辨率模式，未使用预览拍摄、且当前环境非夜景模式的情况下，使用第一分辨率模式响应该拍摄指令。In the case of receiving a shooting instruction, detect whether the user selects the desired resolution mode, when it is detected that the user selects the first resolution mode, or, when the user does not select the required resolution mode, If preview shooting is not used and the current environment is not in the night scene mode, the first resolution mode is used to respond to the shooting instruction.

在第一分辨率模式下，全色滤光片233中的全色子滤光片透过的光线投射至对应的全色像素241上，全色像素241接收穿过全色子滤光片的光线以生成电信号。彩色滤光片234中的彩色子滤光片透过的光线投射至对应的彩色像素242上，彩色像素242穿过对应的彩色子滤光片的光线以生成电信号。In the first resolution mode, the light transmitted by the panchromatic sub-filter in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter. light to generate electrical signals. The light transmitted by the color sub-filter in the color filter 234 is projected onto the corresponding color pixel 242 , and the color pixel 242 generates an electrical signal through the light passing through the corresponding color sub-filter.

合并读出是指将多个像素的像素值求和，或者计算出多个像素的像素值的均值。Combined readout refers to summing the pixel values of multiple pixels, or calculating the average value of the pixel values of multiple pixels.

在一种实施方式中，对于每个全色滤光片233，对各个全色子滤光片对应的全色像素241求平均值，将该平均值读出为第一全色像素值。在另一种实施方式中，对于每个全色滤光片233，对各个全色子滤光片对应的全色像素241进行相加，将相加得到的和读出为第一全色像素值。在其他实施例方式中，电子设备还可以采用其他方式将全色子滤光片对应的全色像素241合并读出第一全色像素值，在此不做限定。In one embodiment, for each panchromatic filter 233, the panchromatic pixels 241 corresponding to each panchromatic sub-filter are averaged, and the average value is read out as the first panchromatic pixel value. In another embodiment, for each panchromatic filter 233, sum the panchromatic pixel 241 corresponding to each panchromatic sub-filter, and read the sum obtained as the first panchromatic pixel value. In other embodiments, the electronic device may also use other methods to combine the panchromatic pixels 241 corresponding to the panchromatic sub-filters to read out the first panchromatic pixel value, which is not limited here.

在一种实施方式中，对于每个彩色滤光片234，对各个彩色子滤光片对应的彩色像素242求平均值，将该平均值读出为第一彩色像素值。在另一种实施方式中，对于每个彩色滤光片234，对各个彩色子滤光片对应的彩色像素进行相加，将相加得到的和读出为第一彩色像素值。在其他实施例方式中，电子设备还可以采用其他方式将彩色子滤光片对应的彩色像素242合并读出第一彩色像素值，在此不做限定。In one embodiment, for each color filter 234, the color pixel 242 corresponding to each color sub-filter is averaged, and the average value is read out as the first color pixel value. In another implementation manner, for each color filter 234, the color pixels corresponding to each color sub-filter are summed, and the sum obtained by the addition is read out as the first color pixel value. In other embodiments, the electronic device may also combine the color pixels 242 corresponding to the color sub-filters in other ways to read out the first color pixel value, which is not limited here.

需要说明的是，对于各个全色滤光片233，合并读出第一全色像素值的方式可以相同，也可以不同。对于各个彩色滤光片234，合并读出第一彩色像素值的方式可以相同，也可以不同。而对于全色滤光片233和彩色滤光片234，合并读出第一全色像素值和第一彩色像素值的方式可以相同，也可以不同。It should be noted that, for each panchromatic filter 233 , the way of combining and reading out the first panchromatic pixel values may be the same or different. For each color filter 234, the way of combining and reading out the first color pixel values may be the same or different. For the panchromatic filter 233 and the color filter 234, the way of combining and reading out the first panchromatic pixel value and the first color pixel value may be the same or different.

电子设备可根据预设像素读取方式，从各个第一全色像素值和各个第一彩色像素值中读取像素值以生成第一目标图像。预设像素读取方式是预先设置的像素读取方式。以滤光片阵列23的最小重复单元为图8的排布方式为例，则生成的第一目标图像如图13所示。The electronic device can read pixel values from each first panchromatic pixel value and each first color pixel value according to a preset pixel reading method to generate a first target image. The preset pixel reading mode is a preset pixel reading mode. Taking the arrangement of the smallest repeating unit of the filter array 23 as shown in FIG. 8 as an example, the generated first target image is shown in FIG. 13 .

在本实施例中，在第一分辨率模式下，将每个全色滤光片233中各个全色子滤光片对应的全色像素241合并读出第一全色像素值，以及将每个彩色滤光片234中各个彩色子滤光片对应的彩色像素242合并读出第一彩色像素值，而全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个第一全色像素值和各个第一彩色像素值，能够生成信息更多、细节解析更清晰的第一目标图像。In this embodiment, in the first resolution mode, the panchromatic pixels 241 corresponding to each panchromatic sub-filter in each panchromatic filter 233 are combined to read out the first panchromatic pixel value, and each The color pixels 242 corresponding to each color sub-filter in each color filter 234 are combined to read the first color pixel value, and the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234 , can integrate the panchromatic channel information into the image to increase the overall light input, so that based on each first panchromatic pixel value and each first color pixel value, the first target image with more information and clearer detail analysis can be generated .

并且，全色滤光片233与彩色滤光片234在每一行和每一列上交替排布，每个全色滤光片233包括N行N列个全色子滤光片，每个彩色滤光片234包括N行N列个同个颜色的彩色子滤光片，N为正整数；而像素阵列24中各个像素与滤光片阵列23的子滤光片对应设置，也即像素阵列24中每一行和每一列上均包括每种颜色的彩色像素，可以提高生成的第一目标图像的每一行每一列的彩色分辨率，使得第一目标图像的颜色更加丰富。And, the panchromatic filter 233 and the color filter 234 are alternately arranged on each row and each column, and each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each color filter The light sheet 234 includes color sub-filters of the same color in N rows and N columns, and N is a positive integer; Each row and each column includes color pixels of each color, which can improve the color resolution of each row and each column of the generated first target image, making the color of the first target image richer.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，子单元中的彩色滤光片排布在子单元的对角线上，子单元中的全色滤光片排布在子单元的反对角线上。In one embodiment, each filter set includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filters in the subunits are arranged on the diagonal of the subunits , the panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits.

子单元包括2行2列个滤光片，子单元中的彩色滤光片排布在子单元的对角线上，子单元中的全色滤光片排布在子单元的反对角线上。也就是说，2行2列个滤光片呈矩阵排列。The sub-unit includes 2 rows and 2 columns of filters, the color filters in the sub-unit are arranged on the diagonal of the sub-unit, and the panchromatic filters in the sub-unit are arranged on the anti-diagonal of the sub-unit . That is to say, the filters in 2 rows and 2 columns are arranged in a matrix.

如图14所示，生成第一目标图像之后，还包括：As shown in Figure 14, after generating the first target image, it also includes:

操作1402，在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并基于各个第二全色像素值生成第一全色图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率。 Operation 1402, in the second resolution mode, combine the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and based on each second panchromatic pixel value value produces the first panchromatic image; the second resolution mode corresponds to a smaller resolution than the first resolution mode.

第二分辨率模式是指对分辨率要求比第一分辨率模式要求低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的二级像素合并读出模式。该第二分辨率模式对应的分辨率、功耗小于第一分辨率模式对应的分辨率、功耗。该第二分辨率模式对应的信噪比、帧率大于第一分辨率模式对应的信噪比、帧率。The second resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the first resolution mode. out mode. The resolution and power consumption corresponding to the second resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode. The signal-to-noise ratio and frame rate corresponding to the second resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.

第二分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如1080p视频预览、应用视频预览等。The second resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a scene with lower resolution requirements such as image capture and video capture under night scenes, but is not limited thereto. The preview modes of video shooting include 1080p video preview, application video preview, etc.

在接收到拍摄指令的情况下，确定该拍摄指令是否为预览拍摄。在该拍摄指令为预览拍摄的情况下，触发第二分辨率模式。或者，电子设备检测当前环境是否为夜景，在当前环境为夜景的情况下，触发第二分辨率模式。或者，在用户选择第二分辨率模式的情况下，触发第二分辨率模式对应的读出模式。When a shooting instruction is received, it is determined whether the shooting instruction is preview shooting. In the case that the shooting instruction is preview shooting, the second resolution mode is triggered. Alternatively, the electronic device detects whether the current environment is a night scene, and if the current environment is a night scene, triggers the second resolution mode. Alternatively, when the user selects the second resolution mode, the readout mode corresponding to the second resolution mode is triggered.

具体地，电子设备将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并根据预设像素读取方式，从各个第二全色像素值读取像素值以生成第一全色图像。Specifically, the electronic device combines the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and according to the preset pixel reading method, from each second panchromatic pixel value Color Pixel Values Pixel values are read to generate a first full-color image.

可以理解的是，第一目标图像中各个像素值是在第一分辨率模式下，将滤光片阵列中每个滤光片中的子对应的像素合并得到的，则第一目标图像中每个像素值对应滤光片阵列中每个滤光片，也对应每个滤光片中的多个子滤光片。并且滤光片阵列的最小重复单元包括多个滤光片组，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，而每个子单元包括的彩色滤光片和全色滤光片又对应第一目标图像中的像素值，则每个子单元可以对应第一目标图像中的多个像素值。It can be understood that each pixel value in the first target image is obtained by combining the sub-corresponding pixels in each filter in the filter array in the first resolution mode, then each pixel value in the first target image A pixel value corresponds to each filter in the filter array, and also corresponds to a plurality of sub-filters in each filter. And the smallest repeating unit of the filter array includes a plurality of filter groups, each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and each subunit includes a color filter The optical filter and the panchromatic filter correspond to pixel values in the first target image, and each subunit can correspond to multiple pixel values in the first target image.

在第二分辨率模式下，电子设备确定每个子单元于第一目标图像中的多个像素值，从多个像素值中获取多个第一全色像素合并读出第二全色像素值，以及从多个像素值中获取多个相同颜色的第一彩色像素值合并读出第二彩色像素值。In the second resolution mode, the electronic device determines a plurality of pixel values of each subunit in the first target image, acquires a plurality of first panchromatic pixels from the plurality of pixel values and reads out a second panchromatic pixel value, And acquiring a plurality of first color pixel values of the same color from the plurality of pixel values and reading out a second color pixel value.

可以理解的是，合并读出可以包括求均值、求和或加权平均等其中一种方式，在此不做限定。It can be understood that combining readout may include one of methods such as averaging, summing, or weighted averaging, which is not limited here.

操作1404，将第一目标图像的每个预设区域中各相同颜色的第一彩色像素值合并读出第二彩色像素值，并基于各个第二彩色像素值生成第一彩色图像。 Operation 1404, combining the first color pixel values of the same color in each preset area of the first target image to read out the second color pixel values, and generating a first color image based on each second color pixel value.

具体地，在第二分辨率模式下，电子设备将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，并根据预设像素读取方式，从各个第二彩色像素值读取像素值以生成第一彩色图像。Specifically, in the second resolution mode, the electronic device combines multiple first color pixel values corresponding to each subunit in the first target image to read out the second color pixel value, and calculates the second color pixel value according to the preset pixel value. In a reading mode, pixel values are read from respective second color pixel values to generate a first color image.

以图13为第一目标图像为例，则生成的第一彩色图像如图15中的1502所示，生成的第一全色图像如1504所示。Taking FIG. 13 as an example of the first target image, the generated first color image is shown as 1502 in FIG. 15 , and the generated first full-color image is shown as 1504 .

操作1406，基于第一全色图像和第一彩色图像，生成第二目标图像。 Operation 1406, based on the first panchromatic image and the first color image, generate a second target image.

具体地，电子设备将第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者将第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。Specifically, the electronic device arranges each row of second panchromatic pixel values in the first full-color image alternately with each row of second color pixel values in the first color image to generate a second target image; or the first full-color image Each column of second panchromatic pixel values in the first color image is alternately arranged with each column of second color pixel values in the first color image to generate a second target image.

图16和图17为第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相间排布得到的第二目标图像的示意图。图18和图19为第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布得到的第二目标图像。16 and 17 are schematic diagrams of the second target image obtained by alternately arranging each row of second panchromatic pixel values in the first full-color image and each row of second color pixel values in the first color image. Fig. 18 and Fig. 19 are the second target image obtained by alternately arranging each column of second panchromatic pixel values in the first panchromatic image and each column of second color pixel values in the first color image.

在另一实施例中，电子设备还可以将第一全色图像和第一彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第二目标图像。其中，合并可以采用求均值、加权求平均或相加求和等其中一种方式。In another embodiment, the electronic device may also combine pixel values at the same position in the first full-color image and the first color image to obtain combined pixel values at corresponding positions, and form the second target image based on the combined pixel values. Wherein, the merging may adopt one of methods such as averaging, weighted averaging, or adding and summing.

在其他实施例中，电子设备还可以采用其他方式生成第二目标图像，在此不做限定。In other embodiments, the electronic device may also use other methods to generate the second target image, which is not limited here.

在本实施例中，在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，以及将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，能够将各个不同的彩色像素242进行混合排列，使得所生成的第二目标图像中的各个第二彩色像素如RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第二分辨率模式下，具有全排列的全色像素，无需做插值，提高了整体的解析力。同时，在全尺寸情形下，各个颜色的彩色像素如第一颜色的像素和第三颜色的像素在对角方向或反对角方向更加分散均衡。In this embodiment, in the second resolution mode, multiple first panchromatic pixel values corresponding to each subunit in the first target image are combined to read out the second panchromatic pixel value, and the first target In the image, multiple first color pixel values of the same color corresponding to each subunit are combined to read out the second color pixel value, and each different color pixel 242 can be mixed and arranged so that the generated second target image The distribution of each second color pixel such as RGB pixel is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained second target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect. Moreover, in the second resolution mode, there are full-arranged panchromatic pixels without interpolation, which improves the overall resolution. At the same time, in the case of full size, the color pixels of each color, such as the pixels of the first color and the pixels of the third color, are more dispersed and balanced in the diagonal direction or anti-diagonal direction.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，子单元中的彩色滤光片排布在子单元的对角线，子单元中的全色滤光片排布在子单元的反对角线；上述方法还包括：在第二分辨率模式下，将每个子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第四全色像素值，并基于各个第四全色像素值生成第三全色图像；将每个子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第四彩色像素值，并基于各个第四彩色像素值生成第三彩色图像；基于第三全色图像和第三彩色图像，生成第四目标图像。In one embodiment, each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on anti-diagonal lines of the subunits; the above method also includes: in the second resolution mode, each panchromatic sub-filter of the plurality of panchromatic filters in each subunit The panchromatic pixels corresponding to the light sheet are combined to read out the fourth panchromatic pixel value, and a third panchromatic image is generated based on each fourth panchromatic pixel value; The color pixels corresponding to the sub-filters are combined to read fourth color pixel values, and a third color image is generated based on each fourth color pixel value; a fourth target image is generated based on the third panchromatic image and the third color image.

其中，合并读出的方式可以是求平均、加权平均或相加等其中一种。Wherein, the combined readout method may be one of averaging, weighted averaging, or addition.

在一个实施例中，基于第三全色图像和第三彩色图像，生成第四目标图像，包括：将第三全色图像中每一行第四全色像素值与第三彩色图像中每一行第四彩色像素值相间排布，生成第四目标图像；或者将第三全色图像中每一列第四全色像素值与第三彩色图像中每一列第四彩色像素值相间排布，生成第四目标图像。In one embodiment, based on the third panchromatic image and the third color image, generating the fourth target image includes: combining the fourth panchromatic pixel value of each row in the third panchromatic image with the fourth panchromatic pixel value of each row in the third color image The four color pixel values are arranged alternately to generate the fourth target image; or the fourth panchromatic pixel values of each column in the third panchromatic image and the fourth color pixel values of each column in the third color image are arranged alternately to generate the fourth target image. target image.

在另一实施例中，电子设备还可以将第三全色图像和第三彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第四目标图像。其中，合并读出可以采用求均值、加权求平均或相加求和等其中一种方式。In another embodiment, the electronic device may also combine pixel values at the same position in the third panchromatic image and the third color image to obtain combined pixel values at corresponding positions, and form a fourth target image based on the combined pixel values. Wherein, the combined readout may adopt one of methods such as averaging, weighted averaging, or adding and summing.

在其他实施例中，电子设备还可以采用其他方式生成第四目标图像，在此不做限定。In other embodiments, the electronic device may also use other methods to generate the fourth target image, which is not limited here.

在本实施例中，在第二分辨率模式下，将每个子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第四全色像素值，以及将每个子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第四彩色像素值，可以更快速生成第三全色图像和第三彩色图像，从而更快速生成第四目标图像。In this embodiment, in the second resolution mode, the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each subunit are combined to read out the fourth panchromatic pixel value, and Combining the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each subunit to read out the fourth color pixel value can generate the third panchromatic image and the third color image more quickly, thereby Generate the fourth target image more quickly.

并且，上述实施例能够将各个不同的彩色像素进行混合排列，使得所生成的第四目标图像中的各个第四彩色像素值如RGB像素分布更均匀，图像质量更高。并且，所得到第四目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。Moreover, the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each fourth color pixel value, such as RGB pixels, in the generated fourth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained fourth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.

在一个实施例中，如图20所示，上述方法还包括：In one embodiment, as shown in Figure 20, the above method further includes:

操作2002，在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个对角线上的第二全色像素值合并读出对角线的第三全色像素值，并基于各个对角线的第三全色像素值生成对角线的第二全色图像；以及，将第一全色图像中对应在同一滤光片组中的多个反对角线上的第二全色像素值合并读出反对角线的第三全色像素值，并基于各个反对角线的第三全色像素值生成反对角线的第二全色图像；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率。 Operation 2002, in the third resolution mode, combine the second panchromatic pixel values corresponding to multiple diagonal lines in the same filter set in the first panchromatic image to read out the third panchromatic pixel values of the diagonal lines color pixel values, and generate a diagonal second panchromatic image based on the third panchromatic pixel values of each diagonal; Combining the second panchromatic pixel values on the line to read out the third panchromatic pixel values of the anti-diagonal lines, and generating the second panchromatic image of the anti-diagonal lines based on the third panchromatic pixel values of the respective anti-diagonal lines; the third resolution The resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the second resolution mode.

第三分辨率模式是指对分辨率要求比第二分辨率模式要求低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的三级像素合并读出模式。该第三分辨率模式对应的分辨率、功耗小于第二分辨率模式对应的分辨率、功耗。该第三分辨率模式对应的信噪比、帧率大于第二分辨率模式对应的信噪比、帧率。The third resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the second resolution mode. It is a three-level pixel binning read with low resolution, low power consumption, high SNR and high frame rate out mode. The resolution and power consumption corresponding to the third resolution mode are smaller than the resolution and power consumption corresponding to the second resolution mode. The signal-to-noise ratio and frame rate corresponding to the third resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the second resolution mode.

第三分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如720p视频预览、应用视频预览等。The third resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a night scene mode for image capture and video capture under night scenes, etc., but is not limited thereto. The preview modes of video shooting include 720p video preview, application video preview, etc.

电子设备根据预设像素读取方式，从各个对角线的第三全色像素值中读取像素值以生成对角线的第二全色图像。电子设备根据预设像素读取方式，从各个反对角线的第三全色像素值中读取像素值以生成反对角线的第二全色图像。The electronic device reads pixel values from the third panchromatic pixel values of each diagonal line according to a preset pixel reading manner to generate a second panchromatic image of the diagonal lines. The electronic device reads pixel values from the third panchromatic pixel values of each anti-diagonal line according to a preset pixel reading manner to generate a second anti-diagonal panchromatic image.

操作2004，将第一彩色图像中对应在同一滤光片组中的多个对角线上的相同颜色的第二彩色像素值合并读出对角线的第三彩色像素值，并基于各个对角线的第三彩色像素值生成对角线的第二彩色图像；以及，将第一彩色图像中对应在同一滤光片组中的多个反对角线上的相同颜色的第二彩色像素值合并读出反对角线的第三彩色像素值，并基于各个反对角线的第三彩色像素值生成反对角线的第二彩色图像。 Operation 2004, combining the second color pixel values of the same color corresponding to multiple diagonal lines in the same filter set in the first color image to read out the third color pixel values of the diagonal lines, and based on each pair The second color image of the diagonal line is generated by the third color pixel value of the diagonal line; Combining and reading out the third color pixel values of the anti-diagonal lines, and generating a second color image of the anti-diagonal lines based on the third color pixel values of the anti-diagonal lines.

电子设备根据预设像素读取方式，从各个对角线的第三彩色像素值中读取像素值以生成对角线的第二彩色图像。The electronic device reads pixel values from the third color pixel values of each diagonal line according to a preset pixel reading method to generate a second color image of the diagonal lines.

电子设备根据预设像素读取方式，从各个反对角线的第三彩色像素值中读取像素值以生成反对角线的第二彩色图像。The electronic device reads pixel values from the third color pixel values of each anti-diagonal line according to a preset pixel reading method to generate a second color image of the anti-diagonal line.

如图21和图22所示，以图15的第一彩色图像和第一全色图像为例，在第三分辨率模式下，电子设备将第一全色图像1504中对应在同一滤光片组中的多个对角线上的第二全色像素值合并读出对角线的第三全色像素值，并基于各个对角线的第三全色像素值生成对角线的第二全色图像2202；以及，将第一全色图像1504中对应在同一滤光片组中的多个反对角线上的第二全色像素值合并读出反对角线的第三全色像素值，并基于各个反对角线的第三全色像素值生成反对角线的第二全色图像2208；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率；将第一彩色图像1502中对应在同一滤光片组中的多个对角线上的相同颜色的第二彩色像素值合并读出线方向的第三彩色像素值，并基于各个对角线的第三彩色像素值生成对角线的第二彩色图像2204；以及，将第一彩色图像1502中对应在同一滤光片组中的多个对角线上的相同颜色的第二彩色像素值合并读出反对角线的第三彩色像素值，并基于各个反对角线的第三彩色像素值生成反对角线的第二彩色图像2206。As shown in Figure 21 and Figure 22, taking the first color image and the first panchromatic image in Figure 15 as an example, in the third resolution mode, the electronic device maps the first panchromatic image 1504 to the same filter The second panchromatic pixel values on the plurality of diagonals in the group are combined to read out the third panchromatic pixel values of the diagonals, and the second panchromatic pixel values of the diagonals are generated based on the third panchromatic pixel values of the respective diagonals. Panchromatic image 2202; and, combining the second panchromatic pixel values corresponding to multiple anti-diagonal lines in the same filter set in the first panchromatic image 1504 to read out the third panchromatic pixel value of anti-diagonal lines , and generate a second anti-diagonal panchromatic image 2208 based on the third panchromatic pixel values of each anti-diagonal line; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode; the first color In the image 1502, the pixel values of the second color corresponding to the same color on multiple diagonals in the same filter group are merged to read out the pixel values of the third color in the direction of the line, and based on the third color pixel values of each diagonal The value generates the second color image 2204 of the diagonal; and, the second color pixel values corresponding to the same color on multiple diagonals in the same filter set in the first color image 1502 are combined to read the anti-diagonal The second color image 2206 of the anti-diagonal line is generated based on the third color pixel value of each anti-diagonal line.

操作2006，基于对角线的第二全色图像、反对角线的第二全色图像、对角线的第二彩色图像和反对角线的第二彩色图像，生成第三目标图像。In operation 2006, a third target image is generated based on the second panchromatic image of the diagonal, the second panchromatic image of the anti-diagonal, the second color image of the diagonal, and the second color image of the anti-diagonal.

具体地，电子设备将对角线的第二全色图像中每一行对角线的第三全色像素值、反对角线的第二全色图像中每一行反对角线的第三全色像素值、对角线的第二彩色图像中每一行对角线的第三彩色像素值和反对角线的第二彩色图像中每一行反对角线的第三彩色像素值相间排布，生成第三目标图像；或者将对角线的第二全色图像中每一列对角线的第三全色像素值、反对角线的第二全色图像中每一列反对角线的第三全色像素值、对角线的第二彩色图像中每一列对角线的第三彩色像素值和反对角线的第二彩色图像中每一列反对角线的第三彩色像素值相间排布，生成第三目标图像。Specifically, the electronic device calculates the third panchromatic pixel value of each diagonal line in the second panchromatic image of the diagonal line and the third panchromatic pixel value of each anti-diagonal line in the second anti-diagonal panchromatic image value, the third color pixel value of each diagonal line in the second color image of the diagonal line and the third color pixel value of each line of the anti-diagonal line in the second color image of the anti-diagonal line are arranged alternately to generate the third The target image; or the third panchromatic pixel value of each column of the diagonal in the second panchromatic image of the diagonal, the third panchromatic pixel value of each column of the anti-diagonal in the second panchromatic image of the anti-diagonal , the third color pixel values of each column of the diagonal in the second color image of the diagonal and the third color pixel values of each column of the anti-diagonal in the second color image of the anti-diagonal are arranged alternately to generate the third target image.

以图22中的对角线的第二全色图像2202、反对角线的第二全色图像2208、对角线的第二彩色图像2204和反对角线的第二彩色图像2206为例，图23是对角线的第二全色图像中每一列对角线的第三全色像素值、反对角线的第二全色图像中每一列反对角线的第三全色像素值、对角线的第二彩色图像中每一列对角线的第三彩色像素值和反对角线的第二彩色图像中每一列反对角线的第三彩色像素值相间排布的一种方式，所生成的第三目标图像。Taking the second panchromatic image 2202 of the diagonal line, the second panchromatic image 2208 of the anti-diagonal line, the second color image 2204 of the diagonal line and the second color image 2206 of the anti-diagonal line in Fig. 22 as examples, Fig. 23 is the third panchromatic pixel value of each column of the diagonal in the second panchromatic image of the diagonal line, the third panchromatic pixel value of each column of the anti-diagonal line in the second panchromatic image of the anti-diagonal line, and the diagonal In the second color image of the line, the third color pixel value of each column of the diagonal line and the third color pixel value of each column of the anti-diagonal line in the second color image of the anti-diagonal line are arranged alternately, the generated The third target image.

在其他实施例中，还可将图22中4个图像的每一列像素值以其他方式相间排布，生成第三目标图像。在其他实施例中，上述的4个图像还可将每一行像素值以其他方式相间排布，生成第三目标图像。In other embodiments, the pixel values in each column of the four images in FIG. 22 may also be arranged alternately in other ways to generate the third target image. In other embodiments, the above four images can also arrange the pixel values of each row alternately in other ways to generate the third target image.

需要说明的是，对角线的第二全色图像2202、反对角线的第二全色图像2208、对角线的第二彩色图像2204和反对角线的第二彩色图像2206中相应位置的像素坐标一致。例如，图2202中wr1wc1位置的像素、图2204中gr1gc1位置的像素、图2206中rbr1rbc1位置的像素和图2208中wr3wc3位置的像素坐标均一致。又如，图2202中wr2wc2位置的像素、图2204中gr2gc2位置的像素、图2206中rbr2rbc2位置的像素和和图2208中wr4wc4位置的像素坐标均一致。It should be noted that, in the second panchromatic image 2202 of the diagonal line, the second panchromatic image 2208 of the anti-diagonal line, the second color image 2204 of the diagonal line and the second color image 2206 of the anti-diagonal line, the The pixel coordinates are consistent. For example, the pixel at position wr1wc1 in Figure 2202, the pixel at gr1gc1 in Figure 2204, the pixel at rbr1rbc1 in Figure 2206, and the pixel coordinates at wr3wc3 in Figure 2208 are all consistent. For another example, the coordinates of the pixel at wr2wc2 in Figure 2202, the pixel at gr2gc2 in Figure 2204, the pixel at rbr2rbc2 in Figure 2206, and the pixel coordinates at wr4wc4 in Figure 2208 are all consistent.

可以理解的是，在生成第三目标图像的过程中，电子设备将上述4个图像的同一坐标的像素排布完成后，再排布其他坐标的像素。而同一坐标的各个像素之间的排布顺序并不限定。It can be understood that, in the process of generating the third target image, the electronic device arranges the pixels of the same coordinates of the above four images before arranging the pixels of other coordinates. However, the arrangement order of the pixels of the same coordinate is not limited.

如图23所示，电子设备将上述的4个图像还可将每一列像素值以其他方式相间排布，则先将图2202中的wc1列、图2204中的gc1列、图2206中的rgc1列和wc3列进行相间排布，排布完成后再将图2202中的wc2列、图2204中的gc2列、图2206中的rgc2列和wc4列进行相间排布，生成第三目标图像。As shown in Figure 23, the electronic device arranges the above-mentioned 4 images and arranges the pixel values of each column alternately in other ways, then firstly arrange the wc1 column in Figure 2202, the gc1 column in Figure 2204, and the rgc1 in Figure 2206 Columns and wc3 columns are alternately arranged, and after the arrangement is completed, the wc2 column in Figure 2202, the gc2 column in Figure 2204, the rgc2 column and wc4 column in Figure 2206 are alternately arranged to generate the third target image.

在另一实施例中，电子设备还可以将对角线的第二全色图像、反对角线的第二全色图像、对角线的第二彩色图像和反对角线的第二彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第三目标图像。其中，合并可以采用求均值、加权求平均或相加求和等其中一种方式。In another embodiment, the electronic device can also combine the second full-color image of the diagonal line, the second full-color image of the anti-diagonal line, the second color image of the diagonal line and the second color image of the anti-diagonal line The pixel values at the same position are combined to obtain combined pixel values at corresponding positions, and a third target image is formed based on each combined pixel value. Wherein, the merging may adopt one of methods such as averaging, weighted averaging, or adding and summing.

在其他实施例中，电子设备还可以采用其他方式生成第三目标图像，在此不做限定。In other embodiments, the electronic device may also use other methods to generate the third target image, which is not limited here.

在本实施例中，在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个对角线上的第二全色像素进行合并，以及将第一全色图像中对应在同一滤光片组中的多个对角线上的第二全色像素进行合并，将第一彩色图像中对应在同一滤光片组中的多个对角线上的第二彩色像素进行合并，以及将第一彩色图像中对应在同一滤光片组中的多个对角线上的第二彩色像素进行合并，能够将各个不同的彩色像素进行混合排列，使得所生成的第三目标图像中的第三彩色像素如RGB像素分布更均匀，图像质量更高。并且，所得到第三目标图像的分辨率和图像尺寸进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了三级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第三分辨率模式下，第三目标图像中包括有全排列的全色像素，可以提高整体的解析力。同时，在第三分辨率模式下，也无需跨越周期做同个颜色的像素的合并，无需做插值，提高了整体的解析力。在全尺寸情形下，各个颜色的彩色像素如第一颜色的像素值和第三颜色的像素值在对角线或反对角线更加分散均衡。其中，全排列指的是每一坐标均有该像素，不需要做插值估计。In this embodiment, in the third resolution mode, the second panchromatic pixels corresponding to multiple diagonal lines in the same filter set in the first panchromatic image are combined, and the first panchromatic pixels The second panchromatic pixels corresponding to multiple diagonals in the same filter set in the color image are combined, and the first color image is corresponding to the second panchromatic pixels on multiple diagonals in the same filter set The two-color pixels are combined, and the second color pixels corresponding to multiple diagonal lines in the same filter group are combined in the first color image, and the different color pixels can be mixed and arranged, so that the generated The third color pixels in the third target image, such as RGB pixels, are more evenly distributed, and the image quality is higher. Moreover, the resolution and image size of the obtained third target image are further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the three-level pixel combination output is lower, Image processing effect with better signal-to-noise ratio. Moreover, in the third resolution mode, the third target image includes fully arranged panchromatic pixels, which can improve the overall resolution. At the same time, in the third resolution mode, there is no need to merge pixels of the same color across cycles, and no interpolation is required, which improves the overall resolution. In the case of full size, the color pixels of each color, such as the pixel values of the first color and the pixel values of the third color, are more dispersed and balanced on the diagonal or anti-diagonal. Among them, the full arrangement means that each coordinate has the pixel, and interpolation estimation is not required.

在一个实施例中，每个滤光片组的彩色滤光片排布在对应滤光片组的对角线及平行于对角线的方向上，每个滤光片组的全色滤光片排布在对应滤光片组的反对角线及平行于反对角线的方向上；上述方法还包括：在第三分辨率模式下，将每个滤光片组中反对角线上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出反对角线的第五全色像素值，并基于各个反对角线的第五全色像素值生成反对角线的第四全色图像；以及，将每个滤光片组中平行于反对角线的方向上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出平行反对角线的第五全色像素值，并基于各个平行反对角线的第五全色像素值生成对角线的第四全色图像；将每个滤光片组中对角线上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出对角线的第五彩色像素值，并基于各个对角线的第五彩色像素值生成对角线的第四彩色图像；以及，将每个滤光片组中平行于对角线的方向上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出平行对角线的第五彩色像素值，并基于各个平行对角线的第五彩色像素值生成反对角线的第四彩色图像；基于反对角线的第四全色图像、对角线的第四全色图像、对角线的第四彩色图像和反对角线的第四彩色图像，生成第五目标图像。In one embodiment, the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group The slices are arranged on the anti-diagonal line of the corresponding filter group and the direction parallel to the anti-diagonal line; the above method also includes: in the third resolution mode, multiple The panchromatic pixels corresponding to each panchromatic sub-filter of a panchromatic optical filter combine to read out the fifth panchromatic pixel value of the anti-diagonal line, and generate the anti-diagonal line based on the fifth panchromatic pixel value of each anti-diagonal line of the fourth panchromatic image; and, the panchromatic pixels corresponding to the respective panchromatic sub-filters of the plurality of panchromatic filters on the direction parallel to the anti-diagonal in each filter set are merged and read out in parallel The fifth panchromatic pixel value of the anti-diagonal, and the fourth panchromatic image of the diagonal is generated based on the fifth panchromatic pixel value of each parallel anti-diagonal; The color pixels corresponding to each color sub-filter of each color filter are merged to read out the fifth color pixel value of the diagonal line, and the fourth color image of the diagonal line is generated based on the fifth color pixel value of each diagonal line and, combining the color pixels corresponding to each color sub-filter of a plurality of color filters in the direction parallel to the diagonal in each filter group to read out the fifth color pixel value parallel to the diagonal , and generate the fourth color image of the anti-diagonal line based on the fifth color pixel value of each parallel diagonal line; based on the fourth panchromatic image of the anti-diagonal line, the fourth panchromatic image of the diagonal line, the first The four-color image and the anti-diagonal fourth-color image generate a fifth target image.

在一个实施例中，基于反对角线的第四全色图像、对角线的第四全色图像、对角线的第四彩色图像和反对角线的第四彩色图像，生成第五目标图像，包括：将反对角线的第四全色图像中每一行反对角线的第五全色像素值、对角线的第四全色图像中每一行平行反对角线的第五全色像素值、对角线的第四彩色图像中每一行对角线的第五彩色像素值和反对角线的第四彩色图像中每一行平行对角线的第五彩色像素值相间排布，生成第五目标图像；或者，将反对角线的第四全色图像中每一列反对角线的第五全色像素值、对角线的第四全色图像中每一列平行反对角线的第五全色像素值、对角线的第四彩色图像中每一列对角线的第五彩色像素值和反对角线的第四彩色图像中每一列平行对角线的第五彩色像素值相间排布，生成第五目标图像。In one embodiment, the fifth target image is generated based on the anti-diagonal fourth panchromatic image, the diagonal fourth panchromatic image, the diagonal fourth color image, and the anti-diagonal fourth color image , including: the fifth panchromatic pixel value of each anti-diagonal line in the fourth panchromatic image of the anti-diagonal line, the fifth panchromatic pixel value of each parallel anti-diagonal line in the fourth panchromatic image of the diagonal line , the fifth color pixel value of each diagonal line in the fourth color image of the diagonal line and the fifth color pixel value of each parallel diagonal line in the fourth color image of the anti-diagonal line are arranged alternately to generate the fifth color pixel value target image; alternatively, the fifth panchromatic pixel value for each column of the anti-diagonal in the anti-diagonal fourth panchromatic image, the fifth panchromatic color for each column of parallel anti-diagonal in the diagonal fourth panchromatic The pixel value, the fifth color pixel value of each diagonal line in the fourth color image of the diagonal line and the fifth color pixel value of each column parallel diagonal line in the fourth color image of the anti-diagonal line are arranged alternately to generate Fifth target image.

在另一实施例中，电子设备还可以将反对角线的第四全色图像、对角线的第四全色图像、对角线的第四彩色图像和反对角线的第四彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第四目标图像。其中，合并读出可以采用求均值、加权求平均或相加求和等其中一种方式。In another embodiment, the electronic device may also combine the anti-diagonal fourth panchromatic image, the diagonal fourth panchromatic image, the diagonal fourth color image and the anti-diagonal fourth color image The pixel values at the same position are combined to obtain combined pixel values at corresponding positions, and a fourth target image is formed based on each combined pixel value. Wherein, the combined readout may adopt one of methods such as averaging, weighted averaging, or adding and summing.

在其他实施例中，电子设备还可以采用其他方式生成第五目标图像，在此不做限定。In other embodiments, the electronic device may also use other methods to generate the fifth target image, which is not limited here.

在本实施例中，在第三分辨率模式下，将每个滤光片组中反对角线上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出反对角线的第五全色像素值，以及将每个滤光片组中平行于所述反对角线的方向上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出平行反对角线的第五全色像素值，可以更快速生成反对角线的第四全色图像和对角线的第四全色图像；将每个滤光片组中对角线上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出对角线的第五彩色像素值，以及将每个滤光片组中平行于所述对角线的方向上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出平行对角线的第五彩色像素值，可以更快速生成对角线的第四彩色图像和反对角线的第四彩色图像，从而更快速生成第五目标图像。In this embodiment, in the third resolution mode, the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters on the anti-diagonal line in each filter group are combined and read out The fifth panchromatic pixel value of the anti-diagonal, and the corresponding panchromatic color of each panchromatic sub-filter of the plurality of panchromatic filters in the direction parallel to the anti-diagonal in each filter group Pixel merging reads out the fifth panchromatic pixel values of the parallel anti-diagonal lines, which can generate the fourth panchromatic image of the anti-diagonal line and the fourth panchromatic image of the diagonal line more quickly; The color pixels corresponding to each color sub-filter of a plurality of color filters on the line are merged to read out the fifth color pixel value of the diagonal line, and the values of the fifth color pixel in each filter group parallel to the diagonal line The color pixels corresponding to each color sub-filter of multiple color filters in the direction are merged and read out the fifth color pixel value parallel to the diagonal, which can generate the fourth color image of the diagonal and the anti-diagonal more quickly The fourth color image of , so as to generate the fifth target image more quickly.

并且，上述实施例能够将各个不同的彩色像素进行混合排列，使得所生成的第五目标图像中的各个第五彩色像素值如RGB像素分布更均匀，图像质量更高。并且，所得到第五目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了三级像素合并输出的功耗更低、信噪比更佳的图像处理效果。Moreover, the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each fifth color pixel value, such as RGB pixels, in the generated fifth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained fifth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the three-level pixel combination output is lower , Better signal-to-noise ratio image processing effect.

在一个实施例中，还提供了另一种图像生成方法，应用于图像传感器，图像传感器包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量；全色滤光片233与彩色滤光片234在最小重复单元230的每一行和每一列上交替排布；像素阵列24中各个像素与滤光片阵列23的滤光片对应设置，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号；该图像生成方法包括：在全分辨率模式下，将每个全色滤光片对应的全色像素读出全分辨率全色像素值，以及将每个彩色滤光片对应的彩色像素读出全分辨率彩色像素值；基于各个全分辨率全色像素值和各个全分辨率彩色像素值，生成全分辨率目标图像。In one embodiment, another image generation method is provided, which is applied to an image sensor. The image sensor includes a filter array 23 and a pixel array 24. The filter array 23 includes a minimum repeating unit 230, and the minimum repeating unit 230 includes A plurality of filter groups, each filter group includes only two colors of color filters 234 and panchromatic filters 233, the amount of light transmitted by the panchromatic filters 233 is greater than that of the color filters 234 The amount of light that passes through; the panchromatic filter 233 and the color filter 234 are arranged alternately on each row and each column of the minimum repeating unit 230; each pixel in the pixel array 24 and the filter of the filter array 23 Correspondingly set, the pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals; The color pixel reads out the full-resolution panchromatic pixel value, and the color pixel corresponding to each color filter reads out the full-resolution color pixel value; based on each full-resolution panchromatic pixel value and each full-resolution color pixel value , generating the full-resolution target image.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片234和全色滤光片233，子单元中的彩色滤光片234排布在子单元的对角线，子单元中的全色滤光片233排布在子单元的反对角线；上述方法还包括：在第一分辨率模式下，将每个子单元中多个全色滤光片对应的全色像素合并读出第六全色像素值，并基于各个第六全色像素值生成第五全色图像；将每个子单元中多个相同颜色的彩色滤光片对应的彩色像素合并读出第六彩色像素值，并基于各个所述第六彩色像素值生成第五彩色图像；基于第五全色图像和第五彩色图像，生成第六目标图像。。In one embodiment, each filter set includes a plurality of subunits, each subunit includes a color filter 234 and a panchromatic filter 233, and the color filters 234 in the subunits are arranged on opposite sides of the subunits. Diagonal line, the panchromatic filter 233 in the subunit is arranged on the anti-diagonal line of the subunit; the above method also includes: in the first resolution mode, a plurality of panchromatic filters corresponding to each subunit Combining panchromatic pixels to read out the sixth panchromatic pixel value, and generating a fifth panchromatic image based on each sixth panchromatic pixel value; combining and reading out the color pixels corresponding to multiple color filters of the same color in each subunit sixth color pixel values, and generate a fifth color image based on each of the sixth color pixel values; generate a sixth target image based on the fifth panchromatic image and the fifth color image. .

其中，基于第五全色图像和第五彩色图像，生成第六目标图像，包括：将第五全色图像中每一行第六全色像素值与第五彩色图像中每一行第六彩色像素值相间排布，生成第六目标图像；或者将第五全色图像中每一列第六全色像素值与第五彩色图像中每一列第六彩色像素值相间排布，生成第六目标图像。Wherein, based on the fifth panchromatic image and the fifth color image, generating the sixth target image includes: combining the sixth panchromatic pixel value of each row in the fifth panchromatic image with the sixth color pixel value of each row in the fifth color image Arranging alternately to generate the sixth target image; or alternately arranging each column of sixth panchromatic pixel values in the fifth panchromatic image and each column of sixth color pixel values in the fifth color image to generate the sixth target image.

在一个实施例中，上述方法还包括：在第二分辨率模式下，将第五全色图像中对应在同一滤光片组中的多个对角线上的第六全色像素值合并读出对角线的第七全色像素值，并基于各个对角线的第七全色像素值生成对角线的第六全色图像；以及，将第五全色图像中对应在同一滤光片组中的多个反对角线上的第六全色像素值合并读出反对角线的第七全色像素值，并基于各个反对角线的第七全色像素值生成反对角线的第六全色图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率；将第五彩色图像中对应在同一滤光片组中的多个对角线上的相同颜色的第六彩色像素值合并读出对角线的第七彩色像素值，并基于各个对角线的第七彩色像素值生成对角线的第六彩色图像；以及，将所述第五彩色图像中对应在同一滤光片组中的多个反对角线上的相同颜色的第六彩色像素值合并读出反对角线的第七彩色像素值，并基于各个反对角线的第七彩色像素值生成反对角线的第六彩色图像；基于反对角线的第六全色图像、对角线的第六全色图像、对角线的第六彩色图像和反对角线的第六彩色图像，生成第七目标图像。In one embodiment, the above method further includes: in the second resolution mode, combining and reading sixth panchromatic pixel values corresponding to multiple diagonal lines in the same filter set in the fifth panchromatic image Get the seventh panchromatic pixel value of the diagonal line, and generate the sixth panchromatic image of the diagonal line based on the seventh panchromatic pixel value of each diagonal line; and, corresponding to the same filter in the fifth panchromatic image Combining the sixth panchromatic pixel values on multiple anti-diagonal lines in the slice group to read out the seventh panchromatic pixel values of the anti-diagonal lines, and generating the seventh panchromatic pixel values of the anti-diagonal lines based on the seventh panchromatic pixel values of the anti-diagonal lines Six full-color images; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode; the fifth color image corresponds to the same color on multiple diagonal lines in the same filter group Combining the pixel values of the sixth color and reading the pixel values of the seventh color of the diagonal, and generating a sixth color image of the diagonal based on the pixel values of the seventh color of each diagonal; and, the fifth color image of the Combining the sixth color pixel values corresponding to the same color on multiple anti-diagonal lines in the same filter set to read out the seventh color pixel values of the anti-diagonal lines, and generating based on the seventh color pixel values of each anti-diagonal line a sixth color image of the anti-diagonal; based on the sixth panchromatic image of the anti-diagonal, the sixth panchromatic image of the diagonal, the sixth color image of the diagonal, and the sixth color image of the anti-diagonal, generating the sixth Seven target images.

在一个实施例中，基于反对角线的第六全色图像、对角线的第六全色图像、对角线的第六彩色图像和反对角线的第六彩色图像，生成第七目标图像，包括：将反对角线的第六全色图像中每一行对角线的第七全色像素值、对角线的第六全色图像中每一行反对角线的第七全色像素值、对角线的第六彩色图像中每一行对角线的第七彩色像素值和反对角线的第六彩色图像中每一行反对角线的第七彩色像素值相间排布，生成第七目标图像；或者将反对角线的第六全色图像中每一列对角线的第七全色像素值、对角线的第六全色图像中每一列反对角线的第七全色像素值、对角线的第六彩色图像中每一列对角线的第七彩色像素值和反对角线的第六彩色图像中每一列反对角线的第七彩色像素值相间排布，生成第七目标图像。In one embodiment, the seventh target image is generated based on the sixth panchromatic image of the anti-diagonal, the sixth panchromatic image of the diagonal, the sixth color image of the diagonal, and the sixth color image of the anti-diagonal , including: the seventh panchromatic pixel value of each diagonal line in the sixth panchromatic image of the anti-diagonal line, the seventh panchromatic pixel value of each anti-diagonal line in the sixth panchromatic image of the diagonal line, The seventh color pixel values of each diagonal line in the sixth color image of the diagonal line and the seventh color pixel value of each anti-diagonal line in the sixth color image of the anti-diagonal line are arranged alternately to generate the seventh target image ; or the seventh panchromatic pixel value of each column in the sixth panchromatic image of the anti-diagonal line, the seventh panchromatic pixel value of each column of the anti-diagonal line in the sixth panchromatic image of the diagonal line, and the The seventh color pixel value of each column of the diagonal line in the sixth color image of the diagonal line and the seventh color pixel value of each column of the anti-diagonal line in the sixth color image of the anti-diagonal line are arranged alternately to generate the seventh target image.

在一个实施例中，每个滤光片组的彩色滤光片排布在对应滤光片组的对角线及平行于对角线的方向上，每个滤光片组的全色滤光片排布在对应滤光片组的反对角线及平行于反对角线的方向上；上述方法还包括：在第二分辨率模式下，将每个滤光片组中反对角线上的多个全色滤光片对应的全色像素合并读出反对角线的第八全色像素值，并基于各个反对角线的第八全色像素值生成反对角线的第七全色图像；以及，将每个滤光片组中平行于反对角线的方向上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出平行反对角线的第八全色像素值，并基于各个平行反对角线的第八全色像素值生成对角线的第七全色图像；将每个滤光片组中对角线上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出对角线的第八彩色像素值，并基于各个对角线的第八彩色像素值生成对角线的第七彩色图像；以及，将每个滤光片组中平行于对角线的方向上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出平行对角线的第八彩色像素值，并基于各个平行对角线的第八彩色像素值生成反对角线的第七彩色图像；基于反对角线的第七全色图像、对角线的第七全色图像、对角线的第七彩色图像和反对角线的第七彩色图像，生成第八目标图像。In one embodiment, the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group The slices are arranged on the anti-diagonal line of the corresponding filter group and the direction parallel to the anti-diagonal line; the above method also includes: in the second resolution mode, multiple Combining the panchromatic pixels corresponding to each panchromatic filter to read out the eighth panchromatic pixel value of the anti-diagonal line, and generating the seventh panchromatic image of the anti-diagonal line based on the eighth panchromatic pixel value of each anti-diagonal line; and , combining the panchromatic pixels corresponding to the panchromatic sub-filters of the multiple panchromatic filters in the direction parallel to the anti-diagonal in each filter group to read out the eighth panchromatic color of the parallel anti-diagonal pixel value, and generate the seventh panchromatic image of the diagonal based on the eighth panchromatic pixel value of each parallel anti-diagonal line; The color pixels corresponding to the sub-filters are combined to read out the eighth color pixel value of the diagonal, and the seventh color image of the diagonal is generated based on the eighth color pixel value of each diagonal; and, each filter The color pixels corresponding to the color sub-filters of the plurality of color filters in the direction parallel to the diagonal in the film group are combined to read out the value of the eighth color pixel on the parallel diagonal, and based on each parallel diagonal The eighth color pixel value of the anti-diagonal generates the seventh color image of the anti-diagonal; based on the seventh panchromatic image of the anti-diagonal, the seventh panchromatic image of the diagonal, the seventh color image of the diagonal and the The seventh color image is used to generate the eighth target image.

其中，基于反对角线的第七全色图像、对角线的第七全色图像、对角线的第七彩色图像和反对角线的第七彩色图像，生成第八目标图像，包括：将反对角线的第七全色图像中每一行反对角线的第八全色像素值、对角线的第七全色图像中每一行平行反对角线的第八全色像素值、对角线的第七彩色图像中每一行对角线的第八彩色像素值和反对角线的第七彩色图像中每一行平行对角线的第八彩色像素值相间排布，生成第八目标图像；或者，将反对角线的第七全色图像中每一列反对角线的第八全色像素值、对角线的第七全色图像中每一列平行反对角线的第八全色像素值、对角线的第七彩色图像中每一列对角线的第八彩色像素值和反对角线的第七彩色图像中每一列平行对角线的第八彩色像素值相间排布，生成第八目标图像。Wherein, the eighth target image is generated based on the seventh panchromatic image of the anti-diagonal line, the seventh panchromatic image of the diagonal line, the seventh color image of the diagonal line and the seventh color image of the anti-diagonal line, including: The eighth panchromatic pixel value of each anti-diagonal line in the seventh panchromatic image of the diagonal line, the eighth panchromatic pixel value of each parallel anti-diagonal line in the seventh panchromatic image of the diagonal line, and the diagonal line The eighth color pixel value of each diagonal line in the seventh color image of the anti-diagonal line and the eighth color pixel value of each parallel diagonal line in the seventh color image of the anti-diagonal line are arranged alternately to generate the eighth target image; or , the eighth panchromatic pixel value of each anti-diagonal line in the seventh panchromatic image of the anti-diagonal line, the eighth panchromatic pixel value of each parallel anti-diagonal line in the seventh panchromatic image of the diagonal line, and The eighth color pixel value of each column of the diagonal line in the seventh color image of the diagonal line and the eighth color pixel value of each column of the parallel diagonal line in the seventh color image of the anti-diagonal line are arranged alternately to generate the eighth target image .

需要说明的是，上述生成第六目标图像、第七目标图像、第八目标图像的原理和上述生成第二目标图像、第三目标图像、第五目标图像的原理类似，在此不做赘述。It should be noted that the principle of generating the sixth target image, the seventh target image, and the eighth target image is similar to the principle of generating the second target image, third target image, and fifth target image, and will not be repeated here.

应该理解的是，虽然图12、图14和图20的流程图中的各个操作按照箭头的指示依次显示，但是这些操作并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些操作的执行并没有严格的顺序限制，这些操作可以以其它的顺序执行。而且，图12、图14和图20中的至少一部分操作可以包括多个子操作或者多个阶段，这些子操作或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子操作或者阶段的执行顺序也不必然是依次进行，而是可以与其它操作或者其它操作的子操作或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the various operations in the flowcharts of FIG. 12 , FIG. 14 and FIG. 20 are displayed sequentially as indicated by the arrows, these operations are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction on the execution of these operations, and these operations can be executed in other orders. Moreover, at least some of the operations in FIG. 12 , FIG. 14 and FIG. 20 may include multiple sub-operations or multiple stages, and these sub-operations or stages are not necessarily performed at the same time, but may be performed at different times. The execution sequence of the sub-operations or stages is not necessarily performed sequentially, but may be performed in turn or alternately with other operations or at least a part of sub-operations or stages of other operations.

图24为一个实施例的图像生成装置的结构框图。如图24所示，提供了一种图像生成装置，应用于图像传感器，该图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，全色滤光片透过的进光量大于彩色滤光片透过的进光量；全色滤光片与彩色滤光片在最小重复单元的每一行和每一列上交替排布；每个全色滤光片包括N行N列个全色子滤光片，每个彩色滤光片包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为大于或等于2的正整数；像素阵列中各个像素与滤光片阵列的子滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号；上述装置包括：读出模块2402和图像生成模块2404，其中：Fig. 24 is a structural block diagram of an image generating device of an embodiment. As shown in Figure 24, an image generating device is provided, which is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups , each filter group includes only two colors of color filters and panchromatic filters, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; the panchromatic filter Sheets and color filters are alternately arranged in each row and column of the smallest repeating unit; each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer greater than or equal to 2; each pixel in the pixel array and the sub-color of the filter array The optical filter is correspondingly arranged, and the pixel array is configured to receive light passing through the optical filter array to generate an electrical signal; the above-mentioned device includes: a readout module 2402 and an image generation module 2404, wherein:

读出模块2402，用于在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素。The readout module 2402 is configured to read out the full-resolution panchromatic pixels from the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic optical filter in the full-resolution mode, and read out the full-resolution panchromatic pixels in the color filter The color pixel corresponding to each color sub-filter reads out the full resolution color pixel.

图像生成模块2404，用于基于各个全分辨率全色像素和各个全分辨率彩色像素，生成全分辨率目标图像。An image generating module 2404, configured to generate a full-resolution target image based on each full-resolution panchromatic pixel and each full-resolution color pixel.

上述图像生成装置，The above-mentioned image generating device,

在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素，而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素和各个全分辨率彩色像素，能够生成信息更多、细节解析更清晰的全分辨率目标图像。In the full-resolution mode, the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out to the full-resolution panchromatic pixel, and each color sub-filter in the color filter The corresponding color pixels read out the full-resolution color pixels, and the amount of light transmitted by the panchromatic filter is greater than that of the color filter, which can integrate the information of the panchromatic channel into the image and improve the overall light input , so that based on each full-resolution panchromatic pixel and each full-resolution color pixel, a full-resolution target image with more information and clearer detail analysis can be generated.

在一个实施例中，上述读出模块2402还用于在第一分辨率模式下，将每个全色滤光片中各个全色子滤光片对应的全色像素合并读出第一全色像素值，以及将每个彩色滤光片中各个彩色子滤光片对应的彩色像素合并读出第一彩色像素值；第一分辨率模式对应的分辨率小于全分辨率模式对应的分辨率；上述图像生成模块2404还用于基于各个第一全色像素值和各个第一彩色像素值，生成第一目标图像。In one embodiment, the above-mentioned readout module 2402 is also used to combine the panchromatic pixels corresponding to each panchromatic sub-filter in each panchromatic filter to read out the first panchromatic pixel in the first resolution mode. The pixel value, and combining the color pixels corresponding to each color sub-filter in each color filter to read out the first color pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode; The image generating module 2404 is further configured to generate a first target image based on each first panchromatic pixel value and each first color pixel value.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，子单元中的彩色滤光片排布在子单元的对角线上，子单元中的全色滤光片排布在子单元的反对角线上；上述读出模块2402还用于在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，上述图像生成模块2404还用于基于各个第二全色像素值生成第一全色图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率；上述读出模块2402还用于将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，上述图像生成模块2404还用于基于各个第二彩色像素值生成第一彩色图像；上述图像生成模块2404还用于基于第一全色图像和第一彩色图像，生成第二目标图像。In one embodiment, each filter set includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filters in the subunits are arranged on the diagonal of the subunits , the panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits; the above-mentioned readout module 2402 is also used to convert the first target image corresponding to each subunit in the second resolution mode A plurality of first panchromatic pixel values are combined to read out a second panchromatic pixel value, and the above-mentioned image generation module 2404 is also used to generate a first panchromatic image based on each second panchromatic pixel value; the resolution corresponding to the second resolution mode It is smaller than the resolution corresponding to the first resolution mode; the above-mentioned readout module 2402 is also used to combine and read out the second color pixel value corresponding to multiple first color pixel values of the same color in each subunit in the first target image The image generation module 2404 is further configured to generate a first color image based on each second color pixel value; the image generation module 2404 is further configured to generate a second target image based on the first panchromatic image and the first color image.

在一个实施例中，上述图像生成模块2404还用于将第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者将第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。In one embodiment, the above-mentioned image generation module 2404 is further configured to alternately arrange each row of second panchromatic pixel values in the first panchromatic image with each row of second color pixel values in the first color image to generate the second target image; or alternately arrange each column of second panchromatic pixel values in the first panchromatic image with each column of second color pixel values in the first color image to generate a second target image.

在一个实施例中，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，子单元中的彩色滤光片排布在子单元的对角线，子单元中的全色滤光片排布在子单元的反对角线；上述读出模块2402还用于在第二分辨率模式下，将每个子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第四全色像素值，上述图像生成模块2404还用于基于各个第四全色像素值生成第三全色图像；上述读出模块2402还用于将每个子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第四彩色像素值，上述图像生成模块2404还用于基于各个第四彩色像素值生成第三彩色图像；上述图像生成模块2404还用于基于第三全色图像和第三彩色图像，生成第四目标图像。In one embodiment, each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits; the above-mentioned readout module 2402 is also used for, in the second resolution mode, each panchromatic filter in each subunit The panchromatic pixels corresponding to the color sub-filters are combined to read out the fourth panchromatic pixel value, and the above-mentioned image generation module 2404 is also used to generate a third panchromatic image based on each fourth panchromatic pixel value; the above-mentioned readout module 2402 is also used to In order to combine the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each subunit to read out the fourth color pixel value, the above-mentioned image generation module 2404 is also used to Generate a third color image; the image generation module 2404 is further configured to generate a fourth target image based on the third panchromatic image and the third color image.

在一个实施例中，上述图像生成模块2404还用于将第三全色图像中每一行第四全色像素值与第三彩色图像中每一行第四彩色像素值相间排布，生成第四目标图像；或者将第三全色图像中每一列第四全色像素值与第三彩色图像中每一列第四彩色像素值相间排布，生成第四目标图像。In one embodiment, the above-mentioned image generation module 2404 is further configured to alternately arrange the fourth panchromatic pixel values in each row in the third panchromatic image and the fourth color pixel values in each row in the third color image to generate the fourth target image; or alternately arrange each column of fourth panchromatic pixel values in the third panchromatic image and each column of fourth color pixel values in the third color image to generate a fourth target image.

在一个实施例中，上述读出模块2402还用于在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个对角线上的第二全色像素值合并读出对角线的第三全色像素值，上述图像生成模块2404还用于基于各个对角线的第三全色像素值生成对角线的第二全色图像；以及，上述读出模块2402还用于将第一全色图像中对应在同一滤光片组中的多个反对角线上的第二全色像素值合并读出反对角线的第三全色像素值，上述图像生成模块2404还用于基于各个反对角线的第三全色像素值生成反对角线的第二全色图像；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率；上述读出模块2402还用于将第一彩色图像中对应在同一滤光片组中的多个对角线上的相同颜色的第二彩色像素值合并读出对角线的第三彩色像素值，上述图像生成模块2404还用于基于各个对角线的第三彩色像素值生成对角线的第二彩色图像；以及，上述读出模块2402还用于将第一彩色图像中对应在同一滤光片组中的多个反对角线上的相同颜色的第二彩色像素值合并读出反对角线的第三彩色像素值，上述读出生成模块2404还用于基于各个反对角线的第三彩色像素值生成反对角线的第二彩色图像；上述图像生成模块2404还用于基于对角线的第二全色图像、反对角线的第二全色图像、对角线的第二彩色图像和反对角线的第二彩色图像，生成第三目标图像。In one embodiment, the above-mentioned readout module 2402 is also used to convert the second panchromatic pixels corresponding to multiple diagonal lines in the same filter set in the first panchromatic image in the third resolution mode Combining and reading out the third panchromatic pixel values of the diagonal lines, the image generation module 2404 is also used to generate a second panchromatic image of the diagonal lines based on the third panchromatic pixel values of the diagonal lines; and, the above read The output module 2402 is also used to combine the second panchromatic pixel values corresponding to multiple anti-diagonal lines in the same filter set in the first panchromatic image and read out the third panchromatic pixel value of the anti-diagonal line, the above The image generating module 2404 is further configured to generate a second panchromatic image of the anti-diagonal line based on the third panchromatic pixel values of each anti-diagonal line; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode; The above-mentioned readout module 2402 is also used to combine the second color pixel values of the same color corresponding to multiple diagonal lines in the same filter set in the first color image to read out the third color pixel value of the diagonal line , the above-mentioned image generation module 2404 is also used to generate a second color image of the diagonal line based on the third color pixel values of each diagonal line; Combining the second color pixel values of the same color on multiple anti-diagonal lines in the light sheet group to read out the third color pixel value of the anti-diagonal lines, the above-mentioned readout generation module 2404 is also used for the third color pixel value based on each anti-diagonal line The color pixel values generate a second color image of the anti-diagonal line; the above-mentioned image generation module 2404 is also used for the second panchromatic image based on the diagonal line, the second panchromatic image of the anti-diagonal line, and the second color image of the diagonal line and the anti-diagonal second color image to generate a third target image.

在一个实施例中，上述图像生成模块2404还用于将对角线的第二全色图像中每一行对角线的第三全色像素值、反对角线的第二全色图像中每一行反对角线的第三全色像素值、对角线的第二彩色图像中每一行对角线的第三彩色像素值和反对角线的第二彩色图像中每一行反对角线的第三彩色像素值相间排布，生成第三目标图像；或者将对角线的第二全色图像中每一列对角线的第三全色像素值、反对角线的第二全色图像中每一列反对角线的第三全色像素值、对角线的第二彩色图像中每一列对角线的第三彩色像素值和反对角线的第二彩色图像中每一列反对角线的第三彩色像素值相间排布，生成第三目标图像。In one embodiment, the above-mentioned image generation module 2404 is further configured to convert the third panchromatic pixel value of each row of the diagonal line in the second panchromatic image of the diagonal line, and each row of the second panchromatic image of the anti-diagonal line The third panchromatic pixel value of the anti-diagonal, the third-color pixel value of each row of the diagonal in the second-color image of the diagonal, and the third-color pixel value of each row of the anti-diagonal second-color image of the anti-diagonal The pixel values are arranged alternately to generate the third target image; or the third panchromatic pixel value of each column in the second panchromatic image of the diagonal line and each column of the second panchromatic image of the anti-diagonal line are opposed The third panchromatic pixel value of the diagonal, the third color pixel value of each column of the diagonal in the second color image of the diagonal, and the third color pixel of each column of the anti-diagonal in the second color image of the anti-diagonal The values are arranged alternately to generate the third target image.

在一个实施例中，每个滤光片组的彩色滤光片排布在对应滤光片组的对角线及平行于对角线的方向上，每个滤光片组的全色滤光片排布在对应滤光片组的反对角线及平行于反对角线的方向上；上述读出模块2402还用于在第三分辨率模式下，将每个滤光片组中反对角线上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出反对角线的第五全色像素值，上述图像生成模块2404还用于基于各个反对角线的第五全色像素值生成反对角线的第四全色图像；以及，上述读出模块2402还用于将每个滤光片组中平行于反对角线的方向上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出平行反对角线的第五全色像素值，上述图像生成模块2404还用于基于各个平行反对角线的第五全色像素值生成对角线的第四全色图像；上述读出模块2402还用于将每个滤光片组中对角线上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出对角线的第五彩色像素值，上述图像生成模块2404还用于基于各个对角线的第五彩色像素值生成对角线的第四彩色图像；以及，上述读出模块2402还用于将每个滤光片组中平行于对角线的方向上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出平行对角线的第五彩色像素值，上述图像生成模块2404还用于基于各个平行对角线的第五彩色像素值生成反对角线的第四彩色图像；上述图像生成模块2404还用于基于反对角线的第四全色图像、对角线的第四全色图像、对角线的第四彩色图像和反对角线的第四彩色图像，生成第五目标图像。In one embodiment, the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group The slices are arranged on the anti-diagonal line of the corresponding filter group and the direction parallel to the anti-diagonal line; the above-mentioned readout module 2402 is also used for, in the third resolution mode, the anti-diagonal line in each filter group The panchromatic pixels corresponding to each panchromatic sub-filter of a plurality of panchromatic optical filters on the merging read out the fifth panchromatic pixel value of the anti-diagonal line, and the above-mentioned image generation module 2404 is also used for each anti-diagonal line The fifth panchromatic pixel value generates a fourth panchromatic image of the anti-diagonal; and, the above-mentioned readout module 2402 is also used to filter a plurality of panchromatic colors in the direction parallel to the anti-diagonal in each filter group The panchromatic pixels corresponding to each panchromatic sub-filter of the sheet are combined to read out the fifth panchromatic pixel value of the parallel anti-diagonal line, and the above-mentioned image generation module 2404 is also used for the fifth panchromatic pixel value based on each parallel anti-diagonal line Generate the fourth panchromatic image of the diagonal; the above-mentioned readout module 2402 is also used to combine the color pixels corresponding to the color sub-filters of the multiple color filters on the diagonal in each filter group Read out the fifth color pixel values of the diagonal lines, the above-mentioned image generation module 2404 is also used to generate the fourth color image of the diagonal lines based on the fifth color pixel values of each diagonal line; and, the above-mentioned readout module 2402 also uses Combining the color pixels corresponding to each color sub-filter of a plurality of color filters in the direction parallel to the diagonal in each filter group to read out the fifth color pixel value parallel to the diagonal, the above The image generation module 2404 is also used for generating the fourth color image of the anti-diagonal line based on the fifth color pixel value of each parallel diagonal line; the image generation module 2404 is also used for the fourth panchromatic image based on the anti-diagonal line, the diagonal The fourth panchromatic image of the line, the fourth color image of the diagonal line and the fourth color image of the anti-diagonal line generate the fifth target image.

在一个实施例中，上述图像生成模块2404还用于将反对角线的第四全色图像中每一行反对角线的第五全色像素值、对角线的第四全色图像中每一行平行反对角线的第五全色像素值、对角线的第四彩色图像中每一行对角线的第五彩色像素值和反对角线的第四彩色图像中每一行平行对角线的第五彩色像素值相间排布，生成第五目标图像；或者，将反对角线的第四全色图像中每一列反对角线的第五全色像素值、对角线的第四全色图像中每一列平行反对角线的第五全色像素值、对角线的第四彩色图像中每一列对角线的第五彩色像素值和反对角线的第四彩色图像中每一列平行对角线的第五彩色像素值相间排布，生成第五目标图像。In one embodiment, the above-mentioned image generation module 2404 is further configured to convert the fifth panchromatic pixel value of each row in the fourth panchromatic image of the anti-diagonal line to the fifth panchromatic pixel value of the diagonal line, and each row of the fourth panchromatic image of the diagonal line The fifth panchromatic pixel value of the parallel anti-diagonal, the fifth color pixel value of each row of the diagonal in the fourth color image of the diagonal, and the fifth color pixel value of each row of the parallel diagonal in the fourth color image of the anti-diagonal The five-color pixel values are arranged alternately to generate the fifth target image; or, the fifth panchromatic pixel value of each column in the anti-diagonal line in the fourth pan-color image of the anti-diagonal line, and the fifth pan-color pixel value of the diagonal line in the fourth pan-color image of the diagonal line The fifth panchromatic pixel value of each column parallel to the anti-diagonal, the fifth color pixel value of each column of the diagonal in the fourth color image of the diagonal and the parallel diagonal of each column of the fourth color image of the anti-diagonal The pixel values of the fifth color are arranged alternately to generate the fifth target image.

上述图像生成装置中各个模块的划分仅仅用于举例说明，在其他实施例中，可将图像生成装置按照需要划分为不同的模块，以完成上述图像生成装置的全部或部分功能。The division of each module in the above image generating device is only for illustration. In other embodiments, the image generating device can be divided into different modules according to needs, so as to complete all or part of the functions of the above image generating device.

关于图像生成装置的具体限定可以参见上文中对于图像生成方法的限定，在此不再赘述。上述图像生成装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。For specific limitations on the image generation device, refer to the above-mentioned limitations on the image generation method, which will not be repeated here. Each module in the above-mentioned image generating device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

图25为一个实施例中电子设备的内部结构示意图。该电子设备可以是手机、平板电脑、笔记本电脑、台式电脑、PDA(Personal Digital Assistant，个人数字助理)、POS(Point of Sales，销售终端)、车载电脑、穿戴式设备等任意终端设备。该电子设备包括通过***总线连接的处理器和存储器。其中，该处理器可以包括一个或多个处理单元。处理器可为CPU(Central Processing Unit,中央处理单元)或DSP(Digital Signal Processing，数字信号处理器)等。存储器可包括非易失性存储介质及内存储器。非易失性存储介质存储有操作***和计算机程序。该计算机程序可被处理器所执行，以用于实现以下各个实施例所提供的一种图像生成方法。内存储器为非易失性存储介质中的操作***计算机程序提供高速缓存的运行环境。Fig. 25 is a schematic diagram of the internal structure of an electronic device in one embodiment. The electronic device can be any terminal device such as mobile phone, tablet computer, notebook computer, desktop computer, PDA (Personal Digital Assistant, personal digital assistant), POS (Point of Sales, sales terminal), vehicle-mounted computer, wearable device, etc. The electronic device includes a processor and memory connected by a system bus. Wherein, the processor may include one or more processing units. The processor can be a CPU (Central Processing Unit, central processing unit) or a DSP (Digital Signal Processing, digital signal processor), etc. The memory may include non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs. The computer program can be executed by a processor to implement an image generation method provided in the following embodiments. The internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium.

本申请实施例中提供的图像生成装置中的各个模块的实现可为计算机程序的形式。该计算机程序可在终端或服务器上运行。该计算机程序构成的程序模块可存储在电子设备的存储器上。该计算机程序被处理器执行时，实现本申请实施例中所描述方法的操作。The implementation of each module in the image generation device provided in the embodiment of the present application may be in the form of a computer program. The computer program can run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the electronic device. When the computer program is executed by the processor, the operations of the methods described in the embodiments of the present application are realized.

本申请实施例还提供了一种计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质，当所述计算机可执行指令被一个或多个处理器执行时，使得所述处理器执行图像生成方法的操作。The embodiment of the present application also provides a computer-readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image generation method.

本申请实施例还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行图像生成方法。The embodiment of the present application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the image generation method.

本申请所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。非易失性存储器可包括ROM(Read-Only Memory，只读存储器)、PROM(Programmable Read-only Memory，可编程只读存储器)、EPROM(Erasable Programmable Read-Only Memory，可擦除可编程只读存储器)、EEPROM(Electrically Erasable Programmable Read-only Memory，电可擦除可编程只读存储器)或闪存。易失性存储器可包括RAM(Random Access Memory，随机存取存储器)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如SRAM(Static Random Access Memory，静态随机存取存储器)、DRAM(Dynamic Random Access Memory，动态随机存取存储器)、SDRAM(Synchronous Dynamic Random Access Memory，同步动态随机存取存储器)、双数据率DDR SDRAM(Double Data Rate Synchronous Dynamic Random Access memory，双数据率同步动态随机存取存储器)、ESDRAM(Enhanced Synchronous Dynamic Random Access memory，增强型同步动态随机存取存储器)、SLDRAM(Sync Link Dynamic Random Access Memory，同步链路动态随机存取存储器)、RDRAM(Rambus Dynamic Random Access Memory，总线式动态随机存储器)、DRDRAM(Direct Rambus Dynamic Random Access Memory，接口动态随机存储器)。Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Non-volatile memory can include ROM (Read-Only Memory, read-only memory), PROM (Programmable Read-only Memory, programmable read-only memory), EPROM (Erasable Programmable Read-Only Memory, erasable programmable read-only memory) Memory), EEPROM (Electrically Erasable Programmable Read-only Memory, Electrically Erasable Programmable Read-only Memory) or flash memory. Volatile memory can include RAM (Random Access Memory, Random Access Memory), which is used as external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as SRAM (Static Random Access Memory, static random access memory), DRAM (Dynamic Random Access Memory, dynamic random access memory), SDRAM (Synchronous Dynamic Random Access Memory , synchronous dynamic random access memory), double data rate DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access memory, double data rate synchronous dynamic random access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory, enhanced synchronous dynamic random access memory access memory), SLDRAM (Sync Link Dynamic Random Access Memory, synchronous link dynamic random access memory), RDRAM (Rambus Dynamic Random Access Memory, bus dynamic random access memory), DRDRAM (Direct Rambus Dynamic Random Access Memory, interface dynamic random access memory) memory).

以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express several implementation modes of the present application, and its description is relatively specific and detailed, but it should not be interpreted as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims

一种图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。An image sensor, characterized in that the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each filter The sheet group only includes color filters of two colors and a panchromatic filter, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; the panchromatic filter The light sheets and the color filters are alternately arranged on each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, each The color filter includes color sub-filters in N rows and N columns, the color sub-filters in N rows and N columns are the same color as the color filter, and N is a positive integer; Each pixel in the pixel array is arranged corresponding to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signals.
根据权利要求1所述的图像传感器，其特征在于，所述每一行和所述每一列上均包括每种颜色的彩色滤光片。The image sensor according to claim 1, wherein each row and each column includes a color filter of each color.
根据权利要求1所述的图像传感器，其特征在于，所述最小重复单元包括2个第一滤光片组和2个第二滤光片组，所述2个第一滤光片组在所述最小重复单元的对角线上排布，所述2个第二滤光片组在所述最小重复单元的反对角线上排布。The image sensor according to claim 1, wherein the minimum repeating unit comprises 2 first filter groups and 2 second filter groups, and the 2 first filter groups are in the Arranged on the diagonal of the minimum repeating unit, and the two second filter groups are arranged on the anti-diagonal of the minimum repeating unit.
根据权利要求1至3中任意一项所述的图像传感器，其特征在于，所述每个滤光片组的彩色滤光片排布在对应滤光片组的对角线及平行于所述对角线的方向上。The image sensor according to any one of claims 1 to 3, wherein the color filters of each filter group are arranged on the diagonal of the corresponding filter group and parallel to the in the direction of the diagonal.
根据权利要求1至3中任意一项所述的图像传感器，其特征在于，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，所述子单元中的所述彩色滤光片排布在所述子单元的对角线，所述子单元中的所述全色滤光片排布在所述子单元的反对角线。The image sensor according to any one of claims 1 to 3, wherein each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the subunits The color filters in the subunits are arranged on the diagonals of the subunits, and the panchromatic filters in the subunits are arranged on the opposite diagonals of the subunits.
根据权利要求1所述的图像传感器，其特征在于，所述彩色滤光片包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片；所述最小重复单元至少包括第一滤光片组和第二滤光片组，所述第一滤光片组包括第一颜色滤光片和第二颜色滤光片，所述第二滤光片组包括第二颜色滤光片和第三颜色滤光片。The image sensor according to claim 1, wherein the color filter includes a first color filter, a second color filter, and a third color filter; the minimum repeating unit includes at least the first color filter A filter group and a second filter group, the first filter group includes a first color filter and a second color filter, and the second filter group includes a second color filter sheet and a third color filter.
根据权利要求6所述的图像传感器，其特征在于，所述第一滤光片组中第二颜色滤光片设置在所述第一滤光片组的对角线上；所述第二滤光片组中第二颜色滤光片设置在所述第二滤光片组的对角线上。The image sensor according to claim 6, wherein the second color filter in the first filter group is arranged on a diagonal line of the first filter group; the second filter The second color filter in the light sheet group is arranged on the diagonal of the second filter group.
根据权利要求7所述的图像传感器，其特征在于，所述第一滤光片组中的第一颜色滤光片设置在平行于所述第一滤光片组的对角线的方向上；所述第二滤光片组中第三颜色滤光片设置在平行于所述第二滤光片组的对角线的方向上。The image sensor according to claim 7, wherein the first color filter in the first filter group is arranged in a direction parallel to the diagonal of the first filter group; The third color filter in the second filter group is arranged in a direction parallel to the diagonal of the second filter group.
根据权利要求1所述的图像传感器，其特征在于，所述N为2，所述最小重复单元包括16行16列256个子滤光片，排布方式为：The image sensor according to claim 1, wherein the N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, arranged in the following manner:

或者or

或者or

或者or

其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。Among them, w represents a panchromatic sub-filter, and a, b, and c all represent color sub-filters.
根据权利要求1所述的图像传感器，其特征在于，所述N为1，所述最小重复单元包括8行8列64个滤光片，排布方式为：The image sensor according to claim 1, wherein the N is 1, and the minimum repeating unit includes 64 optical filters in 8 rows and 8 columns, arranged in the following manner:

或者or

或者or

或者or

其中，w表示全色滤光片，a、b和c均表示彩色滤光片。Among them, w represents a panchromatic filter, and a, b and c all represent color filters.
一种摄像模组，其特征在于，所述摄像模组包括镜头和权利要求1-10中任一项所述的图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号。A camera module, characterized in that the camera module includes a lens and the image sensor according to any one of claims 1-10; the image sensor is used to receive light passing through the lens, the The pixels generate electrical signals in response to the light.
一种电子设备，其特征在于，包括：An electronic device, characterized in that it comprises:

权利要求11所述的摄像模组；及The camera module as claimed in claim 11; and

壳体，所述摄像模组设置在所述壳体上。A casing, the camera module is arranged on the casing.
一种图像生成方法，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；An image generation method applied to an image sensor, characterized in that the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filters Each filter group includes only 2 colors of color filters and panchromatic filters, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter ; The panchromatic filter and the color filter are alternately arranged on each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic elements Each of the color filters includes color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns are the same color as the color filter, and the N It is a positive integer greater than or equal to 2; each pixel in the pixel array corresponds to the sub-filter of the filter array, and the pixel array is configured to receive light passing through the filter array light to generate electrical signals;

所述方法包括：The methods include:

在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；及In the full-resolution mode, the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full-resolution panchromatic pixel value, and each of the color filters is The color pixel corresponding to the color sub-filter reads out the full resolution color pixel value; and

基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。A full resolution target image is generated based on each of said full resolution panchromatic pixel values and each of said full resolution color pixel values.
根据权利要求13所述的方法，其特征在于，所述方法还包括：The method according to claim 13, further comprising:

在第一分辨率模式下，将每个所述全色滤光片中各个全色子滤光片对应的全色像素合并读出第一全色像素值，以及将每个所述彩色滤光片中各个彩色子滤光片对应的彩色像素合并读出第一彩色像素值；所述第一分辨率模式对应的分辨率小于所述全分辨率模式对应的分辨率；及In the first resolution mode, the panchromatic pixels corresponding to each panchromatic sub-filter in each of the panchromatic filters are combined to read out the first panchromatic pixel value, and each of the color filters The color pixels corresponding to each color sub-filter in the chip are combined to read out the first color pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode; and

基于各个所述第一全色像素值和各个所述第一彩色像素值，生成第一目标图像。A first target image is generated based on each of the first panchromatic pixel values and each of the first color pixel values.
根据权利要求14所述的方法，其特征在于，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，所述子单元中的所述彩色滤光片排布在所述子单元的对角线上，所述子单元中的所述全色滤光片排布在所述子单元的反对角线上；The method according to claim 14, wherein each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filter in the subunit The slices are arranged on the diagonal of the sub-unit, and the panchromatic filters in the sub-unit are arranged on the anti-diagonal of the sub-unit;

所述生成第一目标图像之后，还包括：After the first target image is generated, it also includes:

在第二分辨率模式下，将所述第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并基于各个所述第二全色像素值生成第一全色图像；所述第二分辨率模式对应的分辨率小于所述第一分辨率模式对应的分辨率；In the second resolution mode, a plurality of first panchromatic pixel values corresponding to each subunit in the first target image are combined to read out a second panchromatic pixel value, and based on each of the second panchromatic pixel values The pixel value generates a first full-color image; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode;

将所述第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，并基于各个所述第二彩色像素值生成第一彩色图像；及Combining multiple first color pixel values corresponding to the same color in each subunit in the first target image to read out second color pixel values, and generating a first color image based on each of the second color pixel values; and

基于所述第一全色图像和所述第一彩色图像，生成第二目标图像。A second target image is generated based on the first panchromatic image and the first color image.
根据权利要求15所述的方法，其特征在于，所述基于所述第一全色图像和所述第一彩色图像，生成第二目标图像，包括：The method according to claim 15, wherein said generating a second target image based on said first panchromatic image and said first color image comprises:

将所述第一全色图像中每一行第二全色像素值与所述第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者Arranging each row of second panchromatic pixel values in the first panchromatic image alternately with each row of second color pixel values in the first color image to generate a second target image; or

将所述第一全色图像中每一列第二全色像素值与所述第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。Arranging each column of second panchromatic pixel values in the first panchromatic image alternately with each column of second color pixel values in the first color image to generate a second target image.
根据权利要求15所述的方法，其特征在于，所述方法还包括：The method according to claim 15, further comprising:

在第三分辨率模式下，将所述第一全色图像中对应在同一滤光片组中的多个对角线上的第二全色像素值合并读出对角线的第三全色像素值，并基于各个对角线的第三全色像素值生成对角线的第二全色图像；以及，将所述第一全色图像中对应在同一滤光片组中的多个反对角线上的第二全色像素值合并读出反对角线的第三全色像素值，并基于各个反对角线的第三全色像素值生成反对角线的第二全色图像；所述第三分辨率模式对应的分辨率小于所述第二分辨率模式对应的分辨率；In the third resolution mode, combine the second panchromatic pixel values corresponding to multiple diagonals in the same filter set in the first panchromatic image to read out the third panchromatic diagonal pixel value, and generate a diagonal second panchromatic image based on the third panchromatic pixel value of each diagonal; Combining the second panchromatic pixel values on the diagonal lines to read out the third panchromatic pixel values of the anti-diagonal lines, and generating a second panchromatic image of the anti-diagonal lines based on the third panchromatic pixel values of the respective anti-diagonal lines; The resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode;

将所述第一彩色图像中对应在同一滤光片组中的多个对角线上的相同颜色的第二彩色像素值合并读出对角线的第三彩色像素值，并基于各个对角线的第三彩色像素值生成对角线的第二彩色图像；以及，将所述第一彩色图像中对应在同一滤光片组中的多个反对角线上的相同颜色的第二彩色像素值合并读出反对角线的第三彩色像素值，并基于各个反对角线的第三彩色像素值生成反对角线的第二彩色图像；及Combining the second color pixel values of the same color corresponding to multiple diagonal lines in the same filter group in the first color image to read out the third color pixel values of the diagonal lines, and based on each diagonal The second color image of the diagonal line is generated by the third color pixel value of the line; Combining and reading out the third color pixel values of the anti-diagonal lines, and generating a second color image of the anti-diagonal lines based on the third color pixel values of the anti-diagonal lines; and

基于所述对角线的第二全色图像、所述反对角线的第二全色图像、所述对角线的第二彩色图像和所述反对角线的第二彩色图像，生成第三目标图像。Based on the second panchromatic image of the diagonal, the second panchromatic image of the anti-diagonal, the second color image of the diagonal, and the second color image of the anti-diagonal, a third target image.
根据权利要求17所述的方法，其特征在于，所述基于所述对角线的第二全色图像、所述反对角线的第二全色图像、所述对角线的第二彩色图像和所述反对角线的第二彩色图像，生成第三目标图像，包括：The method according to claim 17, wherein the second panchromatic image based on the diagonal, the second panchromatic image of the anti-diagonal, and the second color image of the diagonal and the second color image of the anti-diagonal to generate a third target image comprising:

将所述对角线的第二全色图像中每一行所述对角线的第三全色像素值、所述反对角线的第二全色图像中每一行所述反对角线的第三全色像素值、所述对角线的第二彩色图像中每一行所述对角线的第三彩色像素值和所述反对角线的第二彩色图像中每一行所述反对角线的第三彩色像素值相间排布，生成第三目标图像；或者The third panchromatic pixel value of each row of the diagonal in the second panchromatic image of the diagonal, the third panchromatic pixel value of each row of the anti-diagonal in the second panchromatic image of the anti-diagonal Panchromatic pixel values, third color pixel values of each row of the diagonal in the second color image of the diagonal, and third color pixel values of each row of the anti-diagonal in the second color image of the anti-diagonal The pixel values of the three colors are arranged alternately to generate a third target image; or

将所述对角线的第二全色图像中每一列所述对角线的第三全色像素值、所述反对角线的第二全色图像中每一列所述反对角线的第三全色像素值、所述对角线的第二彩色图像中每一列所述对角线的第三彩色像素值和所述反对角线的第二彩色图像中每一列所述反对角线的第三彩色像素值相间排布，生成第三目标图像。The third panchromatic pixel value of each column of the diagonal line in the second panchromatic image of the diagonal line, the third panchromatic pixel value of each column of the anti-diagonal line in the second panchromatic image of the anti-diagonal line Panchromatic pixel values, third color pixel values of each column of the diagonal in the second color image of the diagonal, and third color pixel values of each column of the anti-diagonal in the second color image of the anti-diagonal The pixel values of the three colors are arranged alternately to generate the third target image.
根据权利要求13所述的方法，其特征在于，每个滤光片组包括多个子单元，每个子单元包括彩色滤光片和全色滤光片，所述子单元中的所述彩色滤光片排布在所述子单元的对角线，所述子单元中的所述全色滤光片排布在所述子单元的反对角线；The method according to claim 13, wherein each filter group includes a plurality of subunits, each subunit includes a color filter and a panchromatic filter, and the color filter in the subunit The slices are arranged on the diagonal of the subunit, and the panchromatic filters in the subunit are arranged on the anti-diagonal of the subunit;

所述方法还包括：The method also includes:

在第二分辨率模式下，将每个子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第四全色像素值，并基于各个所述第四全色像素值生成第三全色图像；In the second resolution mode, the panchromatic pixels corresponding to the respective panchromatic sub-filters of the plurality of panchromatic filters in each subunit are combined to read out the fourth panchromatic pixel value, and based on each of the fourth panchromatic pixel values to generate a third panchromatic image;

将每个子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第四彩色像素值，并基于各个所述第四彩色像素值生成第三彩色图像；及Combining the color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in each subunit to read out a fourth color pixel value, and generating a third color image based on each of the fourth color pixel values; and

基于所述第三全色图像和所述第三彩色图像，生成第四目标图像。A fourth target image is generated based on the third panchromatic image and the third color image.
根据权利要求19所述的方法，其特征在于，所述基于所述第三全色图像和所述第三彩色图像，生成第四目标图像，包括：The method according to claim 19, wherein said generating a fourth target image based on said third panchromatic image and said third color image comprises:

将所述第三全色图像中每一行第四全色像素值与所述第三彩色图像中每一行第四彩色像素值相间排布，生成第四目标图像；或者arranging each row of fourth panchromatic pixel values in the third panchromatic image alternately with each row of fourth color pixel values in the third color image to generate a fourth target image; or

将所述第三全色图像中每一列第四全色像素值与所述第三彩色图像中每一列第四彩色像素值相间排布，生成第四目标图像。Arranging each column of fourth panchromatic pixel values in the third panchromatic image alternately with each column of fourth color pixel values in the third color image to generate a fourth target image.
根据权利要求13所述的方法，其特征在于，每个滤光片组的彩色滤光片排布在对应滤光片组的对角线及平行于所述对角线的方向上，每个滤光片组的全色滤光片排布在对应滤光片组的反对角线及平行于所述反对角线的方向上；The method according to claim 13, wherein the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, each The panchromatic filters of the filter group are arranged on the anti-diagonal line of the corresponding filter group and in a direction parallel to the anti-diagonal line;

所述方法还包括：The method also includes:

在第三分辨率模式下，将每个滤光片组中反对角线上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出反对角线的第五全色像素值，并基于各个反对角线的第五全色像素值生成反对角线的第四全色图像；以及，将每个滤光片组中平行于所述反对角线的方向上的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出平行反对角线的第五全色像素值，并基于各个平行反对角线的第五全色像素值生成对角线的第四全色图像；In the third resolution mode, the panchromatic pixels corresponding to the panchromatic sub-filters of the multiple panchromatic filters on the anti-diagonal line in each filter group are merged to read out the fifth image of the anti-diagonal line. panchromatic pixel values, and generate a fourth panchromatic image of the anti-diagonal based on the fifth panchromatic pixel value of each anti-diagonal; Combining the panchromatic pixels corresponding to each panchromatic sub-filter of multiple panchromatic filters to read out the fifth panchromatic pixel value of parallel anti-diagonal lines, and generating the fifth panchromatic pixel value based on each parallel anti-diagonal line the fourth panchromatic image of the diagonal;

将每个滤光片组中对角线上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出对角线的第五彩色像素值，并基于各个对角线的第五彩色像素值生成对角线的第四彩色图像；以及，将每个滤光片组中平行于所述对角线的方向上的多个彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出平行对角线的第五彩色像素值，并基于各个平行对角线的第五彩色像素值生成反对角线的第四彩色图像；及Combining the color pixels corresponding to each color sub-filter of a plurality of color filters on the diagonal in each filter group to read out the fifth color pixel value of the diagonal, and based on the The fifth color pixel value generates a fourth color image of the diagonal; and, corresponding color sub-filters of a plurality of color filters in a direction parallel to the diagonal in each filter group Combining and reading out the fifth color pixel values of the parallel diagonal lines, and generating the fourth color image of the anti-diagonal lines based on the fifth color pixel values of each parallel diagonal line; and

基于所述反对角线的第四全色图像、所述对角线的第四全色图像、所述对角线的第四彩色图像和所述反对角线的第四彩色图像，生成第五目标图像。Based on the fourth panchromatic image of the anti-diagonal, the fourth panchromatic image of the diagonal, the fourth color image of the diagonal, and the fourth color image of the anti-diagonal, a fifth target image.
根据权利要求21所述的方法，其特征在于，所述基于所述反对角线的第四全色图像、所述对角线的第四全色图像、所述对角线的第四彩色图像和所述反对角线的第四彩色图像，生成第五目标图像，包括：The method according to claim 21, wherein the fourth panchromatic image based on the anti-diagonal, the fourth panchromatic image of the diagonal, and the fourth color image of the diagonal and the anti-diagonal fourth color image to generate a fifth target image comprising:

将所述反对角线的第四全色图像中每一行所述反对角线的第五全色像素值、所述对角线的第四全色图像中每一行所述平行反对角线的第五全色像素值、所述对角线的第四彩色图像中每一行所述对角线的第五彩色像素值和所述反对角线的第四彩色图像中每一行所述平行对角线的第五彩色像素值相间排布，生成第五目标图像；或者，The fifth panchromatic pixel value of each row of the anti-diagonal in the fourth panchromatic image of the diagonal, the fifth panchromatic pixel value of each row of the parallel anti-diagonal in the fourth panchromatic image of the diagonal Five panchromatic pixel values, a fifth color pixel value for each row of the diagonal in the fourth color image of the diagonal and a fifth color pixel value for each row of the anti-diagonal fourth color image of the parallel diagonal The fifth color pixel values of are arranged alternately to generate the fifth target image; or,

将所述反对角线的第四全色图像中每一列所述反对角线的第五全色像素值、所述对角线的第四全色图像中每一列所述平行反对角线的第五全色像素值、所述对角线的第四彩色图像中每一列所述对角线的第五彩色像素值和所述反对角线的第四彩色图像中每一列所述平行对角线的第五彩色像素值相间排布，生成第五目标图像。The fifth panchromatic pixel value of each column of the anti-diagonal line in the fourth panchromatic image of the anti-diagonal line, the fifth panchromatic pixel value of each column of the parallel anti-diagonal line in the fourth panchromatic image of the diagonal line five panchromatic pixel values, a fifth color pixel value for each column of said diagonal in said fourth color image of said diagonal, and said parallel diagonal for each column of said anti-diagonal fourth color image The pixel values of the fifth color are arranged alternately to generate the fifth target image.
一种图像生成装置，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组中仅包括2种颜色的彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；所述全色滤光片与所述彩色滤光片在所述最小重复单元的每一行和每一列上交替排布；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；An image generating device applied to an image sensor, characterized in that the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filters Each filter group includes only 2 colors of color filters and panchromatic filters, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter ; The panchromatic filter and the color filter are alternately arranged on each row and each column of the minimum repeating unit; each of the panchromatic filters includes N rows and N columns of panchromatic elements Each of the color filters includes color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns are the same color as the color filter, and the N It is a positive integer greater than or equal to 2; each pixel in the pixel array corresponds to the sub-filter of the filter array, and the pixel array is configured to receive light passing through the filter array light to generate electrical signals;

所述装置包括：The devices include:

读出模块，用于在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素；及The readout module is used to read out the full-resolution panchromatic pixels from the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read the full-resolution panchromatic pixels of the color filter The color pixel corresponding to each color sub-filter in the light sheet reads out the full resolution color pixel; and

图像生成模块，用于基于各个所述全分辨率全色像素和各个所述全分辨率彩色像素，生成全分辨率目标图像。An image generating module, configured to generate a full-resolution target image based on each of the full-resolution panchromatic pixels and each of the full-resolution color pixels.
一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，其特征在于，所述计算机程序被所述处理器执行时，使得所述处理器执行如权利要求13至22中任一项所述的图像生成方法的操作。An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, wherein, when the computer program is executed by the processor, the processor can execute any of the following claims 13-22 An operation of the described image generation method.
一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求13至22中任一项所述的方法的操作。A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the operation of the method according to any one of claims 13 to 22 is realized.
一种计算机程序产品，包括计算机程序，其特征在于，该计算机程序被处理器执行时实现如权利要求13至22中任一项所述的方法的操作。A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the operation of the method according to any one of claims 13 to 22 is realized.