WO2021051354A1

WO2021051354A1 - Hdr image generating method, filter array, image sensor, image processing chip, and photographing device

Info

Publication number: WO2021051354A1
Application number: PCT/CN2019/106723
Authority: WO
Inventors: 徐彬; 周游; 张树汉
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-19
Filing date: 2019-09-19
Publication date: 2021-03-25
Also published as: CN112204948A

Abstract

An HDR image generating method, a filter array (31, 812), an image sensor (81), an image processing chip (82), and a photographing device. By means of the HDR image generating method, an HDR image can be obtained in real time during a photographing process. The method comprises: receiving pixel values of pixel points of an image after filtering processing acquired by the image sensor (81), wherein the image sensor (81) comprises a photosensitive pixel point array (32, 811) and the filter array (31, 812) covering the photosensitive pixel point array (32, 811), the filter array (31, 812) comprises a plurality of minimal repeating units arranged in an array form, the minimal repeating unit comprises at least two types of filter units, different types of filter units have different light transmittances, and a single filter unit corresponds to a single photosensitive pixel point (S402); determining a preset region from the image after filtering processing, and taking pixel points in the preset region as sub-pixel points, and synthesizing a pixel value of a single pixel point of the HDR image on the basis of pixel values of the sub-pixel points (S404).

Description

HDR图像生成方法、滤光片阵列、图像传感器、图像处理芯片以及摄像装置HDR image generation method, filter array, image sensor, image processing chip and imaging device

技术领域Technical field

本申请涉及图像处理技术领域，尤其涉及一种HDR图像生成方法、滤光片阵列、图像传感器、图像处理芯片以及摄像装置。This application relates to the field of image processing technology, and in particular to an HDR image generation method, a filter array, an image sensor, an image processing chip, and an imaging device.

背景技术Background technique

HDR(High Dynamic Range Imaging，高动态范围成像，简称HDRI或HDR)技术是用来实现比普通数字图像技术更大曝光动态范围，也就是更大的明暗差别的一种技术。通过HDR技术拍摄得到的HDR图像色彩更加鲜明，更加接近人眼的观看效果，因而被广泛应用各种摄像装置中。HDR (High Dynamic Range Imaging, HDRI or HDR for short) technology is a technology used to achieve a greater dynamic range of exposure than ordinary digital imaging technology, that is, a greater difference between light and dark. The HDR image captured by HDR technology has more vivid colors and is closer to the viewing effect of the human eye, so it is widely used in various camera devices.

目前在拍摄HDR图像时，有的技术通过采集多帧不同曝光时长的图像，然后再进行后期的融合处理，这种方式需要采集多张图像，拍摄时间较长，且还需对图像进一步融合处理，比较繁琐。也有的技术通过控制不同子像素曝光时长的方式来得到实时的HDR图像，但是这种技术只适用于卷帘快门的图像传感器，对于全局快门的图像传感器则不适用。然而，对于机器人、无人机上等本身会有大姿态运动的产品，其图像传感器大多数采用全局快门，因而这种技术在这些产品上不适用。因此，有必要对HDR技术加以改进，提供一种适用于全局快门产品的实时HDR成像技术。At present, when shooting HDR images, some technologies collect multiple frames of images with different exposure durations, and then perform post-fusion processing. This method requires multiple images to be collected, the shooting time is longer, and the images need to be further fused. , More cumbersome. There are also technologies that obtain real-time HDR images by controlling the exposure time of different sub-pixels, but this technology is only applicable to image sensors with rolling shutters, and not applicable to image sensors with global shutters. However, for robots, unmanned aerial vehicles and other products that have large gestures, most of their image sensors use global shutters, so this technology is not applicable to these products. Therefore, it is necessary to improve HDR technology and provide a real-time HDR imaging technology suitable for global shutter products.

发明内容Summary of the invention

有鉴于此，本申请提供一种HDR图像生成方法、滤光片阵列、图像传感器、图像处理芯片以及摄像装置。In view of this, the present application provides an HDR image generation method, a filter array, an image sensor, an image processing chip, and an imaging device.

根据本申请实施例的第一方面，提供一种HDR图像生成方法，所述方法包括：According to a first aspect of the embodiments of the present application, there is provided a method for generating an HDR image, the method including:

接收图像传感器采集的滤光处理后的图像的各像素点的像素值，其中，所述图像传感器包括感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括至少两类滤光片单元，不同类别的所述滤光片单元的透光率不同，单个所述滤光片单元对应单个所述感光像素点；Receive the pixel value of each pixel of the filtered image collected by the image sensor, wherein the image sensor includes an array of photosensitive pixel points and a filter array covering the array of photosensitive pixel points, and the filter The array includes a plurality of minimum repeating units arranged in an array form. The minimum repeating unit includes at least two types of filter units. The light transmittances of the filter units of different types are different, and a single filter unit Corresponding to a single said photosensitive pixel;

从所述滤光处理后的图像中确定一预设区域，将所述预设区域中的所述像素点作为子像素点，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。Determine a preset area from the filtered image, use the pixels in the preset area as sub-pixels, and synthesize a single HDR image based on the pixel values of the sub-pixels The pixel value of the pixel.

根据本申请实施例的第二方面，提供一种用于生成HDR图像的滤光片阵列，所述滤光片阵列包括：According to a second aspect of the embodiments of the present application, there is provided a filter array for generating an HDR image, the filter array including:

多个以阵列形态排布的最小重复单元；Multiple smallest repeating units arranged in an array;

所述最小重复单元包括至少两类滤光片单元，不同类别的滤光片单元的透光率不同。The minimum repeating unit includes at least two types of filter units, and different types of filter units have different light transmittances.

根据本申请实施例的第三方面，提供一种图像传感器，所述图像传感器包括：According to a third aspect of the embodiments of the present application, there is provided an image sensor, the image sensor including:

感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括至少两类滤光片，不同类别的滤光片的透光率不同，单个所述滤光片单元对应单个所述感光像素点。A photosensitive pixel point array and a filter array covering the photosensitive pixel point array, the filter array including a plurality of minimum repeating units arranged in an array, the minimum repeating unit including at least two types of filters Different types of filters have different light transmittances, and a single filter unit corresponds to a single photosensitive pixel.

根据本申请实施例的第四方面，提供一种图像处理芯片，所述图像处理芯片包括处理器、存储器和存储在所述存储器上的计算机程序，所述处理器执行所述计算程序本说明书实施例中任意一项所述的方法。According to a fourth aspect of the embodiments of the present application, an image processing chip is provided. The image processing chip includes a processor, a memory, and a computer program stored on the memory. The processor executes the calculation program. The method described in any of the examples.

根据本申请实施例的第五方面，提供一种摄像装置，包括图像传感器和图像处理芯片，所述图像传感器包括感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括少两类滤光片，不同类别的滤光片的透光率不同，单个所述滤光片单元对应单个所述感光像素点；According to a fifth aspect of the embodiments of the present application, there is provided an imaging device, including an image sensor and an image processing chip, the image sensor including a photosensitive pixel point array and a filter array covering the photosensitive pixel point array, the The filter array includes a plurality of minimum repeating units arranged in an array form. The minimum repeating unit includes at least two types of filters. Different types of filters have different light transmittances, and a single filter unit corresponds to A single said photosensitive pixel;

所述图像传感器采集到滤光处理后的图像的各像素点的像素值后，发送给所述图像处理芯片，所述图像处理芯片从所述滤光处理后的图像中确定一预设区域，将所述预设区域中的感光像素点作为子像素点，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。After the image sensor collects the pixel value of each pixel of the filtered image and sends it to the image processing chip, the image processing chip determines a preset area from the filtered image, Taking the photosensitive pixels in the preset area as sub-pixels, and synthesizing the pixel values of individual pixels of the HDR image based on the pixel values of the sub-pixels.

根据本申请实施例的第六方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现本说明书实施例任意一项所述的方法。According to a sixth aspect of the embodiments of the present application, there is provided a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method described in any one of the embodiments of this specification is implemented.

应用本申请的方案，通过在图像传感器的感光像素点阵列上添加滤光片阵列，采用不同透光率的滤光片单元对到达感光像素点的光线进行过滤处理，使得各像素点的像素值呈现更大的明暗差异，然后以过滤处理后的图像上的预设区域内的像素点作为子像素点，合成HDR图像的单个像素点，以得到曝光范围更大的HDR图像。通过这种方式，可以在摄像过程中实时地得到HDR图像，无需进行后期的图像融合处理，且既适用于卷帘快门传感器，也适用于全局快门传感器。Applying the solution of this application, by adding a filter array to the photosensitive pixel array of the image sensor, filter units with different light transmittances are used to filter the light reaching the photosensitive pixel, so that the pixel value of each pixel is It presents a larger light and dark difference, and then uses the pixels in the preset area on the filtered image as sub-pixels to synthesize individual pixels of the HDR image to obtain an HDR image with a larger exposure range. In this way, HDR images can be obtained in real time during the shooting process, without the need for post-image fusion processing, and it is suitable for both the rolling shutter sensor and the global shutter sensor.

附图说明Description of the drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

图1a是本申请一示例性实施例中的一种添加拜耳阵列的图像传感器的结构示意图。Fig. 1a is a schematic structural diagram of an image sensor with a Bayer array added in an exemplary embodiment of the present application.

图1b是本申请一示例性实施例中的一种拜耳阵列的结构示意图。Fig. 1b is a schematic structural diagram of a Bayer array in an exemplary embodiment of the present application.

图2是本申请一示例性实施例中的一种滤光片阵列的结构示意图。Fig. 2 is a schematic structural diagram of a filter array in an exemplary embodiment of the present application.

图3是本申请一示例性实施例中的一种图像的结构示意图。Fig. 3 is a schematic diagram of an image structure in an exemplary embodiment of the present application.

图4是本申请一示例性实施例中的一种HDR图像生成方法的流程图。Fig. 4 is a flowchart of a method for generating an HDR image in an exemplary embodiment of the present application.

图5a-5d是本申请一示例性实施例中的一种滤光片阵列的结构示意图。5a-5d are schematic diagrams of the structure of a filter array in an exemplary embodiment of the present application.

图6a-6c是本申请一示例性实施例中的一种多个子像素点合成HDR图像的单个像素点的示意图。Figures 6a-6c are schematic diagrams of a single pixel of an HDR image synthesized by multiple sub-pixels in an exemplary embodiment of the present application.

图7是本申请一示例性实施例中的一种图像处理芯片的结构示意图。Fig. 7 is a schematic structural diagram of an image processing chip in an exemplary embodiment of the present application.

图8是本申请一示例性实施例中的一种摄像装置的结构示意图。Fig. 8 is a schematic structural diagram of a camera device in an exemplary embodiment of the present application.

具体实施方式detailed description

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

采用HDR技术拍摄得到的HDR图像相比于普通的图像具有更大的曝光动态范围，图像中高光区域和暗部区域显示的细节更加清晰，明暗对比度较大，因而得到的图像色彩更加鲜明，效果更好。目前HDR技术已广泛应用于各种摄像产品。有的技术是在图像拍摄过程中采集多帧不相同曝光率的图像，然后后期再通过软件对多张不同曝光率的图像进行合成，得到HDR图像，这种方式在拍摄过程中由于一次要采集多帧图像，拍摄时间较长，且无法实时获取图像，需要多采集到的图像经过后期的图像融合处理才能得到HDR图像。还有的技术是在摄像过程中通过控制像素点的曝光时长以得到实时的HDR图像。例如，索尼的Quad Bayer Coding HDR(实现高动态范围)技术是一种实时获取HDR图像的技术，这种技术的原理是在采集图像时，控制不同子像素的曝光时长，然后再通过曝光时长不同的子像素合成HDR图像。但是这种技术只适用于卷帘快门的图像传感器，对于全局快门的图像传感器并不适用。然后，对于无人机、机器人等移动的产品，多采用全局快门，因此，有必要提供一种HDR技术，使其也可以适用这些产品。Compared with ordinary images, HDR images captured by HDR technology have a larger dynamic range of exposure. The highlights and dark areas in the image show clearer details, and the contrast between light and dark is larger. Therefore, the color of the image obtained is more vivid and the effect is better. Great. At present, HDR technology has been widely used in various camera products. Some technology is to collect multiple frames of images with different exposure rates during the image shooting process, and then use software to synthesize multiple images with different exposure rates in the later stage to obtain HDR images. This method is due to the collection of images during the shooting process. For multi-frame images, the shooting time is long and the images cannot be obtained in real time. It is necessary to obtain HDR images through post-image fusion processing of the multi-collected images. Another technique is to obtain real-time HDR images by controlling the exposure time of pixels during the shooting process. For example, Sony’s Quad Bayer Coding HDR (Realization of High Dynamic Range) technology is a technology for real-time acquisition of HDR images. The principle of this technology is to control the exposure time of different sub-pixels when capturing images, and then use different exposure time. The sub-pixels of the composite HDR image. However, this technique is only applicable to image sensors with rolling shutters, and is not applicable to image sensors with global shutters. Then, for mobile products such as drones and robots, global shutters are often used. Therefore, it is necessary to provide an HDR technology that can also be applied to these products.

为了便于理解本申请的各实施例，先简单介绍一下拜耳阵列。图像传感器采集的图像的像素点通常只记录各像素点的光强度，而没法记录光的颜色，因而采集的图像是黑白的。为了得到彩色的图像，需要在图像传感器上加一个微透镜层，镜头中的光线透过微透镜层后到达图像传感器的感光素点阵列，如图1a所示。这个微透镜层通常是由红、绿、蓝三种颜色的彩色滤光片构成的一个阵列，即拜耳阵列，如图1b所示。通过在图像传感器的每个子像素前面添加对应颜色的彩色滤光片，可以使得不同的子像素感应对应的颜色，最后再通过四个子像素点的像素值合成彩色图片的一个像素点的像素值，以及通过猜色算法确定彩色图片的各像素点的颜色，以得到最终的彩色图片。In order to facilitate the understanding of the various embodiments of the present application, first briefly introduce the Bayer array. The pixels of the image collected by the image sensor usually only record the light intensity of each pixel, but cannot record the color of the light, so the collected image is black and white. In order to obtain a color image, a microlens layer needs to be added to the image sensor. The light in the lens passes through the microlens layer and reaches the photosensitive pixel array of the image sensor, as shown in Figure 1a. This microlens layer is usually an array composed of three color filters of red, green, and blue, that is, the Bayer array, as shown in Figure 1b. By adding a color filter of the corresponding color in front of each sub-pixel of the image sensor, different sub-pixels can sense the corresponding color, and finally the pixel value of one pixel of the color picture is synthesized by the pixel values of the four sub-pixels. And the color of each pixel of the color picture is determined by the color guessing algorithm to obtain the final color picture.

类似于拜耳阵列，首先，本申请实施例提供了一种用于生成HDR图像的滤光片阵列，该滤光片阵列的结构与拜耳阵列类似，与拜耳阵列不同的是拜耳阵列是由红绿绿蓝的彩色滤光片构成，而本申请实施例提供的滤光片阵列由不同透光率的滤光片构成，该滤光片阵列可以覆盖于感光素点阵列上，用于对镜头中采集的光线进行滤光处理，从而实现HDR图像的生成。可以理解，本申请实施例中的滤光片阵列中的滤光片数量不受拜耳阵列的限制，例如可以为四个，也可以为更多个或更少个。Similar to the Bayer array, first, the embodiments of the application provide a filter array for generating HDR images. The structure of the filter array is similar to the Bayer array. The difference from the Bayer array is that the Bayer array is composed of red, green, and green. Blue color filter, while the filter array provided by the embodiment of the application is composed of filters with different light transmittances, the filter array can be covered on the photosensitive pixel array, used to capture the lens The light is filtered to achieve the generation of HDR images. It can be understood that the number of filters in the filter array in the embodiment of the present application is not limited by the Bayer array, for example, it may be four, or it may be more or less.

本申请实施例提供的滤光片阵列包括多个以阵列排布的最小重复单元，最小重复单元中包括至少两类滤光片单元，不同类别的滤光片单元对应不同的透光率。图2为本申请实施例中的一种滤光片阵列的示意图，需要指出的是，图2所展示的仅是一种示意性的滤光片阵列示意图，本申请的滤光片阵列可以是由至少两类不同透光率的滤光片单元构成的最小重复单元按任意形式排布的滤光片阵列，并不局限于图2所示的结构。其中，滤光片单元的透光率可以在0-100％的范围内任一透光率，最小重复单元可以是由滤光片单元构成的矩形区域、正方形区域以及其他形状的区域，本申请不作限制。最小重复单元中的滤光片的类别是两类或者两类以上，并且每个最小重复单元中每个类别的滤光片单元可以是一个或者多个。The filter array provided by the embodiment of the present application includes a plurality of minimum repeating units arranged in an array, and the minimum repeating unit includes at least two types of filter units, and different types of filter units correspond to different light transmittances. Figure 2 is a schematic diagram of a filter array in an embodiment of the application. It should be pointed out that the diagram shown in Figure 2 is only a schematic diagram of a schematic filter array, and the filter array of the present application may be The filter array in which the smallest repeating unit composed of at least two types of filter units with different light transmittances is arranged in any form is not limited to the structure shown in FIG. 2. Among them, the light transmittance of the filter unit can be any light transmittance in the range of 0-100%, and the smallest repeating unit can be a rectangular area, a square area, and other shapes formed by the filter unit. This application No restrictions. The types of filters in the smallest repeating unit are two or more types, and there may be one or more filter units of each category in each smallest repeating unit.

其次，本申请实施例还提供了一种图像传感器，如图3所示，所述图像传感器包括感光像素点阵列32，以及位于所述感光像素点阵列的滤光片阵列31，当然，图像传感器还包括用于感光所需要的其他器件。所述滤光片阵列包括多个以阵列排布的最小重复单元，最小重复单元中包括至少两类滤光片单元，不同类别的滤光片单元对应不同的透光率，单个所述滤光片单元对应单个所述感光像素点。在摄像过程中，可以通过图像传感器上面的感光器件来记录光线的强度，感光器件上有许多感光像素点，以阵列形态排布，称之为感光像素点阵列，感光像素点阵列中的每个感光像素点会采集光强，从而得到图像的各个像素点的像素值。类似于采用拜耳阵列获得彩色图像的方式，本申请实施例提供的图像传感器在感光像素点阵列上面添加了一层微透镜层，该微透镜层即为滤光片阵列，滤光片阵列的每一个滤光片单元对应感光像素点阵列中的一个像素点，即一个滤光片单元的尺寸与一个感光像素点的尺寸一致，并且覆盖于感光像素点之上。当光线从镜头穿过，可以先通过滤光片单元过滤掉一部分光线后，再达到感光像素点阵列，以便各感光像素点记录滤光处理后的像素值。Secondly, the embodiment of the present application also provides an image sensor. As shown in FIG. 3, the image sensor includes a photosensitive pixel point array 32 and a filter array 31 located in the photosensitive pixel point array. Of course, the image sensor It also includes other devices needed for photosensitive. The filter array includes a plurality of smallest repeating units arranged in an array, and the smallest repeating unit includes at least two types of filter units. Different types of filter units correspond to different light transmittances, and a single filter unit The slice unit corresponds to a single said photosensitive pixel. In the imaging process, the light intensity can be recorded by the photosensitive device on the image sensor. There are many photosensitive pixels on the photosensitive device, arranged in an array, which is called the photosensitive pixel dot array. Each of the photosensitive pixel dot arrays The photosensitive pixels will collect the light intensity to obtain the pixel value of each pixel of the image. Similar to the way of using Bayer arrays to obtain color images, the image sensor provided in the embodiments of the present application adds a layer of microlens layer on the photosensitive pixel array, and the microlens layer is the filter array. One filter unit corresponds to one pixel in the photosensitive pixel point array, that is, the size of one filter unit is consistent with the size of one photosensitive pixel and covers the photosensitive pixel. When light passes through the lens, a part of the light can be filtered by the filter unit before reaching the photosensitive pixel point array, so that each photosensitive pixel point records the pixel value after the filtering process.

同样的，所述感光像素点阵列上覆盖的滤光片阵列可以是由至少两类不同透光率的滤光片单元构成的最小重复单元按任意形式排布构成。通过使用至少两种不同的透光率的滤光片单元，可以过滤掉不同程度的光线，使最终到达感光像素点的光线强度呈现更大的差别，使得各像素点的亮度值呈现更大的差异。最小重复单元可以是由滤光片单元构成的矩形区域、正方形区域以及其他形状的区域，本申请不作限制。最小重复单元中的滤光片的类别是两类或者两类以上，并且每个最小重复单元中每个类别的滤光片单元可以是一个或者多个。Similarly, the filter array covered on the photosensitive pixel dot array may be a minimum repeating unit composed of at least two types of filter units with different light transmittances arranged in any form. By using at least two different light transmittance filter units, different degrees of light can be filtered out, so that the light intensity that finally reaches the photosensitive pixel has a greater difference, so that the brightness value of each pixel is greater difference. The minimum repeating unit may be a rectangular area, a square area, or other shapes formed by filter units, which is not limited in this application. The types of filters in the smallest repeating unit are two or more types, and there may be one or more filter units of each category in each smallest repeating unit.

另外，本申请实施例还提供了一种HDR图像生成方法，该HDR图像生成方法的流程图如图4所示，包括以下步骤：In addition, an embodiment of the present application also provides a method for generating an HDR image. The flow chart of the method for generating an HDR image is shown in FIG. 4 and includes the following steps:

S402、接收图像传感器采集的滤光处理后的图像的各像素点的像素值，其中，所述图像传感器包括感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括至少两类滤光片单元，不同类别的所述滤光片单元的透光率不同，单个所述滤光片单元对应单个所述感光像素点；S402. Receive the pixel value of each pixel of the filtered image collected by the image sensor, where the image sensor includes an array of photosensitive pixel points and a filter array covering the array of photosensitive pixel points, and the filter The light sheet array includes a plurality of smallest repeating units arranged in an array, and the smallest repeating unit includes at least two types of filter units. The light transmittances of the filter units of different types are different. The slice unit corresponds to a single said photosensitive pixel;

S404、从所述滤光处理后的图像中确定一预设区域，将所述预设区域中的所述像素点作为子像素点，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。S404. Determine a preset area from the filtered image, use the pixels in the preset area as sub-pixels, and synthesize the HDR image based on the pixel values of the sub-pixels The pixel value of a single pixel.

本申请实施例提供的HDR图像生成方法可以用各种对图像信号进行处理的图像处理芯片，比如ISP芯片、FPGA芯片以及ASIC芯片等。该方法不仅可以用于各种帘卷快门的图像传感器，也适用于各种全局快门的图像传感器，以便在摄像过程中实时的生成HDR图像。The HDR image generation method provided by the embodiments of the present application can use various image processing chips for processing image signals, such as ISP chips, FPGA chips, and ASIC chips. This method can be used not only for image sensors of various rolling shutters, but also for image sensors of various global shutters, so as to generate HDR images in real time during the shooting process.

为了获取实时的HDR图像，本申请实施例使用的图像传感器的感光像素点阵列上覆盖了一层微透镜层，该微透镜层可以是由多个最小重复单元以阵列形态排布构成的滤光片阵列，用于对到达感光像素点阵列的光线进行过滤处理。其中，最小重复单元包括至少两类滤光片单元，不同类别的滤光片单元的透光率不同，单个滤光片单元对应于单个像素点，也就是一个滤光片单元的尺寸与一个像素点的尺寸一致，并且一个滤光片单元覆盖于一个像素点上，与对应的像素点重叠。当光线透过镜头后，会经过滤光片阵列的滤光处理，然后到达感光像素点进行感光，感光像素点便可以记录滤光处理后的图像的各个像素点的像素值。通过使用至少两种不同的透光率的滤光片单元，可以根据需要过滤光线，使最终到达感光像素点的光线强度呈现更大的差别，使得各像素点的亮度值呈现更大的动态范围，从而得到曝光动态范围更大的HDR图像。In order to obtain real-time HDR images, the photosensitive pixel array of the image sensor used in the embodiments of the present application is covered with a microlens layer, which may be a filter composed of a plurality of smallest repeating units arranged in an array. The sheet array is used to filter the light reaching the photosensitive pixel point array. Among them, the smallest repeating unit includes at least two types of filter units. Different types of filter units have different light transmittances. A single filter unit corresponds to a single pixel, that is, the size of a filter unit and a pixel. The size of the dots is the same, and a filter unit covers a pixel point and overlaps the corresponding pixel point. After the light passes through the lens, it is filtered by the filter array, and then reaches the photosensitive pixel for light exposure. The photosensitive pixel can record the pixel value of each pixel of the filtered image. By using at least two filter units with different light transmittances, the light can be filtered as needed, so that the intensity of the light that finally reaches the photosensitive pixel has a greater difference, so that the brightness value of each pixel has a larger dynamic range , Resulting in an HDR image with a larger dynamic range of exposure.

当然，在采用滤光片阵列过滤处理后，临近的各像素点由于采用不同透光率的滤光片单元处理过，像素值会存在较大差异，为了让最终的图像过渡更加自然，还可以进一步的对这些像素点进行处理，得到最终的HDR图像。例如，在一种实施方式中，在接收到图像传感器采集到的经过滤处理后的图像的各像素点的像素值后，可以从过滤处理后的图像中确定出一个预设区域，然后将预设区域的像素点作为子像素点，合成HDR图像的单个像素点。其中，子像素点为图像传感器采集的滤光处理后的各像素点，预设区域可以根据对HDR图像效果的实际需求和滤光片阵列的排布结构来确定，比如可以将过滤处理后的图像中的每四个像素点作为子像素点，合成HDR图像的一个像素点，也可以将每两个像素点作为子像素点，合成HDR图像的一个像素点。Of course, after the filter array is used for filtering, the neighboring pixels have been processed by filter units with different light transmittances, and the pixel values will have large differences. In order to make the final image transition more natural, you can These pixels are further processed to obtain the final HDR image. For example, in one embodiment, after receiving the pixel value of each pixel of the filtered image collected by the image sensor, a preset area may be determined from the filtered image, and then the preset area may be determined from the filtered image. Set the pixels of the area as sub-pixels, and synthesize a single pixel of the HDR image. Among them, the sub-pixels are the pixels collected by the image sensor after the filtering process. The preset area can be determined according to the actual requirements for the HDR image effect and the arrangement structure of the filter array. For example, the filtered Every four pixels in the image are used as sub-pixels to synthesize one pixel of the HDR image, or every two pixels can be used as sub-pixels to synthesize one pixel of the HDR image.

通过滤光片阵列中不同透光率的滤光片单元对各个感光像素点采集的光线进行滤光处理，可以使每个感光像素点接收到的光线强度呈现出更大的明暗差异，比如部分感光像素点覆盖有高透光率的滤光片，其感应的光线较强，最后像素点的亮度也较大，部分像素点覆盖有低透光率的滤光片单元，其感应的光线较弱，最后像素点的亮度也较小。然后将这些滤光处理后的像素点作为子像素点，进一步合成HDR图像的各个像素点，即可得到曝光范围更大的HDR图像。Through the filter unit with different transmittance in the filter array, the light collected by each photosensitive pixel can be filtered, so that the light intensity received by each photosensitive pixel can show a greater difference between light and dark, such as some The light-sensitive pixels are covered with filters with high light transmittance, and the light that it senses is stronger, and the brightness of the final pixels is also larger. Some pixels are covered with filter units with low light transmittance, and the light it senses is higher. Weak, the brightness of the final pixel is also smaller. Then, these filtered pixels are used as sub-pixels, and each pixel of the HDR image is further synthesized to obtain an HDR image with a larger exposure range.

对于每个滤光片阵列，其最小重复单元中包括的滤光片单元的透光率以及滤光片单元的排布无疑会影响最终的HDR图像的效果。在某些实施例中，最小重复单元可以包括由滤光片单元构成的矩形区域、正方形区域或者是其他形状的区域，可以根据实际需求设置，本申请不作限制。比如，在某些实施例中，如图5a所示，最小重复单元可以是由一个透光率为100％和一个透光率为10％的滤光片构成的矩形区域。当然，如图5b所示，最小重复单元也可以是由一个透光率为100％、一个透光率为10％和两个透光率为50％的滤光片单元构成的正方形区域。当然，滤光片单元的透光率可以根据实际需求选择，透光率取值范围可以是0-100％中的任意值。For each filter array, the light transmittance of the filter units included in the smallest repeating unit and the arrangement of the filter units will undoubtedly affect the effect of the final HDR image. In some embodiments, the minimum repeating unit may include a rectangular area, a square area, or an area of other shapes formed by filter units, which can be set according to actual needs, and this application does not limit it. For example, in some embodiments, as shown in FIG. 5a, the smallest repeating unit may be a rectangular area formed by a filter with a light transmittance of 100% and a light transmittance of 10%. Of course, as shown in FIG. 5b, the smallest repeating unit can also be a square area composed of a filter unit with a light transmittance of 100%, a light transmittance of 10%, and two filter units with a light transmittance of 50%. Of course, the light transmittance of the filter unit can be selected according to actual requirements, and the light transmittance range can be any value from 0-100%.

由于高透光率的滤光片单元可以透过更多的光线，因而最后感光像素点感应到的光强更强，像素点的亮度也较大。而低透光率的滤光片单元会过滤掉更多的光线，透过的光线少，感光像素点感应的光强较弱，使得最后像素点的亮度会比较小。因此，为了使图像获得更大的动态范围，即更明显的明暗差别，像素点间的像素值差别可以更大，即接收到的光线的强度差更大。在某些实施例中，最小重复单元至少包括第一类滤光片单元和第二类滤光片单元，其中，第一类滤光片单元的透光率可以大于第一预设阈值，第二类滤光片单元的透光率单元可以小于第二预设阈值，使得两类滤光片单元的透光率呈现出一定差值，最终像素点的亮度值才能获得较大的范围。比如，可以设置第一预设阈值为一个比较高的透光率，以得到亮度较大的像素点，比如可以设置成95％，可以设置第二预设阈值为一个比较低的透光率，以得到亮度较小的像素点，比如可以设置成5％。当然，第一预设阈值和第二预设阈值可以根据实际应用场景灵活去设定，本申请不作限制。Since the filter unit with high light transmittance can transmit more light, the light intensity sensed by the final photosensitive pixel is stronger, and the brightness of the pixel is also greater. The filter unit with low light transmittance will filter out more light and transmit less light, and the light intensity induced by the photosensitive pixel will be weaker, so that the brightness of the final pixel will be smaller. Therefore, in order to obtain a larger dynamic range of the image, that is, a more obvious difference between light and dark, the pixel value difference between pixels can be larger, that is, the intensity difference of the received light is larger. In some embodiments, the smallest repeating unit includes at least a first-type filter unit and a second-type filter unit, wherein the light transmittance of the first-type filter unit may be greater than the first preset threshold, and the The light transmittance unit of the second type filter unit may be smaller than the second preset threshold, so that the light transmittance of the two types of filter unit presents a certain difference, and the final brightness value of the pixel point can obtain a larger range. For example, the first preset threshold can be set to a relatively high light transmittance to obtain pixels with greater brightness, for example, it can be set to 95%, and the second preset threshold can be set to a relatively low light transmittance. In order to obtain pixels with lower brightness, for example, it can be set to 5%. Of course, the first preset threshold and the second preset threshold can be flexibly set according to actual application scenarios, which is not limited in this application.

当然，为了得到不同亮度取值的像素点，最小重复单元中滤光片的种类可以是两种或者多种，在某些实施例，最小重复单元除了包括第一类滤光片单元和第二类滤光片单元，还可以包括第三类滤光片单元，第三类滤光单元的透光率位于第一预设阈值和第二预设阈值之间。比如，第一预设阈值90％，第二预设阈值为5％，则第三类滤光片的透光率可以取5％-95％之间的一个值。当然，为了得到更多不同亮度取值的子像素点，以便最终合成的HDR图像更加精细，最小重复单元也可以包括四类、五类或者更多类别的滤光片单元，本申请不作限制。比如，如图5a所示，最小重复单元包括透光率为10％和透光率为100％的两类滤光片单元。如图5b所示，最小重复单元包括透光率为10％、透光率为100％以及透光率为50％的三类滤光片单元。如图5c所示，最小重复单元包括透光率为10％、透光率为100％、透光率为40％以及透光率70％的四类滤光片单元。如图5d所示，最小重复单元还包括透光率为10％、透光率为20％、透光率为30％、透光率为40％、透光率为50％、透光率为60％、透光率为70％、透光率为80％、透光率100％的九类滤光片单元。当然，以上只是示例性的例子，最小重复单元中滤光片的类别可以根据实际需求灵活设置。Of course, in order to obtain pixels with different brightness values, the types of filters in the minimum repeating unit can be two or more. In some embodiments, the minimum repeating unit includes the first type of filter unit and the second type of filter unit. The class filter unit may further include a third type filter unit, and the light transmittance of the third type filter unit is between the first preset threshold and the second preset threshold. For example, if the first preset threshold is 90% and the second preset threshold is 5%, the light transmittance of the third type filter can take a value between 5% and 95%. Of course, in order to obtain more sub-pixels with different brightness values so that the final synthesized HDR image is more refined, the minimum repeating unit may also include four, five or more types of filter units, which is not limited in this application. For example, as shown in FIG. 5a, the smallest repeating unit includes two types of filter units with a light transmittance of 10% and a light transmittance of 100%. As shown in FIG. 5b, the smallest repeating unit includes three types of filter units with a light transmittance of 10%, a light transmittance of 100%, and a light transmittance of 50%. As shown in FIG. 5c, the smallest repeating unit includes four types of filter units with a light transmittance of 10%, a light transmittance of 100%, a light transmittance of 40%, and a light transmittance of 70%. As shown in Figure 5d, the smallest repeating unit also includes a light transmittance of 10%, a transmittance of 20%, a transmittance of 30%, a transmittance of 40%, a transmittance of 50%, and a transmittance of 50%. Nine types of filter units with 60%, 70% transmittance, 80% transmittance, and 100% transmittance. Of course, the above are only illustrative examples, and the types of filters in the smallest repeating unit can be flexibly set according to actual needs.

此外，在某些实施例中，最小重复单元可以包括多类滤光片单元，每类滤光片单元的数量可以是一个，也可以是多个。比如，如图5a所示，最小重复单元由两类滤光片构成，每类滤光片单元的数量为一个。如图5b所示，最小重复单元中透光率为50％的滤光片的数量也可以是两个。In addition, in some embodiments, the smallest repeating unit may include multiple types of filter units, and the number of each type of filter unit may be one or more. For example, as shown in Figure 5a, the smallest repeating unit is composed of two types of filters, and the number of each type of filter unit is one. As shown in Fig. 5b, the number of filters with a light transmittance of 50% in the smallest repeating unit can also be two.

在某些实施例中，为了得到亮度较大的子像素点，第一类滤光片单元可以是透光率为100％的滤光片单元，对于镜头入射的光线完全透过，使得像素点的亮度较大。为了得到亮度较小但又至于太暗而出现欠曝的像素点，第二类滤光片单元可以是透光率为10％的滤光片单元，通过透光率为100％的滤光片单元和透光率为10％的滤光片单元来调整子像素点的亮度范围差值，使得各像素点的亮度值呈现更大的动态范围，从而得到曝光动态范围更大的HDR图像。In some embodiments, in order to obtain sub-pixels with greater brightness, the first type of filter unit may be a filter unit with a light transmittance of 100%, and the light incident on the lens is completely transmitted, so that the pixel The brightness is greater. In order to obtain pixels with low brightness but too dark and underexposed, the second type of filter unit can be a filter unit with a light transmittance of 10%, and a filter with a light transmittance of 100%. The unit and the filter unit with a light transmittance of 10% adjust the brightness range difference of the sub-pixels, so that the brightness value of each pixel presents a larger dynamic range, thereby obtaining an HDR image with a larger exposure dynamic range.

通过图像传感器采集到过滤处理后的图像的各像素点的像素值后，可以从过滤处理后的图像中选出一预设区域，将预设区域的像素点作为子像素点来合成HDR图像的单个像素点。其中，预设区域可以根据实际需求来选取，比如可以是由两个子像素点构成的区域，四个子像素点构成的区域，或者是更多子像素点构成区域。当然，由于每个子像素点上添加了不同透光率的滤光片单元，相邻的子像素点的像素值的亮度差异可能比较大，为了使得最后生成的HDR图像各个部分的过渡可以更加自然，在某些实施例中，在选择预设区域时，预设区域中的各个子像素点上覆盖的滤光片单元的类别可以与最小重复单元中包括的滤光片单元的类别一致，比如，最小重复单元包含透光率为10％、40％、50％以及100％四种滤光片单元，在选取预设区域时，可以选择覆盖有透光率为10％、40％、70％以及100％四种滤光片单元的四个子像素构成的区域作为预设区域，并使用这四个子像素的像素值合成HDR图像的单个像素点的像素值。这样便可以保证HDR图像的每个像素点都是由覆盖了相同种类的滤光片单元的子像素点合成得到，使得HDR图像的各像素点的亮度过渡自然些。After the pixel value of each pixel of the filtered image is collected by the image sensor, a preset area can be selected from the filtered image, and the pixels of the preset area are used as sub-pixels to synthesize the HDR image. Single pixel. Among them, the preset area can be selected according to actual needs, for example, it can be an area composed of two sub-pixels, an area composed of four sub-pixels, or an area composed of more sub-pixels. Of course, because filter units with different light transmittances are added to each sub-pixel, the brightness difference of the pixel values of adjacent sub-pixels may be relatively large, in order to make the transition of each part of the final HDR image more natural In some embodiments, when the preset area is selected, the type of the filter unit covered on each sub-pixel point in the preset area may be the same as the type of the filter unit included in the smallest repeating unit, such as , The smallest repeating unit includes four filter units with transmittance of 10%, 40%, 50% and 100%. When selecting a preset area, you can choose to cover with transmittance of 10%, 40%, 70% And the area formed by the four sub-pixels of the 100% four filter units is used as the preset area, and the pixel values of the four sub-pixels are used to synthesize the pixel value of a single pixel of the HDR image. In this way, it can be ensured that each pixel of the HDR image is synthesized by sub-pixels covering the same type of filter unit, so that the brightness transition of each pixel of the HDR image is natural.

例如，在一个实施例中，如图6a所示，滤光片阵列中的最小重复单元可以是由1个透光率为10％和1个透光率为100％的滤光片单元构成的矩形区域，在另一个实施例中，滤光片阵列中的最小重复单元也可以是由2个透光率为10％和2个透光率为100％的滤光片单元构成的正方形区域。当最小重复单元为由1个透光率为10％和1个透光率为100％的滤光片单元构成的矩形区域时，在选取预设区域时，也就是选取合成每个HDR图像的像素点的子像素点时，可以选取覆盖有透光率为10％的滤光片单元的子像素点以及覆盖有透光率为100％的滤光片单元的子像素点(如图6a中的虚线框选的矩形区域)，并用这两个子像素点的像素值合成HDR图像的一个像素点(图6a中的黑色点)。当然，在某些实施例中，如图如6b所示，当最小重复单元为由2个透光率为10％和2个透光率为100％的滤光片单元构成的矩形区域时，在选取预设区域时，也可以选取相邻的2个覆盖有透光率为10％的滤光片单元的子像素点以及2个覆盖有透光率为100％的滤光片单元的子像素点(如图6b中虚线框选的矩形区域)，并用这四个子像素点的像素值合成HDR图像的一个像素点(例如可以为四个子像素点中心位置的点，如图6b所示)。For example, in an embodiment, as shown in FIG. 6a, the smallest repeating unit in the filter array may be composed of a filter unit with a light transmittance of 10% and a filter unit with a light transmittance of 100%. Rectangular area. In another embodiment, the smallest repeating unit in the filter array can also be a square area composed of two filter units with a light transmittance of 10% and two filter units with a light transmittance of 100%. When the smallest repeating unit is a rectangular area composed of a filter unit with a light transmittance of 10% and a filter unit with a light transmittance of 100%, when selecting the preset area, that is, selecting the composite of each HDR image When the pixel points are sub-pixel points, you can select sub-pixel points covered with a filter unit with a light transmittance of 10% and sub-pixel points covered with a filter unit with a light transmittance of 100% (as shown in Figure 6a) The rectangular area selected by the dashed line of ), and use the pixel values of these two sub-pixels to synthesize a pixel of the HDR image (the black dot in Figure 6a). Of course, in some embodiments, as shown in Figure 6b, when the smallest repeating unit is a rectangular area composed of 2 filter units with a light transmittance of 10% and two filter units with a light transmittance of 100%, When selecting the preset area, you can also select two adjacent sub-pixels covered with a filter unit with a light transmittance of 10% and two sub-pixels covered with a filter unit with a light transmittance of 100%. Pixels (the rectangular area selected by the dotted line in Figure 6b), and use the pixel values of these four sub-pixels to synthesize a pixel of the HDR image (for example, it can be a point at the center of the four sub-pixels, as shown in Figure 6b) .

在另外一个实施例中，如图6c所示，滤光片阵列中的最小重复单元是由1个透光率为10％、1个透光率为100％以及2个透光率为50％的滤光片单元构成的正方形区域。选取合成每个HDR图像的像素点的子像素点时，可以选取覆盖有透光率为10％、透光率为100％以及透光率为50％的滤光片单元的子像素点(如图6c中的虚线框选的正方形区域)来合成HDR图像的单个像素点(图6c中位于四个子像素点中心的黑色点)。当然，选取的子像素点可以是任意相邻的四个分别覆盖有透光率为10％、透光率为100％以及2个透光率为50％的滤光片单元的子像素点。In another embodiment, as shown in FIG. 6c, the smallest repeating unit in the filter array is composed of one with a transmittance of 10%, one with a transmittance of 100%, and two with a transmittance of 50%. The filter unit constitutes a square area. When selecting the sub-pixels that synthesize the pixels of each HDR image, you can select the sub-pixels covered with a filter unit with a light transmittance of 10%, a light transmittance of 100%, and a light transmittance of 50% (such as The square area selected by the dashed line in FIG. 6c) is used to synthesize a single pixel point of the HDR image (the black point located in the center of the four sub-pixel points in FIG. 6c). Of course, the selected sub-pixel points can be any four adjacent sub-pixel points covered with filter units with a light transmittance of 10%, a light transmittance of 100%, and two filter units with a light transmittance of 50%, respectively.

在某些实施例中，预设区域的尺寸可以与最小重复单元尺寸一致，当然也可以不一致。比如，当最小重复单元的结构如图5a所示时，最小重复单元可以是2个透光率为10％和2个透光率为100％构成的正方形区域，在选取预设区域时，即在选取用于合成HDR图像像素点的子像素点所构成的区域时，可以只选取覆盖有透光率为10％和透光率为100％的两个子像素构成矩形区域作为预设区域，也就是说，可以只选取相邻的覆盖有透光率为10％和透光率为100％的两个子像素点来合成HDR图像的单个像素点。而当最小重复单元的结构如图5b所示时，则最小重复单元是1 个透光率为10％、1个透光率40％、1个透光率为70％、1个透光率100％构成的正方形区域，在选取用于合成HDR图像像素点的子像素点所构成的区域时，可以选取覆盖有四种透光率的滤光片单元相邻的四个子像素点构成的正方区域作为预设区域，通过上述四个子像素点来合成一个HDR图像的单个像素点。当然，在某些情况，也可以选取更多的子像素点来合成HDR图像的单个像素点，比如选取9个或者16个子像素来合成一个HDR图像的像素点，具体可以根据实际需求去设置。In some embodiments, the size of the preset area may be the same as the minimum repeating unit size, of course, it may also be different. For example, when the structure of the smallest repeating unit is shown in Figure 5a, the smallest repeating unit can be 2 square areas with a light transmittance of 10% and 2 light transmittances of 100%. When the preset area is selected, that is When selecting the area composed of sub-pixels for synthesizing HDR image pixels, you can select only the rectangular area covered by two sub-pixels with a light transmittance of 10% and a light transmittance of 100% as the preset area. That is, it is possible to select only two adjacent sub-pixels covered with a light transmittance of 10% and a light transmittance of 100% to synthesize a single pixel of the HDR image. When the structure of the smallest repeating unit is shown in Figure 5b, the smallest repeating unit is 1 light transmittance 10%, 1 light transmittance 40%, 1 light transmittance 70%, 1 light transmittance A square area composed of 100%. When selecting an area composed of sub-pixels for synthesizing HDR image pixels, you can select a square composed of four adjacent sub-pixels covering a filter unit with four light transmittances. The area is used as a preset area, and a single pixel of an HDR image is synthesized through the above four sub-pixels. Of course, in some cases, more sub-pixels can be selected to synthesize a single pixel of an HDR image, for example, 9 or 16 sub-pixels are selected to synthesize a pixel of an HDR image, which can be specifically set according to actual needs.

在某些实施例中，在确定出预设区域以及合成HDR图像的单个像素点的各子像素点后。可以预先确定各子像素点的权重，然后基于各子像素点的权重以及各子像素点的像素值合成所述HDR图像的单个像素点的像素值。比如，合成HDR图像的单个像素点的子像素点分别覆盖有透光率为10％以及100％的滤光片单元，确定出覆盖有透光率为10％滤光片单元的子像素点权重为60％，像素值为53，而覆盖有透光率为100％滤光片单元的权重为40％，像素值为198，则合成的HDR图像的像素点的像素值为:53*60％+198**40％＝114。In some embodiments, after determining the preset area and each sub-pixel of a single pixel of the synthesized HDR image. The weight of each sub-pixel may be determined in advance, and then the pixel value of a single pixel of the HDR image is synthesized based on the weight of each sub-pixel and the pixel value of each sub-pixel. For example, the sub-pixels of a single pixel of the composite HDR image are respectively covered with filter units with a transmittance of 10% and 100%, and the weights of the sub-pixels covered with a filter unit with a transmittance of 10% are determined. 60%, the pixel value is 53, and the weight of the filter unit covered with 100% light transmittance is 40%, and the pixel value is 198, the pixel value of the synthesized HDR image is: 53*60% +198**40%=114.

在某些实施例中，各子像素点的权重可以基于第一指定子像素点的像素值来确定。第一指定子像素点可以是经过某个特定透光率的滤光片单元进行滤光处理的子像素点。比如，在某些实施例中，第一指定子像素点可以是覆盖有透光率为50％的滤光片单元的子像素点。通过透光率为50％的滤光片单元滤光处理后的子像素点的像素值可以确定预设区域中的各子像素点的权重，进而计算HDR图像的像素点的像素值。假设感光像素点阵列上覆盖的滤光阵列如图5b所示，最小重复单元中包含透光率为10％、透光率为100％以及2个透光率为50％的滤光片单元，子像素点分别覆盖有透光率为10％、透光率为100％以及透光率为50％的滤光片单元，在计算权重时，可以将通过透光率为50％的滤光片单元滤光处理的子像素点视为为正常曝光，将通过透光率为100％的滤光片单元滤光处理的子像素点视为过曝，将通过透光率为10％的滤光片单元滤光处理的子像素点视为欠曝，则可以根据正常曝光的子像素点的像素值来确定其他子像素点的权重，比如，覆盖有透光率为10％的滤光片单元的子像素对应的权重可以通过如下计算公式计算：In some embodiments, the weight of each sub-pixel may be determined based on the pixel value of the first designated sub-pixel. The first designated sub-pixel point may be a sub-pixel point that has been filtered by a filter unit with a specific light transmittance. For example, in some embodiments, the first designated sub-pixel may be a sub-pixel covered with a filter unit with a light transmittance of 50%. The pixel values of the sub-pixels filtered by the filter unit with a light transmittance of 50% can determine the weight of each sub-pixel in the preset area, and then calculate the pixel value of the pixel of the HDR image. Assuming that the filter array covered on the photosensitive pixel point array is shown in Figure 5b, the smallest repeating unit contains 10% light transmittance, 100% light transmittance, and 2 filter units with 50% light transmittance. The sub-pixels are respectively covered with a filter unit with a light transmittance of 10%, a light transmittance of 100%, and a light transmittance of 50%. When calculating the weight, the filter unit with a light transmittance of 50% can be passed The sub-pixels processed by the unit filter are regarded as normal exposure, and the sub-pixels processed by the filter unit with a transmittance of 100% are regarded as overexposed, and the filter with a transmittance of 10% is regarded as overexposed. The sub-pixels processed by the filter unit are regarded as under-exposed, and the weights of other sub-pixels can be determined according to the pixel value of the normally exposed sub-pixels, for example, a filter unit covered with a light transmittance of 10% The weight corresponding to the sub-pixel of can be calculated by the following calculation formula:

weight＝0.5+(127–val50％)*0.002，weight=0.5+(127–val50%)*0.002,

其中，weight指覆盖有透光率为10％的滤光片单元的子像素对应的权重，val50％指覆盖有透光率为50％的滤光片单元的像素值；Wherein, weight refers to the weight corresponding to the sub-pixel covered with a filter unit with a light transmittance of 10%, and val50% refers to the pixel value of a filter unit with a light transmittance of 50%;

确定权重后，合成的HDR图像的像素点的像素值的可以采用如下公式计算得到：After the weights are determined, the pixel values of the pixels of the synthesized HDR image can be calculated using the following formula:

合成的HDR图像的像素点的像素值val＝(val10％*weight)+(val100％*(1-weight))。The pixel value of the pixel of the synthesized HDR image is val=(val10%*weight)+(val100%*(1-weight)).

在某些实施例中，摄像过程中某些子像素点可能会出现过曝或者欠曝的现象，致使这些子像素点的亮度值过高或者过低，在一定程度上会影响最终合成的HDR图像的效果。为了避免这个问题，在采用子像素点合成HDR图像的像素点的像素值时，当某个子像素点的像素值大于第一阈值或者小于第二阈值，可以认为该子像素点出现过曝或欠曝的现象，因而在合成HDR图像的像素点的像素值时，可以将该子像素点的权重设置为0，即不考虑该过曝或欠曝的子像素点的像素值。其中，所述第一阈值可以设置为一个较大的像素值，比如250(以8bit为例，像素值取值为0-255)，所述第二阈值可以设置为一个较小的像素值，比如25(以8bit为例，像素值取值为0-255)。当然，以上只是示意性实施例，第一阈值和第二阈值可以根据实际情况具体设定。In some embodiments, some sub-pixels may be over-exposed or under-exposed during the imaging process, causing the brightness values of these sub-pixels to be too high or too low, which will affect the final composite HDR to a certain extent. The effect of the image. To avoid this problem, when sub-pixels are used to synthesize the pixel values of HDR images, when the pixel value of a certain sub-pixel is greater than the first threshold or less than the second threshold, it can be considered that the sub-pixel is overexposed or underexposed. Therefore, when synthesizing the pixel value of the pixel of the HDR image, the weight of the sub-pixel can be set to 0, that is, the pixel value of the over-exposed or under-exposed sub-pixel is not considered. Wherein, the first threshold can be set to a larger pixel value, such as 250 (taking 8bit as an example, the pixel value is 0-255), and the second threshold can be set to a smaller pixel value, For example, 25 (take 8bit as an example, the pixel value is 0-255). Of course, the above are only exemplary embodiments, and the first threshold and the second threshold can be specifically set according to actual conditions.

以下用一个具体实施例进一步解释，假设滤光片阵列的最小重复单元如图6a所示，包括2个透光率为10％和2个透光率为100％的滤光片单元，在合成HDR图像时，选取相邻的两个分别覆盖有透光率为10％和透光率为100％的滤光片单元的子像素点来合成HDR图像的单个像素点，当覆盖有透光率为100％的子像素点出现过曝现象时，亮度较大时，比如覆盖有透光率为100％的子像素点的像素值大于250，这时就不考虑该子像素点的像素值，取覆盖有透光率为10％的子像素点的像素值作为HDR图像的单个像素点的像素值。当覆盖有透光率为10％的子像素点出现欠曝现象，亮度较小时，比如覆盖有透光率为10％的子像素点的像素值小于25，这时就不考虑该子像素点的像素值，取覆盖有透光率为100％的子像素点的像素值作为HDR图像的单个像素点的像素值。The following uses a specific embodiment for further explanation. It is assumed that the smallest repeating unit of the filter array is shown in Figure 6a, including 2 filter units with a light transmittance of 10% and 2 filter units with a light transmittance of 100%. For HDR images, select two adjacent sub-pixels covered with filter units with a transmittance of 10% and 100% respectively to synthesize a single pixel of the HDR image. When covered with transmittance When the 100% sub-pixel is overexposed, when the brightness is large, for example, the pixel value of the sub-pixel covered with 100% transmittance is greater than 250, then the pixel value of the sub-pixel is not considered. The pixel value of a sub-pixel covered with a light transmittance of 10% is taken as the pixel value of a single pixel of the HDR image. When the sub-pixels covered with 10% transmittance are under-exposed and the brightness is low, for example, the pixel value of the sub-pixels covered with 10% transmittance is less than 25, then the sub-pixels are not considered Take the pixel value of a sub-pixel covered with a light transmittance of 100% as the pixel value of a single pixel of the HDR image.

在某些实施例中，在确定出用于合成HDR图像的单个像素点的各个子像素点后，可以基于第二指定子像素点的像素值范围、和预先设置的像素值映射表来确定所述HDR图像的单个像素点的像素值，其中，第二指定子像素点例如可以是未经过滤光处理的像素点，例如可以是覆盖有透光率为100％的滤光片单元的子像素点，所述像素值映射表用于记录所述第二指定子像素点的像素值范围与所述HDR图像单个像素点的像素值的对应关系。比如，可以参考所有子像素点中的某一个像素点的像素值取值，来确定合成的HDR图像的像素点的像素值。在一个实施例中，可以事先根据经验设置一个像素值映射表，记录该子像素点的像素值在不同范围时，合成的HDR图像的像素点的像素值的合适的取值。比如，该子像素点的像素值在0-50时，HDR图像的像素点的像素值为取值为40，该子像素点的像素值在50-100时，HDR图像的像素点的像素值为取值为70。当然，也可以是该子像素点的像素值在0-50时，HDR图像的像素点的像素值取覆盖有透光率为70％的滤光片单元的子像素点，该子像素点的像素值在50-100时，HDR图像的像素点的像素值取覆盖有透光率为40％的滤光片单元的子像素点。具体可以根据最小重复单元的透光片类别和结构去设置像素值映射表。In some embodiments, after determining each sub-pixel used to synthesize a single pixel of the HDR image, the determination can be made based on the pixel value range of the second designated sub-pixel and a preset pixel value mapping table. The pixel value of a single pixel of the HDR image, where the second designated sub-pixel may be, for example, a pixel that has not been filtered, for example, it may be a sub-pixel covered with a filter unit with a light transmittance of 100% The pixel value mapping table is used to record the correspondence between the pixel value range of the second designated sub-pixel and the pixel value of a single pixel of the HDR image. For example, the pixel value of a certain pixel of all sub-pixels can be referred to to determine the pixel value of the pixel of the synthesized HDR image. In an embodiment, a pixel value mapping table can be set up in advance based on experience to record the appropriate value of the pixel value of the pixel point of the synthesized HDR image when the pixel value of the sub-pixel point is in different ranges. For example, when the pixel value of the sub-pixel is 0-50, the pixel value of the HDR image is 40, and when the pixel value of the sub-pixel is 50-100, the pixel value of the HDR image The value is 70. Of course, it can also be that when the pixel value of the sub-pixel is 0-50, the pixel value of the pixel of the HDR image is the sub-pixel covered with a filter unit with a light transmittance of 70%. When the pixel value is 50-100, the pixel value of the pixel of the HDR image is taken as the sub-pixel covered with a filter unit with a light transmittance of 40%. Specifically, the pixel value mapping table can be set according to the type and structure of the light-transmitting sheet of the smallest repeating unit.

在某些实施例中，第二指定子像素点可以是未经过滤光处理像素点，比如可以是覆盖有透光率为100％的滤光片单元的子像素点。例如，在一个实施方式中，最小重复单元如图5c所示，是由一个透光率为10％、一个透光率为100％、一个透光率为40％以及一个透光率为70％的滤光片单元构成的正方形区域，因此，可以选择覆盖有上述四种滤光片单元的四个子像素点来合成HDR图像的单个像素点。在计算HDR图像的像素点时，可以采用梯度取值的方式，比如都参照覆盖有透光率为100％的滤光片单元的子像素点的像素值来确定HDR图像的单个像素点(以8bit为例，像素值取值为0-255)，具体如下：In some embodiments, the second designated sub-pixel point may be a pixel point without filtering treatment, for example, it may be a sub-pixel point covered with a filter unit with a light transmittance of 100%. For example, in one embodiment, the smallest repeating unit is shown in FIG. 5c, which is composed of a light transmittance of 10%, a light transmittance of 100%, a light transmittance of 40%, and a light transmittance of 70%. Therefore, four sub-pixels covered by the above-mentioned four filter units can be selected to synthesize a single pixel of the HDR image. When calculating the pixel points of the HDR image, the gradient value method can be used. For example, the pixel values of the sub-pixel points covered with a filter unit with a light transmittance of 100% are used to determine a single pixel point of the HDR image (in 8bit as an example, the pixel value is 0-255), the details are as follows:

当val100％值在0～63时，HDR图像像素点的像素值＝val100％When the value of val100% is between 0 and 63, the pixel value of the HDR image pixel = val100%

当val100％值在64～127时，HDR图像像素点的像素值＝val40％When the value of val100% is between 64 and 127, the pixel value of the HDR image pixel = val40%

当val100％值在128～191时，HDR图像像素点的像素值＝val70％When the value of val100% is between 128 and 191, the pixel value of the HDR image pixel = val70%

当val100％值在192～255时，HDR图像像素点的像素值＝val100％When the value of val100% is between 192 and 255, the pixel value of the HDR image pixel = val100%

其中，val100％指覆盖有透光率为100％的滤光片单元的子像素点的像素值，val40％指覆盖有透光率为40％的滤光片单元的子像素点的像素值，val70％指覆盖有透光率为70％的滤光片单元的子像素点的像素值。Among them, val100% refers to the pixel value of a sub-pixel covered with a filter unit with a light transmittance of 100%, and val40% refers to the pixel value of a sub-pixel covered with a filter unit with a light transmittance of 40%. val70% refers to the pixel value of a sub-pixel covered with a filter unit with a light transmittance of 70%.

可以理解，上述取值仅为示例性说明，可以根据实际需要计算HDR图像的像素点的像素值，在此不作限定It can be understood that the above values are only exemplary, and the pixel values of the pixels of the HDR image can be calculated according to actual needs, which are not limited here.

在某些实施例中，通过滤光处理后的子像素点合成的HDR图像可以是灰度图像，也可以是彩色图像。In some embodiments, the HDR image synthesized by the sub-pixels after the filtering process may be a grayscale image or a color image.

在某些实施例中，如果合成的HDR图像是彩色图像的话，该彩色图像可以通过拜耳阵列得到。即图像传感器上除了覆盖一个滤光片阵列，还覆盖了一个拜耳阵列，通过拜耳阵列过滤掉红、蓝、绿三色的光线，得到子像素点，然后通过四个覆盖有红、绿、绿、蓝三色的子像素点的像素值合成一个彩色图像的像素点的像素值，该像素值可以通过红、绿、绿、蓝四个子像素点进行相应计算得到。In some embodiments, if the synthesized HDR image is a color image, the color image can be obtained through a Bayer array. That is, in addition to covering a filter array, the image sensor is also covered with a Bayer array. The Bayer array filters out the red, blue, and green light to obtain sub-pixels, and then passes through the four covered with red, green, and green. The pixel values of the sub-pixels of the three colors of blue and blue are synthesized into the pixel value of the pixel of a color image, and the pixel value can be obtained by corresponding calculation of the four sub-pixels of red, green, green, and blue.

在某些实施例中，为了得到彩色的HDR图像，图像传感器上同时覆盖有滤光片阵列和拜耳阵列，通过上述两个阵列对入射的光线进行双重滤光处理。例如，滤光片阵列覆盖于感光像素点阵列之上，拜耳阵列覆盖于滤光片阵列之上，拜耳阵列的每个彩色滤光片单元对应所述最小重复单元。在一个实施例中，拜耳阵列是由红、绿、绿、蓝四个滤光片构成的正方形区域以阵列形态排布构成，其中，拜耳阵列的每个彩色滤光片对应一个滤光片阵列的最小重复单元。举个例子，滤光片阵列是由一个透光率为10％、一个透光率为100％、一个透光率为40％以及一个透光率为70％的滤光片单元构成的最小重复单元以阵列形态排布而成，那么拜耳阵列的每个彩色滤光片的尺寸与一个最小重复单元的尺寸一致，并且重叠。比如红色滤光片对应一个最小重复单元，两个绿色滤光片分别对应一个最小重复单元，然后蓝色滤光片也对应一个最小重复单元。经过两个阵列双重滤光处理后，然后再用双重滤光处理后的子像素点来合成最终的HDR彩色图像的像素点。合成时，可以先用最小重复单元对应的四个子像素点合成一个中级像素点，再用红、绿、绿、蓝四个彩色滤光片覆盖的四个中级像素点合成彩色HDR图像的单个像素点，以得到最终的彩色图像。In some embodiments, in order to obtain a color HDR image, the image sensor is covered with a filter array and a Bayer array at the same time, and the incident light is double-filtered through the two arrays. For example, the filter array covers the photosensitive pixel dot array, the Bayer array covers the filter array, and each color filter unit of the Bayer array corresponds to the minimum repeating unit. In one embodiment, the Bayer array is composed of a square area composed of four filters of red, green, green, and blue arranged in an array form, wherein each color filter of the Bayer array corresponds to a filter array The smallest repeating unit. For example, the filter array is the smallest repetition of a filter unit with a transmittance of 10%, a transmittance of 100%, a transmittance of 40%, and a transmittance of 70%. The units are arranged in an array, so the size of each color filter of the Bayer array is consistent with the size of a smallest repeating unit, and overlaps. For example, the red filter corresponds to a minimum repeating unit, two green filters respectively correspond to a minimum repeating unit, and then the blue filter also corresponds to a minimum repeating unit. After two arrays of double filter processing, the sub-pixels after double filter processing are then used to synthesize the pixels of the final HDR color image. When synthesizing, the four sub-pixels corresponding to the smallest repeating unit can be used to synthesize an intermediate pixel, and then the four intermediate pixels covered by the four color filters of red, green, green and blue can be used to synthesize a single pixel of the color HDR image Click to get the final color image.

本申请实施例提供的HDR图像生成方法，在图像传感器的感光像素点阵列中覆盖滤光片阵列，通过滤光片阵列中至少两类不同透光率的滤光片单元对光线进行过滤处理，使到达感光像素点的光线强度呈现更大的区别，然后通过感光像素点阵列采集到的滤光处理后的子像素点来合成HDR图像的像素点，以便得到曝光范围更大的HDR图像。通过这种方式，可以实时的获得HDR图像，无需进行图像的融合处理，并且既能适用全局快门的图像传感器，也能适用于卷帘快门的图像传感器。The HDR image generation method provided by the embodiment of the present application covers the light-sensitive pixel array of the image sensor with a filter array, and filters the light through at least two types of filter units with different light transmittance in the filter array, The intensity of light reaching the photosensitive pixel points is more different, and then the pixel points of the HDR image are synthesized through the filtered sub-pixel points collected by the photosensitive pixel point array, so as to obtain an HDR image with a larger exposure range. In this way, HDR images can be obtained in real time without image fusion processing, and it can be applied to both global shutter image sensors and rolling shutter image sensors.

此外，本申请实施例还提供了一种图像处理芯片，该图像处理芯片用于接收图像传感器采集的滤光处理后的各子像素点的像素值，然后采用这些子像素点合成HDR图像，如图7所示，该图像处理芯片包括处理702器、内存704以及非易失存储器706，所述图像处理芯片，通常根据该芯片的实际功能，还可以包括其他硬件，对此不再赘述。非易失存储器706上存储有计算机程序，所述处理702器执行所述计算程序执行说明书实施例中任意一项所述的方法，具体实施细节可参考上述HDR图像的生成方法中的各实施例，在此不再赘述。In addition, the embodiments of the present application also provide an image processing chip, which is used to receive the pixel value of each sub-pixel after filtering and processing collected by the image sensor, and then use these sub-pixels to synthesize an HDR image, such as As shown in FIG. 7, the image processing chip includes a processor 702, a memory 704, and a non-volatile memory 706. The image processing chip may generally include other hardware according to the actual function of the chip, which will not be repeated here. A computer program is stored on the non-volatile memory 706, and the processor 702 executes the calculation program to execute the method described in any one of the embodiments of the specification. For specific implementation details, please refer to each embodiment in the above HDR image generation method. , I won’t repeat it here.

进一步地，本申请实施例还提供了一种摄像装置，如图8所示，该摄像装置包括图像传感器81和图像处理芯片82，图像传感器81包括感光像素点阵列811以及覆盖于所述感光像素点阵列的滤光片阵列812，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括少两类滤光片，不同类别的滤光片的透光率不同，单个所述滤光片单元对应单个所述感光像素点；Further, an embodiment of the present application also provides a camera device. As shown in FIG. 8, the camera device includes an image sensor 81 and an image processing chip 82. The image sensor 81 includes a photosensitive pixel dot array 811 and covers the photosensitive pixel. A dot-array filter array 812. The filter array includes a plurality of smallest repeating units arranged in an array. The smallest repeating unit includes less than two types of filters. Different types of filters can transmit light Different rates, a single said filter unit corresponds to a single said photosensitive pixel;

在某些实施例中，所述预设区域中的子像素点上覆盖的滤光片的类别与所述最小重复单元中滤光片的类别相同。In some embodiments, the type of the filter covered on the sub-pixel points in the preset area is the same as the type of the filter in the smallest repeating unit.

在某些实施例中，所述最小重复单元包括第一类滤光片单元和第二类滤光片单元，其中，第一类滤光片单元的透光率大于第一预设阈值，第二类滤光片单元的透光率小于第二预设阈值。In some embodiments, the minimum repeating unit includes a first-type filter unit and a second-type filter unit, wherein the light transmittance of the first-type filter unit is greater than a first preset threshold, and the first type of filter unit has a light transmittance greater than a first preset threshold. The light transmittance of the second type filter unit is less than the second preset threshold.

在某些实施例中，所述最小重复单元还包括第三类滤光片单元，第三类滤光片单元的透光率位于第一类滤光片单元的透光率和第二类滤光片单元的透光率之间。In some embodiments, the minimum repeating unit further includes a third-type filter unit, and the light transmittance of the third-type filter unit is located between the light transmittance of the first-type filter unit and the second-type filter unit. Between the light transmittance of the light sheet unit.

在某些实施例中，所述第一类滤光片单元的透光率为100％，所述第二类滤光片单元的透光率为10％。In some embodiments, the light transmittance of the first type filter unit is 100%, and the light transmittance of the second type filter unit is 10%.

在某些实施例中，所述最小重复单元中的不同类别的滤光片单元为一个或多个。In some embodiments, there are one or more different types of filter units in the minimum repeating unit.

在某些实施例中，所述最小重复单元为所述滤光片单元构成的正方形区域。In some embodiments, the smallest repeating unit is a square area formed by the filter unit.

在某些实施例中，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值包括：In some embodiments, synthesizing the pixel value of a single pixel of the HDR image based on the pixel value of the sub-pixel includes:

确定各子像素点的权重；Determine the weight of each sub-pixel;

基于所述权重以及所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。The pixel value of a single pixel of the HDR image is synthesized based on the weight and the pixel value of the sub-pixel.

在某些实施例中，所述权重基于第一指定子像素点的像素值计算得到。In some embodiments, the weight is calculated based on the pixel value of the first designated sub-pixel.

在某些实施例中，所述第一指定子像素点为覆盖有透光率为50％的滤光片单元的子像素点。In some embodiments, the first designated sub-pixel is a sub-pixel covered with a filter unit with a light transmittance of 50%.

在某些实施例中，当所述子像素点的像素值大于第一阈值或小于第二阈值时，则将所述子像素点的权重设置为0。In some embodiments, when the pixel value of the sub-pixel is greater than the first threshold or less than the second threshold, the weight of the sub-pixel is set to zero.

在某些实施例中，所述基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值，包括：In some embodiments, the synthesizing the pixel value of a single pixel of the HDR image based on the pixel value of the sub-pixel includes:

基于覆盖有透光率为100％的滤光片单元的子像素点的像素值确定所述HDR图像的单个像素点的像素值；Determining the pixel value of a single pixel of the HDR image based on the pixel value of the sub-pixel covered with a filter unit with a light transmittance of 100%;

基于第二指定子像素点的像素值范围和预先设置的像素值映射表确定所述HDR图像的单个像素点的像素值，其中，所述像素值映射表用于记录所述第二指定子像素点的像素值范围与所述HDR图像单个像素点的像素值的对应关系。Determine the pixel value of a single pixel of the HDR image based on the pixel value range of the second designated sub-pixel and a preset pixel value mapping table, where the pixel value mapping table is used to record the second designated sub-pixel Correspondence between the pixel value range of a point and the pixel value of a single pixel point of the HDR image.

在某些实施例中，所述第二指定子像素点为未经过滤光处理的子像素点。In some embodiments, the second designated sub-pixel is a sub-pixel that has not been filtered.

在某些实施例中，所述预设区域的尺寸与所述最小重复单元的尺寸一致。In some embodiments, the size of the predetermined area is consistent with the size of the smallest repeating unit.

在某些实施例中，在某些实施例中，所述过滤处理后的HDR图像为灰度图像或彩色图像。In some embodiments, in some embodiments, the HDR image after the filtering process is a grayscale image or a color image.

在某些实施例中，当所述过滤处理后的HDR图像为彩色图像时，所述图像传感器还包括一个拜耳阵列，所述彩色图像基于所述拜耳阵列得到。In some embodiments, when the filtered HDR image is a color image, the image sensor further includes a Bayer array, and the color image is obtained based on the Bayer array.

在某些实施例中，所述拜耳阵列覆盖于所述滤光片阵列之上，所述拜耳阵列的每个彩色滤光片单元对应所述最小重复单元。In some embodiments, the Bayer array covers the filter array, and each color filter unit of the Bayer array corresponds to the smallest repeating unit.

本申请实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现本说明说实施例中任意一项所述的方法，具体实施细节可参考上述HDR图像的生成方法中的各实施例，在此不再赘述。The embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any one of the embodiments of this description is implemented. For specific implementation details, please refer to the above The various embodiments in the method for generating an HDR image will not be repeated here.

本申请可采用在一个或多个其中包含有程序代码的存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。计算机可用存储介质包括永久性和非永久性、可移动和非可移动媒体，可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括但不限于：相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program codes. Computer usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.

需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. The terms "including", "including" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, articles, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

以上对本申请实施例所提供的方法和装置进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。The methods and devices provided in the embodiments of the present application are described in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. Core idea; At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a limitation of the present invention .

Claims

一种HDR图像生成方法，其特征在于，所述方法包括：An HDR image generation method, characterized in that the method includes:

接收图像传感器采集的滤光处理后的图像的各像素点的像素值，其中，所述图像传感器包括感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括至少两类滤光片单元，不同类别的所述滤光片单元的透光率不同，单个所述滤光片单元对应单个所述感光像素点；Receive the pixel value of each pixel of the filtered image collected by the image sensor, wherein the image sensor includes an array of photosensitive pixel points and a filter array covering the array of photosensitive pixel points, and the filter The array includes a plurality of minimum repeating units arranged in an array form. The minimum repeating unit includes at least two types of filter units. The light transmittances of the filter units of different types are different, and a single filter unit Corresponding to a single said photosensitive pixel;

从所述滤光处理后的图像中确定一预设区域，将所述预设区域中的所述像素点作为子像素点，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。Determine a preset area from the filtered image, use the pixels in the preset area as sub-pixels, and synthesize a single HDR image based on the pixel values of the sub-pixels The pixel value of the pixel.
根据权利要求1所述的HDR图像生成方法，所述预设区域中的子像素点上覆盖的滤光片的类别与所述最小重复单元中滤光片的类别相同。The HDR image generation method according to claim 1, wherein the type of the filter covered on the sub-pixel points in the preset area is the same as the type of the filter in the smallest repeating unit.
根据权利要求1所述的HDR图像生成方法，其特征在于，所述最小重复单元包括第一类滤光片单元和第二类滤光片单元，其中，第一类滤光片单元的透光率大于第一预设阈值，第二类滤光片单元的透光率小于第二预设阈值。The HDR image generation method according to claim 1, wherein the minimum repeating unit comprises a first type filter unit and a second type filter unit, wherein the light transmission of the first type filter unit The light transmittance of the second-type filter unit is less than the second preset threshold.
根据权利要求3所述的HDR图像生成方法，其特征在于，所述最小重复单元还包括第三类滤光片单元，第三类滤光片单元的透光率位于第一类滤光片单元的透光率和第二类滤光片单元的透光率之间。The HDR image generation method according to claim 3, wherein the minimum repeating unit further comprises a third-type filter unit, and the light transmittance of the third-type filter unit is located in the first-type filter unit Between the transmittance and the transmittance of the second type of filter unit.
根据权利要求3所述的HDR图像生成方法，其特征在于，所述第一类滤光片单元的透光率为100％，所述第二类滤光片单元的透光率为10％。3. The HDR image generation method according to claim 3, wherein the light transmittance of the first type filter unit is 100%, and the light transmittance of the second type filter unit is 10%.
根据权利要求1所述的HDR图像生成方法，其特征在于，所述最小重复单元中的不同类别的滤光片单元为一个或多个。The method for generating an HDR image according to claim 1, wherein there are one or more filter units of different types in the minimum repeating unit.
根据权利要求1所述的HDR图像生成方法，其特征在于，所述最小重复单元为所述滤光片单元构成的正方形区域。The HDR image generation method according to claim 1, wherein the smallest repeating unit is a square area formed by the filter unit.
根据权利要求1-7任一项所述的HDR图像生成方法，其特征在于，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值包括：The HDR image generation method according to any one of claims 1-7, wherein the synthesizing the pixel value of a single pixel of the HDR image based on the pixel value of the sub-pixel comprises:

确定各子像素点的权重；Determine the weight of each sub-pixel;

基于所述权重以及所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。The pixel value of a single pixel of the HDR image is synthesized based on the weight and the pixel value of the sub-pixel.
根据权利要求8所述的HDR图像生成方法，其特征在于，所述权重基于第一指定子像素点的像素值计算得到。8. The method for generating an HDR image according to claim 8, wherein the weight is calculated based on the pixel value of the first designated sub-pixel.
根据权利要求9所述的HDR图像生成方法，其特征在于，所述第一指定子像素点为覆盖有透光率为50％的滤光片单元的子像素点。The HDR image generation method according to claim 9, wherein the first designated sub-pixel point is a sub-pixel point covered with a filter unit with a light transmittance of 50%.
根据权利要求8所述的HDR图像生成方法，其特征在于，当所述子像素点的像素值大于第一阈值或小于第二阈值时，则将所述子像素点的权重设置为0。The HDR image generation method according to claim 8, wherein when the pixel value of the sub-pixel is greater than a first threshold or less than a second threshold, the weight of the sub-pixel is set to zero.
根据权利要求1-7任一项所述的HDR图像生成方法，其特征在于，所述基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值，包括：The method for generating an HDR image according to any one of claims 1-7, wherein the synthesizing the pixel value of a single pixel of the HDR image based on the pixel value of the sub-pixel comprises:

基于第二指定子像素点的像素值范围和预先设置的像素值映射表确定所述HDR图像的单个像素点的像素值，其中，所述像素值映射表用于记录所述第二指定子像素点的像素值范围与所述HDR图像单个像素点的像素值的对应关系。Determine the pixel value of a single pixel of the HDR image based on the pixel value range of the second designated sub-pixel and a preset pixel value mapping table, where the pixel value mapping table is used to record the second designated sub-pixel Correspondence between the pixel value range of a point and the pixel value of a single pixel point of the HDR image.
根据权利要求12任一项所述的HDR图像生成方法，其特征在于，所述第二指定子像素点为未经过滤光处理的子像素点。The method for generating an HDR image according to any one of claims 12, wherein the second designated sub-pixel is a sub-pixel that has not been filtered.
根据权利要求1所述的HDR图像生成方法，其特征在于，所述预设区域的尺寸与所述最小重复单元的尺寸一致。The HDR image generation method according to claim 1, wherein the size of the preset area is consistent with the size of the smallest repeating unit.
根据权利要求1所述的HDR图像生成方法，其特征在于，The HDR image generation method according to claim 1, wherein:

所述HDR图像为灰度图像或彩色图像。The HDR image is a grayscale image or a color image.
根据权利要求15所述的HDR图像生成方法，当所述HDR图像为彩色图像时，所述图像传感器还包括一个拜耳阵列，所述彩色图像基于所述拜耳阵列得到。The HDR image generation method according to claim 15, when the HDR image is a color image, the image sensor further includes a Bayer array, and the color image is obtained based on the Bayer array.
根据权利要求16所述的HDR图像生成方法，所述拜耳阵列覆盖于所述滤光片阵列之上，所述拜耳阵列的每个彩色滤光片单元对应所述最小重复单元。The HDR image generation method according to claim 16, wherein the Bayer array covers the filter array, and each color filter unit of the Bayer array corresponds to the minimum repeating unit.
一种用于生成HDR图像的滤光片阵列，其特征在于，所述滤光片阵列包括：A filter array for generating HDR images, wherein the filter array includes:

多个以阵列形态排布的最小重复单元；Multiple smallest repeating units arranged in an array;

所述最小重复单元包括至少两类滤光片单元，不同类别的滤光片单元的透光率不同。The minimum repeating unit includes at least two types of filter units, and different types of filter units have different light transmittances.
一种图像传感器，其特征在于，所述图像传感器包括：An image sensor, characterized in that the image sensor comprises:

感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括至少两类滤光片，不同类别的滤光片的透光率不同，单个所述滤光片单元对应单个所述感光像素点。A photosensitive pixel point array and a filter array covering the photosensitive pixel point array, the filter array including a plurality of minimum repeating units arranged in an array, the minimum repeating unit including at least two types of filters Different types of filters have different light transmittances, and a single filter unit corresponds to a single photosensitive pixel.
根据权利要求19所述的图像传感器，所述图像传感器还包括一个拜耳阵列。The image sensor according to claim 19, said image sensor further comprising a Bayer array.
一种图像处理芯片，其特征在于，所述图像处理芯片包括处理器、存储器和存储在所述存储器上的计算机程序，所述处理器执行所述计算程序执行权利要求1-17所述的方法。An image processing chip, wherein the image processing chip includes a processor, a memory, and a computer program stored on the memory, and the processor executes the calculation program to execute the method of claims 1-17 .
一种摄像装置，其特征在于，包括图像传感器和图像处理芯片，所述图像传感器包括感光像素点阵列以及覆盖于所述感光像素点阵列的滤光片阵列，所述滤光片阵列包括多个以阵列形态排布的最小重复单元，所述最小重复单元包括少两类滤光片，不同类别的滤光片的透光率不同，单个所述滤光片单元对应单个所述感光像素点；An imaging device, characterized by comprising an image sensor and an image processing chip, the image sensor comprising a photosensitive pixel point array and a filter array covering the photosensitive pixel point array, the filter array comprising a plurality of The smallest repeating unit arranged in an array, the smallest repeating unit includes at least two types of filters, different types of filters have different light transmittances, and a single filter unit corresponds to a single photosensitive pixel;

所述图像传感器采集到滤光处理后的图像的各像素点的像素值后，发送给所述图像处理芯片，所述图像处理芯片从所述滤光处理后的图像中确定一预设区域，将所述预设区域中的感光像素点作为子像素点，并基于所述子像素点的像素值合成所述HDR图像的单个像素点的像素值。The image sensor collects the pixel value of each pixel of the filtered image and sends it to the image processing chip, and the image processing chip determines a preset area from the filtered image, Taking the photosensitive pixels in the preset area as sub-pixels, and synthesizing the pixel values of individual pixels of the HDR image based on the pixel values of the sub-pixels.
一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现权利要求1至17任意一项所述的方法。A computer-readable storage medium with a computer program stored thereon, and when the program is executed by a processor, the method according to any one of claims 1 to 17 is realized.