TWI730587B - An image format conversion method and device based on rgb-ir image sensor - Google Patents
An image format conversion method and device based on rgb-ir image sensor Download PDFInfo
- Publication number
- TWI730587B TWI730587B TW109101009A TW109101009A TWI730587B TW I730587 B TWI730587 B TW I730587B TW 109101009 A TW109101009 A TW 109101009A TW 109101009 A TW109101009 A TW 109101009A TW I730587 B TWI730587 B TW I730587B
- Authority
- TW
- Taiwan
- Prior art keywords
- pixel
- image
- component
- rgb
- unit
- Prior art date
Links
Images
Landscapes
- Facsimile Image Signal Circuits (AREA)
- Color Image Communication Systems (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
Description
本發明涉及圖像格式轉換技術領域，尤其涉及一種基於RGB-IR圖像感測器的圖像格式轉換方法及裝置。 The present invention relates to the technical field of image format conversion, in particular to an image format conversion method and device based on an RGB-IR image sensor.
RGB-IR感測器係一種新型的圖像感測器，其配置有RGB-IR濾鏡。 請參閱圖1A、圖1B及圖1C，圖1A及圖1B係兩種常見的RGB-IR的濾鏡排列方式，圖1C係四種傳統的拜耳彩色濾鏡陣列(Bayer color filter array，Bayer CFA)的排列方式。相比於傳統的拜耳彩色濾鏡陣列(Bayer CFA)，RGB-IR濾鏡將部分濾鏡替換為紅外濾鏡(Infrared，IR)。多出來的紅外濾鏡有利於感測器夜間補光，進而實現監控系統的夜視技術。 The RGB-IR sensor is a new type of image sensor, which is equipped with an RGB-IR filter. Please refer to Figure 1A, Figure 1B and Figure 1C. Figure 1A and Figure 1B show two common RGB-IR filter arrangements. Figure 1C shows four traditional Bayer color filter arrays (Bayer CFA). ) Arrangement. Compared with the traditional Bayer color filter array (Bayer CFA), the RGB-IR filter replaces some filters with infrared filters (Infrared, IR). The extra infrared filter is helpful for the sensor to fill light at night, and then realize the night vision technology of the surveillance system.
然而，每種濾鏡只允許一種基本顏色的光通過。因此，當需要從RGB-IR圖像感測器得到RGB彩色圖像時，如何將RGB-IR圖像中的每個單顏色分量的像素轉換為具有RGB顏色分量的像素成為一個必須解決的問題。 However, each filter only allows light of one basic color to pass through. Therefore, when the RGB color image needs to be obtained from the RGB-IR image sensor, how to convert each single color component pixel in the RGB-IR image into a pixel with RGB color components becomes a problem that must be solved. .
有鑒於此，有必要提供一種基於RGB-IR圖像感測器的圖像格式轉換方法及裝置。 In view of this, it is necessary to provide an image format conversion method and device based on an RGB-IR image sensor.
本發明提供一種基於RGB-IR圖像感測器的圖像格式轉換方法，所述圖像格式轉換方法包括以下步驟：(a)獲取一RGB-IR圖像； (b)選取所述RGB-IR圖像上的像素為中心像素點，並以所述中心像素點為中心，選取一個像素單元；(c)對所述中心像素點進行插值處理，以獲得所述中心像素點的缺失顏色分量；(d)根據所述步驟(b)及步驟(c)，依次對所述RGB-IR圖像上的每個像素進行插值處理，以獲得每個像素的缺失顏色分量；(e)輸出經過插值處理後的圖像。 The present invention provides an image format conversion method based on an RGB-IR image sensor. The image format conversion method includes the following steps: (a) acquiring an RGB-IR image; (b) Select the pixel on the RGB-IR image as the central pixel, and with the central pixel as the center, select a pixel unit; (c) Interpolate the central pixel to obtain the The missing color component of the central pixel; (d) according to the steps (b) and (c), sequentially interpolate each pixel on the RGB-IR image to obtain the missing color component of each pixel Color component; (e) output the image after interpolation processing.
進一步的，所述RGB-IR圖像的每一像素包含一種基本顏色分量，所述基本顏色分量為R分量、G分量、B分量及IR分量中的任意一種。 Further, each pixel of the RGB-IR image includes a basic color component, and the basic color component is any one of R component, G component, B component, and IR component.
進一步的，所述缺失顏色分量為R分量、G分量及B分量或所述R分量、G分量及B分量的任意兩種顏色分量。 Further, the missing color components are R, G, and B components or any two color components of the R, G, and B components.
進一步的，執行所述步驟(b)之前，還包括：判斷所述像素單元內每個像素的基本顏色分量，並確定所述中心像素點的缺失顏色分量。 Further, before performing the step (b), the method further includes: determining the basic color component of each pixel in the pixel unit, and determining the missing color component of the center pixel.
進一步的，所述像素單元包括邊緣像素單元、IR像素單元及完全像素單元。 Further, the pixel unit includes an edge pixel unit, an IR pixel unit, and a complete pixel unit.
進一步的，所述邊緣像素單元的中心像素點為所述RGB-IR圖像的任一邊緣像素；所述邊緣像素單元包括空白像素，所述空白像素不做參考。 Further, the center pixel of the edge pixel unit is any edge pixel of the RGB-IR image; the edge pixel unit includes blank pixels, and the blank pixels are not for reference.
進一步的，所述IR像素單元的中心像素點為所述RGB-IR圖像的任一IR像素，所述IR像素的缺失顏色分量為R分量、G分量及B分量。 Further, the center pixel of the IR pixel unit is any IR pixel of the RGB-IR image, and the missing color components of the IR pixel are R component, G component, and B component.
進一步的，所述完全像素單元的中心像素點為所述RGB-IR圖像上除了邊緣像素及IR像素的任一像素。 Further, the central pixel of the complete pixel unit is any pixel on the RGB-IR image except for edge pixels and IR pixels.
進一步的，步驟(d)中，從所述RGB-IR圖像的左上角的像素開始插值，當一行像素插值完畢，再從下一行像素的左邊開始插值，直至整個 RGB-IR圖像所有的像素插值完成。 Further, in step (d), interpolation is started from the pixels in the upper left corner of the RGB-IR image, and when the interpolation of one row of pixels is completed, the interpolation is started from the left of the next row of pixels until the entire All pixels of the RGB-IR image are interpolated.
本發明另一方面還提供一種基於RGB-IR圖像感測器的圖像格式轉換裝置，包括：第一獲取單元，用以從一RGB-IR圖像感測器獲取一RGB-IR圖像；第二獲取單元，用以從所述RGB-IR圖像中獲取一中心像素點，並以所述中心像素點為中心，選取一像素單元；插值處理單元，用以對所述中心像素點進行插值，補充所述中心像素點的缺失顏色分量；及圖像輸出單元，用以輸出經過插值處理後的圖像。 Another aspect of the present invention also provides an image format conversion device based on an RGB-IR image sensor, including: a first acquisition unit for acquiring an RGB-IR image from an RGB-IR image sensor The second acquisition unit is used to acquire a central pixel point from the RGB-IR image, and a pixel unit is selected with the central pixel point as the center; the interpolation processing unit is used to compare the central pixel point Perform interpolation to supplement the missing color components of the central pixel; and an image output unit for outputting an image after interpolation processing.
本發明藉由每個像素為中心像素點，逐個獲取像素單元，並以所述像素單元內與所述中心像素點的缺失顏色分量同一顏色的顏色分量的強度值均值，賦值予所述中心像素點，從而得到所述中心像素點的缺失顏色分量，最終完成所述RGB-IR圖像每個像素的單顏色分量至多顏色分量的轉換，解決了RGB-IR圖像單元的圖像格式轉換問題。 The present invention uses each pixel as the central pixel to obtain pixel units one by one, and assigns the average value of the intensity value of the color component of the same color as the missing color component of the central pixel in the pixel unit to the central pixel Point, so as to obtain the missing color component of the center pixel, and finally complete the conversion of the single color component to the multi-color component of each pixel of the RGB-IR image, which solves the image format conversion problem of the RGB-IR image unit .
100:基於RGB-IR圖像感測器的圖像格式轉換裝置 100: Image format conversion device based on RGB-IR image sensor
10:第一獲取單元 10: The first acquisition unit
20:第二獲取單元 20: Second acquisition unit
30:插值處理單元 30: Interpolation processing unit
40:圖像輸出單元 40: Image output unit
圖1A為一種傳統的RGB-IR濾鏡陣列排列示意圖；圖1B為另一種傳統的RGB-IR濾鏡陣列排列示意圖。 FIG. 1A is a schematic diagram of a conventional RGB-IR filter array arrangement; FIG. 1B is a schematic diagram of another conventional RGB-IR filter array arrangement.
圖1C為傳統的拜耳濾鏡陣列的示意圖。 FIG. 1C is a schematic diagram of a traditional Bayer filter array.
圖2係本發明較佳實施方式提供的基於RGB-IR圖像感測器的圖像格式轉換方法的流程圖。 2 is a flowchart of an image format conversion method based on an RGB-IR image sensor provided by a preferred embodiment of the present invention.
圖3係本發明實施方式中求取所述邊緣像素單元的中心像素點的缺失顏色分量的示意圖。 FIG. 3 is a schematic diagram of calculating the missing color component of the center pixel of the edge pixel unit in the embodiment of the present invention.
圖4係本發明實施方式中求取所述IR像素單元的中心像素點的缺失顏色分量的示意圖。 FIG. 4 is a schematic diagram of calculating the missing color component of the center pixel of the IR pixel unit in the embodiment of the present invention.
圖5係本發明實施方式中求取所述完全像素單元的中心像素點的缺失顏色分量的示意圖。 FIG. 5 is a schematic diagram of obtaining the missing color component of the central pixel of the complete pixel unit in the embodiment of the present invention.
圖6係本發明較佳實施方式提供的基於RGB-IR圖像感測器的圖像格式轉換裝置的功能框圖。 Fig. 6 is a functional block diagram of an image format conversion device based on an RGB-IR image sensor provided by a preferred embodiment of the present invention.
下面將結合本發明實施方式中之附圖，對本發明實施方式中之技術方案進行清楚、完整地描述，顯然，所描述之實施方式僅僅係本發明一部分實施方式，而非全部之實施方式。基於本發明中之實施方式，本領域普通技術人員於沒有付出創造性勞動前提下所獲得之所有其他實施方式，都屬於本發明保護之範圍。 The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.
除非另有定義，本文所使用的所有的技術和科學術語與屬於本發明的技術領域的技術人員通常理解的含義相同。本文中於本發明的說明書中所使用的術語只為了描述具體的實施例的目的，不旨於於限制本發明。 Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.
下面結合附圖，對本發明的一些實施方式作詳細說明。於不衝突的情況下，下述的實施例及實施例中的特徵可以相互組合。 In the following, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.
請參閱圖2，本發明較佳實施方式提供一種基於RGB-IR圖像感測器的圖像格式轉換方法。 Please refer to FIG. 2, a preferred embodiment of the present invention provides an image format conversion method based on an RGB-IR image sensor.
可以理解，彩色圖像的每個像素都由一組R、G、B顏色分量來記錄和表達。但圖像感測器只能感應光強而不能分辨色彩資訊。因此需要於RGB-IR圖像感測器前加一個彩色濾鏡陣列，從而使每個像素只能得到一種基本顏色分量。當需要將RGB-IR輸出的RGB-IR圖像轉換成RGB圖像時，每一像素缺失的RGB顏色分量藉由插值得到。於本發明中，為了敘述方便，下文中將圖像感測器上覆蓋紅色濾鏡的像素稱為R像素，類似地，將圖像感測器上覆蓋綠色濾鏡、藍色濾鏡及紅外濾鏡的像素分別稱為G像素、B像素及IR像素。 It can be understood that each pixel of a color image is recorded and expressed by a set of R, G, and B color components. However, the image sensor can only sense light intensity and cannot distinguish color information. Therefore, it is necessary to add a color filter array before the RGB-IR image sensor, so that each pixel can only obtain one basic color component. When it is necessary to convert the RGB-IR image output by the RGB-IR into an RGB image, the missing RGB color components of each pixel are obtained by interpolation. In the present invention, for the convenience of description, the pixels covered with the red filter on the image sensor are referred to as R pixels in the following. Similarly, the image sensor is covered with the green filter, the blue filter and the infrared The pixels of the filter are called G pixels, B pixels, and IR pixels, respectively.
可以理解，根據不同的需求，所述流程圖中步驟的順序可以改變，某些步驟可以省略。所述圖像格式轉換方法包括以下步驟： It can be understood that according to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The image format conversion method includes the following steps:
步驟S1：獲取一RGB-IR圖像。 Step S1: Obtain an RGB-IR image.
可以理解，所述RGB-IR圖像由若干像素組成。每一像素包含一種基本顏色分量，且每一顏色分量對應有一強度值。所述基本顏色分量可為R分量、G分量、B分量及IR分量中的任意一種。例如，R像素的基本顏色分量為R分量。所述R像素對應有一R強度值。所述R強度值決定所述R像素的顏色強度。又如，IR像素的基本顏色分量為IR分量，其並不包含其他顏色分量，例如R分量、G分量及B分量。 It can be understood that the RGB-IR image is composed of several pixels. Each pixel includes a basic color component, and each color component corresponds to an intensity value. The basic color component may be any one of R component, G component, B component and IR component. For example, the basic color component of the R pixel is the R component. The R pixel corresponds to an R intensity value. The R intensity value determines the color intensity of the R pixel. For another example, the basic color component of the IR pixel is the IR component, which does not include other color components, such as R component, G component, and B component.
可以理解，所述RGB-IR圖像中的R像素、G像素、B像素及IR像素，可按照圖1A或圖1B中所示的兩種濾鏡排列方式的任意一種有序排列。於本實施例中，所述RGB-IR圖像中的R像素、G像素、B像素及IR像素，按照圖1A中的第一種濾鏡排列方式進行排列。 It can be understood that the R pixels, G pixels, B pixels, and IR pixels in the RGB-IR image can be arranged in an orderly manner according to any one of the two filter arrangements shown in FIG. 1A or FIG. 1B. In this embodiment, the R pixels, G pixels, B pixels, and IR pixels in the RGB-IR image are arranged according to the first filter arrangement in FIG. 1A.
步驟S2：選取所述RGB-IR圖像上的像素為中心像素點，並以所述中心像素點為中心，選取一個像素單元。 Step S2: Select a pixel on the RGB-IR image as a central pixel, and with the central pixel as a center, select a pixel unit.
可以理解，每一RGB-IR圖像，具有不同的圖像尺寸，通常圖像尺寸以M*N表示。其中M表示圖片寬度，即圖像上每一行具有的像素個數。N表示圖片高度，即圖片上每一列具有的像素個數。如此，所述RGB-IR圖像可依據其像素構成一圖像座標系，其中每個像素具有一圖像座標。 It can be understood that each RGB-IR image has a different image size, and the image size is usually represented by M*N. Where M represents the width of the picture, that is, the number of pixels in each line of the image. N represents the height of the picture, that is, the number of pixels in each column of the picture. In this way, the RGB-IR image can form an image coordinate system according to its pixels, wherein each pixel has an image coordinate.
例如，請一併參閱圖3，針對所述RGB-IR圖像，可以左上角的像素為原點，分別建立u軸及v軸。其中u軸用以表示所述像素所處的列數。所述v軸，用以表示所述像素所處的行數。如此，每一像素可藉由座標(u，v)表示該像素於所述RGB-IR圖像中的位置。例如，所述RGB-IR圖像的左上角像素座標為(1，1)，所述左上角往下一個像素的座標為(1，2)。以此類推， 可獲得整幅RGB-IR圖像中每個像素的座標。可以理解，所述u的取值範圍為1至M，所述v的取值範圍為1至N。 For example, please refer to FIG. 3 together. For the RGB-IR image, the pixel in the upper left corner can be the origin, and the u-axis and v-axis can be established respectively. The u axis is used to indicate the number of columns where the pixel is located. The v axis is used to indicate the number of rows where the pixel is located. In this way, each pixel can represent the position of the pixel in the RGB-IR image by coordinates (u, v). For example, the coordinates of the upper left pixel of the RGB-IR image are (1, 1), and the coordinates of the next pixel from the upper left corner are (1, 2). And so on, The coordinates of each pixel in the entire RGB-IR image can be obtained. It can be understood that the value range of u is 1 to M, and the value range of v is 1 to N.
優選地，所述像素單元的選取可根據具體需求進行調整，例如為可選取包含3*3個像素的像素單元、包含5*5個像素的像素單元或包含5*5個像素的像素單元。於本實施例中，所述像素單元包含5*5個像素。 Preferably, the selection of the pixel unit can be adjusted according to specific requirements, for example, a pixel unit containing 3*3 pixels, a pixel unit containing 5*5 pixels, or a pixel unit containing 5*5 pixels can be selected. In this embodiment, the pixel unit includes 5*5 pixels.
可以理解，於步驟S2中，每選取一個中心像素點，同時獲得所述中心像素點的座標。由於所述像素單元以所述中心像素點為中心，故由所述中心像素點的座標，可推知所述像素單元內每一像素的座標。 It can be understood that, in step S2, each time a central pixel is selected, the coordinates of the central pixel are obtained at the same time. Since the pixel unit is centered on the central pixel point, the coordinates of each pixel in the pixel unit can be inferred from the coordinates of the central pixel point.
步驟S3：對所述中心像素點進行插值處理，以獲得所述中心像素點的缺失顏色分量。 Step S3: Perform interpolation processing on the central pixel to obtain the missing color component of the central pixel.
可以理解，於本實施例中，由於所述RGB-IR圖像中每個像素具有基本顏色分量，故對於所述中心像素點，只需補充該中心像素點RGB分量中的缺失顏色分量。例如，當所述中心像素點為R像素時，所述中心像素點的基本顏色分量為R分量，故只需補充所述中心像素點的G分量及B分量即可。當所述中心像素點為IR像素時，所述中心像素點的基本顏色分量為IR分量，故需補充所述中心像素點的R分量、G分量及B分量。 It can be understood that in this embodiment, since each pixel in the RGB-IR image has a basic color component, for the central pixel, it is only necessary to supplement the missing color component in the RGB component of the central pixel. For example, when the central pixel is an R pixel, the basic color component of the central pixel is the R component, so only the G and B components of the central pixel need to be supplemented. When the central pixel is an IR pixel, the basic color component of the central pixel is an IR component, so the R, G, and B components of the central pixel need to be supplemented.
可以理解，於本實施例中，所述RGB-IR圖像上的每個像素有兩個缺失顏色分量或三個缺失顏色分量。所述插值過程，係選取所述像素單元中所有R分量的R強度值的均值、所有G分量的G強度值的均值、所有B分量的B強度值的均值中的三個值或兩個值，賦值予所述中心像素點，使所述中心像素點的缺失顏色分量具有相應的強度值，從而獲取所述中心像素點缺失的顏色分量。 It can be understood that, in this embodiment, each pixel on the RGB-IR image has two missing color components or three missing color components. The interpolation process is to select three or two of the average of the R intensity values of all R components, the average of G intensity values of all G components, and the average of B intensity values of all B components in the pixel unit. , Assigning values to the central pixel, so that the missing color component of the central pixel has a corresponding intensity value, so as to obtain the missing color component of the central pixel.
因此，對所述中心像素點插值之前，還包括判斷所述像素單元內每個像素的基本顏色分量，並確定所述中心像素點的缺失顏色分量。判斷完成 之後，藉由計算所述像素單元內，與所述中心像素點的缺失顏色分量相同顏色的顏色分量的強度值均值，並將所述強度值均值賦值予所述中心像素點，從而獲得所述中心像素點的缺失顏色分量，完成插值過程。 Therefore, before interpolating the central pixel, it also includes determining the basic color component of each pixel in the pixel unit, and determining the missing color component of the central pixel. Judgment is complete After that, by calculating the average intensity value of the color component of the same color as the missing color component of the central pixel in the pixel unit, and assigning the average intensity value to the central pixel, the The missing color component of the center pixel completes the interpolation process.
可以理解，由於所述RGB-IR圖像中的每個像素有序排列，故可由所述像素單元內每一像素的圖像座標判斷所述每一像素的基本顏色分量。所述判斷方法為：以所述RGB-IR圖像的像素的u座標及v座標向量分別除以4，可得到餘數0或1或2或3。當u座標被整除後得到的餘數為1，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的u軸座標為1；當u座標被整除後得到的餘數為2，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的u軸座標為2；當u座標被整除後得到的餘數為3，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的u軸座標為3；當u座標被整除後得到的餘數為0，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的u軸座標為4。同理可得，當v座標被整除後得到的餘數為1，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的v軸座標為1；當v座標被整除後得到的餘數為2，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的v軸座標為2；當v座標被整除後得到的餘數為3，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的v軸座標為3；當v座標被整除後得到的餘數為0，則與所述像素具有相同顏色分量的圖1A中像素矩陣的像素的v軸座標為4。 It can be understood that since each pixel in the RGB-IR image is arranged in an orderly manner, the basic color component of each pixel can be determined by the image coordinates of each pixel in the pixel unit. The judgment method is: dividing the u-coordinate and v-coordinate vectors of the pixels of the RGB-IR image by 4 respectively to obtain a remainder of 0 or 1 or 2 or 3. When the u-coordinate is divided evenly and the remainder is 1, then the u-axis coordinate of the pixel in the pixel matrix in Figure 1A that has the same color component as the pixel is 1; when the u-coordinate is divided evenly, the remainder is 2, then The u-axis coordinates of the pixels in the pixel matrix in Figure 1A that have the same color components are 2; when the u-coordinate is divisible, the remainder is 3, then the pixel in the pixel matrix in Figure 1A that has the same color component as the pixel has the same color component as that of the pixel matrix in Figure 1A. The u-axis coordinate of the pixel is 3; when the remainder obtained by dividing the u-coordinate is 0, the u-axis coordinate of the pixel in the pixel matrix in FIG. 1A that has the same color component as the pixel is 4. In the same way, when the v coordinate is divided evenly, the remainder is 1, then the v-axis coordinate of the pixel in the pixel matrix in Figure 1A that has the same color component as the pixel is 1; the remainder is obtained when the v coordinate is divided evenly If the pixel has the same color component as the pixel, the v-axis coordinate of the pixel in the pixel matrix in Figure 1A is 2; when the v coordinate is divisible, the remainder is 3, and the pixel has the same color component as the pixel. The v-axis coordinate of the pixel in the pixel matrix in 1A is 3; when the v-coordinate is divisible, the remainder is 0, and the v-axis coordinate of the pixel in the pixel matrix in FIG. 1A that has the same color component as the pixel is 4.
例如，對於所述RGB-IR圖像中座標為(16，1)的像素，將所述像素的u座標與v座標分別除以4。由於16可被4除且餘數為0，1被4除餘數為1，故座標為(16，1)的像素包含的顏色通道與圖1A中圖像座標為(4，1)的像素具有相同的基本顏色分量。 For example, for a pixel with a coordinate of (16, 1) in the RGB-IR image, the u coordinate and v coordinate of the pixel are divided by 4 respectively. Since 16 can be divided by 4 and the remainder is 0, and 1 is divided by 4, the remainder is 1, so the pixel with the coordinate (16, 1) contains the same color channel as the pixel with the coordinate (4, 1) in the image in Figure 1A The basic color components.
可以理解，當所述中心像素點處於圖像的邊緣位置，即所述中心像素點為邊緣像素時，則選取的像素單元將包含空白像素。於本實施例中，所 述空白像素不做參考，即所述像素單元計算RGB各顏色通道的通道均值時不將所述空白像素的個數計入。於本實施例中，根據選取的中心像素點的不同，將所述像素單元分為邊緣像素單元、IR像素單元及完全像素單元。 It can be understood that when the central pixel point is at an edge position of the image, that is, when the central pixel point is an edge pixel, the selected pixel unit will include blank pixels. In this embodiment, the The blank pixels are not for reference, that is, the pixel unit does not count the number of blank pixels when calculating the channel average of each color channel of RGB. In this embodiment, the pixel units are divided into edge pixel units, IR pixel units, and full pixel units according to the difference of the selected central pixel points.
當選取的中心像素點為邊緣像素時，劃定的像素單元為邊緣像素單元，所述邊緣像素單元包含部分空白像素。可以理解，所述邊緣像素的確定與所述像素單元的大小有關。當所述像素單元為n*n像素單元時，且n/2的整數商為c，則所述RGB-IR圖像中的前c行、前c列、最後c行及最後c列的所有像素為邊緣像素。請參閱圖3，於本實施例中，所述像素單元為5*5像素單元，由於5/2的整數商為2，則所述RGB-IR圖像中的前兩列、前兩行、最後兩列及最後兩行的所有像素均為邊緣像素。 When the selected center pixel point is an edge pixel, the defined pixel unit is an edge pixel unit, and the edge pixel unit includes a part of blank pixels. It can be understood that the determination of the edge pixel is related to the size of the pixel unit. When the pixel unit is an n*n pixel unit, and the integer quotient of n/2 is c, then all of the first c rows, the first c columns, the last c rows and the last c columns in the RGB-IR image The pixels are edge pixels. Please refer to FIG. 3. In this embodiment, the pixel unit is a 5*5 pixel unit. Since the integer quotient of 5/2 is 2, the first two columns and the first two rows of the RGB-IR image All the pixels in the last two columns and the last two rows are edge pixels.
當選取的中心像素點為IR像素時，以所述中心像素點選取的像素單元為IR像素單元。 When the selected central pixel point is an IR pixel, the pixel unit selected by the central pixel point is an IR pixel unit.
當選取的中心像素點既不為邊緣像素，也不為IR像素時，所述像素單元為完全像素單元。 When the selected central pixel is neither an edge pixel nor an IR pixel, the pixel unit is a complete pixel unit.
可以理解，每選取一個中心像素點，同時獲得所述中心像素點的座標。藉由所述中心像素點的座標，可判斷所述中心像素點的基本顏色分量、缺失顏色分量，及包含所述中心像素點的像素單元內的每個像素的座標。由於所述RGB-IR圖像上的每一像素有序排列，故由所述像素單元內的每一像素的座標，可判斷所述像素單元內每個像素的基本顏色分量。 It can be understood that each time a central pixel is selected, the coordinates of the central pixel are obtained at the same time. Based on the coordinates of the central pixel, it is possible to determine the basic color component, the missing color component of the central pixel, and the coordinates of each pixel in the pixel unit including the central pixel. Since each pixel on the RGB-IR image is arranged in an orderly manner, the basic color component of each pixel in the pixel unit can be determined from the coordinates of each pixel in the pixel unit.
可以理解，所述RGB-IR圖像上像素的缺失顏色分量為RGB三種顏色分量中缺失的分量，即所述缺失顏色分量為R分量、G分量及B分量中的任意兩種顏色分量或全部顏色分量。 It can be understood that the missing color components of pixels on the RGB-IR image are missing components among the three color components of RGB, that is, the missing color components are any two or all of the R component, G component, and B component. Color components.
所述RGB-IR圖像上的R像素、G像素及B像素分別具有兩個缺失顏色分量，IR像素具有三個缺失顏色分量。進一步的，所述缺失顏色分量分 別對應一顏色強度值，藉由對所述中心像素點的缺失顏色分量的顏色強度值進行賦值，從而使得所述中心像素點獲得相應的缺失顏色分量。可以理解，所述像素單元包含與所述缺失顏色分量相同顏色的對應顏色分量。 The R pixel, G pixel, and B pixel on the RGB-IR image have two missing color components, respectively, and the IR pixel has three missing color components. Further, the missing color component points Do not correspond to a color intensity value, by assigning a value to the color intensity value of the missing color component of the central pixel, so that the central pixel obtains the corresponding missing color component. It can be understood that the pixel unit includes a corresponding color component of the same color as the missing color component.
具體地，統計所述像素單元內對應顏色分量的總數S a 、及所述像素單元內對應顏色分量的強度值的總和S b 。則所述對應顏色分量的顏色強度值的均值S 0=[S b /S a ] (1) Specifically, the sum S b, corresponding to the color components of the pixel unit count the total number of S a, and the pixel unit corresponding to the color component intensity values. Then the average value of the color intensity values of the corresponding color components S 0 =[ S b / S a ] (1)
於本實施方式中，對所述顏色強度值的均值進行向下取整，即對於S b /S a 求得的實際值，忽略其小數點後的值，只取整數。 In the present embodiment, the mean of the color intensity value is rounded down, i.e. the actual value of S b / S a is obtained, ignoring the value after the decimal point, only rounded.
將所述S 0賦值於所述中心像素點的RGB分量中的缺失顏色分量，則所述中心像素點獲取所述缺失顏色分量。 The S 0 is assigned to the missing color component in the RGB components of the central pixel, and the central pixel acquires the missing color component.
步驟S4：根據所述步驟S2及步驟S3，依次對所述RGB-IR圖像上的每個像素進行插值處理，以獲得每個像素的缺失顏色分量。 Step S4: According to the steps S2 and S3, interpolation processing is sequentially performed on each pixel on the RGB-IR image to obtain the missing color component of each pixel.
本實施例中，從所述RGB-IR圖像的左上角的像素開始插值，當一行像素插值完畢，再從下一行像素的左邊開始插值，直至整個RGB-IR圖像所有的像素插值完成。 In this embodiment, the interpolation is started from the pixels in the upper left corner of the RGB-IR image, and when the interpolation of one row of pixels is completed, the interpolation is started from the left of the next row of pixels, until all the pixels of the entire RGB-IR image are interpolated.
S5：輸出經過差值處理後的圖像。 S5: Output the image after the difference processing.
可以理解，於本實施例中，所述經過插值處理後的圖像中的每一個像素包含有R分量、G分量及B分量。 It can be understood that, in this embodiment, each pixel in the image after interpolation processing includes an R component, a G component, and a B component.
請參閱圖3至圖5，下面以像素單元分別為邊緣像素單元、所述像素單元為IR像素單元及所述像素單元為IR像素單元為例，分別對步驟S3進行具體說明。 Referring to FIGS. 3 to 5, step S3 will be specifically described below by taking the pixel units as edge pixel units, the pixel units being IR pixel units, and the pixel units being IR pixel units as examples.
其中，圖3表示當所述像素單元為邊緣像素單元時，求取所述中心像素點的缺失顏色分量的示意圖。圖3中“B1”表示所述像素為B像素，且 所述B像素的強度值為1。同理，“IR6”表示所述像素為IR像素，且所述IR像素的強度值為6。可以理解，圖3至圖5中標出的部分像素強度值僅為方便說明書的敘述，其他像素的強度值於圖中未標示。可以理解，於步驟S3中已判斷出所述邊緣像素單元內每一像素的基本顏色分量。 Wherein, FIG. 3 shows a schematic diagram of obtaining the missing color component of the center pixel when the pixel unit is an edge pixel unit. "B1" in Figure 3 indicates that the pixel is a B pixel, and The intensity value of the B pixel is 1. Similarly, “IR6” means that the pixel is an IR pixel, and the intensity value of the IR pixel is 6. It can be understood that the intensity values of some of the pixels marked in FIGS. 3 to 5 are only for convenience of description, and the intensity values of other pixels are not marked in the figures. It can be understood that the basic color component of each pixel in the edge pixel unit has been determined in step S3.
請參閱圖3，圖3中選取的中心像素點為G像素，則所述中心像素點的缺失顏色分量為R分量及B分量。因此，於本實施例中，需統計所述邊緣像素單元內R分量及B分量的強度值均值。 Please refer to FIG. 3. The selected center pixel in FIG. 3 is a G pixel, and the missing color components of the center pixel are R and B components. Therefore, in this embodiment, the average intensity values of the R component and the B component in the edge pixel unit need to be counted.
於本實施例中，所述邊緣像素單元包含2個R分量，強度值分別為3和9。即所述兩個R分量的強度值的總和為12。所述邊緣像素單元包含兩個B分量，強度值分別為1和11，即所述兩個B分量的強度值的總和為12。將所述像素單元內R分量的總數2及所述R分量的強度值的總和12代入所述公式(1)，得到所述邊緣像素單元內的R分量的強度值均值SR
同理，將所述像素單元內B分量的總數2及所述B分量的強度值的總和12代入所述公式(1)，得到所述邊緣像素單元內的B分量的強度值均值
將所述R分量的強度值均值及所述B分量的強度值均值分別對應賦值予所述中心像素點缺失的R分量及B分量，則所述中心像素點由原來的B像素轉換為具有R分量、G分量及B分量的RGB像素，且所述中心像素點的RGB向量為(6，2，6)。 The average intensity value of the R component and the average intensity value of the B component are respectively assigned to the missing R component and the B component of the central pixel, and the central pixel is converted from the original B pixel to having R RGB pixels of component, G component, and B component, and the RGB vector of the center pixel is (6, 2, 6).
請參閱圖4，圖4為當所述像素單元為IR像素單元時，求取所述中心像素點的缺失顏色分量的示意圖。圖4中選取的中心像素點為IR像素，則所述中心像素點的缺失顏色分量為R分量、G分量及B分量。因此，於本實施 例中，需統計所述IR像素單元內R分量、G分量及B分量的強度值均值。 Please refer to FIG. 4, which is a schematic diagram of obtaining the missing color component of the central pixel when the pixel unit is an IR pixel unit. The central pixel selected in FIG. 4 is an IR pixel, and the missing color components of the central pixel are R component, G component, and B component. Therefore, in this implementation In an example, the average intensity values of the R component, G component, and B component in the IR pixel unit need to be counted.
於本實施例中，所述IR像素單元內包含2個R分量、12個G分量及兩個B分量。同理，由公式(1)可得：所述IR像素單元內的R分量的強度值均值為：
所述IR像素單元內的G分量的強度值均值為：
所述IR像素單元內的B分量的強度值均值為：
將所述R分量的強度值均值、B分量的強度值均值及所述B分量的強度值均值分別對應賦值予所述中心像素點缺失的R分量、G分量及B分量，則所述中心像素點由原來的B像素轉換為具有R分量、G分量及B分量的RGB像素，且所述中心像素點的RGB向量為(13，13，13)。 The average intensity value of the R component, the average intensity value of the B component, and the average intensity value of the B component are respectively assigned to the missing R component, G component, and B component of the central pixel, then the central pixel The point is converted from the original B pixel to an RGB pixel with R component, G component and B component, and the RGB vector of the central pixel is (13, 13, 13).
請參閱圖5，圖5為所述像素單元為完全像素單元時，求取所述中心像素點的缺失顏色分量的示意圖。圖5中選取的中心像素點為G像素，則所述中心像素點的缺失顏色分量為R分量及B分量。因此，於本實施例中，需統計所述完全像素單元內R分量及B分量的強度值均值。 Please refer to FIG. 5, which is a schematic diagram of obtaining the missing color component of the central pixel when the pixel unit is a complete pixel unit. The selected central pixel in FIG. 5 is a G pixel, and the missing color components of the central pixel are R and B components. Therefore, in this embodiment, the average intensity values of the R component and the B component in the complete pixel unit need to be counted.
於本實施例中，所述完全像素單元內包含3個R分量及3個B分量。同理，由公式(1)可得：所述IR像素單元內的R分量的強度值均值為：
所述IR像素單元內的B分量的強度值均值為：
將所述R分量的強度值均值及所述B分量的強度值均值，分別對 應賦值予所述中心像素點缺失的R分量及B分量，則所述中心像素點由原來的B像素轉換為具有R分量、G分量及B分量的RGB像素，且所述中心像素點的RGB向量為(11，13，14)。 The average value of the intensity value of the R component and the average value of the intensity value of the B component are compared to Should be assigned to the missing R and B components of the central pixel, the central pixel is converted from the original B pixel to an RGB pixel with R, G, and B components, and the RGB of the central pixel The vector is (11, 13, 14).
可以理解，本發明藉由對所述RGB-IR圖像上的每一像素進行插值處理，以補充所述每一像素的缺失顏色向量，從而使所述圖像上的每一像素均具有R分量、G分量及B分量。 It can be understood that the present invention supplements the missing color vector of each pixel by performing interpolation processing on each pixel on the RGB-IR image, so that each pixel on the image has R Component, G component and B component.
請參閱圖6，本發明另一方面還提供一種基於RGB-IR圖像感測器的圖像格式轉換裝置100。所述圖像格式轉換裝置100包括第一獲取單元10、第二獲取單元20、插值處理單元30及圖像輸出單元40。
Please refer to FIG. 6, another aspect of the present invention also provides an image
所述第一獲取單元10用以從一RGB-IR圖像感測器獲取一RGB-IR圖像。所述RGB-IR圖像中的像素，按照圖1A或圖1B兩種RGB-IR濾鏡排列方式中的任一種有序排列。
The
所述第二獲取單元20，用以獲取所述RGB-IR圖像上的像素作為中心像素點，並以所述中心像素點為中心，選取一個像素單元。
The second acquiring
可以理解，每選取一個中心像素點，同時獲得所述中心像素點的座標。由於所述像素單元以所述中心像素點為中心，故由所述中心像素點的座標，可推知所述像素單元內每一像素的座標。 It can be understood that each time a central pixel is selected, the coordinates of the central pixel are obtained at the same time. Since the pixel unit is centered on the central pixel point, the coordinates of each pixel in the pixel unit can be inferred from the coordinates of the central pixel point.
所述像素單元為正方形的像素單元，且包含奇數個像素。例如，所述像素單元可包含3*3個像素、5*5個像素或7*7個像素中的一種。可以理解，所述像素單元的大小可根據實際需要進行調整。 The pixel unit is a square pixel unit and includes an odd number of pixels. For example, the pixel unit may include one of 3*3 pixels, 5*5 pixels, or 7*7 pixels. It can be understood that the size of the pixel unit can be adjusted according to actual needs.
所述插值處理單元30，用以對所述中心像素點插值，得到所述中心像素點的缺失顏色分量。
The
所述插值處理單元30包括判斷單元31、統計單元32及轉換單元33。
The
所述判斷單元31用以對所述像素單元內每個像素的座標進行計算，從而判斷所述像素單元內每個像素的基本顏色分量，並確定所述中心像素點的缺失顏色分量。所述統計單元32用以統計所述像素單元內缺失顏色分量的總數及強度值總和，從而計算所述像素單元內對應所述缺失顏色分量的強度值均值。所述轉換單元32用以將所述強度值均值賦值予所述中心像素點，從而得到所述中心像素點的缺失顏色分量。 The determining unit 31 is used for calculating the coordinates of each pixel in the pixel unit, thereby determining the basic color component of each pixel in the pixel unit, and determining the missing color component of the center pixel. The statistics unit 32 is used to count the total number of missing color components in the pixel unit and the sum of the intensity values, so as to calculate the average value of the intensity values corresponding to the missing color components in the pixel unit. The conversion unit 32 is used for assigning the average value of the intensity value to the central pixel, so as to obtain the missing color component of the central pixel.
本發明藉由以每個像素為中心像素點，逐個獲取像素單元，並以所述像素單元內與所述中心像素點的缺失顏色分量同一顏色的顏色分量的強度值均值，賦值予所述中心像素點，從而得到所述中心像素點的缺失顏色分量，最終完成所述RGB-IR圖像每個像素的單顏色分量至多顏色分量的轉換，解決了RGB-IR圖像單元的圖像格式轉換問題。 The present invention acquires pixel units one by one by taking each pixel as the central pixel point, and assigns the average value of the intensity value of the color component of the same color as the missing color component of the central pixel point in the pixel unit to the center Pixel points, so as to obtain the missing color component of the central pixel point, and finally complete the conversion of the single-color component to the multi-color component of each pixel of the RGB-IR image, which solves the image format conversion of the RGB-IR image unit problem.
以上實施方式僅用以說明本發明的技術方案而非限制，儘管參照以上較佳實施方式對本發明進行了詳細說明，本領域的普通技術人員應當理解，可以對本發明的技術方案進行修改或等同替換都不應脫離本發明技術方案的精神和範圍。本領域技術人員還可於本發明精神內做其它變化等用於本發明的設計，只要其不偏離本發明的技術效果均可。這些依據本發明精神所做的變化，都應包含於本發明所要求保護的範圍之內。 The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced. None should deviate from the spirit and scope of the technical solution of the present invention. Those skilled in the art can also make other changes within the spirit of the present invention for the design of the present invention, as long as they do not deviate from the technical effects of the present invention. These changes made in accordance with the spirit of the present invention should all fall within the scope of protection claimed by the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW109101009A TWI730587B (en) | 2020-01-11 | 2020-01-11 | An image format conversion method and device based on rgb-ir image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW109101009A TWI730587B (en) | 2020-01-11 | 2020-01-11 | An image format conversion method and device based on rgb-ir image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TWI730587B true TWI730587B (en) | 2021-06-11 |
| TW202127367A TW202127367A (en) | 2021-07-16 |
Family
ID=77517205
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW109101009A TWI730587B (en) | 2020-01-11 | 2020-01-11 | An image format conversion method and device based on rgb-ir image sensor |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI730587B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686111A (en) * | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
| CN108282644A (en) * | 2018-02-14 | 2018-07-13 | 北京飞识科技有限公司 | A kind of single camera imaging method and device |
| CN109276231A (en) * | 2018-11-15 | 2019-01-29 | 豪威科技(上海)有限公司 | A kind of imaging system and imaging method |
-
2020
- 2020-01-11 TW TW109101009A patent/TWI730587B/en not_active IP Right Cessation
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686111A (en) * | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
| CN108282644A (en) * | 2018-02-14 | 2018-07-13 | 北京飞识科技有限公司 | A kind of single camera imaging method and device |
| CN109276231A (en) * | 2018-11-15 | 2019-01-29 | 豪威科技(上海)有限公司 | A kind of imaging system and imaging method |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202127367A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101006731B (en) | Image processing device and method, and imaging device | |
| KR100982685B1 (en) | System and method for a high performance color filter mosaic array | |
| US9262805B2 (en) | Method and device for processing image in Bayer format | |
| US8854511B2 (en) | Apparatus and method for image processing and storage medium, and image pickup apparatus | |
| CN103416067B (en) | Imaging device | |
| EP3902242B1 (en) | Image sensor and signal processing method | |
| CN101335900A (en) | Image processing device, image processing method, program, and imaging device | |
| CN103597821A (en) | Image pickup apparatus and signal value correction method | |
| US7969479B2 (en) | Method and device for detecting color temperature | |
| CN104717474A (en) | Image processing method and module | |
| US9160999B2 (en) | Image processing device and imaging device | |
| US8970748B2 (en) | Imaging device, storage medium storing a control program for imaging device, and control method for imaging device | |
| CN109285113B (en) | Improved color image interpolation method based on gradient | |
| US20150055861A1 (en) | Methods and Systems for Image Demosaicing | |
| CN108513112B (en) | Image processing method based on color filter array | |
| US7324707B2 (en) | Interpolation method for generating pixel color | |
| TWI730587B (en) | An image format conversion method and device based on rgb-ir image sensor | |
| CN103259960A (en) | Data interpolation method and device and image output method and device | |
| US20080316343A1 (en) | Method and Apparatus For Allowing Access to Individual Memory | |
| CN113115010B (en) | Image format conversion method and device based on RGB-IR image sensor | |
| US6970608B1 (en) | Method for obtaining high-resolution performance from a single-chip color image sensor | |
| JP4805294B2 (en) | Color binning to reduce the image resolution of digital images | |
| US20050094006A1 (en) | Imaging apparatuses, image data processing methods, and articles of manufacture | |
| Lai et al. | Color Interpolation with Full Resolution for Hybrid RGB-IR CMOS Sensor | |
| TWI400961B (en) | Algorithm of interpolation applied to ymcg color filter array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |