TWI423670B - Image sampling method and image processing method thereof - Google Patents
Image sampling method and image processing method thereof Download PDFInfo
- Publication number
- TWI423670B TWI423670B TW98138645A TW98138645A TWI423670B TW I423670 B TWI423670 B TW I423670B TW 98138645 A TW98138645 A TW 98138645A TW 98138645 A TW98138645 A TW 98138645A TW I423670 B TWI423670 B TW I423670B
- Authority
- TW
- Taiwan
- Prior art keywords
- pixel
- field
- integer
- pixel units
- during
- Prior art date
Links
Landscapes
- Color Television Image Signal Generators (AREA)
Description
本發明是有關於一種影像取樣及處理方法,且特別是有關於一種藉由隔行掃描(interlaced scanning)以及在交替配置列中的鋸齒掃描(serrated scanning)取樣影像的方法。The present invention relates to an image sampling and processing method, and more particularly to a method of sampling images by interlaced scanning and serrated scanning in alternately arranged columns.
影像感測器用以將聚焦在感測器上的光學影像轉譯成電子訊號。電荷耦合元件(charge-coupled device,CCD)為一種眾所周知的影像感測器。由於需要CCD的特定製造過程,因此包含CCD的積體電路將十分昂貴。另外,CCD也造成大的功率消耗。最近,一種藉由CMOS製程生產的主動畫素感測器(Active -Pixel Sensor,APS)吸引到許多注意,由於其具有整合電路的能力,例如將控制電路、驅動電路以及訊號處理電路整合至單一感應晶片。CMOS APS的好處在於,相較CCD其具有低功率消耗、並與其週邊相關的電子晶片的製程具相容性以及較低的生產成本。The image sensor is used to translate an optical image focused on the sensor into an electronic signal. A charge-coupled device (CCD) is a well-known image sensor. The integrated circuit including the CCD will be very expensive due to the specific manufacturing process of the CCD. In addition, the CCD also causes large power consumption. Recently, an Active-Pixel Sensor (APS) produced by the CMOS process has attracted a lot of attention due to its integrated circuit capability, such as integrating control circuits, drive circuits, and signal processing circuits into a single Inductive wafer. The advantage of CMOS APS is that it has lower power consumption than CCDs, and has compatibility with the peripherals of the electronic wafers and lower production costs.
然而,CMOS影像感測器的影像品質受到雜訊的影響將降低其效能。這些雜訊包括熱雜訊(即KTC雜訊)、1/f雜訊以及固定圖形雜訊(fixed pattern noise)。熱雜訊為有關於影像資料的取樣,1/f雜訊有關於用於放大影像訊號的電路,而固定圖形雜訊則有關於畫素陣列中行之間的不均勻性。這些雜訊造成CMOS APS相較於CCD具有較低的低靈敏度或取樣的低動態範圍的主要因素。改善靈敏度及動態範圍的其中一方法為在固定畫面速率下儘可能增加曝光時間。However, the image quality of CMOS image sensors is affected by noise and will reduce its performance. These noises include thermal noise (ie, KTC noise), 1/f noise, and fixed pattern noise. Thermal noise is about sampling of image data, 1/f noise is about the circuit used to amplify the image signal, and fixed graphics noise is about the unevenness between the lines in the pixel array. These noises cause the CMOS APS to have a lower low sensitivity or a lower dynamic range of sampling than the CCD. One way to improve sensitivity and dynamic range is to increase the exposure time as much as possible at a fixed picture rate.
圖1為一影像感測器的示意圖。請參照圖1,影像感測器100包括排列成列與行的多個畫素單元110。畫素單元110包括光二極體Pd以及電晶體T1~T3。假設隔行掃描系統符合美國國家電視標準委員會(National Television Standards Committee,NTSC)制定的標準,影像可被分為一奇圖場(odd field)與一偶圖場(even field)來顯示,且隔行掃描系統的畫面速率為每圖場1/60秒。為便於說明,以畫素單元110在(X1,Y1)為例,畫素單元110的座標標示為(X,Y)。參照圖1,在電晶體T1的控制下,光二極體Pd的節點N1最初為重置至參考電壓Vref,其中電晶體T1在列線RST1為主動時被開啟。在足夠的曝光時間後,控制線RD1主動開啟電晶體T3,如此在節點N1經由源極跟隨電晶體T2轉譯的光二極體電壓可經由行線RT1被讀出。然後,光二極體電壓將被取樣並保持在一接著的相關雙重取樣(correlated doubled sampling,CDS)電路(未繪示)。依照上述原理,每列的光二極體被曝光以產生電壓訊號至相應的行線,以回應列線的狀態和相應的控制線狀態。Figure 1 is a schematic diagram of an image sensor. Referring to FIG. 1, the image sensor 100 includes a plurality of pixel units 110 arranged in columns and rows. The pixel unit 110 includes a photodiode Pd and transistors T1 to T3. Assuming that the interlaced scanning system conforms to the standards set by the National Television Standards Committee (NTSC), images can be displayed as an odd field and an even field, and interlaced. The system's picture rate is 1/60 second per field. For convenience of explanation, the pixel unit 110 takes (X1, Y1) as an example, and the coordinates of the pixel unit 110 are denoted by (X, Y). Referring to FIG. 1, under the control of the transistor T1, the node N1 of the photodiode Pd is initially reset to the reference voltage Vref, wherein the transistor T1 is turned on when the column line RST1 is active. After a sufficient exposure time, the control line RD1 actively turns on the transistor T3, so that the photodiode voltage translated at the node N1 via the source follower transistor T2 can be read out via the row line RT1. The photodiode voltage will then be sampled and held in a subsequent correlated doubled sampling (CDS) circuit (not shown). In accordance with the principles described above, each column of photodiodes is exposed to generate a voltage signal to a corresponding row line in response to the state of the column line and the corresponding control line state.
關於影像感測器100,典型的拜耳模型(Bayer pattern)彩色濾光片安排部署在畫素單元110上,如此每個畫素單元110僅感測紅、綠、藍其中任何一方的影像資訊。為了使每個畫素顯示全彩影像,一種稱為色彩內插(color interpolation)的算術計算須被使用在畫素矩陣上。圖2為自圖1之影像感測器100取樣的畫素陣列示意圖。請參照圖2,以3X3畫素矩陣210a為例,三個連續的列資料須進行色彩內插。為了便於說明,畫素在(X,Y)的影像資料標示為紅資料(R)、綠資料(G)或藍資料(B),其中影像資料藉由取樣影像資訊而獲得。當畫素矩陣210a展示時,畫素資料在三列X1~X3與三行Y1~Y3的交叉點為用以***在(X2,Y2)的畫素。例如,在(X2,Y2)的畫素的紅資料為畫素在(X1,Y2)和(X3,Y2)的紅資料的平均。在(X2,Y2)的畫素的綠資料為在(X1,Y1),(X1,Y3),(X2,Y2),(X3,Y1)和(X3,Y3)的畫素的綠資料的平均。在(X2,Y2)的畫素的藍資料為在(X2,Y1)和(X2,Y3)的畫素的藍資料的平均Regarding the image sensor 100, a typical Bayer pattern color filter arrangement is disposed on the pixel unit 110 such that each pixel unit 110 senses only image information of any of red, green, and blue. In order for each pixel to display a full color image, an arithmetic calculation called color interpolation must be used on the pixel matrix. 2 is a schematic diagram of a pixel array sampled from the image sensor 100 of FIG. 1. Referring to FIG. 2, taking the 3×3 pixel matrix 210a as an example, three consecutive columns of data must be color interpolated. For convenience of explanation, the image data of the pixel in (X, Y) is marked as red data (R), green data (G) or blue data (B), wherein the image data is obtained by sampling image information. When the pixel matrix 210a is displayed, the intersection of the pixel data in the three columns X1 to X3 and the three rows Y1 to Y3 is a pixel for insertion in (X2, Y2). For example, the red data of the pixels in (X2, Y2) is the average of the red data of the pixels in (X1, Y2) and (X3, Y2). The green data of the pixels in (X2, Y2) are the green data of the pixels in (X1, Y1), (X1, Y3), (X2, Y2), (X3, Y1) and (X3, Y3). average. The blue data of the pixels in (X2, Y2) is the average of the blue data of the pixels in (X2, Y1) and (X2, Y3)
請同時參照圖1及圖2,藉由光柵掃描(raster scanning)和隔行掃描,畫素單元110在奇圖場期間依X1,X3,X5,...X511的順序被掃瞄並被取樣,且對應的影像資料可被獲得以顯示奇圖場。接著,畫素單元110在偶圖場期間依X2,X4,X6,...X512的順序被掃瞄並被取樣,且對應的影像資料可被獲得以顯示偶圖場。只有奇列在奇圖場期間被掃瞄,且只有偶列在偶圖場期間被掃瞄。明顯地,需要額外的畫面緩衝器儲存奇圖場資料或偶圖場資料以執行色彩內插,而其佔據了顯著的成本,且更進一步來說,色彩內插需要至少三個連續列資料,因此色彩內插不能即時被執行。Referring to FIG. 1 and FIG. 2 simultaneously, by raster scanning and interlaced scanning, the pixel unit 110 is scanned and sampled in the order of X1, X3, X5, . . . , X511 during the odd field. And the corresponding image data can be obtained to display the odd field. Next, the pixel unit 110 is scanned and sampled in the order of X2, X4, X6, ..., X512 during the even field, and the corresponding image data can be obtained to display the even field. Only the odd columns are scanned during the odd field and only the even columns are scanned during the even field. Obviously, an additional picture buffer is needed to store odd field data or even field data to perform color interpolation, which occupies significant cost, and further, color interpolation requires at least three consecutive columns of data. Therefore color interpolation cannot be performed immediately.
請參照圖1,至於隔行掃描系統,由於光二極體Pd在每條奇列可於奇圖場期間的開始時間至下一奇圖場期間的開始時間的期間被曝光,因此畫素單元110的最大曝光時間為1/30秒。若在隔行掃描系統中使用漸進的掃描,畫素單元110在奇圖場期間及偶圖場期間依X1,X2,X3,...X512的順序被取樣。因此,其需要不只至少三個線緩衝器儲存畫素資料以進行色彩內插,還需每個畫素單元縮減一半的曝光時間,如此在黑暗環境中影像的訊雜比將被減低。如何藉由維持曝光時間的最大值以增強畫素單元的影像品質,並排除隔行掃描系統之畫面緩衝器的額外成本花費已成為CMOS感測器晶片設計者須改善的一項重要課題。Referring to FIG. 1, as for the interlaced scanning system, since the photodiode Pd is exposed during the start time of each odd column during the odd field period to the start time of the next odd field period, the pixel unit 110 is exposed. The maximum exposure time is 1/30 second. If a progressive scan is used in the interlaced scanning system, the pixel unit 110 is sampled in the order of X1, X2, X3, ..., X512 during the odd field and during the even field. Therefore, it requires more than at least three line buffers to store pixel data for color interpolation, and each pixel unit needs to be reduced by half of the exposure time, so that the image-to-noise ratio of the image will be reduced in a dark environment. How to enhance the image quality of the pixel unit by maintaining the maximum exposure time and eliminating the extra cost of the image buffer of the interlaced scanning system has become an important issue for the CMOS sensor chip designer to improve.
因此,本發明提供一種影像感測器的影像取樣方法,影像感測器藉由在交替配置列中的鋸齒掃描取樣畫素單元。此取樣方法適用於隔行掃描系統且有益於接著的即時色彩內插而不需花費畫面緩衝器的額外成本。Therefore, the present invention provides an image sampling method for an image sensor, which scans a pixel unit by sawtooth scanning in an alternate configuration column. This sampling method is suitable for interlaced scanning systems and is beneficial for subsequent instant color interpolation without the additional cost of the picture buffer.
本發明提出一種影像感測器的影像取樣方法。影像感測器包括在多列與多行的交叉點上的多個畫素單元。在奇圖場期間或偶圖場期間,取樣在奇列與奇行交叉點的畫素單元和在偶列與偶行交叉點的畫素單元。在剩下的偶圖場期間或奇圖場期間,取樣在偶列和奇行交叉點的畫素單元和在奇列和偶行交叉點的畫素單元。The invention provides an image sampling method for an image sensor. The image sensor includes a plurality of pixel units at intersections of multiple columns and multiple rows. During the odd field or during the even field, the pixel elements at the intersection of the odd and odd lines and the pixel units at the intersection of the even and even lines are sampled. During the remaining even field or during the odd field, the pixel elements at the intersection of the even and odd lines and the pixel units at the intersection of the odd and even lines are sampled.
本發明提出一種用以色彩內插的影像處理方法。首先,提供一影像感測器,其包括在多列與多行的交叉點上的多個畫素單元,且各畫素單元感測紅、綠、藍的影像資訊其中之一。接著,在奇圖場期間或偶圖場期間,取樣在奇列與奇行交叉點的畫素單元和在偶列與偶行交叉點的畫素單元。另外,在剩下的偶圖場期間或奇圖場期間,取樣在偶列與奇行交叉點的畫素單元和在奇列與偶行交叉點的畫素單元。繼之,利用在特定區塊中的多個畫素資料***特定畫素至特定區塊中,其中畫素資料為在奇圖場期間或偶圖場期間取樣於畫素單元。The invention proposes an image processing method for color interpolation. First, an image sensor is provided that includes a plurality of pixel units at intersections of a plurality of columns and a plurality of rows, and each pixel unit senses one of red, green, and blue image information. Next, during the odd field or the even field, the pixel elements at the intersection of the odd and odd lines and the pixel units at the intersection of the even and even lines are sampled. In addition, during the remaining even field or during the odd field, the pixel elements at the intersection of the even and odd lines and the pixel units at the intersection of the odd and even lines are sampled. Then, a plurality of pixel data in a specific block is used to insert a specific pixel into a specific block, wherein the pixel data is sampled in the pixel unit during the odd field or the even field.
基於上述,本發明提供適於隔行掃描系統取樣影像感測器的影像取樣方法。在隔行掃描系統中,影像感測器感測的影像可被分為奇圖場和偶圖場。奇圖場包含取樣自在奇列與奇行交叉點的畫素單元和在偶列與偶行交叉點的畫素單元的畫素資料。奇圖場包含取樣自在偶列與奇行交叉點的畫素單元和在奇列與偶行交叉點的畫素單元的畫素資料。由於各畫素單元感測光線的主要色彩中其一的影像資訊,本發明提供影像處理方法可利用對上述在奇圖場或偶圖場的畫素資料進行色彩內插,以在相對較少的記憶緩衝器***一全彩影像Based on the above, the present invention provides an image sampling method suitable for an interlaced scanning system sampled image sensor. In an interlaced scanning system, an image sensed by an image sensor can be divided into an odd field and an even field. The odd field contains the pixel data of the pixel unit sampled at the intersection of the odd and odd lines and the pixel unit at the intersection of the even and even lines. The odd field contains the pixel data of the pixel unit sampled at the intersection of the even and odd lines and the pixel unit at the intersection of the odd and even lines. Since each pixel unit senses image information of one of the main colors of the light, the present invention provides an image processing method that can perform color interpolation on the pixel data in the odd field or the even field to be relatively small. Memory buffer inserted into a full color image
為讓本發明之上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明如下。The above described features and advantages of the present invention will be more apparent from the following description.
圖3為根據本發明一實施例之影像感測器的示意圖。請參照圖3,影像感測器300包括在m列及n行的交叉點的多個畫素單元310。在此假設本實施例具有512列(列R1~列R512),以及512行(行C1~行512),亦即m=512、n=512,且影像感測器300被CMOS主動畫素感測器(APS).所實現。在本實施例所揭露的內容中,R1,R3,R5,...R511意指奇列,而R2,R4,R6,...R512意指偶列,而C1,C3,C5,...C511意指奇行,C2,C4,C6,...C512意指偶行。影像感測器300中的畫素單元310用以轉譯一聚焦在其上的光影像為電子訊號。為了顯示影像,畫素單元310所感測到的影像資訊須被取樣至相應的畫素資料,之後再對畫素資料進行後期處理。本實施例之影像取樣方法適於隔行掃描系統,亦即影像被分為一奇圖場和一偶圖場而進行顯示。3 is a schematic diagram of an image sensor in accordance with an embodiment of the present invention. Referring to FIG. 3, the image sensor 300 includes a plurality of pixel units 310 at intersections of m columns and n rows. It is assumed here that the embodiment has 512 columns (column R1 to column R512), and 512 rows (rows C1 to 512), that is, m=512, n=512, and the image sensor 300 is sensed by the CMOS main animation. Detector (APS). In the content disclosed in this embodiment, R1, R3, R5, ... R511 means odd columns, and R2, R4, R6, ... R512 means even columns, and C1, C3, C5, .. .C511 means odd lines, C2, C4, C6, ... C512 means even lines. The pixel unit 310 in the image sensor 300 is configured to translate an optical image focused thereon into an electronic signal. In order to display the image, the image information sensed by the pixel unit 310 must be sampled to the corresponding pixel data, and then the pixel data is post-processed. The image sampling method of this embodiment is suitable for an interlaced scanning system, that is, the image is divided into an odd field and an even field for display.
請繼續參照圖3,如圖中的虛線箭號所示,位於奇列R1,R3,R5,...R511和奇行C1,C3,C5,...C511交叉點的畫素單元310,和位於偶列R2,R4,R6,...R512和偶行C2,C4,C6...C512交叉點的畫素單元310在奇圖場期間被取樣。在本實施例中,畫素單元310藉由在交替配置列中的鋸齒掃描被取樣。為了便於說明,每個畫素單元310在第i列和第j的交叉點表示為(i,j)。在(2x-1,2y-1)的畫素單元310和在(2x,2y)的畫素單元310在奇圖場期間依序地被取樣,其中x為1~(m/2)的整數,而y為相應於各個x變數的1~(n/2)的整數。舉例來說,為了組成奇圖場的第一條線(亦即x=1),在(R1,C1),(R2,C2),(R1,C3),(R2,C4),...(R1,C511)以及(R2,C512)的畫素單元310被取樣。為了組成奇圖場的第二條線(亦即x=2),在(R3,C1),(R4,C2),(R3,C3),(R4,C4),...(R3,C511)以及(R4,C512)的畫素單元310被取樣,以此類推組成奇圖場的第x條線。Please continue to refer to FIG. 3, as shown by the dotted arrow in the figure, the pixel unit 310 located at the intersection of the odd columns R1, R3, R5, ... R511 and the odd lines C1, C3, C5, ... C511, and The pixel unit 310 located at the intersection of the even columns R2, R4, R6, ... R512 and the even rows C2, C4, C6 ... C512 is sampled during the odd field. In the present embodiment, the pixel unit 310 is sampled by zigzag scanning in an alternate configuration column. For convenience of explanation, each pixel unit 310 is represented as (i, j) at the intersection of the ith column and the jth. The pixel unit 310 at (2x-1, 2y-1) and the pixel unit 310 at (2x, 2y) are sequentially sampled during the odd field, where x is an integer of 1~(m/2) And y is an integer of 1~(n/2) corresponding to each x variable. For example, to form the first line of the odd field (ie, x=1), at (R1, C1), (R2, C2), (R1, C3), (R2, C4),... The pixel units 310 of (R1, C511) and (R2, C512) are sampled. In order to form the second line of the odd field (ie, x=2), at (R3, C1), (R4, C2), (R3, C3), (R4, C4), ... (R3, C511) And the pixel unit 310 of (R4, C512) is sampled, and so on, to form the xth line of the odd field.
另外,如實線箭號所示,位於偶列R2,R4,R6,...R512和奇行C1,C3,C5,...C511交叉點的畫素單元310,和位於奇列R3,R5,...R511和偶行C2,C4,C6...C512交叉點的畫素單元310在偶圖場期間被取樣。在本實施例中,位於(2x,2y-1)的畫素單元310和位於(2x+1,2y)的畫素單元310在偶圖場期間被取樣。舉例來說,為了組成偶圖場的第一條線(亦即x=1),在(R2,C1),(R3,C2),(R2,C3),(R3,C4),...(R2,C511)以及(R3,C512)的畫素單元310被取樣。為了組成偶圖場的第二條線(亦即x=2),在(R4,C1),(R5,C2),(R4,C3),(R5,C4),...(R4,C511)以及(R5,C512)的畫素單元310被取樣,以此類推組成奇圖場的第x條線。In addition, as indicated by the solid arrow, the pixel unit 310 located at the intersection of the even columns R2, R4, R6, ... R512 and the odd rows C1, C3, C5, ... C511, and the odd columns R3, R5, The pixel unit 310 of the intersection of ... R511 and the even lines C2, C4, C6 ... C512 is sampled during the even field. In the present embodiment, the pixel unit 310 at (2x, 2y-1) and the pixel unit 310 at (2x+1, 2y) are sampled during the even field. For example, to form the first line of the even field (ie, x=1), at (R2, C1), (R3, C2), (R2, C3), (R3, C4),... The pixel units 310 of (R2, C511) and (R3, C512) are sampled. In order to form the second line of the even field (ie, x=2), at (R4, C1), (R5, C2), (R4, C3), (R5, C4), ... (R4, C511) And the pixel unit 310 of (R5, C512) is sampled, and so on, to form the xth line of the odd field.
在本實施例中,影像感測器300符合拜耳模型(Bayer pattern),亦即每個畫素單元310只感測光線的多個主要色彩之一的影像資訊,例如紅(R)、綠(G)、藍(B),而紅、綠、藍的比例在拜耳模型中為1:2:1。因此,每個畫素單元310可包括一佈置於其上的彩色濾光片。圖4A為圖3之影像感測器300所感測到的畫素陣列示意圖。請同時參照圖3和圖4A,藉由安排適當的彩色濾光片,在多行中的其中一行(例如行C2)的畫素單元310感測綠影像資訊(G),且在鄰近行C2的行(例如行C1)中每兩個鄰近的畫素單元310,交替地感測紅影像資訊(R)和藍影像資訊(B)。另外,在鄰近C2的其他行(例如行C3)中每兩個鄰近的畫素單元310,交替地感測藍影像資訊(B)和紅影像資訊(R)。因此,畫素陣列400中每筆紅(R)、綠(G)、藍(B)其中之一的畫素資料藉由取樣相應的畫素單元310而被獲得。一般而言,藉由如畫素陣列400形成的影像常被稱為原影像,因此色彩內插應可被使用於諸如此類的原影像以顯示一全彩影像。In this embodiment, the image sensor 300 conforms to the Bayer pattern, that is, each pixel unit 310 senses only image information of one of a plurality of main colors of light, such as red (R), green ( G), blue (B), and the ratio of red, green, and blue is 1:2:1 in the Bayer model. Thus, each pixel unit 310 can include a color filter disposed thereon. 4A is a schematic diagram of a pixel array sensed by the image sensor 300 of FIG. 3. Referring to FIG. 3 and FIG. 4A simultaneously, by arranging an appropriate color filter, the pixel unit 310 of one of the plurality of rows (for example, the line C2) senses the green image information (G), and is adjacent to the line C2. Each of the two neighboring pixel units 310 in the row (e.g., row C1) alternately senses red image information (R) and blue image information (B). In addition, blue image information (B) and red image information (R) are alternately sensed for every two adjacent pixel units 310 in other rows adjacent to C2 (e.g., row C3). Therefore, pixel data of one of each of red (R), green (G), and blue (B) in the pixel array 400 is obtained by sampling the corresponding pixel unit 310. In general, images formed by, for example, pixel array 400 are often referred to as original images, so color interpolation should be used for such original images to display a full color image.
如前所述,影像感測器300藉由隔行掃描和在交替配置列中的鋸齒掃描被取樣。接著的實施例提供一種色彩內插的影像處理方法。圖4B為根據圖3實施例之畫素陣列對應奇圖場的示意圖。請參照圖4B,對應奇圖場的畫素資料藉由在奇圖場期間取樣畫素單元310而被獲得。假設一特定的區塊,例如3×5的區塊被用於在色彩內插時涵蓋畫素資料。首先,當獲得奇圖場的第二條線時,色彩內插被使用在3×5的區塊410a中的畫素資料。依據上述在奇圖場的掃描順序,3×5的區塊410a包括在(R1,C1),(R2,C2),(R1,C3),(R2,C4),(R1,C5),(R3,C1),(R3,C3)和(R3,C5)的畫素,而此些畫素被用以***一特定的畫素,例如在(R2,C3)的畫素,其為在3×5的區塊410a中央的畫素。舉例來說,在(R2,C3)被***的畫素紅資料為在3×5的區塊410a中畫素紅資料的平均值,在(R2,C3)被***的畫素綠資料為在3×5的區塊410a中畫素綠資料的平均值,在(R2,C3)被***的畫素藍資料為在3×5的區塊410a中畫素藍資料的平均值。因此,在(R2,C3)的畫素包含全色彩。As previously described, image sensor 300 is sampled by interlaced scanning and zigzag scanning in alternating configuration columns. The following embodiments provide an image processing method for color interpolation. 4B is a schematic diagram of a corresponding pixel field corresponding to a pixel array according to the embodiment of FIG. 3. Referring to FIG. 4B, the pixel data corresponding to the odd field is obtained by sampling the pixel unit 310 during the odd field. Suppose a particular block, such as a 3x5 block, is used to cover pixel data when color interpolation. First, when the second line of the odd field is obtained, the color interpolation is used for the pixel data in the 3x5 block 410a. According to the above scanning sequence in the odd field, the 3×5 block 410a is included in (R1, C1), (R2, C2), (R1, C3), (R2, C4), (R1, C5), ( R3, C1), (R3, C3) and (R3, C5) pixels, and these pixels are used to insert a specific pixel, such as the pixel in (R2, C3), which is at 3 The pixel in the center of the block 410a of ×5. For example, the pixel red data inserted in (R2, C3) is the average value of the prime red data in the 3×5 block 410a, and the pixel green data inserted in (R2, C3) is The average value of the pixel green data in the block 3a of 3 × 5, and the pixel blue data inserted in (R2, C3) is the average value of the blue data in the block 410a of 3 × 5. Therefore, the pixels in (R2, C3) contain full color.
接著,將3×5的區塊410a往右移動一行並對3×5的區塊410b中的畫素資料進行色彩內插。根據上述在奇圖場期間的掃描順序,3×5的區塊410b包括在(R2,C2),(R1,C3),(R2,C4),(R1,C5),(R2,C6),(R3,C3)以及(R3,C5)的畫素,而此些畫素資料被用以***在(R2,C4)的畫素,其為在3×5的區塊410b中央的畫素。在(R2,C4)被***的畫素紅資料為在3×5的區塊410b中畫素紅資料的平均值,而在(R2,C4)被***的畫素紅資料為在3×5的區塊410b中畫素藍資料的平均值。在(R2,C4)被***的畫素綠資料可為在(R2,C4)的固有的畫素綠資料或在3×5的區塊410b中畫素綠資料的平均值。因此,在(R2,C4)的畫素包含全色彩。值得注意的是,兩個線緩衝器或一個線緩衝器外加幾個緩衝暫存器將足以儲存畫素資料以供進行色彩內插時使用,且色彩內插幾乎可及時地被執行。Next, the 3x5 block 410a is shifted to the right by one line and the pixel data in the 3x5 block 410b is color interpolated. According to the above scanning sequence during the odd field, the 3×5 block 410b is included in (R2, C2), (R1, C3), (R2, C4), (R1, C5), (R2, C6), (R3, C3) and (R3, C5) pixels, and such pixel data is used to insert pixels at (R2, C4), which are pixels in the center of the 3x5 block 410b. The pixel red data inserted in (R2, C4) is the average value of the prime red data in the 3×5 block 410b, and the pixel red data inserted in (R2, C4) is 3×5. The average of the blue data in block 410b. The pixel green data inserted at (R2, C4) may be an average of the pixel green data in (R2, C4) or the pixel green data in the 3x5 block 410b. Therefore, the pixels in (R2, C4) contain full color. It is worth noting that two line buffers or one line buffer plus several buffer registers will be sufficient to store the pixel data for color interpolation, and color interpolation can be performed almost in time.
圖4C為根據圖3實施例之畫素陣列對應偶圖場的示意圖。請參照圖4C,在畫素陣列中相應於偶圖場的畫素資料在偶圖場期間藉由取樣畫素單元310而獲得。當獲得偶圖場的第二條線時,畫素內插被用於3×5的區塊410c中的畫素資料。根據上述在偶圖場期間的掃描順序,3×5的區塊410c包括在(R2,C1),(R3,C2),(R2,C3),(R3,C4),(R2,C5),(R4,C1),(R4,C3)以及(R4,C5)的畫素,而此些畫素資料被用以***在(R3,C3)的畫素。在對3×5的區塊410c進行畫素內插後,將3×5的區塊410c向右移動一行,以獲得3×5的區塊410d。3×5的區塊410d包括在(R3,C2),(R2,C3),(R3,C4),(R2,C5),(R3,C6),(R4,C3)以及(R4,C5)的畫素,而此些畫素資料被用以***在(R3,C4)的畫素。畫素內插的方式與圖4之實施例相同,因此不再贅述。4C is a schematic diagram of a corresponding pixel field corresponding to a pixel array according to the embodiment of FIG. 3. Referring to FIG. 4C, the pixel data corresponding to the even field in the pixel array is obtained by sampling the pixel unit 310 during the even field. When the second line of the even field is obtained, pixel interpolation is used for the pixel data in the 3x5 block 410c. According to the above scanning sequence during the even field, the 3×5 block 410c is included in (R2, C1), (R3, C2), (R2, C3), (R3, C4), (R2, C5), (R4, C1), (R4, C3) and (R4, C5) pixels, and these pixel data are used to insert pixels in (R3, C3). After pixel interpolation is performed on the 3 × 5 block 410c, the 3 × 5 block 410c is shifted to the right by one line to obtain a 3 × 5 block 410d. The 3×5 block 410d is included in (R3, C2), (R2, C3), (R3, C4), (R2, C5), (R3, C6), (R4, C3), and (R4, C5). The pixels are used, and the pixel data is used to insert the pixels in (R3, C4). The manner of pixel interpolation is the same as that of the embodiment of FIG. 4, and therefore will not be described again.
為了使通常熟習此技藝者易於實現本發明,在此舉另一影像取樣方法的實施例。圖5為依據本發明另一實施例之畫素感測器的示意圖。請參照圖5,畫素感測器500包括在m列及n行的交叉點的多個畫素單元510(例如m=512、n=512)。如圖5的虛線箭號所示,位於偶列R2,R4,R6,...R512和奇行C1,C3,C5,...C511交叉點的畫素單元510,和位於奇列R1,R3,R5,...R511和偶行C2,C4,C6...C512交叉點的畫素單元510在奇圖場期間被取樣。每個畫素單元510在第i列和第j的交叉點表示為(i,j)。在(2x,2y-1)的畫素單元510和在(2x-1,2y)的畫素單元510在奇圖場期間依序地被取樣,其中x為1~(m/2)的整數,而y為相應於各個x變數的1~(n/2)的整數。舉例來說,為了組成奇圖場的第一條線(亦即x=1),在(R2,C1),(R1,C2),(R2,C3),(R1,C4),...(R2,511)以及(R1,C512)的畫素單元510被取樣。為了組成奇圖場的第二條線(亦即x=2),在(R4,C1),(R3,C2),(R4,C3),(R3,C4),...(R4,C511)以及(R3,C512)的畫素單元510被取樣,以此類推,組成奇圖場的第x條線。In order to make it easier for those skilled in the art to practice the invention, another embodiment of an image sampling method is presented herein. FIG. 5 is a schematic diagram of a pixel sensor according to another embodiment of the present invention. Referring to FIG. 5, the pixel sensor 500 includes a plurality of pixel units 510 (for example, m=512, n=512) at the intersection of the m columns and the n rows. As shown by the dotted arrow in Fig. 5, the pixel unit 510 located at the intersection of the even columns R2, R4, R6, ... R512 and the odd rows C1, C3, C5, ... C511, and the odd columns R1, R3 The pixel units 510 of the R5, ... R511 and even C2, C4, C6 ... C512 intersections are sampled during the odd field. Each pixel unit 510 is represented as (i, j) at the intersection of the ith column and the jth. The pixel unit 510 at (2x, 2y-1) and the pixel unit 510 at (2x-1, 2y) are sequentially sampled during the odd field, where x is an integer of 1~(m/2) And y is an integer of 1~(n/2) corresponding to each x variable. For example, to form the first line of the odd field (ie, x=1), at (R2, C1), (R1, C2), (R2, C3), (R1, C4),... The pixel units 510 of (R2, 511) and (R1, C512) are sampled. In order to form the second line of the odd field (ie, x=2), at (R4, C1), (R3, C2), (R4, C3), (R3, C4), ... (R4, C511) And the pixel unit 510 of (R3, C512) is sampled, and so on, to form the xth line of the odd field.
另外,如圖5之實線箭號所示,畫素單元510位於奇列R3,R5,R7,...R511和奇行C1,C3,C5,...C511的交叉點,和畫素單元510位於偶列R2,R4,R6,...R512和偶行C2,C4,C6...C512的交叉點在偶圖場期間被取樣。在本實施例中,位於(2x+1,2y-1)的畫素單元510和位於(2x,2y)的畫素單元510在偶圖場期間被取樣。舉例來說,為了組成偶圖場的第一條線(亦即x=1),在(R3,C1),(R2,C2),(R3,C3),(R2,C4),...(R3,511)以及(R4,C512)的畫素單元510被取樣。為了組成偶圖場的第二條線(亦即x=2),在(R5,C1),(R4,C2),(R5,C3),(R4,C4),...(R5,C511)以及(R4,C512)的畫素單元510被取樣,以此類推,組成奇圖場的第x條線。In addition, as shown by the solid arrows in FIG. 5, the pixel unit 510 is located at the intersection of the odd columns R3, R5, R7, ..., R511 and the odd lines C1, C3, C5, ... C511, and the pixel unit. The intersection of 510 at the even columns R2, R4, R6, ... R512 and the even rows C2, C4, C6...C512 is sampled during the even field. In the present embodiment, the pixel unit 510 at (2x+1, 2y-1) and the pixel unit 510 at (2x, 2y) are sampled during the even field. For example, to form the first line of the even field (ie, x=1), at (R3, C1), (R2, C2), (R3, C3), (R2, C4),... The pixel elements 510 of (R3, 511) and (R4, C512) are sampled. In order to form the second line of the even field (ie, x=2), at (R5, C1), (R4, C2), (R5, C3), (R4, C4), ... (R5, C511) And the pixel unit 510 of (R4, C512) is sampled, and so on, to form the xth line of the odd field.
根據上述所揭露實施例,可歸納為下列步驟。圖6為根據本發明一實施例之影像取樣方法的流程圖。請參照圖6,在步驟S601中,在奇圖場期間或偶圖場期間,取樣位於奇列與奇行交叉點的畫素單元和位於偶列與偶行交叉點的畫素單元。在步驟S602中,在剩下的奇圖場期間或偶圖場期間,取樣在偶列和奇行交叉點的畫素單元和在奇列和偶行交叉點的畫素單元。值得注意的是,奇圖場和偶圖場的顯示順序決定於隔行掃描系統。上述實施例以奇圖場早於偶圖場為例,然本發明不以此順序為限。According to the above disclosed embodiments, the following steps can be summarized. 6 is a flow chart of an image sampling method according to an embodiment of the invention. Referring to FIG. 6, in step S601, during the odd field or the even field, the pixel unit located at the intersection of the odd column and the odd line and the pixel unit located at the intersection of the even column and the even line are sampled. In step S602, during the remaining odd field or even field, the pixel units at the intersection of the even and odd lines and the pixel units at the intersection of the odd and even lines are sampled. It is worth noting that the order in which the odd and even fields are displayed is determined by the interlaced scanning system. In the above embodiment, the odd field is earlier than the even field, but the present invention is not limited to this order.
圖7為根據本發明一實施例之影像處理方法的流程圖。請參照圖7,首先,提供一影像感測器(步驟S701),其中影像感測器在多列與多行的交叉點具有多個畫素單元。每個畫素單元感測光線的多個主要色彩之一的影像資訊,例如紅、綠、藍。在步驟S702中,在奇圖場期間或偶圖場期間,取樣在畫素單元奇列與奇行交叉點的畫素單元以及在偶列與偶行交叉點的畫素單元。在步驟S703中,利用在特定區塊中的多個畫素資料***一特定畫素至特定區塊中,其中此些畫素資料為在步驟S702中取樣於畫素單元的畫素資料。在步驟S704中,在剩下的一個奇圖場期間或偶圖場期間,取樣在偶列與奇行交叉點的畫素單元和在奇列與偶行交叉點的畫素單元。在步驟S705中,利用在特定區塊中的多個畫素資料***特定畫素至特定區塊中,其中此些畫素資料為在步驟S704中取樣於畫素單元的畫素資料。FIG. 7 is a flow chart of an image processing method according to an embodiment of the invention. Referring to FIG. 7, first, an image sensor is provided (step S701), wherein the image sensor has a plurality of pixel units at intersections of multiple columns and multiple rows. Each pixel unit senses image information of one of a plurality of dominant colors of light, such as red, green, and blue. In step S702, during the odd field or the even field, the pixel units at the intersection of the odd and odd lines of the pixel unit and the pixel unit at the intersection of the even and even lines are sampled. In step S703, a specific pixel is inserted into a specific block by using a plurality of pixel data in a specific block, wherein the pixel data is pixel data sampled in the pixel unit in step S702. In step S704, during the remaining one of the odd field or the even field, the pixel unit at the intersection of the even and odd lines and the pixel unit at the intersection of the odd and even lines are sampled. In step S705, a plurality of pixel data in a specific block is inserted into a specific pixel, wherein the pixel data is pixel data sampled in the pixel unit in step S704.
綜上所述,上述實施例提供一種適用於隔行掃描系統的影像取樣方法,在奇圖場期間和偶圖場期間藉由在交替配置列中的鋸齒掃描取樣影像感測器。另外,為了顯示全彩影像,在奇圖場期間(或偶圖場期間)自影像感測器的畫素單元取樣並被特定區塊所涵蓋的畫素資料,被用來在特定區塊中***特定畫素。假設隔行掃描系統在上述實施例中符合美國國家電視標準委員會(NTSC)制定的標準或相位轉換線(Phase Alternating Line,PAL)系統,以致奇圖場期間和偶圖場期間為1/60秒(NTSC)或1/50秒(PAL)。因此,影像感測器的曝光時間最大值可達到1/30秒(NTSC)或1/25秒(PAL),也就是奇圖場期間的開始時間至下一奇圖場期間的開始時間之間的時間。另外,由於在三個連續列中的畫素資料為用來執行色彩內插,相較於圖1之先前技術需要大容量的畫面緩衝器以儲存奇圖場資料或偶圖場資料以供進行色彩內插使用,上述實施例僅需兩個線緩衝器或一個線緩衝器外加幾個緩衝暫存器便足以儲存畫素資料以供進行色彩內插時使用。In summary, the above embodiments provide an image sampling method suitable for an interlaced scanning system, in which the image sensor is sampled by zigzag scanning in alternately arranged columns during the odd field and during the even field. In addition, in order to display full-color images, pixel data sampled from the pixel unit of the image sensor and covered by the specific block during the odd field (or during the even field) is used in a specific block. Insert a specific pixel. It is assumed that the interlaced scanning system conforms to the standard or Phase Alternating Line (PAL) system established by the National Television Standards Committee (NTSC) in the above embodiment, so that the odd field period and the even field period are 1/60 second ( NTSC) or 1/50 second (PAL). Therefore, the maximum exposure time of the image sensor can reach 1/30 second (NTSC) or 1/25 second (PAL), which is between the start time of the odd field and the start time of the next odd field. time. In addition, since the pixel data in the three consecutive columns is used to perform color interpolation, a prior art technique of FIG. 1 requires a large-capacity picture buffer to store odd field data or even field data for processing. For color interpolation, the above embodiment requires only two line buffers or one line buffer plus several buffer registers to store pixel data for color interpolation.
雖然本發明已以實施例揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明之精神和範圍內,當可作些許之更動與潤飾,故本發明之保護範圍當視後附之申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.
100、300、500...影像感測器100, 300, 500. . . Image sensor
110、310、510...畫素單元110, 310, 510. . . Pixel unit
T1~T3...電晶體T1~T3. . . Transistor
Pd...光二極體Pd. . . Light diode
RST1~RST512...列線RST1~RST512. . . Column line
RD1~RD512...控制線RD1~RD512. . . Control line
RT1~RT512...行線RT1~RT512. . . Line
N1...節點N1. . . node
Vref...參考電壓Vref. . . Reference voltage
210a...畫素矩陣210a. . . Pixel matrix
410a~410d...區塊410a~410d. . . Block
R1~R512...列R1~R512. . . Column
C1~C512...行C1~C512. . . Row
S601~S602...影像取樣方法的步驟S601~S602. . . Steps for image sampling method
S701~S705...影像處理方法的步驟S701~S705. . . Image processing method steps
圖1為一影像感測器的示意圖。Figure 1 is a schematic diagram of an image sensor.
圖2為取樣自圖1之影像感測器100的畫素陣列示意圖。2 is a schematic diagram of a pixel array sampled from the image sensor 100 of FIG.
圖3為根據本發明一實施例之影像感測器的示意圖。3 is a schematic diagram of an image sensor in accordance with an embodiment of the present invention.
圖4A為圖3之影像感測器300所感測到的畫素陣列示意圖。4A is a schematic diagram of a pixel array sensed by the image sensor 300 of FIG. 3.
圖4B為根據圖3實施例之畫素陣列對應奇圖場的示意圖。4B is a schematic diagram of a corresponding pixel field corresponding to a pixel array according to the embodiment of FIG. 3.
圖4C為根據圖3實施例之畫素陣列對應偶圖場的示意圖。4C is a schematic diagram of a corresponding pixel field corresponding to a pixel array according to the embodiment of FIG. 3.
圖5為依據本發明另一實施例之畫素感測器的示意圖。FIG. 5 is a schematic diagram of a pixel sensor according to another embodiment of the present invention.
圖6為根據本發明一實施例之影像取樣方法的流程圖。6 is a flow chart of an image sampling method according to an embodiment of the invention.
圖7為根據本發明一實施例之影像處理方法的流程圖。FIG. 7 is a flow chart of an image processing method according to an embodiment of the invention.
S601~S602...影像取樣方法的步驟S601~S602. . . Steps for image sampling method
Claims (17)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW98138645A TWI423670B (en) | 2009-11-13 | 2009-11-13 | Image sampling method and image processing method thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW98138645A TWI423670B (en) | 2009-11-13 | 2009-11-13 | Image sampling method and image processing method thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201117603A TW201117603A (en) | 2011-05-16 |
TWI423670B true TWI423670B (en) | 2014-01-11 |
Family
ID=44935283
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW98138645A TWI423670B (en) | 2009-11-13 | 2009-11-13 | Image sampling method and image processing method thereof |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI423670B (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6229578B1 (en) * | 1997-12-08 | 2001-05-08 | Intel Corporation | Edge-detection based noise removal algorithm |
US6833868B1 (en) * | 1998-12-10 | 2004-12-21 | Imec Vzw | Method and device for determining corrected color aspects of a pixel in an imaging device |
US6876388B1 (en) * | 2000-02-02 | 2005-04-05 | Taiwan Advanced Sensors Corporation | Interlaced alternating pixel design for high sensitivity CMOS Image sensors |
-
2009
- 2009-11-13 TW TW98138645A patent/TWI423670B/en not_active IP Right Cessation
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6229578B1 (en) * | 1997-12-08 | 2001-05-08 | Intel Corporation | Edge-detection based noise removal algorithm |
US6833868B1 (en) * | 1998-12-10 | 2004-12-21 | Imec Vzw | Method and device for determining corrected color aspects of a pixel in an imaging device |
US6876388B1 (en) * | 2000-02-02 | 2005-04-05 | Taiwan Advanced Sensors Corporation | Interlaced alternating pixel design for high sensitivity CMOS Image sensors |
Also Published As
Publication number | Publication date |
---|---|
TW201117603A (en) | 2011-05-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR100911231B1 (en) | Cmos image sensor for suppressing degradation of spatial resolution and generating compressed image signals | |
US7961239B2 (en) | CMOS image sensor with interpolated data of pixels | |
KR100855957B1 (en) | Solid state image sensing device compensating brightness of the side display area and driving method thereof | |
US8310573B2 (en) | Solid-state imaging device, signal processing method thereof and image capturing apparatus | |
US20200260054A1 (en) | Solid state imaging device and electronic apparatus | |
US6867806B1 (en) | Interlace overlap pixel design for high sensitivity CMOS image sensors | |
JP2001298748A (en) | Solid-state image pickup device and its drive method, and camera system | |
TW201245767A (en) | Solid-state imaging device and camera system | |
CN101262553A (en) | Image sensor possessing temperature sensor and its driving method | |
TWI690209B (en) | Solid-state imaging device, imaging device and electronic equipment | |
US8947565B2 (en) | Solid-state image capture element and image capture device comprising said solid-state image capture element, and image capture control method and image capture control program | |
JP2010161484A (en) | Solid-state imaging apparatus, method for processing signal of solid-state imaging apparatus, and imaging apparatus | |
JP2010136226A (en) | Solid-state imaging device, method of processing signal of the same, and image capturing apparatus | |
EP1137262A2 (en) | Interlaced alternating pixel design for high sensitivity CMOS image sensors | |
CN102065245B (en) | Image sampling method and image processing method for image sensor | |
US8169521B2 (en) | Image sampling method and image processing method thereof | |
TWI423670B (en) | Image sampling method and image processing method thereof | |
KR100686930B1 (en) | CMOS image sensor | |
JP2009124524A (en) | Imaging apparatus, and captured pixel data correcting method and program | |
JP2005204240A (en) | Solid-state imaging device and its drive method | |
JP2003348605A (en) | Color video camera | |
JP2007189556A (en) | Solid-state image pickup device and driving method thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
MM4A | Annulment or lapse of patent due to non-payment of fees |