1288364 11934twfl .doc/006 96-3-3 玖、發明說明： 潑日日所屬之技術領域 本發明是有關於影像縮放的方法，且較特別的是，有 關於一種利用影像內插法處理影像解析度的方法。 先前技艦 多媒體通訊已經成爲現代網路中非常重要的一環。藉 由網際網路(Internet)及無線通訊，人們可面對面交談並且 互相交換資訊◦然而，因爲數位影像(digital image)及視訊 序列(video sequence)具有較大資料量，而且其網路頻寬爲 有限，所以多媒體通訊的品質會下降。如果低解析度的視 訊序列是在編碼器(encoder)中編碼，而且在解碼器(decoder) 中使用內插（interpolation)技術，將低解析度的視訊序列放 大成高解析度的視訊序列，藉此可節省網路頻寬。 簡要地說，影像內插是一種有關影像放大的技術。目 前已經提出的常見內插法則包括零-順序內插法(zero-order interpolation)、線性內插法（linear interpolation)、以及二次 方卷積內插法（cubic convolution interpolation)，等等。爲解 決人工效應（artifact effects)問題，目前已提出多種使用像 素特性(pixel feature)內插影像的內插法則。然而，決定像 素特性的方法需要較高的計算複雜度，而且無法在視訊序 1288364 11934twfl .doc/006 96-3-3 列中達到實時(real-time)(30訊框(frame)/每秒)的影像放大。 爲解決影像內插的人工效應，目前已經有多種法則提 出。這些法則可根據空間域方向（spatial domain direction)， 使用由該方向變量所決定的方向性加權（directional weights)，將沿各個方向的內插値（interpolated values)組合 在一起。該些法則同時也包含其中的內插像素是分類成兩 個 decimations 的卷積型（convolution-based)。不同的 decimation是分別使用不同的過瀘器（filter)內插。此外，在 習知技藝中也採用將內插***成兩個方向性階段 (directional stages)的卷積型及中央型（median-based)的混 合型。 新邊定向內插（new edge-directed interpolation，NEDI) 法則是一種產生良好的影像放大主觀品質的良好法則。 NEDI法則的基本觀念是從一低解析度影像中，估計本地協 變係數（local covariance coefficient)，接下來使用這些本地 協變係數，來配合在基於低解析度協變與高解析度協變之 間的幾何對偶性（geometric duality)的一較高解析度的內 插。協變型（covariance-based)內插法則具有較高的計算複 雜度。 高解析度協變是根據其本質的，，幾何對偶性”，從其低 1288364 11934twfl.doc/006 96-3-3 解析度對應物件（counterpart)中估計所得。幾何對偶性是鍋 合解析度不同但沿相同方向的像素對(pair of pixels)的高解 析度協變與低解析度協變之間的對應關係 (correspondence)。第1圖係顯示一個當內插交錯點陣 (interlacing lattice) Γ2/仏々+7以成形Γ2/，々時，高解析度協變 A/，Q與低解析度協變4，&之間的幾何對偶性。幾何對偶 性可方便估計2-D訊號的本地協變，而不需明確地估計邊 緣方向（edge orientation)。當從點陣]^(/+y· = even)內插交 錯點陣= odd)時，在第2圖中也可發現相似的幾何 對偶性。事實上，第1圖和第2圖直到縮放係數爲21/2，以 及旋轉係數爲7Γ /4皆爲同構(isomorphic)。 低解析度協變4，&可輕易地從低解析度影像的一本 地視窗中，使用以下的傳統協變方法估計而得： k = 4cTcJ =如 〇 ⑴ 其中爲包含在本地視窗中的像素从的資料 向量，而C爲VxM2的資料矩陣，其第k個行向量是沿對角 方向，第四個最接近h的鄰居。72/ + 7,2; + 7的內插値可由下 列公式計算而得： Y2i+l,2j+l = ΣΣα^+/2(/+/) ( 2 ) λ-=0 1=0 根據傳統的韋納過濾理論(Wiener filtering theory)，最佳最 1288364 11934twfl.doc/006 96-3-3 小平均値平方誤差（optimal minimum mean squared root, MMSE)線性內插係數可由下列公式計算而得： a = R~Jr (3) 根據（1)及(3)，ά可由下列公式計算而得： a = {cTcY(cTy) (4) 其中，NEDI法則可用來將灰階影像的大小，沿每個相 位，放大二的冪次方的倍數。在放大係數爲二的基本範例 中，調整大小的方案包括兩個步驟：第一個步驟是從點陣 6^·內插交錯點陣r2/+/，2y.+7;第二個步驟是從點陣= even)內插另一個交錯點陣= 〇dd)。 即使目前已經提出多種內插法則，但目前使用的內插 法則仍須耗費大量的計算時間來放大影像。而快速的內插 法則仍在硏發當中。 發明內容 有鑑於此，本發明提供一種將影像的低解析度像素 •，放大(zooming)成高解析度像素的影像內插法。本 發明考慮面向邊(edge-orient)，在確保影像品質維持在可接 受的程度的條件下加速內插。 本發明提供一種利用影像內插法處理影像解析度的方 法，該方法包括首先接收低解析度像素7iV。接下來，根據 1288364 11934twfl.doc/006 像素h,.，々及相鄰像素所得的像素差與一臨界値比較所得 的差’決疋β像的—同質區（h0ni0gen0Us area)及一邊緣區 (edge area) ’小於該臨界値者該像素歸屬於該同質區，大 於該臨界値者屬於該邊緣區。接下來，使用一第一內插法 則’內插屬於同質區的像素，以及使用一第二內插法 則’內插屬於邊緣區的像素72i.，27.。 在上述的內插法中，決定影像的同質區及邊緣區的步 驟根據下列二個變數 AYl - \ Y2i,2j — Y2 i +2 p,2j + 2q\ 5 p, q g {(〇,l), (l,〇)} ? = + —匕2/ + 2丨，以及 Δ Y3 — \ Y2i,2j ~ ^2i + 2,2y4-2| 是否滿足來決定該像素是在同質區或邊緣區中： 如果<臨界値，則 像素 ^2i+p,2j + q 是在同質區中 否則 像素+ 是在邊緣區中，當做邊緣像素的其中之 如果h <臨界値，而且G <臨界値，貝[J 像素 是在同質區中 如果Zl h〈臨界値’則 1288364 11934twfl .doc/006 96-3-3 像素是在同質區中 如果<臨界値，則 像素匕/+/力·+/是在同質區中 否則 像素 Y2i + l,2j + l 是在據邊緣區中，當做邊緣像素的其中 之一。 在上述的內插法中，第二內插法則可包含將沿著具有 最小差方向的像素•，內插到鄰近像素（neighboring pixels)中。 在上述的內插法中，該已定的邊緣像素並未包含在像 素6⑶·其中之一的鄰近像素中。 在上述的內插法則中，該最小差是藉由從下列 的四個差中選出一最小値所決定： diffi = \Y2i]，2j-Y2i+1，2j\, diff2 - \Y2i^>2J^ - = 1匕/，々_7 - 72/ 2;+7|，以及 dlff4 - 1^2/ + 7,27-7 - Y2i-L2j + l\^ 其中包含邊緣像素的其中之一的差會將於省略。 在上述的內插法中，像素}是藉由在具最小像素差方 向上計算/ 2 所得。 1288364 11934twfl .doc/006 96-3-3 熟習相關技藝者當知上述的一般說明以及下述的詳細 說明’都僅爲用來說明本發明的範例，其目的爲提供本發 明申請專利範圍的詳細說明。 爲3襄本發明之上述和其他目的、特徵、和優點能更明 顯易懂，下文特以較佳實施例，並配合所附圖式，作詳細 說明如下： 實施方式： 如上所述，目前已經有多種影像內插法則被提出。然 而’傳統的方法至少會增加計算工作負載。爲解決此問題， 本發明提出一種用於影像放大的新內插法則。該法則對影 像放大可提供良好的主觀品質（subjective quality)，並且可 降低計算複雜度。 爲設計該新法則，至少必須考慮降低計算複雜度以及 提供良好主觀品質兩目標。雖然本發明所提供的新內插法 則，最少可有效應用於視訊序列以及視訊會議 (videoconference)。然而，本發明所提供的影像內插法則， 亦可用於放大固定影像。 該影像內插法則最好可根據分析本地結構，來內插影 像。原始影像會被動態地分割成兩個區域：同質區及邊緣 區。而且在不同區中的內插像素是分別對應於不同的內插 法則。 本發明的法則使用一臨界値來決定內插像素是屬於同 質區或邊緣區。其中，該臨界値大約爲像素値完整顯示範 1288364 11934twfl.doc/006 96-3-3 圍的10%。舉例來說，如果灰階影像的完整顯示範圍爲Ο 到255，則臨界値爲25。本發明的法則包括如第3圖所示 的100和102兩個步驟。在第一步驟100的步驟104中， 已經分別定義在3x3的視窗中的水平、垂直及對角線方向 的差。請參考第4圖之（a)〜(c)所示，接下來會一個接一個 的決定在三個方向上的差。如果像素差小於臨界値，則該 像素屬於一同質區，其中該同質像素是使用雙線性(bilinear) 內插法則所內插（步驟106)。如果像素差大於臨界値，則該 像素屬於一邊緣區。在第一步驟100之後，接下來部分剩 餘未內插的像素會屬於邊緣區。接下來，使用本發明所提 出的法則內插邊緣像素，該法則使用鄰近像素資訊以內插 邊緣像素。而且該些鄰近像素包含原始像素及在第一步驟 100中內插的像素。 第5圖係顯示在Lena影像上執行本發明的第一步驟 100之後所得的結果。在影像邊緣上存在部分非內插像素。 在本發明的第二步驟102中，邊緣像素是使用所有包含原 始像素及在第一步驟中內插的像素的鄰近像素所內插。如 第6圖之(a)〜⑷所示，鄰近像素包含黑點(black-points)、灰 點（gray-points)、以及斑點（spot-points)。最小像素差隱含像 素之間的最大關係。此外，邊緣像素是沿著最小差方向內 插。如果所有斑點都屬於在第一步驟1〇〇中內插的像素， 則最小差會出現在穿越白點(邊緣像素）的四個方向上。如果 有任何斑點屬於邊緣像素，則最小差會出現在除邊緣像素 之外的其他方向上。以下將詳細說明本發明所提出的法則。 11 1288364 96-3-3 11934twfl.doc/006 請參考第7圖所示，假設大小爲H x W的低解析度影 像Z，會放大成大小爲2H X 2W的高解析度影像7。 是從不，y所放大，而且藉由使用像素差準則，已經決定同質 像素是在匕+ ^，匕⑶“及Γ2/ + Λ27.〇上。如果這些像素是同 質像素，則這些像素是使用雙線性內插法則所內插。其中， 該像素差準則如下所述： △ Y! = |Y2i,2j — Y2i + 2p,2j + 2q| △ Y2 = |Y2i + 2,2j — Y2i，2j + 2|1288364 11934twfl .doc/006 96-3-3 玖, Invention Description: TECHNICAL FIELD The present invention relates to a method of image scaling, and more particularly, to image resolution using image interpolation Degree method. Previous technology ships multimedia communication has become a very important part of modern networks. Through the Internet and wireless communication, people can talk face to face and exchange information with each other. However, because digital images and video sequences have a large amount of data, and their network bandwidth is Limited, so the quality of multimedia communication will decline. If the low-resolution video sequence is encoded in an encoder and the interpolation technique is used in the decoder, the low-resolution video sequence is amplified into a high-resolution video sequence. This saves network bandwidth. Briefly, image interpolation is a technique for image magnification. Common interpolation rules that have been proposed so far include zero-order interpolation, linear interpolation, and cubic convolution interpolation. In order to solve the problem of artifact effects, various interpolation rules for interpolating images using pixel features have been proposed. However, the method of determining pixel characteristics requires high computational complexity and cannot achieve real-time in the sequence of video sequence 1288364 11934twfl .doc/006 96-3-3 (30 frames per second) ) The image is enlarged. In order to solve the artificial effect of image interpolation, various rules have been proposed. These rules combine interpolated values in all directions using directional weights determined by the direction variable according to the spatial domain direction. These rules also include that the interpolated pixels are convolution-based classified into two decimations. Different decimations are interpolated using different filter filters. In addition, a convolution type and a median-based hybrid type in which interpolation is split into two directional stages are also employed in the prior art. The new edge-directed interpolation (NEDI) rule is a good rule to produce good subjective quality of image magnification. The basic idea of the NEDI rule is to estimate the local covariance coefficient from a low-resolution image, and then use these local covariance coefficients to match the covariation based on low-resolution covariance and high-resolution. A higher resolution interpolation of geometric duality. The covariance-based interpolation method has a high computational complexity. High-resolution covariation is based on its nature, geometric duality, estimated from its lower counterpart, which is 1288364 11934twfl.doc/006 96-3-3 resolution. The geometric duality is the pot resolution. Correspondence between high-resolution covariation and low-resolution covariance of different pairs of pixels in the same direction. Figure 1 shows an interlacing lattice when interpolating lattice Γ2/仏々+7 to form Γ2/, 々, high-resolution covariant A/, Q and low-resolution covariance 4, & geometrical duality. Geometric duality can easily estimate 2-D signal Local covariation without explicitly estimating the edge orientation. When interpolating the interlaced lattice = odd) from the lattice ^(/+y· = even), it can also be found in Figure 2. Similar geometric duality. In fact, Figures 1 and 2 are isomorphic until the scaling factor is 21/2, and the rotation coefficient is 7Γ / 4. Low resolution covariation 4, & From the local window of the low-resolution image, estimate using the following traditional covariation method: k = 4 cTcJ = as 〇(1) where is the data vector of the pixel from the local window, and C is the data matrix of VxM2, the kth row vector is the diagonal direction, the fourth neighbor closest to h. 72/ + 7,2; + 7 interpolation can be calculated by the following formula: Y2i+l, 2j+l = ΣΣα^+/2(/+/) ( 2 ) λ-=0 1=0 according to the traditional Wei Wiener filtering theory, best 1288364 11934twfl.doc/006 96-3-3 The optimal minimum mean squared root (MMSE) linear interpolation coefficient can be calculated by the following formula: a = R~Jr (3) According to (1) and (3), ά can be calculated by the following formula: a = {cTcY(cTy) (4) where NEDI's law can be used to size the grayscale image along each phase. , a multiple of the power of the second power. In the basic example of the amplification factor of two, the resizing scheme consists of two steps: the first step is to interpolate the interlaced lattice r2/+/ from the lattice 6^·, 2y.+7; the second step is to interpolate another interlaced lattice = 〇dd) from lattice = even). Even though various interpolation rules have been proposed so far, the interpolation currently used Nevertheless to spend a lot of computing time to enlarge the image. Quickly interpolation rule still send them WH. SUMMARY OF THE INVENTION In view of the above, the present invention provides an image interpolation method of zooming in a low-resolution pixel of an image into a high-resolution pixel. The present invention contemplates edge-orientation and accelerates interpolation while ensuring that image quality is maintained at an acceptable level. The present invention provides a method of processing image resolution using image interpolation, the method comprising first receiving a low resolution pixel 7iV. Next, according to the difference between the pixel difference of 1288364 11934twfl.doc/006 pixels h, ., 相邻 and adjacent pixels and a threshold ', the homogenous region (h0ni0gen0Us area) and an edge region ( Edge area) 'The pixel is smaller than the threshold, and the pixel belongs to the homogenous region, and the larger than the threshold belongs to the edge region. Next, a pixel belonging to the homogeneous region is interpolated using a first interpolation method, and pixels 72i., 27. belonging to the edge region are interpolated using a second interpolation rule. In the above interpolation method, the steps of determining the homogenous region and the edge region of the image are based on the following two variables AYl - \ Y2i, 2j - Y2 i + 2 p, 2j + 2q \ 5 p, qg {(〇, l) , (l,〇)} ? = + —匕2/ + 2丨, and Δ Y3 — \ Y2i,2j ~ ^2i + 2,2y4-2| Satisfied to determine whether the pixel is in the homogenous or marginal region : If <critical 値, then the pixel ^2i+p, 2j + q is in the homogenous region, otherwise the pixel + is in the edge region, as the edge pixel if h < critical 値, and G < , Bay [J pixel is in the homogenous region if Zl h <critical 値' then 1288364 11934twfl.doc/006 96-3-3 pixels are in the homogenous region if <critical 値, then pixel 匕 / + / force · + / is in the homogenous area, otherwise the pixel Y2i + l, 2j + l is one of the edge pixels in the edge region. In the interpolation described above, the second interpolation rule may include interpolating pixels along the direction having the smallest difference into neighboring pixels. In the above interpolation method, the predetermined edge pixel is not included in the adjacent pixel of one of the pixels 6(3). In the above interpolation rule, the minimum difference is determined by selecting a minimum 从 from the following four differences: diffi = \Y2i], 2j-Y2i+1, 2j\, diff2 - \Y2i^> 2J^ - = 1匕/,々_7 - 72/ 2;+7|, and dlff4 - 1^2/ + 7,27-7 - Y2i-L2j + l\^ which contains one of the edge pixels The difference will be omitted. In the above interpolation method, the pixel} is obtained by calculating / 2 in the direction of the smallest pixel difference. 1288364 11934 TW 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 Description. The above and other objects, features, and advantages of the present invention will become more apparent and understood by the appended claims appended claims claim A variety of image interpolation rules have been proposed. However, the traditional approach at least increases the computational workload. To solve this problem, the present invention proposes a new interpolation rule for image enlargement. This rule provides good subjective quality for image magnification and reduces computational complexity. In order to design this new rule, at least the two objectives of reducing computational complexity and providing good subjective quality must be considered. Although the new interpolation method provided by the present invention is at least effective for video sequences and videoconferences. However, the image interpolation rule provided by the present invention can also be used to enlarge a fixed image. The image interpolation method preferably interpolates the image based on the analysis of the local structure. The original image is dynamically divided into two regions: the homogenous region and the marginal region. Moreover, the interpolated pixels in different regions correspond to different interpolation rules, respectively. The law of the present invention uses a threshold 决定 to determine whether the interpolated pixel belongs to a homogenous or marginal region. Among them, the critical 値 is approximately 10% of the total area of the pixel 1 1288364 11934twfl.doc/006 96-3-3. For example, if the full display range of grayscale images is Ο to 255, the critical 値 is 25. The law of the present invention includes two steps of 100 and 102 as shown in Fig. 3. In step 104 of the first step 100, the difference in horizontal, vertical and diagonal directions in the 3x3 window has been defined, respectively. Please refer to (a) to (c) of Figure 4, and the next one will determine the difference in three directions. If the pixel difference is less than the critical 値, the pixel belongs to a homogenous region, wherein the homogeneous pixel is interpolated using a bilinear interpolation rule (step 106). If the pixel difference is greater than the critical 値, the pixel belongs to an edge region. After the first step 100, the remaining pixels that are not interpolated will belong to the edge region. Next, the edge pixels are interpolated using the rules proposed by the present invention, which uses neighboring pixel information to interpolate the edge pixels. Moreover, the neighboring pixels include the original pixels and the pixels interpolated in the first step 100. Figure 5 shows the results obtained after performing the first step 100 of the present invention on the Lena image. There are some non-interpolated pixels on the edge of the image. In a second step 102 of the invention, the edge pixels are interpolated using all adjacent pixels comprising the original pixel and the pixels interpolated in the first step. As shown in (a) to (4) of Fig. 6, adjacent pixels include black-points, gray-points, and spot-points. The minimum pixel difference implies the largest relationship between pixels. In addition, the edge pixels are interpolated along the direction of the smallest difference. If all the spots belong to the pixels interpolated in the first step 1〇〇, the minimum difference will appear in the four directions that traverse the white point (edge pixels). If any spots are edge pixels, the smallest difference will appear in other directions than the edge pixels. The rules proposed by the present invention will be described in detail below. 11 1288364 96-3-3 11934twfl.doc/006 Please refer to Figure 7. Assume that the low-resolution image Z of size H x W is magnified into a high-resolution image 7 of size 2H X 2W. Never, y is magnified, and by using the pixel difference criterion, it has been decided that the homogenous pixels are at 匕+^, 匕(3)" and Γ2/ + Λ27.〇. If these pixels are homogeneous pixels, then these pixels are used. The bilinear interpolation rule is interpolated. The pixel difference criterion is as follows: △ Y! = |Y2i, 2j — Y2i + 2p, 2j + 2q| △ Y2 = |Y2i + 2,2j — Y2i, 2j + 2|
△ Y3 = |Y2i，2j — Y2i + 2,2j + 2| 如果△ Y! <臨界値，則 Y2i + p，2j + q = (Y2i，2j + Y2i + 2p，2j+2q) / 2 (5) 否則 Y2i + p,2j + q 爲邊緣像素 （6) where p,q e{{〇fl)}(l0)} 如果△ Y2 <臨界値，而且△ Y3 <臨界値，貝[j Δ Ymin = min{ Δ Y2? Δ Y3}△ Y3 = |Y2i,2j — Y2i + 2,2j + 2| If △ Y! < critical 値, then Y2i + p, 2j + q = (Y2i, 2j + Y2i + 2p, 2j + 2q) / 2 ( 5) Otherwise Y2i + p, 2j + q is the edge pixel (6) where p, qe{{〇fl)}(l0)} If △ Y2 < critical 値, and △ Y3 < critical 値, shell [j Δ Ymin = min{ Δ Y2? Δ Y3}
如果 Δ Ymin = Δ Y2 Y2i + l，2j + l = (Y2i + 2,2j + Y2i，2j + 2) / 2 (7) 否則 Y2i+l，2j + l = (Y2i，2j + Y2i + 2,2j+2) / 2 (8) 如果△ Y2 <臨界値，則 Y2i+l，2j + l = (Y2i + 2,2j + Y2i,2j + 2) / 2 (9) 而如果△ Y3 <臨界値，則 Y2i+l，2j+l = (Y2i，2j + Y2i + 2,2j + 2) / 2 (10) 12 1288364 11934twfl.doc/006 96_3·3 那麼 Y2i+l，2j + l爲邊緣像素。 Ο1) 邊緣像素爲在依據像素差準則判定，以及執行下列步驟之 後未內插的像素。 以下將引用一種本發明用於邊緣像素的面向邊調適內 插（Edge-Oriented Adaptive Interpolation)作爲範例加以說 明。 爲求得最小差，以下將說明兩個範例。其中，第一範 例是所有鄰近像素都已經在第一步驟內插。該法則將邊緣 像素內插至四個方向的鄰近像素。首先估計沿著四個方向 的最小差。最小差的方向代表邊緣像素是面向第8圖所示 的方向。最小差法則的程序如下所述： diff! = | Y2i.U2j - Y2i+1?2j| diff2 = |Y2i-l,2j-l - Y2i+l,2j + l| diff3 = |Υ2ί 2Η - Y2i,2j+1| diff4 = |Y2i + l，2j-l - Y2i-l，2j + l| = min{diffk}? for k= 1-4 (12) 在第二範例中，有部分鄰近像素並未在第一步驟中內 插。換言之，部分鄰近像素亦屬於邊緣像素。最小差是使 用剩餘的鄰近像素所求得。以下將說明第二範例。請參考 第8圖所示，舉例來說，假設像素72/_7,2/+/及r2/，2y.d在第 一步驟中並未內插，則最小差是藉由省略這兩個方向所求 得。其程序如下所述： diffi = I Y2i.l52j - Y2i+l,2j| 1288364 11934twfl.doc/006 96-3-3 diff2 = |Y2i-l,2j-l - Y2i+l,2j+i| diffmin - min{diffk}? for k= 1- 2 (13) 從上述的範例中可發現山瓦心的方向。舉例來說，如 果發現吨爲^^，則^〃“.與…，&.緊密相關，而且 r2,.，27.是使用下列公式（14)內插·· ^2i,2j = (^21-1,2] + ^2i+l,2j) ^ ^ (14) 本發明的影像內插法則可使用實驗驗証。這些實驗比 較零順序（Zero-order)、雙線性（Bilinear)、雙三次（Bicubic)、 新邊定向內插（NEDI)、以及本發明所提出的法則的主觀 (subjective)及客觀（objective)品質。這些實驗包含測試六個 灰階影像，包括 Pepper、Milkdrop、Tiffany、Comta卜 Jet、 及Lena，以及三個彩色影像，包括Pepper、Jet、及Lena。 其測試目標爲即時內插與良好主觀品質。測試結果如下所 述。 其中，零順序、雙線性、及雙三次內插法則係爲熟知 的線性內插法，而這些法則的PSNR及計算複雜度的比較 結果，係分別顯示於第1表及第4表中。接下來，同質區 的內插法則會根據上述比較結果決定。雙線性及雙三次內 插法則的PSNR類似，但是雙三次內插法則的計算複雜度 較雙線性內插法則爲高。因此，本發明所提出的法則在第 一步驟採用雙線性內插法則。 14 1288364 11934twfl .doc/006 96-3-3 第1表顯示對六個影像使用不同法則而得的同質區的 PSNR平均値。 零順序 (dB)__ 雙線性 (dB) 雙三次 _ (dB) PSNR 26.61 29.96 __ 30.05 第9圖之(a)〜⑴係顯示邊緣測試步驟的主觀品質。在 第9圖之(a)〜(f)中，所內插的是簡單影像，而且結果顯示 NEDI法則的品質比零順序、雙線性、及雙三次內插法則還 好。本發明提出法則的結果與NEDI法則類似。第10圖之 ⑷〜(f)及第11圖之⑷〜(f)顯不部分的Lena及Pepper影像 的主觀品質。第10圖之(a)〜⑴及第11圖之(a)〜(f)涉及影像 邊緣，而且其內插影像在邊緣上較爲平順。此外，在第1〇 圖之（a)〜(f)中可看出肩膀及帽子邊緣，而在第11圖之（a)〜(f) 中可看出胡椒。第2表顯示將256 X 256的樣本放大成512 X 512，以及將128 X 128的樣本放大成512 X 512的灰階影 像客觀品質。第3表顯示將256 X 256的樣本放大成512 X 5 12,以及將128 X 128的樣本放大成5 12x5 12的彩色影像 客觀品質。本發明所提出的法則幾乎在所有灰階影像中都 具有較佳的客觀品質。根據上述結果，可發現NEDI法則 的客觀品質較差，但其主觀品質較佳。NEDI法則將同質區 的失真隱藏，而且人眼對同質區並不敏感。因此，本發明 15 1288364 11934twfl.doc/006 所提出的法則的客觀品質較nedi法則還好，而且其主觀 品質與NEDI法則類似。 第2表顯示灰階影像的PSNR(dB)比較結果。 256x256 致 512x51 .2 128x128 到 512x512 Peppe r Milkdro P Tiffany Comtal Jet Lena Pepper Milkdr 〇P Tiffany Comtal Jet Lena 噴序 27.53 29.85 27.87 25.39 26.61 29.01 24.39 26.36 26.12 22.01 23.47 25.87 32.19 34.18 29.90 29.60 30.79 34.17 27.52 29.60 27.77 24.58 25.91 28.67 雙三次 32.27 34.31 29.86 29.73 31.19 34.68 27.50 29.66 27.61 24.50 25.97 28.75 NEDI 28.54 29.20 28.58 27.06 28.97 30.10 23.94 24.94 24.86 22.21 23.62 24.98 本發明 提出法 則 33.35 35.34 29.99 29.95 30.77 33.95 28.96 31.16 27.90 24.77 25.99 28.56 第3表顯示彩色影像的PSNR(dB)比較結果。 256x256 到 512x512 128x128 到 512x512 Pepper Jet Lena Pepper Jet Lena 零順序 26.14 25.93 28.04 23.34 23.19 25.14 雙線性 30.01 29.35 32.57 26.25 25.37 27.77 雙三次 29.95 29.62 32.86 26.14 25.38 27.78 NEDI 27.06 27.41 29.11 23.17 23.00 24.46 本發明 提出法 則 30.84 29.45 32.32 27.47 25.49 27.64 16 1288364 11934twfl.doc/006 96-3^ 第4表顯示計算複雜度的比較結果(假設內插n個徵 素）。 加 減 乘 除 位移 反向 零順序 - - - 雙線性 3n 3η όϋι - - - 雙三次 27η 45η 135η 9η - - NEDI 同質 η ------ - η - 邊緣 4η 1288η - - 4η 本發明 提出法 則 同質 η - - η - 邊緣 η 4η ——--- - η -If Δ Ymin = Δ Y2 Y2i + l, 2j + l = (Y2i + 2, 2j + Y2i, 2j + 2) / 2 (7) Otherwise Y2i+l, 2j + l = (Y2i, 2j + Y2i + 2, 2j+2) / 2 (8) If △ Y2 < critical 値, then Y2i+l, 2j + l = (Y2i + 2, 2j + Y2i, 2j + 2) / 2 (9) and if △ Y3 < Critical 値, then Y2i+l, 2j+l = (Y2i, 2j + Y2i + 2, 2j + 2) / 2 (10) 12 1288364 11934twfl.doc/006 96_3·3 Then Y2i+l, 2j + l is the edge Pixel. Ο1) Edge pixels are pixels that are not interpolated after being judged according to the pixel difference criterion and after the following steps are performed. An edge-Oriented Adaptive Interpolation for edge pixels of the present invention will be described below as an example. In order to find the minimum difference, two examples will be explained below. Among them, the first example is that all neighboring pixels have been interpolated in the first step. This rule interpolates edge pixels into adjacent pixels in four directions. First estimate the smallest difference along the four directions. The direction of the smallest difference represents that the edge pixel is facing the direction shown in Fig. 8. The procedure for the minimum difference rule is as follows: diff! = | Y2i.U2j - Y2i+1?2j| diff2 = |Y2i-l,2j-l - Y2i+l,2j + l| diff3 = |Υ2ί 2Η - Y2i, 2j+1| diff4 = |Y2i + l,2j-l - Y2i-l,2j + l| = min{diffk}? for k= 1-4 (12) In the second example, some neighboring pixels are not Interpolated in the first step. In other words, some of the neighboring pixels also belong to the edge pixels. The minimum difference is obtained using the remaining neighboring pixels. The second example will be explained below. Please refer to FIG. 8. For example, if the pixels 72/_7, 2/+/ and r2/, 2y.d are not interpolated in the first step, the minimum difference is by omitting the two directions. Asked for. The procedure is as follows: diffi = I Y2i.l52j - Y2i+l, 2j| 1288364 11934twfl.doc/006 96-3-3 diff2 = |Y2i-l,2j-l - Y2i+l,2j+i| diffmin - min{diffk}? for k= 1- 2 (13) From the above example, the direction of the heart of the hill can be found. For example, if ton is found to be ^^, then 〃". is closely related to ..., &., and r2,., 27. is interpolated using the following formula (14) · ^2i, 2j = (^ 21-1, 2] + ^2i+l, 2j) ^ ^ (14) The image interpolation method of the present invention can be experimentally verified. These experiments compare Zero-order, Bilinear, Bi Bicubic, New Edge Oriented Interpolation (NEDI), and the subjective and objective qualities of the proposed law. These experiments included testing six grayscale images, including Pepper, Milkdrop, Tiffany, Comta, Jet, and Lena, as well as three color images, including Pepper, Jet, and Lena. The test targets are immediate interpolation and good subjective quality. The test results are as follows. Among them, zero order, bilinear, and double The three interpolation rules are well-known linear interpolation methods, and the comparison results of PSNR and computational complexity of these rules are shown in Tables 1 and 4. The interpolation of the homogenous regions is then based on The above comparison results are determined. The PSNR of bilinear and bicubic interpolation is similar. However, the computational complexity of the bicubic interpolation rule is higher than that of the bilinear interpolation rule. Therefore, the law proposed by the present invention adopts the bilinear interpolation rule in the first step. 14 1288364 11934twfl .doc/006 96-3 -3 Table 1 shows the PSNR average 同 of the homogeneous region using different rules for six images. Zero order (dB)__ Bilinear (dB) Bicubic _ (dB) PSNR 26.61 29.96 __ 30.05 Figure 9 (a) ~ (1) shows the subjective quality of the edge test step. In (a) to (f) of Fig. 9, the simple image is interpolated, and the result shows that the quality of the NEDI rule is higher than the zero order, bilinear And the bicubic interpolation method is fine. The result of the proposed rule of the present invention is similar to the NEDI rule. The subjective aspects of the Lena and Pepper images of the (4) to (f) and (11) to (f) Quality. (a) to (1) in Figure 10 and (a) to (f) in Figure 11 relate to the edge of the image, and the interpolated image is smoother on the edge. In addition, in the first figure (a) The shoulders and the edge of the hat can be seen in ~(f), and the pepper can be seen in (a) to (f) in Figure 11. The second table shows the sample of 256 X 256. This is magnified to 512 X 512 and the 128 X 128 sample is magnified to an objective quality of 512 X 512 grayscale image. Table 3 shows a sample of 256 X 256 amplified to 512 X 5 12 and a sample of 128 X 128 Zoom into the objective quality of 5 12x5 12 color images. The law proposed by the present invention has better objective quality in almost all gray scale images. Based on the above results, it can be found that the objective quality of the NEDI rule is poor, but its subjective quality is better. The NEDI rule hides the distortion of the homogeneous region and the human eye is not sensitive to the homogeneous region. Therefore, the objective quality of the proposed rule of the present invention is better than the nedi rule, and its subjective quality is similar to the NEDI rule. Table 2 shows the PSNR (dB) comparison of grayscale images. 256x256 512x51 .2 128x128 to 512x512 Peppe r Milkdro P Tiffany Comtal Jet Lena Pepper Milkdr 〇P Tiffany Comtal Jet Lena Spray sequence 27.53 29.85 27.87 25.39 26.61 29.01 24.39 26.36 26.12 22.01 23.47 25.87 32.19 34.18 29.90 29.60 30.79 34.17 27.52 29.60 27.77 24.58 25.91 28.67 Bicubic 32.27 34.31 29.86 29.73 31.19 34.68 27.50 29.66 27.61 24.50 25.97 28.75 NEDI 28.54 29.20 28.58 27.06 28.97 30.10 23.94 24.94 24.86 22.21 23.62 24.98 The present invention proposes a law 33.35 35.34 29.99 29.95 30.77 33.95 28.96 31.16 27.90 24.77 25.99 28.56 Table 3 shows the color image PSNR (dB) comparison results. 256x256 to 512x512 128x128 to 512x512 Pepper Jet Lena Pepper Jet Lena Zero sequence 26.14 25.93 28.04 23.34 23.19 25.14 Bilinear 30.01 29.35 32.57 26.25 25.37 27.77 Bicubic 29.95 29.62 32.86 26.14 25.38 27.78 NEDI 27.06 27.41 29.11 23.17 23.00 24.46 The present invention proposes a law 30.84 29.45 32.32 27.47 25.49 27.64 16 1288364 11934twfl.doc/006 96-3^ The fourth table shows the comparison of the computational complexity (assuming that n elements are interpolated). Addition, subtraction, multiplication and division, displacement, reverse zero order - - - bilinear 3n 3η όϋι - - - bicubic 27η 45η 135η 9η - - NEDI homogenous η ------ - η - edge 4η 1288η - - 4η The present invention proposes the law homogeneity η - - η - edge η 4η ——--- - η -
每一法則的計算複雜度顯示於第4表中。根據第4表 所示，NEDI法則在邊緣區具有最高的計算複雜度。相較於 所有其他法則，本發明所提出法則的計算複雜度最低。The computational complexity of each rule is shown in Table 4. According to Table 4, the NEDI rule has the highest computational complexity in the marginal zone. The proposed algorithm has the lowest computational complexity compared to all other laws.
綜合上述說明，本發明所提出的法則至少可成功達成 兩個目標，也就是可即時內插及提供與邊緣方向內插相似 的主觀品質。因此，本發明所提出法則可應用於需要将 QCIF轉換爲CIF大小或將CIF轉換爲4CIF大小的視訊會 議上，而且可提升視訊會議的品質。 雖然本發明已以較佳實施例揭露如上，然其並非用以 限定本發明，任何熟習此技藝者，在不脫離本發明之精神 和範圍內，當可作各種之更動與潤飾，因此本發明之保護 17 1288364 11934twfl.doc/006 96-3- 3 範圍當視後附之申請專利範圍所界定者爲準。 圖式簡單說明 第1圖係顯示一個當內插交錯點陣以成形 時，高解析度協變及^與低解析度協變4，j之間的 幾何對偶性的示意圖。 第2圖係顯示一個當從點陣}^·，7·(/+/ = even)內插交錯點 陣= odd)時的一個類似的幾何對偶性的示意圖。 第3圖係顯示一流程圖，用來說明根據本發明一較佳 實施例的一個影像內插法。 第4圖之（a)〜(c)係顯示根據本發明一較佳實施例，在 三個方向中的像素差的示意圖。 第5圖係顯示根據本發明一較佳實施例，執行內插法 則第一步驟之後的Lena影像的結果。 第6圖之（a)〜(c)係顯示一個鄰近像素的示意圖。 第7圖係顯示一個內插同質像素的示意圖。 第8圖係顯示一個內插邊緣像素的示意圖。 第9圖之(a)〜(f)係顯示使用不同法則對人工影像A執 行內插的結果。 第10圖之（a)〜(f)係顯示一個部分Lena影像主觀品質 的示意圖。 第11圖之（a)〜(f)係顯示一個部分Pepper影像主觀品質 的示意圖。 第1表顯示對六個影像使用不同法則而得的同質區的 1288364 11934twfl.doc/006 96-3-3 PSNR平均値。 第2表顯示灰階影像的PSNR(dB)比較結果。 第3表顯示彩色影像的PSNR(dB)比較結果。 第4表顯示計算複雜度的比較結果(假設內插η個像 素）。 圖式標記說明= 100:第一步驟，102:第二步驟，104··偵測同質區， 106:內插同質區，108 :內插邊緣像素。Based on the above description, the proposed rule of the present invention achieves at least two goals, namely, immediate interpolation and providing subjective quality similar to edge direction interpolation. Therefore, the proposed rule can be applied to a video conference that requires converting a QCIF to a CIF size or converting a CIF to a 4CIF size, and can improve the quality of the video conference. While the present invention has been described above by way of a preferred embodiment, it is not intended to limit the invention, and the present invention may be modified and modified without departing from the spirit and scope of the invention. Protection 17 1288364 11934twfl.doc/006 96-3- 3 The scope is subject to the definition of the scope of the patent application. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing the geometrical duality between a high-resolution covariation and a low-resolution covariant 4, j when interpolating an interlaced lattice to form. Figure 2 shows a similar geometric duality when interpolating the interlaced matrix = odd) from the lattice}^·,7·(/+/ = even). Figure 3 is a flow chart showing an image interpolation method in accordance with a preferred embodiment of the present invention. (a) to (c) of Fig. 4 are diagrams showing pixel differences in three directions according to a preferred embodiment of the present invention. Figure 5 is a diagram showing the results of performing a Lena image after the first step of the interpolation method in accordance with a preferred embodiment of the present invention. (a) to (c) of Fig. 6 show a schematic diagram of a neighboring pixel. Figure 7 shows a schematic diagram of an interpolated homogeneous pixel. Figure 8 is a schematic diagram showing an interpolated edge pixel. (a) to (f) of Fig. 9 show the results of performing interpolation on the artificial image A using different laws. (a) to (f) of Fig. 10 show a schematic diagram of the subjective quality of a partial Lena image. Figure 11 (a) ~ (f) shows a schematic diagram of the subjective quality of a partial Pop image. Table 1 shows the PSNR average of 1288364 11934 twfl.doc/006 96-3-3 for the homogenous region using different rules for the six images. Table 2 shows the PSNR (dB) comparison of grayscale images. Table 3 shows the PSNR (dB) comparison of color images. Table 4 shows the comparison of the computational complexity (assuming η pixels are interpolated). Schematic Description = 100: First Step, 102: Second Step, 104··Detecting the homogeneous region, 106: Interpolating the homogeneous region, 108: Interpolating the edge pixels.
1919