TWI384876B - Method for upscaling images and videos and associated image processing device - Google Patents
Method for upscaling images and videos and associated image processing device Download PDFInfo
- Publication number
- TWI384876B TWI384876B TW98106352A TW98106352A TWI384876B TW I384876 B TWI384876 B TW I384876B TW 98106352 A TW98106352 A TW 98106352A TW 98106352 A TW98106352 A TW 98106352A TW I384876 B TWI384876 B TW I384876B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- interpolation
- amplification
- edge
- input image
- Prior art date
Links
Landscapes
- Image Processing (AREA)
Description
本發明係關於一種圖像及視頻(video)的放大(upscaling)方法，尤指一種將低解析度的輸入圖像放大且顯示在高解析度的視頻設備上時，能使放大的輸出圖像仍具有抗鋸齒、邊緣銳利、細節豐富及對比度強等特點的方法。The present invention relates to an image and video upscaling method, and more particularly to an enlarged output image when a low-resolution input image is enlarged and displayed on a high-resolution video device. It still has anti-aliasing, sharp edges, rich detail and strong contrast.
近年來，各式數位攝像裝置(如：數位相機及攝影機等)不斷推陳出新，不僅圖像品質愈來愈高、產品體積愈來愈小，其市場價格亦日漸便宜，因此，該等數位攝像裝置已日益普及，而成為人們日常生活及工作中不可或缺的一重要工具。以目前市場上頗為流行的具攝像功能的行動電話為例，該等行動電話除設有CCD或CMOS攝像單元，用以拍攝圖像外，尚設有一小尺寸的液晶顯示螢幕，用以顯示所拍攝的圖像，供使用者瀏覽。一般而言，當使用者利用行動電話拍攝景物時，所拍攝的圖像儲存在其內所插設的一記憶卡內，使用者在行動電話上點選圖像時，行動電話將自記憶卡內讀取欲瀏覽的圖像，並經重新編碼後，將圖像的縮小圖顯示在液晶顯示螢幕上，如此，使用者即可在液晶顯示螢幕上對圖像進行縮小、放大、拖曳或調整頁面大小等操控動作。因此，無論前述數位攝像裝置上儲存的圖像是幾百萬個位元組(如：1.2Mega byte)，或數十萬個位元組(如：120K byte)，該等圖像均係被儲存在其上所設的一儲存裝置(如：記憶卡、硬碟或隨身碟)內，使用者每次點選欲瀏覽的圖像時，數位攝像裝置再自儲存裝置內讀取圖像，並經重新編碼，將原始圖像縮小成數萬個位元組(如：75K byte)或數千個位元組(如：7.5K byte)的縮小圖後，始將縮小圖顯示在小尺寸的液晶顯示螢幕上，供使用者瀏覽，並令使用者可依實際需要，對縮小圖進行縮小、放大、拖曳或頁面大小調整等操控動作。In recent years, various digital camera devices (such as digital cameras and video cameras) have been continuously updated. Not only are image quality getting higher and higher, the product volume is getting smaller and smaller, and the market price is becoming cheaper. Therefore, these digital camera devices are becoming cheaper. It has become increasingly popular and has become an indispensable tool in people's daily life and work. Take the mobile phone with camera function, which is quite popular on the market, as an example. In addition to the CCD or CMOS camera unit, the mobile phone has a small LCD screen for displaying images. The captured image is for the user to browse. Generally, when a user uses a mobile phone to take a scene, the captured image is stored in a memory card inserted therein, and when the user clicks on the image on the mobile phone, the mobile phone will be self-made from the memory card. After reading the image to be browsed and re-encoding, the reduced image of the image is displayed on the liquid crystal display screen, so that the user can reduce, enlarge, drag or adjust the image on the liquid crystal display screen. Control actions such as page size. Therefore, regardless of whether the image stored on the aforementioned digital camera device is several million bytes (eg, 1.2 Mega byte), or hundreds of thousands of bytes (eg, 120K byte), the images are all Stored in a storage device (such as a memory card, hard disk or flash drive) set on the computer, the digital camera device reads the image from the storage device each time the user clicks on the image to be browsed. After re-encoding, the original image is reduced to a tens of thousands of bytes (such as: 75K byte) or thousands of bytes (such as: 7.5K byte), and the reduced image is displayed in a small size. The LCD screen is displayed on the screen for the user to browse, and the user can perform operations such as zooming out, zooming in, dragging, or resizing the thumbnail according to actual needs.
由於，該等數位攝像裝置的圖像解析度及擷取速度已愈來愈高，使得各式數位攝像裝置已被廣泛地應用在各專業領域，如：刑事、生物、醫藥及天文科學...等，用以保存許多重要的事證，如：刑事案件的關鍵線索、證物或犯罪現場等存證圖像；生物科學的新發現或實驗結果；醫學上X光片或電腦斷層圖像等供醫療人員判斷病徵的資料，因此，如何為專業研究人員保存實驗室或其他研究領域中的重要事證，並以數位圖像格式存檔，作為日後實驗或研究時，查閱或比對的重要資料，且在對數位圖像進行放大時，能有效降低圖像失真，提供高解析度且有利於辨識的數位圖像，供解讀及研判數位圖像內所呈現的相關特徵，即成為各該專業領域人士非常重視的一項課題。Since the image resolution and the capture speed of these digital camera devices have become higher and higher, various digital camera devices have been widely used in various fields of expertise, such as criminal, biological, medical and astronomical sciences. Etc., to preserve many important documents, such as: critical clues in criminal cases, evidence of evidence or crime scenes; new discoveries or experimental results in biological sciences; medical X-rays or computed tomography images, etc. For medical personnel to judge the symptoms of the disease, therefore, how to save important documents in the laboratory or other research fields for professional researchers, and archive them in digital image format, as important data for review or comparison in future experiments or research And when the digital image is enlarged, the image distortion can be effectively reduced, and the digital image with high resolution and favorable recognition can be provided for interpreting and judging the relevant features presented in the digital image, that is, becoming the professional A topic that people in the field attach great importance to.
按，圖像處理業者及設計人員對於圖像及視頻(video)放大技術，已展開了很多年的研究，無論是從初期的線性放大技術，到後來基於邊緣(edge)的放大技術，新的理論及應用方法不斷地被開發出來，其中常用的線性放大技術包括雙線性插值運算(Bilinear Interpolation)、雙立方插值運算(Bicubic Interpolation)及蘭索斯法(Lanzcos)等，而基於邊緣的放大技術中最具代表性者則是NEDI(New Edge-Directed Interpolation)，惟，該等放大技術仍存在諸多缺點，其中前述線性放大技術容易造成邊緣鋸齒(blocky edge)效應、細節損失及邊緣模糊(blurry)等問題，而前述基於邊緣的放大技術係沿著圖像的邊緣根據梯度方向(gradient direction)進行插值(interpolation)，雖可解決部分的邊緣鋸齒和模糊等問題，但是，該等基於邊緣的放大技術對於邊緣方向(edge direction)的準確性有著較大的依賴，容易出現因邊緣方向不準確，而造成插值錯誤的現象，特別是，在圖像中細節豐富且邊緣雜亂的區域中，插值錯誤的問題格外明顯且嚴重；此外，該等基於邊緣的放大技術，為了保持其結果的準確性，往往必需使用龐大的運算，易造成圖像處理效率不彰的問題。最後，由於目前使用的放大技術一般均是採用鄰域(neighborhood)圖元(pixel)的加權求和，來進行插值計算，對原圖像具有低通濾波的作用，故在圖像被放大後，不可避免地會造成原圖像上應有的銳度和細節資訊(高頻部分)的損失。有鑒於此，圖像處理業者為恢復原圖像的品質，往往會在圖像被放大後，對圖像進行一些增強和恢復處理，但是該等處理又產生了邊緣過沖(overshoot)及振鈴效應(ringing effect)等新的瑕疵。According to image processing companies and designers, image and video amplification technology has been researched for many years, from the initial linear amplification technology to the later edge-based amplification technology. Theories and application methods have been continuously developed. The commonly used linear amplification techniques include Bilinear Interpolation, Bicubic Interpolation and Lanzcos, while edge-based amplification. The most representative of the technology is NEDI (New Edge-Directed Interpolation). However, these amplification techniques still have many shortcomings, in which the aforementioned linear amplification technique is prone to edge jagged edge effects, loss of detail and edge blurring ( Blurr) and the like, and the aforementioned edge-based magnification technique performs interpolation according to the gradient direction along the edge of the image, although some problems such as edge aliasing and blurring can be solved, but the edges are based on edges. The amplification technique has a large dependence on the accuracy of the edge direction, which is prone to edge The direction is not accurate, which causes interpolation errors. In particular, in areas with rich details and cluttered edges, the problem of interpolation errors is particularly obvious and serious. In addition, these edge-based amplification techniques maintain their results. The accuracy of the image often requires the use of huge calculations, which is easy to cause inefficiency in image processing. Finally, since the currently used amplification techniques generally use a weighted summation of neighboring pixels (pixels) to perform interpolation calculation, the original image has a low-pass filtering effect, so after the image is enlarged Inevitably, it will cause the loss of sharpness and detail information (high frequency part) on the original image. In view of this, in order to restore the quality of the original image, the image processing industry tends to perform some enhancement and recovery processing on the image after the image is enlarged, but these processes generate edge overshoot and ringing. New flaws such as the ringing effect.
故，如何提供一種全新的圖像放大技術，使得解析度較低的圖像及視頻被放大且顯示在解析度較高的視頻設備上時，仍能呈現清晰且有利於辨識相關特徵的數位圖像，即成為各圖像處理業者及設計人員刻正努力研究且亟待解決的一重要課題。Therefore, how to provide a new image enlargement technology, so that the lower resolution image and video are enlarged and displayed on a higher resolution video device, still can display a clear and advantageous digital image for identifying relevant features. For example, it has become an important issue that every image processing company and designer is trying to study and urgently need to solve.
有鑑於此，本發明之一目的，在於提供一種圖像及視頻的放大方法與相關之圖像處理裝置，以改進前述傳統圖像放大技術的缺點。In view of the above, it is an object of the present invention to provide an image and video enlargement method and related image processing apparatus to improve the disadvantages of the aforementioned conventional image enlargement technique.
本發明揭露一種圖像及視頻的放大方法，其係應用在一圖像處理裝置上，以在一數位圖像被輸入該圖像處理裝置後，該圖像處理裝置能依預定的放大比例，輸出一放大的數位圖像，該方法包括提供一預處理模組及一複合放大模組，其中該預處理模組係對該輸入圖像執行一高通濾波處理，提取該輸入圖像的高頻部分，用於對放大後的圖像進行高頻補償，且對該輸入圖像執行一圖像分解處理，利用梯度運算元提取該輸入圖像的梯度，且根據預設之一固定門限，將該輸入圖像分解成平坦區域和邊緣區域，並在輸入圖像上標記該二區域，該複合放大模組係對原輸入圖像、平坦區域、邊緣區域及高頻部分，分別進行放大處理，其中對於原輸入圖像及平坦區域係使用簡單插值運算，如：雙立方插值運算(bicubic interpolation)，進行放大處理，對於邊緣區域及高頻部分則同時使用複雜插值運算和簡單插值運算，複雜插值運算是對邊緣區域及高頻部分的圖元使用方向插值(directional interpolation)運算，進行放大處理，嗣，對該方向插值運算的結果，進行可信度處理(confidence process)，即邊緣方向愈明確，方向插值運算結果的可信度愈高，否則，方向插值運算結果的可信度愈低，再根據計算得到的可信度，對方向插值運算的結果和簡單插值運算的結果進行加權求和(weighted sum)，最後，對原輸入圖像、平坦區域、邊緣區域及高頻部分的放大結果，進行融合，即能在運算工作量較小、複雜度較低及速度較高的情形下，依預定的放大比例，輸出一具有抗鋸齒、邊緣銳利、細節豐富及對比度強等特點的輸出圖像。The invention discloses an image and video enlargement method, which is applied to an image processing device, and after a digital image is input into the image processing device, the image processing device can be according to a predetermined enlargement ratio. Outputting an enlarged digital image, the method comprising: providing a preprocessing module and a composite amplification module, wherein the preprocessing module performs a high pass filtering process on the input image to extract a high frequency of the input image a portion for performing high frequency compensation on the enlarged image, performing an image decomposition process on the input image, extracting a gradient of the input image by using a gradient operation element, and fixing a threshold according to one of presets, The input image is decomposed into a flat area and an edge area, and the two areas are marked on the input image, and the composite amplification module respectively amplifies the original input image, the flat area, the edge area, and the high frequency part, For the original input image and the flat region, a simple interpolation operation, such as a bicubic interpolation, is performed for the edge region and the high frequency portion. At the same time, complex interpolation operation and simple interpolation operation are used at the same time. The complex interpolation operation uses the directional interpolation operation on the edges of the edge region and the high-frequency portion to perform amplification processing, and the result of the interpolation operation in the direction is performed. Confidence processing, that is, the more the edge direction is clear, the higher the credibility of the direction interpolation operation result is. Otherwise, the credibility of the direction interpolation operation result is lower, and then the direction is interpolated according to the calculated credibility. The result of the operation and the result of the simple interpolation operation are weighted sum. Finally, the original input image, the flat region, the edge region, and the high-frequency portion are amplified, and the operation is small. In the case of low complexity and high speed, the output image with the characteristics of anti-aliasing, sharp edges, rich details and strong contrast is output according to the predetermined magnification ratio.
本發明另揭露一種圖像處理裝置，用以在一數位圖像被輸入該圖像處理裝置後，能依據一放大比例，輸出一放大的數位圖像。該圖像處理裝置包括：一預處理模組，用以對該輸入圖像執行一高通濾波處理，以提取該輸入圖像的高頻部分，該高頻部分是用於對該輸入圖像的放大結果進行高頻補償，該預處理模組並對該輸入圖像執行一圖像分解處理，該圖像分解處理係利用一梯度運算元提取該輸入圖像的圖像梯度，且根據預設之一固定門限，將該輸入圖像分解成平坦區域和邊緣區域，並在該輸入圖像上標記該二區域；一複合放大模組，其對於原輸入圖像及該平坦區域係使用一簡單插值運算，進行放大處理，對於該邊緣區域及該高頻部分係分別使用一複雜插值運算和該簡單插值運算，進行放大處理，接著，對該複雜插值運算的結果，進行可信度處理，其中，邊緣方向愈明確，該複雜插值運算的結果的可信度愈高，否則，該複雜插值運算的結果的可信度愈低，最後，根據該可信度，對該複雜插值運算的結果和該簡單插值運算的結果進行加權求和；以及一融合處理單元，用以將原輸入圖像、平坦區域、邊緣區域及高頻部分的放大結果，融合成為該輸出圖像。The present invention further discloses an image processing apparatus for outputting an enlarged digital image according to an amplification ratio after a digital image is input to the image processing apparatus. The image processing apparatus includes: a preprocessing module configured to perform a high pass filtering process on the input image to extract a high frequency portion of the input image, the high frequency portion being used for the input image Amplifying the result to perform high frequency compensation, and the preprocessing module performs an image decomposition process on the input image, wherein the image decomposition process extracts an image gradient of the input image by using a gradient operation element, and according to the preset a fixed threshold, the input image is decomposed into a flat area and an edge area, and the two areas are marked on the input image; a composite amplification module that uses a simple input image and the flat area Interpolation operation, performing amplification processing, using a complex interpolation operation and the simple interpolation operation for the edge region and the high frequency portion, respectively performing amplification processing, and then performing reliability processing on the result of the complex interpolation operation, wherein The more clear the edge direction, the higher the credibility of the result of the complex interpolation operation. Otherwise, the credibility of the result of the complex interpolation operation is lower, and finally, according to the credibility And performing a weighted summation of the result of the complex interpolation operation and the result of the simple interpolation operation; and a fusion processing unit for merging the original input image, the flat region, the edge region, and the amplification result of the high frequency portion into the Output image.
茲為令 貴審查委員能對本發明之目的、處理程式及其功效，能有更進一步之認識與瞭解，特列舉實施例，並配合圖示，詳細說明如下：In order to enable the review committee to have a better understanding and understanding of the purpose, processing program and its effects of the present invention, the examples are illustrated and accompanied by the illustrations, which are described in detail as follows:
本發明係一種圖像及視頻的放大方法，請參閱第1圖所示，該方法係應用在一圖像處理裝置1上，以在一數位圖像被輸入該圖像處理裝置1後，該圖像處理裝置1能依預定的放大比例，輸出一放大的數位圖像，使得一解析度較低的輸入圖像及視頻被放大，且被顯示在一解析度較高的視頻設備上時，仍能呈現出清晰且有利於辨識相關特徵的數位圖像。在本發明之一較佳實施例中，參閱第1圖所示，該方法包括提供一預處理模組(preprocess module)10及一複合放大模組(composite up-scaling module)30，參閱第2圖所示，其中該預處理模組10係對一輸入圖像執行一高通濾波處理11，提取該輸入圖像的高頻部分，以在後續處理過程中，利用該高頻部分對該輸入圖像的放大結果進行高頻補償，且對該輸入圖像執行一圖像分解處理12，利用梯度運算元，將該輸入圖像分解成平坦區域和邊緣區域，該複合放大模組30係對原輸入圖像、平坦區域、邊緣區域及高頻部分，分別進行放大處理，其中對於原輸入圖像及平坦區域係使用簡單插值運算，對於邊緣區域及高頻部分則使用複雜插值運算，最後，對原輸入圖像、平坦區域、邊緣區域及高頻部分的放大結果，進行一融合處理33，即能依預定的放大比例，產生一輸出圖像。The present invention relates to an image and video enlargement method. Referring to FIG. 1, the method is applied to an image processing apparatus 1 after a digital image is input to the image processing apparatus 1. The image processing apparatus 1 can output an enlarged digital image according to a predetermined magnification ratio, so that a lower resolution input image and video are enlarged and displayed on a higher resolution video device. Digital images that are clear and useful for identifying relevant features can still be presented. In a preferred embodiment of the present invention, as shown in FIG. 1, the method includes providing a preprocess module 10 and a composite up-scaling module 30, see section 2. As shown in the figure, the pre-processing module 10 performs a high-pass filtering process 11 on an input image, and extracts a high-frequency portion of the input image to utilize the high-frequency portion of the input image during subsequent processing. The amplification result of the image is subjected to high frequency compensation, and an image decomposition processing 12 is performed on the input image, and the input image is decomposed into a flat region and an edge region by using a gradient operation unit, and the composite amplification module 30 is paired with the original The input image, the flat region, the edge region, and the high-frequency portion are respectively subjected to amplification processing, wherein simple interpolation operation is used for the original input image and the flat region, and complex interpolation operation is used for the edge region and the high-frequency portion, and finally, The fusion result of the original input image, the flat area, the edge area, and the high frequency portion is subjected to a blending process 33, that is, an output image can be generated according to a predetermined magnification ratio.
由於，目前使用的傳統插值放大技術都具有低通濾波的特性，輸入圖像被放大後，放大的輸出圖像必然會損失掉輸入圖像中的高頻資訊，在該實施例中，復參閱第1圖所示，該預處理模組10為了防止高頻資訊的過分損失，以為後續處理做好準備工作，會對輸入圖像先進行該高通濾波處理11，參閱第2圖所示，以提取輸入圖像中高頻成分，在輸入圖像的放大處理過程中，該高頻部分與該邊緣區域是使用相同的插值運算法被放大，當原輸入圖像、平坦區域、邊緣區域及高頻部分都完成放大後，再對其進行該融合處理(fusion)33，以補償輸入圖像在放大過程中損失的高頻部份。Since the conventional interpolation amplification technique currently used has the characteristics of low-pass filtering, after the input image is enlarged, the amplified output image necessarily loses the high-frequency information in the input image. In this embodiment, the reference is repeated. As shown in FIG. 1 , in order to prevent excessive loss of high-frequency information, the pre-processing module 10 performs preparatory work for subsequent processing, and performs high-pass filtering processing on the input image first. Referring to FIG. 2, Extracting high-frequency components in the input image. During the amplification process of the input image, the high-frequency portion and the edge region are enlarged using the same interpolation algorithm, when the original input image, flat region, edge region, and high frequency After the amplification is partially completed, the fusion 33 is performed to compensate for the high frequency portion of the input image lost during the amplification process.
此外，由於人眼往往對圖像中梯度較強的邊緣區域特別敏感，因此，在該實施例中，為了減少後續插值放大處理的運算量，以簡化及加速整體的運算效能，僅對圖像中人眼特別敏感的邊緣區域使用複雜插值運算(如：方向插值運算(directional interpolation))，進行高精確度的放大處理，對於圖像中平坦區域則使用簡單插值運算，如：雙立方插值運算(bicubic interpolation)，進行放大處理，此一權宜作法，最終仍能在不影響視覺效果的前提下，獲得清晰的放大圖像。另，由於圖像中較為雜亂的邊緣區域(如：草地等)，其邊緣方向不易準確判斷，且人眼對於該等雜亂的邊緣區域的放大處理是否準確，也不會太感興趣，所以，在該實施例中，對於該等雜亂的邊緣區域也同時使用簡單插值運算，以簡化插值放大處理的複雜度及運算量。為了實現前述目的，參閱第3圖所示，該實施例在對輸入圖像執行該圖像分解處理12時，係先針對輸入圖像進行一圖像分解121，以分解出平坦區域和邊緣區域，其作法係利用梯度運算元，提取高頻成分的梯度Grd (x )，在該實施例中係使用sobel運算元(operator)，提取高頻成分的梯度Grd (x )，然後，根據人眼對梯度變化的敏感程度，使用固定門限ThreshD 為16，根據下列公式(1)分解出平坦區域(plane)和邊緣區域(edge)，並對高頻成分的圖元(pixel)進行標記Label (x )：In addition, since the human eye is often particularly sensitive to the edge regions with strong gradients in the image, in this embodiment, in order to reduce the amount of computation of the subsequent interpolation amplification processing, to simplify and accelerate the overall computational performance, only the image The edge region that is particularly sensitive to the human eye uses complex interpolation operations (such as directional interpolation) for high-precision amplification, and simple interpolation operations for flat regions in the image, such as bicubic interpolation (bicubic interpolation), the amplification process, this expedient method, in the end can still obtain a clear magnified image without affecting the visual effect. In addition, due to the more chaotic edge regions in the image (such as grass, etc.), the edge direction is not easy to accurately judge, and the human eye is not too interested in the amplification processing of the disordered edge regions, so, In this embodiment, simple interpolation operations are also used for the chaotic edge regions to simplify the complexity and computational complexity of the interpolation amplification process. In order to achieve the foregoing object, referring to FIG. 3, when performing the image decomposition processing 12 on an input image, the embodiment first performs an image decomposition 121 on the input image to decompose the flat region and the edge region. The method uses a gradient operation element to extract a gradient of the high-frequency component Grd ( x ). In this embodiment, a sobel operator is used to extract a gradient of the high-frequency component Grd ( x ), and then, according to the human eye. sensitivity to changes in gradient, using the fixed threshold ThreshD 16, (1) a decomposition of the following formula flat area (Plane) and the edge region (edge), and the high-frequency component element (pixel) labeled Label (x ):
；嗣，復參閱第3圖所示，再針對該邊緣區域內較為雜亂的邊緣圖元，進行一去雜亂處理122，去掉雜亂的邊緣圖元，其作法係先提取某一邊緣圖元的一預定範圍的鄰域M ×N ，再統計該鄰域內邊緣圖元的數目Nedge ，且根據下列公式(2)，在判斷出該鄰域內邊緣圖元的數目超過一預定數目(太多或太少)時，即刪除該邊緣圖元：; 嗣, refer to Figure 3, and then perform a de-scrambling process 122 for the more chaotic edge elements in the edge region, and remove the cluttered edge primitives. The method is to extract one of the edge primitives first. a predetermined range of neighborhoods M × N , and then counting the number Nedge of the inner edge primitives in the neighborhood, and judging that the number of edge primitives in the neighborhood exceeds a predetermined number according to the following formula (2) (too much or When there are too few), the edge element is deleted:
if (Nedge >ThrH or Nedge <ThrL )remove the edge ;......................(2) If ( Nedge > ThrH or Nedge < ThrL ) remove the edge ;......................(2)
，其中ThrL =min(M ,N )，ThrH =0.8．M．N 。最後，為了擴大邊緣區域，復參閱第3圖所示，該實施例乃對邊緣圖元向四周進行一形態學膨脹處理(dilation)123，在該實施例中係使用十字結構元(Cross structure element)，對邊緣圖元進行該形態學膨脹處理123，以擴大邊緣區域。, where ThrL = min( M , N ), ThrH = 0.8. M. N. Finally, in order to enlarge the edge region, as shown in Fig. 3, this embodiment performs a morphological dilation 123 on the periphery of the edge primitive, and in this embodiment, a cross structure element is used. The morphological expansion process 123 is performed on the edge primitives to enlarge the edge regions.
在該實施例中，參閱第1及3圖所示，該複合放大模組30係對原輸入圖像、平坦區域、邊緣區域及高頻部分，分別進行放大處理，其中對於邊緣區域及高頻部分係使用第一複合插值放大模組31，進行插值放大及增益處理，對於原輸入圖像及平坦區域係使用第二複合插值放大模組32進行插值放大處理，第一複合插值放大模組31和第二複合插值放大模組32中所使用的插值放大運算法可為相同，且在實際運算過程中，第一複合插值放大模組31和第二複合插值放大模組32中的插值放大運算可同時進行，以節省整體的運算量，最後，再對該第一複合插值放大模組31及第二複合插值放大模組32輸出的圖像，進行融合(fusion)處理33，整個計算過程依下列公式(3)，即能依預定的放大比例，產生一高解析度的輸出圖像：In this embodiment, as shown in FIGS. 1 and 3, the composite amplification module 30 performs amplification processing on the original input image, the flat region, the edge region, and the high frequency portion, respectively, for the edge region and the high frequency. In part, the first composite interpolation amplification module 31 is used for interpolation amplification and gain processing. For the original input image and the flat region, the second composite interpolation amplification module 32 is used for interpolation and amplification processing, and the first composite interpolation amplification module 31 is used. The interpolation amplification algorithm used in the second composite interpolation amplification module 32 can be the same, and in the actual operation, the interpolation and amplification operations in the first composite interpolation amplification module 31 and the second composite interpolation amplification module 32 are performed. Simultaneously, the overall calculation amount can be saved. Finally, the image outputted by the first composite interpolation amplification module 31 and the second composite interpolation amplification module 32 is subjected to a fusion process 33, and the entire calculation process is performed. The following formula (3) can produce a high-resolution output image according to a predetermined magnification ratio:
HR (x )=LR (x )*Hp (x )*CUp (x )．Gain +LR (x )*CUp (x ) ................(3) HR ( x )= LR ( x )* Hp ( x )* CUp ( x ). Gain + LR ( x )* CUp ( x ) ................(3)
，其中HR (x )是高解析度的輸出圖像，LR (x )是低解析度的輸入圖像，Hp (x )是預處理模組10使用的高通濾波函數，CUp (x )是該第一複合插值放大模組31和第二複合插值放大模組32中使用的插值放大函數，Gain 是該第一複合插值放大模組31中使用的常數增益因數。Where HR ( x ) is a high-resolution output image, LR ( x ) is a low-resolution input image, and Hp ( x ) is a high-pass filter function used by the pre-processing module 10, CUp ( x ) is the The interpolation amplification function used in the first composite interpolation amplification module 31 and the second composite interpolation amplification module 32, Gain is a constant gain factor used in the first composite interpolation amplification module 31.
一般言，該等複合插值放大模組31及32主要係執行輸入圖像的放大功能，目前使用的插值放大技術只能將輸入圖像放大成偶數倍，若要放大成任意倍，則要結合使用降採樣(down sample)技術。現在以放大兩倍為例，說明該等複合插值放大模組31及32的功能，參閱第4圖所示為一幅經過插值放大得到的高解析度的圖像，其中標記為黑色的圖元是直接從低解析度的輸入圖像中拷貝而來的圖元，其他圖元則是根據黑色圖元插值而來的圖元，在本發明的插值放大運算過程中，係先計算標記為灰色的圖元，最後，再計算白色圖元，且本發明針對由原輸入圖像分解出的邊緣區域及平坦區域，係使用不同的插值放大運算，其中對於邊緣區域使用複雜插值運算(smart interpolation)，如：方向插值運算，對於平坦區域則使用簡單插值運算(simple interpolation)如：雙立方插值運算。Generally speaking, the composite interpolation amplification modules 31 and 32 mainly perform the amplification function of the input image, and the interpolation amplification technique currently used can only enlarge the input image to an even multiple, and if it is to be enlarged to any multiple, it is combined. Use the down sample technique. Now, taking the magnification twice as an example, the functions of the composite interpolation amplification modules 31 and 32 will be described. Referring to FIG. 4, a high-resolution image obtained by interpolation and magnification is shown. It is a primitive that is directly copied from a low-resolution input image. Other primitives are primitives that are interpolated from black primitives. In the interpolation and amplification operation of the present invention, the calculation is first marked as gray. Finally, the white primitive is recalculated, and the present invention uses different interpolation and amplification operations for the edge region and the flat region which are decomposed from the original input image, wherein a smart interpolation is used for the edge region. For example, direction interpolation operation, for flat areas, simple interpolation is used, such as: bicubic interpolation operation.
在該實施例中，該第一複合插值放大模組31分別對預處理模組10輸出的邊緣區域及高頻部分進行插值放大處理，該第二複合插值放大模組32分別對預處理模組10輸出的原輸入圖像及平坦區域進行插值放大處理，第一複合插值放大模組31和第二複合插值放大模組32可使用相同的插值放大運算法，參閱第5圖所示，唯一不同的地方在於第一複合插值放大模組31的放大結果要做增益處理。由於，該預處理模組10已經在原輸入圖像上標記了邊緣區域和平坦區域，根據該等標記，第一複合插值放大模組31和第二複合插值放大模組32會針對圖像中的圖元進行邊緣區域的判斷處理51，若被標記為邊緣區域，即採用複雜插值運算52，若被標記為平坦區域，則採用簡單插值運算(如：雙立方插值運算)53，其中該複雜插值運算52的運算結構，參閱第6圖所示，係針對處於邊緣區域的當前圖元，同時進行方向插值運算521(directional interpolation)和簡單插值運算53(如：雙立方插值運算)，然後，針對方向插值運算521的結果，執行可信度(confidence)及權重(weight)計算522，可信度計算的原則係邊緣方向愈明確者，方向插值運算521結果的可信度愈高，否則，可信度愈低；最後，根據可信度及權重計算522得到的可信度，再對方向插值運算521的結果和簡單插值運算53的結果，進行加權求和(weighted sum)處理523，並予輸出。In this embodiment, the first composite interpolation amplification module 31 respectively performs interpolation and amplification processing on the edge region and the high frequency portion of the output of the preprocessing module 10, and the second composite interpolation amplification module 32 respectively processes the preprocessing module. 10 output original input image and flat area are subjected to interpolation and amplification processing, and the first composite interpolation amplification module 31 and the second composite interpolation amplification module 32 can use the same interpolation amplification algorithm, as shown in FIG. 5, the only difference The point is that the amplification result of the first composite interpolation amplification module 31 is to be subjected to gain processing. Because the pre-processing module 10 has marked the edge region and the flat region on the original input image, according to the markers, the first composite interpolation amplification module 31 and the second composite interpolation amplification module 32 are targeted to the image. The primitive performs the edge region judging process 51. If it is marked as an edge region, the complex interpolation operation 52 is employed, and if it is marked as a flat region, a simple interpolation operation (for example, a bicubic interpolation operation) 53 is employed, wherein the complex interpolation is performed. The operation structure of the operation 52, as shown in FIG. 6, is for the current primitive in the edge region, and simultaneously performs a directional interpolation operation 521 (a directional interpolation) and a simple interpolation operation 53 (for example, a bicubic interpolation operation), and then, The result of the direction interpolation operation 521 is performed to perform a confidence and a weight calculation 522. The principle of the reliability calculation is that the edge direction is more clear, and the reliability of the direction interpolation operation 521 is higher. Otherwise, The lower the reliability; finally, the credibility obtained by calculating 522 based on the credibility and weight, and then the result of the direction interpolation operation 521 and the result of the simple interpolation operation 53 A weighted sum process 523 is processed and output.
在該實施例中，方向插值運算521是先根據處於邊緣區域的當前圖元的梯度值，估算出其邊緣方向，然後，沿著其邊緣方向獲取其他圖元，以進行插值放大運算，以下係以第7圖所示的灰色點x為例，說明如何使用方向插值運算521，將灰色圖元x的插值範本旋轉45度，而得到第7圖所示白色圖元的運算過程：In this embodiment, the direction interpolation operation 521 first estimates the edge direction according to the gradient value of the current primitive in the edge region, and then acquires other primitives along the edge direction thereof to perform interpolation and amplification operations. Taking the gray point x shown in Fig. 7 as an example, how to use the direction interpolation operation 521 to rotate the interpolation template of the gray primitive x by 45 degrees, and obtain the operation process of the white primitive shown in Fig. 7:
首先，復參閱第7圖所示，使用該灰色圖元x鄰域的12個圖元P 0 ~P 11 ，依照下列公式(4)，對其進行插值計算DPx ，由於其插值計算係從六個方向進行判斷，而該等圖元P 0 ~P 11 的排列更接近於圓形，故具有更高的精確度：First, referring to Figure 7, using the 12 primitives P 0 ~ P 11 of the gray primitive x neighborhood, interpolate the DPx according to the following formula (4), since the interpolation calculation is from six The direction is judged, and the arrangement of the primitives P 0 ~ P 11 is closer to a circle, so it has higher precision:
，其中a i 為該等鄰域圖元P 0 ~P 11 的加權係數，依照下列公式(5)，計算其六個方向上的梯度值：, where a i is the weighting coefficient of the neighboring primitives P 0 ~ P 11 , and the gradient values in six directions are calculated according to the following formula (5):
Dir 0 =|P 0 -P 3 |，Dir 1 =|P 1 -P 2 |，Dir 2 =|P 4 -P 7 |，Dir 3 =|P 5 -P 6 |，Dir 4 =|P 8 -P 11 |，Dir 5 =|P 9 -P 10 | ...................(5) Dir 0 =| P 0 - P 3 |, Dir 1 =| P 1 - P 2 |, Dir 2 =| P 4 - P 7 |, Dir 3 =| P 5 - P 6 |, Dir 4 =| P 8 - P 11 |, Dir 5 =| P 9 - P 10 | ...................(5)
，若公式(5)中有些鄰域圖元P 0 ~P 11 還未計算出，可先用簡單插值運算53(如：雙立方插值運算)估算出替代值。然後，使用該等梯度值，依照下列公式(6)，計算各圖元的權重a i ：If some of the neighborhood primitives P 0 ~ P 11 in equation (5) have not yet been calculated, a simple interpolation operation 53 (eg, a double-cubic interpolation operation) may be used to estimate the substitute value. Then, using the gradient values, the weight a i of each primitive is calculated according to the following formula (6):
，俟計算出該等加權係數ai 後，便可根據公式(4)，再對方向插值運算521的結果和簡單插值運算53的結果，進行加權求和(weighted sum)處理523，計算出複雜插值運算52的結果DPx ，並予輸出。After calculating the weighting coefficients a i , the weighted sum processing 523 can be performed on the result of the direction interpolation operation 521 and the result of the simple interpolation operation 53 according to the formula (4) to calculate the complexity. The result of interpolation operation 52 is DPx and is output.
此外，本發明為了提高方向插值運算521的魯棒性(robustness)，以防止在邊緣方向不明顯的邊緣區域，產生錯誤的鬼(Ghost)點，參閱第2圖所示，乃針對該第一複合插值放大模組31和第二複合插值放大模組32中簡單插值運算及方向插值運算的放大結果，採用下列公式(7)，進行融合處理33，以依預定的放大比例，產生一高解析度的輸出圖像：In addition, in order to improve the robustness of the direction interpolation operation 521, the present invention prevents an edge region that is not noticeable in the edge direction from generating an erroneous ghost (Ghost) point, as shown in FIG. 2, for the first The amplification result of the simple interpolation operation and the direction interpolation operation in the composite interpolation amplification module 31 and the second composite interpolation amplification module 32 is performed by the following formula (7), and the fusion processing 33 is performed to generate a high resolution according to a predetermined amplification ratio. Output image:
HPx =(1-fMix )．SPx +fMix．DPx ..............................(7) HPx = (1- fMix ). SPx + fMix. DPx ..............................(7)
，其中HPx 是最終輸出圖像的圖元值，SPx 是針對該輸入圖像圖元的簡單插值運算的放大結果，DPx 是針對該輸入圖像的方向插值運算的放大結果，fMix 是採用下列公式(8)獲得的融合係數：, where HPx is the primitive value of the final output image, SPx is the amplification result of the simple interpolation operation for the input image primitive, DPx is the amplification result of the direction interpolation operation for the input image, and fMix adopts the following formula (8) The fusion coefficient obtained:
另，由於，在前述插值放大運算過程中所使用的插值放大函數均具有低通濾波的特性，會導致放大圖像的邊緣發生模糊的問題，因此，本發明為了提高放大圖像邊緣的清晰度，乃需要對圖像進行一銳化處理34，復參閱第2圖所示，且為了避免在銳化過程中在邊緣產生過沖(overshoot)現象，本發明特別使用下列公式(9)所示的非線性高通濾波器，如：有限脈波響應(Finite Impulse Response，簡稱FIR)的高通濾波器：In addition, since the interpolation amplification function used in the foregoing interpolation and amplification operation has low-pass filtering characteristics, the edge of the enlarged image may be blurred, and therefore, the present invention improves the sharpness of the edge of the enlarged image. It is necessary to perform a sharpening process 34 on the image, as shown in FIG. 2, and in order to avoid an overshoot phenomenon at the edge during the sharpening process, the present invention particularly uses the following formula (9). A nonlinear high-pass filter, such as a finite pulse response (Finite Impulse Response, FIR) high-pass filter:
SHP (x )=Medain (LocMαx (x ).LocMin (x ),Fir (x )) .....................(9) SHP ( x )= Medain ( LocMαx ( x ). LocMin ( x ), Fir ( x )) .....................(9)
，其中SHP (x )是銳化的結果，Medain ()函數是取中值操作，LocMax (x )和LocMin (x )是當前圖元鄰域內的最大和最小值，Fir (x )是具有非線性高通特性的FIR濾波器。如此，將高通濾波的結果限制在局部鄰域內的最大及最小值之間，即能有效防止銳化過程中的邊緣過沖現象。, where SHP ( x ) is the result of sharpening, Medain () function is the median operation, LocMax ( x ) and LocMin ( x ) are the maximum and minimum values in the neighborhood of the current primitive, Fir ( x ) is A nonlinear high-pass characteristic FIR filter. In this way, the result of the high-pass filtering is limited between the maximum and minimum values in the local neighborhood, that is, the edge overshoot phenomenon during the sharpening process can be effectively prevented.
如此，使用本發明的方法對低解析度的輸入圖像進行放大，以顯示在高解析度的視頻設備上時，不僅能使放大運算量較其他基於邊緣的方法為小，且在複雜度較低及速度較高的情形下，使放大的輸出圖像仍具有抗鋸齒、邊緣銳利、細節豐富及對比度強等特點，以在對數位圖像進行放大時，能有效降低圖像失真，提供高解析度且有利於辨識相關特徵的數位圖像。Thus, using the method of the present invention to amplify a low-resolution input image to display on a high-resolution video device, not only can the amplification operation be smaller than other edge-based methods, but also in complexity. In the case of low and high speed, the amplified output image still has anti-aliasing, sharp edges, rich details and strong contrast, which can effectively reduce image distortion and improve the image when zooming in on the digital image. Resolution and facilitates identification of digital images of related features.
當前述之圖像及視頻的放大方法以軟體實施時，可藉由硬體的協助來加速執行。具體來說，第1圖之預處理模組10與複合放大模組30以軟體的程式碼實施，而該程式碼可藉由一硬體之圖像處理器(graphics processing unit,GPU)來執行。若該圖像處理器支援通用平行計算，亦即通用圖像處理器(General Purpose GPU,GPGPU)，便可以更快的速度執行該程式碼，提升圖像及視頻的放大效率。例如，若使用有支援CUDA技術(Compute Unified Device Architecture，由NVIDIA公司所開發的並行運算技術)的圖像處理器，則可將預處理模組10與複合放大模組30所執行的前述各項運算，如高通濾波處理11、簡單插值運算53、複雜插值運算52、融合處理33、銳化處理34...等等，編寫為CUDA核函數(kernel function)的程式碼，而有支援CUDA技術的圖像處理器在執行這些CUDA核函數時，可產生多個執行緒(thread)來同時執行同一個CUDA核函數，此種並行運算的方式可大幅提高效率。When the aforementioned image and video amplification method is implemented in software, the execution can be accelerated by the assistance of the hardware. Specifically, the pre-processing module 10 and the composite amplifying module 30 of FIG. 1 are implemented by software code, and the code can be executed by a hardware graphics processing unit (GPU). . If the image processor supports general parallel computing, that is, a General Purpose GPU (GPGPU), the code can be executed at a faster speed to improve the image and video amplification efficiency. For example, if an image processor supporting a CUDA technology (Compute Unified Device Architecture, parallel computing technology developed by NVIDIA Corporation) is used, the foregoing items executed by the preprocessing module 10 and the composite amplification module 30 can be used. Operations, such as high-pass filter processing 11, simple interpolation operation 53, complex interpolation operation 52, fusion processing 33, sharpening processing 34, etc., are written as CUDA kernel function (kernel function) code, and support CUDA technology When performing these CUDA kernel functions, the image processor can generate multiple threads to execute the same CUDA kernel function at the same time. This parallel operation can greatly improve the efficiency.
前述之圖像及視頻的放大方法可應用於影像後處理(post processing)領域，例如，應用於視頻播放軟體中，如第8A圖所示。第8A圖係顯示一視頻播放軟體的架構，其中，視頻播放器(video player)80包含視頻解碼器(video decoder)81、預處理模組10、複合放大模組30以及視頻渲染器(video renderer)82。視頻解碼器81可從各種視頻來源取得影像資料，以對其進行解碼；預處理模組10與複合放大模組30則將解碼後的影像資料予以放大；視頻渲染器82則利用影像渲染(rendering)技術，如微軟(Microsoft)的Direct3D技術，將所放大的影像資料繪製顯示。The aforementioned image and video enlargement method can be applied to the field of post processing, for example, to a video playback software, as shown in FIG. 8A. 8A is a diagram showing a video playback software architecture in which a video player 80 includes a video decoder 81, a preprocessing module 10, a composite amplification module 30, and a video renderer. ) 82. The video decoder 81 can obtain image data from various video sources to decode the same; the pre-processing module 10 and the composite amplification module 30 amplify the decoded image data; and the video renderer 82 uses image rendering (rendering) Technology, such as Microsoft's Direct3D technology, draws and displays the magnified image data.
一般而言，當第8A圖的架構置於電腦中運作時，視頻解碼器81係將解碼後的影像資料存在電腦的系統記憶體中，而預處理模組10與複合放大模組30則對於系統記憶體所儲存之解碼影像資料進行放大。預處理模組10與複合放大模組30亦可整合於視頻解碼器81中。然而，若電腦具有支援CUDA技術的圖像處理器，且預處理模組10與複合放大模組30係以CUDA核函數的程式碼來實施，則將預處理模組10與複合放大模組30整合於視頻渲染器82中，如第8B圖所示，可進一步提升硬體加速的效果。這是因為，當視頻解碼器81將解碼後的影像資料傳遞到視頻渲染器82時，視頻渲染器82會建立Direct3D表面(surface)(此處係以Direct3D技術為例說明)，用以顯示該影像資料，而這些表面係儲存於顯示記憶體中。另一方面，CUDA技術有支援Direct3D互用性(interoperability)機制，使得預處理模組10與複合放大模組30可直接對Direct3D表面進行處理，因此，將預處理模組10與複合放大模組30整合於視頻渲染器82中，使其可直接對儲存於顯示記憶體的Direct3D表面進行放大，有助於提昇效率，而避免浪費時間於系統記憶體與顯示記憶體間的資料傳輸。In general, when the architecture of FIG. 8A is placed in a computer, the video decoder 81 stores the decoded image data in the system memory of the computer, and the pre-processing module 10 and the composite amplification module 30 The decoded image data stored in the system memory is enlarged. The pre-processing module 10 and the composite amplification module 30 can also be integrated into the video decoder 81. However, if the computer has an image processor that supports CUDA technology, and the pre-processing module 10 and the composite amplification module 30 are implemented by the code of the CUDA kernel function, the pre-processing module 10 and the composite amplification module 30 are implemented. Integrated into the video renderer 82, as shown in FIG. 8B, the effect of hardware acceleration can be further improved. This is because, when the video decoder 81 passes the decoded image data to the video renderer 82, the video renderer 82 establishes a Direct3D surface (here, using Direct3D technology as an example) to display the Image data, and these surface images are stored in display memory. On the other hand, the CUDA technology supports the Direct3D interoperability mechanism, so that the pre-processing module 10 and the composite amplification module 30 can directly process the Direct3D surface. Therefore, the pre-processing module 10 and the composite amplification module are The 30 is integrated into the video renderer 82 to directly zoom in on the Direct3D surface stored in the display memory, which helps to improve efficiency and avoid wasting time on data transfer between the system memory and the display memory.
第9圖係第8B圖之視頻播放軟體架構之一具體實施例的方塊圖，其中，視頻播放器90包含視頻解碼器91及視頻渲染器92，而視頻渲染器92包含放大模組921及融合器與呈現器(mixer and presenter)922。第9圖的架構係藉由有支援CUDA技術的圖像處理器來實現，而放大模組921係以CUDA核函數的程式碼來實施。視頻渲染器92從視頻解碼器91接收解碼後的影像資料後，會建立Direct3D表面存於顯示記憶體中，而放大模組921會從顯示記憶體中之Direct3D表面提取影像紋理(texture)(即方塊9210)，將其區分為亮度分量(即Y分量)與色度分量(即U/V分量)，以分別執行不同的放大處理：亮度分量依序經過去噪點(de-noising，即方塊9211)、去塊(de-blocking，即方塊9212)、邊緣插值運算(edge interpolation，即方塊9213)、抗鋸齒(anti-aliasing，即方塊9214)、邊緣銳化(edge sharpening，即方塊9215)等處理；色度分量則經過雙線性插值運算(bilinear interpolation，即方塊9216)。最後，放大模組921依據放大後的亮度分量與色度分量，設定所要顯示的影像紋理(即方塊9217)，再送至融合器與呈現器922進行繪製顯示。對於前述之方塊9211~9216，可分別編寫適當的CUDA核函數來實施，如此，當圖像處理器在執行這些CUDA核函數時，便可以並行運算的方式執行，以大幅提昇放大模組921的運作效率。Figure 9 is a block diagram of one embodiment of a video playback software architecture of Figure 8B, wherein video player 90 includes video decoder 91 and video renderer 92, and video renderer 92 includes amplification module 921 and fusion. Mixer and presenter 922. The architecture of Figure 9 is implemented by an image processor that supports CUDA technology, and the amplification module 921 is implemented by the code of the CUDA kernel function. After the video renderer 92 receives the decoded image data from the video decoder 91, the Direct3D surface is created in the display memory, and the zoom module 921 extracts the image texture from the Direct3D surface in the display memory (ie, Block 9210), which is divided into a luminance component (ie, a Y component) and a chrominance component (ie, a U/V component) to perform different amplification processes: the luminance component sequentially passes through a denoising point (ie, 9211) ), de-blocking (block 9212), edge interpolation (block 9213), anti-aliasing (block 9214), edge sharpening (block 9215), etc. Processing; the chrominance component is subjected to bilinear interpolation (block 9216). Finally, the amplification module 921 sets the image texture to be displayed according to the enlarged luminance component and the chrominance component (ie, block 9217), and sends it to the fuser and the renderer 922 for drawing display. For the foregoing blocks 9211~9216, an appropriate CUDA kernel function can be separately written to implement, so that when the image processor executes these CUDA kernel functions, it can be executed in parallel to greatly increase the amplification module 921. Operational efficiency.
以上所述係利用較佳實施例詳細說明本發明，而非限制本發明之範圍。凡熟知此類技藝人士皆能明瞭，可根據以上實施例之揭示而做出諸多可能變化，仍不脫離本發明之精神和範圍。The above description of the present invention is intended to be illustrative of the preferred embodiments of the invention. It will be apparent to those skilled in the art that many variations are possible in light of the above embodiments without departing from the spirit and scope of the invention.
1．．．圖像處理裝置1. . . Image processing device
10．．．預處理模組10. . . Preprocessing module
11．．．高通濾波處理11. . . High-pass filtering
12．．．圖像分解處理12. . . Image decomposition processing
121．．．圖像分解121. . . Image decomposition
122．．．去雜亂處理122. . . Go to messy processing
123．．．形態學膨脹處理123. . . Morphological expansion
30．．．複合放大模組30. . . Composite amplifier module
31．．．第一複合插值放大模組31. . . First composite interpolation amplification module
32．．．第二複合插值放大模組32. . . Second composite interpolation amplification module
33．．．融合處理33. . . Fusion processing
34．．．銳化處理34. . . Sharpening
51．．．邊緣區域的判斷處理51. . . Judgment processing of edge regions
52．．．複雜插值運算52. . . Complex interpolation
53．．．簡單插值運算53. . . Simple interpolation
521．．．方向插值運算521. . . Direction interpolation
522．．．可信度及權重計算522. . . Credibility and weight calculation
523．．．加權求和處理523. . . Weighted summation
80,90．．．視頻播放器80,90. . . Video player
81,91．．．視頻解碼器81,91. . . Video decoder
82,92．．．視頻渲染器82,92. . . Video renderer
921．．．放大模組921. . . Amplification module
922．．．融合器與呈現器922. . . Fusion and renderer
9210．．．提取影像紋理9210. . . Extract image texture
9211．．．去噪點9211. . . Denoising
9212．．．去塊9212. . . Deblocking
9213．．．邊緣插值運算9213. . . Edge interpolation
9214．．．抗鋸齒9214. . . Anti-aliasing
9215．．．邊緣銳化9215. . . Edge sharpening
9216．．．雙線性插值運算9216. . . Bilinear interpolation
9217．．．設定影像紋理9217. . . Set image texture
第1圖係本發明的圖像處理裝置的架構示意圖；1 is a schematic structural view of an image processing apparatus of the present invention;
第2圖係本發明的圖像處理裝置的細部架構示意圖；2 is a schematic diagram showing a detailed structure of an image processing apparatus of the present invention;
第3圖係本發明的圖像分解處理的細部架構示意圖；Figure 3 is a detailed schematic diagram of the image decomposition processing of the present invention;
第4圖係在本發明之一最佳實施例中一幅經過插值放大得到的高解析度的圖像示意圖；Figure 4 is a schematic diagram of a high resolution image obtained by interpolation amplification in a preferred embodiment of the present invention;
第5圖係該實施例中第一複合插值放大模組及第二複合插值放大模組的架構示意圖；FIG. 5 is a schematic structural diagram of a first composite interpolation amplification module and a second composite interpolation amplification module in the embodiment;
第6圖係該實施例中複雜插值放大運算的細部架構示意圖；Figure 6 is a detailed schematic diagram of a complex interpolation amplification operation in the embodiment;
第7圖係該實施例中一幅經過方向插值運算得到的高解析度的圖像示意圖；Figure 7 is a schematic diagram of a high resolution image obtained by direction interpolation in the embodiment;
第8A圖係一視頻播放軟體的架構的示意圖；Figure 8A is a schematic diagram of the architecture of a video playback software;
第8B圖係將第8A圖之預處理模組與複合放大模組整合於視頻渲染器的示意圖；及8B is a schematic diagram of integrating the pre-processing module and the composite amplification module of FIG. 8A into a video renderer; and
第9圖係第8B圖之視頻播放軟體架構之一具體實施例的方塊圖。Figure 9 is a block diagram of one embodiment of a video playback software architecture of Figure 8B.
10．．．預處理模組10. . . Preprocessing module
11．．．高通濾波處理11. . . High-pass filtering
12．．．圖像分解處理12. . . Image decomposition processing
30．．．複合放大模組30. . . Composite amplifier module
31．．．第一複合插值放大模組31. . . First composite interpolation amplification module
32．．．第二複合插值放大模組32. . . Second composite interpolation amplification module
33．．．融合處理33. . . Fusion processing
34．．．銳化處理34. . . Sharpening
Claims (22)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW98106352A TWI384876B (en) | 2009-02-27 | 2009-02-27 | Method for upscaling images and videos and associated image processing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW98106352A TWI384876B (en) | 2009-02-27 | 2009-02-27 | Method for upscaling images and videos and associated image processing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201032592A TW201032592A (en) | 2010-09-01 |
| TWI384876B true TWI384876B (en) | 2013-02-01 |
Family
ID=44854941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW98106352A TWI384876B (en) | 2009-02-27 | 2009-02-27 | Method for upscaling images and videos and associated image processing device |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI384876B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI490818B (en) * | 2013-01-02 | 2015-07-01 | Chunghwa Picture Tubes Ltd | Method and system for weighted image enhancement |
| TWI723123B (en) * | 2017-01-23 | 2021-04-01 | 香港商斑馬智行網絡（香港）有限公司 | Image fusion method, device and equipment |
| CN109919847B (en) * | 2017-12-13 | 2023-03-14 | 彩优微电子(昆山)有限公司 | Method for improving quality of amplified image |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040201862A1 (en) * | 2003-04-14 | 2004-10-14 | Bailey James Ray | Maximizing performance in a hardware image scaling module |
| TW200744365A (en) * | 2006-05-17 | 2007-12-01 | Realtek Semiconductor Corp | Method and related apparatus for determining image characteristics |
| US20080049045A1 (en) * | 2006-06-20 | 2008-02-28 | Sunplus Technology Co., Ltd. | Image scaling system capable of saving memory |
| US20080055338A1 (en) * | 2006-08-30 | 2008-03-06 | Ati Technologies Inc. | Multi-stage edge-directed image scaling |
-
2009
- 2009-02-27 TW TW98106352A patent/TWI384876B/en active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040201862A1 (en) * | 2003-04-14 | 2004-10-14 | Bailey James Ray | Maximizing performance in a hardware image scaling module |
| TW200744365A (en) * | 2006-05-17 | 2007-12-01 | Realtek Semiconductor Corp | Method and related apparatus for determining image characteristics |
| US20080049045A1 (en) * | 2006-06-20 | 2008-02-28 | Sunplus Technology Co., Ltd. | Image scaling system capable of saving memory |
| US20080055338A1 (en) * | 2006-08-30 | 2008-03-06 | Ati Technologies Inc. | Multi-stage edge-directed image scaling |
Non-Patent Citations (2)
| Title |
|---|
| Lei Zhang, Member, and Xiaolin Wu,"An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion",IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 8, AUGUST 2006 * |
| Zaifeng Shi, Suying Yao,and Yingchun Zhao,"A Novel Video Image Scaling Algorithm Based on Morphological Edge Interpolation", IEEE Int. Conference Neural Networks & Signal Processing Zhenjiang, China, June 8~10, 2008 * |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201032592A (en) | 2010-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8131117B2 (en) | Method for magnifying images and videos and associated image processing device | |
| CN101815157B (en) | Image and video amplification method and relevant image processing device | |
| Wang et al. | Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation | |
| CN108765343B (en) | Image processing method, device, terminal and computer readable storage medium | |
| Dai et al. | Softcuts: a soft edge smoothness prior for color image super-resolution | |
| US8917948B2 (en) | High-quality denoising of an image sequence | |
| US7613363B2 (en) | Image superresolution through edge extraction and contrast enhancement | |
| CN108154474B (en) | Super-resolution image reconstruction method, device, medium and equipment | |
| US8351730B2 (en) | System and method for scaling images | |
| US8837854B2 (en) | Image processing method for boundary resolution enhancement | |
| Parsania et al. | A review: Image interpolation techniques for image scaling | |
| Joshi et al. | Super-resolution imaging: use of zoom as a cue | |
| KR101028628B1 (en) | Image texture filtering method, storage medium of storing program for executing the same and apparatus performing the same | |
| TWI384876B (en) | Method for upscaling images and videos and associated image processing device | |
| WO2016004667A1 (en) | Super-resolution reconstruction method for enhancing smoothness and definition of video image | |
| He et al. | A new improved seam carving content aware image resizing method | |
| WO2018040437A1 (en) | Picture processing method and apparatus | |
| CN106558021B (en) | Video enhancement method based on super-resolution technology | |
| CN108510438A (en) | A kind of image, video amplification method and system | |
| Yau et al. | Robust deep learning-based multi-image super-resolution using inpainting | |
| Abu et al. | Image projection over the edge | |
| WO2011158225A1 (en) | System and method for enhancing images | |
| Tsai et al. | EESRGAN: Efficient & Effective Super-Resolution Generative Adversarial Network | |
| Goswami | Comparison of image interpolation methods for image zooming | |
| Abu et al. | Image super-resolution via discrete tchebichef moment |