TWI779948B

TWI779948B - Lens dirt detection method for camera module

Info

Publication number: TWI779948B
Application number: TW110145118A
Authority: TW
Inventors: 賴志宏; 李建慶; 傅楸善
Original assignee: 大陸商廣州立景創新科技有限公司
Priority date: 2020-12-02
Filing date: 2021-12-02
Publication date: 2022-10-01
Also published as: CN112583999A; CN112583999B; TW202224404A

Abstract

The present disclosure provides a lens dirt detection method for a camera module. The lens dirt detection method includes: obtaining an image taken by a camera module; performing a gray-scale processing on the image to obtain a gray-scale image; performing a high-pass filter processing and a contrast processing on the gray-scale image to obtain a to-be-tested image; dividing the to-be-tested image into a plurality of blocks; calculating a luminance average value for luminance values of a plurality of pixels in each block; calculating a luminance gradient between each block and a plurality of neighboring blocks for each block; and determining a corresponding block is a dirt if the luminance gradient is greater than a luminance threshold value.

Description

相機模組的鏡頭髒污偵測方法Lens Dirty Detection Method for Camera Module

本發明是有關於一種影像處理技術，尤其是一種相機模組的鏡頭髒污偵測方法。The invention relates to an image processing technology, in particular to a lens dirty detection method of a camera module.

相機模組的鏡頭一般是由透鏡組成的用於在底片或投影布上形成影像的光學裝置，可廣泛的應用於攝影機、數位相機、或具有攝影或拍攝功能的手機、電腦等電子設備。The lens of a camera module is generally an optical device composed of a lens and used to form an image on a film or projection cloth. It can be widely used in video cameras, digital cameras, or electronic devices such as mobile phones and computers with photography or filming functions.

由於鏡頭廣泛的被應用，因此對於鏡頭的量產以及其質量的需求也相對提升。然而，在鏡頭的生產及裝配的過程中，難免會有灰塵、皮屑、污垢、水漬、刮痕、壞點等異物或髒污情形出現在鏡頭上，導致相機模組最後的成像會出現不需要的陰影，而影響成像的質量，並造成鏡頭生產的良率下降。因此，對鏡頭進行精準的髒污檢測變成極為重要。Due to the widespread use of lenses, the demand for mass production and quality of lenses has also increased relatively. However, in the process of lens production and assembly, it is inevitable that dust, dander, dirt, water stains, scratches, dead pixels and other foreign objects or dirt will appear on the lens, resulting in the final imaging of the camera module will appear Unnecessary shadows affect the quality of imaging and cause a decrease in the yield of lens production. Therefore, it is extremely important to perform accurate dirt detection on the lens.

然而，由於鏡頭的髒污的位置與形狀是微小且不易偵測的。因而，一般在進行鏡頭的髒污檢測時，會對相機模組之經由鏡頭所拍攝的影像進行影像處理而突顯出髒污以方便判斷。然而，此影像處理方式會將影像的干擾波紋（例如自然波紋）也同時突顯出來，而會影響對髒污的偵測（例如會致使髒污檢測發生錯誤判斷的情形）。However, the location and shape of the dirt on the lens are tiny and difficult to detect. Therefore, generally, when detecting the dirt of the lens, image processing is performed on the image captured by the camera module through the lens to highlight the dirt so as to facilitate judgment. However, this image processing method will also highlight the interference ripples (such as natural ripples) of the image at the same time, which will affect the detection of dirt (for example, it will cause wrong judgments in the dirt detection).

鑑於上述，本發明提供一種相機模組的鏡頭髒污偵測方法，可在對鏡頭進行髒污偵測時，消除相機模組經由鏡頭所拍攝的影像的干擾波紋，並精準的偵測出鏡頭的髒污。In view of the above, the present invention provides a lens dirt detection method of a camera module, which can eliminate the interference ripples of the image captured by the camera module through the lens when the lens is dirty detected, and accurately detect the lens dirty.

依據一些實施例，相機模組的鏡頭髒污偵測方法包含取得相機模組拍攝的一影像；對影像執行一灰階處理以取得一灰階影像；對灰階影像執行一高通濾波處理及一對比處理以取得一待測影像；將待測影像區分為多個區塊；對每一區塊中的複數像素的亮度值計算一亮度平均值；對於每個區塊計算其與相鄰的多個區塊的一亮度梯度；以及當亮度梯度大於一亮度門檻值時，判斷對應的區塊為一髒污。According to some embodiments, the lens contamination detection method of the camera module includes obtaining an image captured by the camera module; performing a grayscale processing on the image to obtain a grayscale image; performing a high-pass filtering process and a grayscale image on the grayscale image. Compare and process to obtain an image to be tested; divide the image to be tested into multiple blocks; calculate a brightness average value for the brightness values of the complex pixels in each block; a brightness gradient of a block; and when the brightness gradient is greater than a brightness threshold value, it is determined that the corresponding block is dirty.

綜上所述，依據本發明之實施例，藉由對影像進行灰階處理以及高通濾波處理以消除干擾波紋而降低髒污偵測的誤判。由於影像中每個像素之間並不一定是連續的，因此藉由對影像中的多個像素組合成的區塊進行亮度比對，以準確的判斷出鏡頭的髒污及鏡頭的髒污位置（例如判斷出髒污的邊緣以界定出髒污的位置及大小），而提升相機模組生產良率。To sum up, according to the embodiment of the present invention, by performing grayscale processing and high-pass filtering processing on the image to eliminate interference ripples and reduce misjudgment of dirt detection. Since each pixel in the image is not necessarily continuous, the brightness of the block composed of multiple pixels in the image is compared to accurately determine the dirt of the lens and the dirty position of the lens (For example, determine the edge of the dirt to define the location and size of the dirt), so as to improve the production yield of the camera module.

在本文中使用了某些詞彙來指稱特定的元件。所屬領域中具有通常知識者應可理解，硬體製造商可能會用不同的名詞來稱呼同一個元件。應以元件在功能上的差異來作為區分的準則，並不以名稱的差異來作為區分元件的方式。在本文中所提及的「包含」係為一開放式的用語，故應解釋成「包含但不限定於」。此外，「耦接」一詞在此係包含任何直接及間接的電氣連接手段，因此，若文中描述一第一裝置耦接於一第二裝置，則代表第一裝置可直接電氣連接於第二裝置，或者透過其他裝置或連接手段間接地電氣連接至第二裝置。Certain terms are used herein to refer to particular elements. It should be understood by those skilled in the art that hardware manufacturers may use different terms to refer to the same component. The difference in function of the components should be used as the criterion for distinguishing the components, and the difference in the name should not be used as the way to distinguish the components. The "comprising" mentioned in this article is an open-ended term, so it should be interpreted as "including but not limited to". In addition, the term "coupled" here includes any direct and indirect means of electrical connection. Therefore, if it is described in the text that a first device is coupled to a second device, it means that the first device can be directly electrically connected to the second device. device, or indirectly electrically connected to a second device through other devices or connection means.

參照圖1，圖1係為本發明一實施例之鏡頭髒污偵測的系統的方塊示意圖。鏡頭髒污偵測的系統包含待測的一相機模組100以及影像處理裝置200。相機模組100耦接影像處理裝置200。所述相機模組100可為設置於攝影裝置、數位相機、智慧型手機、電腦等具有攝影或取像功能的電子設備。相機模組100可包含鏡頭（鏡頭模組）以及影像感測器，以對物體進行攝影或取像，而產生物體的影像。若鏡頭具有髒污時，則會產生具有髒污陰影的影像。所述鏡頭可為由一或多個光學玻璃組成的透鏡組。所述影像感測器可為將光訊號轉換為類比電訊號的感光裝置，例如感光耦合裝置（Charge Coupled Device，CCD）、互補性氧化金屬半導體感光裝置（Complementary Metal-Oxide Semiconductor，CMOS）。影像處理裝置200自相機模組100取得其產生的影像，並對影像進行對應的處理，以產生處理後的結果。具體來說，影像處理裝置200執行本發明之相機模組100的鏡頭髒污偵測方法，以判斷相機模組100的鏡頭是否有髒污而產生一結果。所述影像處理裝置200可為桌上型電腦、筆記型電腦、嵌入式運算系統等具有影像處理功能的運算裝置。Referring to FIG. 1 , FIG. 1 is a schematic block diagram of a lens dirt detection system according to an embodiment of the present invention. The lens contamination detection system includes a camera module 100 and an image processing device 200 to be tested. The camera module 100 is coupled to the image processing device 200 . The camera module 100 can be an electronic device with a photographing or image capturing function installed in a photographic device, a digital camera, a smart phone, a computer, and the like. The camera module 100 may include a lens (lens module) and an image sensor to photograph or capture an object to generate an image of the object. If the lens is dirty, it will produce images with dirty shadows. The lens can be a lens group composed of one or more optical glasses. The image sensor can be a photosensitive device that converts light signals into analog electrical signals, such as a charge coupled device (CCD), a complementary metal oxide semiconductor photosensitive device (Complementary Metal-Oxide Semiconductor, CMOS). The image processing device 200 obtains the generated image from the camera module 100 and performs corresponding processing on the image to generate a processed result. Specifically, the image processing device 200 executes the lens dirt detection method of the camera module 100 of the present invention to determine whether the lens of the camera module 100 is dirty to generate a result. The image processing device 200 can be a computing device with image processing functions such as a desktop computer, a notebook computer, and an embedded computing system.

參照圖2，圖2係為本發明一實施例之相機模組100的鏡頭髒污偵測方法的流程圖。合併參照圖3，圖3係為本發明一實施例之相機模組100拍攝白色背景模板所產生的影像的示意圖。首先，取得相機模組100拍攝的一影像（步驟S201）。具體來說，影像處理裝置200取得相機模組100經由鏡頭所拍攝的影像。例如，取得相機模組100透過拍攝白色背景模板所產生的影像。所述白色背景模板可以為白板、白紙、白布、白牆等。在一些實施例中，相機模組100的鏡頭可以緊貼於一目標光屏以進行拍攝而產生影像。所述目標光屏可為一表面光滑的穩定平行光源。透過拍攝目標光屏能夠排除背景模板本身紋理對於後續髒污偵測造成的影響，並能進一步提高拍攝亮度，從而使髒污更加準確且清晰地呈現於影像中，以利後續的髒污偵測。Referring to FIG. 2 , FIG. 2 is a flowchart of a method for detecting lens contamination of the camera module 100 according to an embodiment of the present invention. Referring to FIG. 3 together, FIG. 3 is a schematic diagram of an image generated by the camera module 100 shooting a white background template according to an embodiment of the present invention. Firstly, an image captured by the camera module 100 is obtained (step S201 ). Specifically, the image processing device 200 obtains the image captured by the camera module 100 through the lens. For example, an image generated by the camera module 100 by shooting a white background template is obtained. The white background template can be a white board, white paper, white cloth, white wall, etc. In some embodiments, the lens of the camera module 100 can be closely attached to a target light screen to take pictures and generate images. The target light screen can be a stable parallel light source with a smooth surface. The impact of the texture of the background template itself on the subsequent dirt detection can be eliminated through the shooting target light screen, and the shooting brightness can be further improved, so that the dirt can be more accurately and clearly presented in the image, which is convenient for the subsequent dirt detection .

合併參照圖4，圖4係為本發明一實施例之灰階影像的示意圖。接著，對影像執行一灰階處理以取得一灰階影像（步驟S203）。具體來說，影像處理裝置200將影像的三原色光模式（RGB color model）的三個圖像通道轉換為單一圖像通道以產生灰階影像。例如，將影像中的每個像素的RGB值根據式1或式2來計算明亮度（Luminance 或 Luma）而轉換成灰階影像。其中式1與式2的Y表示明亮度，R代表單位像素的紅色通道亮度，G代表單位像素的綠色通道亮度，B代表單位像素的藍色通道亮度。Referring to FIG. 4 together, FIG. 4 is a schematic diagram of a grayscale image according to an embodiment of the present invention. Next, a grayscale process is performed on the image to obtain a grayscale image (step S203 ). Specifically, the image processing device 200 converts the three image channels of the RGB color model into a single image channel to generate a gray scale image. For example, the RGB value of each pixel in the image is converted into a grayscale image by calculating the brightness (Luminance or Luma) according to Equation 1 or Equation 2. Among them, Y in Formula 1 and Formula 2 represents the brightness, R represents the brightness of the red channel of the unit pixel, G represents the brightness of the green channel of the unit pixel, and B represents the brightness of the blue channel of the unit pixel.

……（式1）

……(Formula 1)

……（式2）

... (Formula 2)

在取得灰階影像後，對灰階影像執行一高通濾波處理及一對比處理以取得一待測影像（步驟S205）。具體來說，影像處理裝置200將灰階影像透過高通濾波器提取其中的高頻分量，並將提取的高頻分量與灰階影像疊加，以產生強化影像中高頻細節的一高通濾波影像。在一些實施例中，高通濾波處理可為高斯濾波、中值濾波、最大值濾波、最小值濾波、反銳化遮掩（Unsharp Masking）或雙邊濾波（Bilateral Filter）等高通濾波的影像處理方式。藉由將影像轉換為適當的灰階影像，並將灰階影像透過適當的高通濾波處理，能夠消除影像中的干擾波紋（例如由影像中心所產生的同心圓波紋），以降低對於後續髒污偵測造成的影響（例如降低髒污偵測誤判的機會）。After the grayscale image is obtained, a high-pass filtering process and a comparison process are performed on the grayscale image to obtain an image to be tested (step S205 ). Specifically, the image processing device 200 extracts high-frequency components from the gray-scale image through a high-pass filter, and superimposes the extracted high-frequency components on the gray-scale image to generate a high-pass filtered image that enhances high-frequency details in the image. In some embodiments, the high-pass filtering process may be Gaussian filtering, median filtering, maximum filtering, minimum filtering, unsharp masking, or bilateral filtering. By converting the image into an appropriate gray-scale image and passing the gray-scale image through appropriate high-pass filter processing, the interference ripples in the image (such as concentric circle ripples generated by the center of the image) can be eliminated to reduce the subsequent contamination Impact of detection (e.g. reduced chance of false detection of contamination).

合併參照圖5，圖5係為本發明一實施例之待測影像的示意圖。續，在影像處理裝置200取得高通濾波影像後，對高通濾波影像執行對比處理以取得待測影像。具體來說，影像處理裝置200對高通濾波影像進一步執行對比度的調節而產生待測影像。例如強化影像明暗對比，以使高通濾波影像中具有亮度值（Intensity）較小的像素其亮度值變得更小，而具有亮度值較大的像素其亮度值變得更大，以進一步強化高通濾波影像的細節（例如突顯出高通濾波影像中的髒污陰影）。在一些實施例中，影像處理裝置200可以對高通濾波影像進行對比平衡的調節，以重新分布高通濾波影像中的亮部與陰影，而使亮部與陰影之間達到平衡。所述對比處理例如但不限於全域型對比增強（Global Enhancement Methods）、直方圖均衡對比增強（Histogram Equalization）、自適應直方圖均衡對比增強（Adaptive Histogram Equalization）、限制對比度自適應直方圖均衡化（Contrast Limited Adaptive Histogram Equalization）等。Combined with reference to FIG. 5 , FIG. 5 is a schematic diagram of an image to be tested according to an embodiment of the present invention. Next, after the image processing device 200 obtains the high-pass filtered image, a comparison process is performed on the high-pass filtered image to obtain the image to be tested. Specifically, the image processing device 200 further performs contrast adjustment on the high-pass filtered image to generate the image to be tested. For example, the contrast between light and dark of the image is enhanced, so that the pixels with a smaller brightness value (Intensity) in the high-pass filter image have a smaller brightness value, and the pixels with a larger brightness value have a larger brightness value to further enhance the high-pass filter. Filter image details (eg accentuate dirty shadows in high-pass filtered images). In some embodiments, the image processing device 200 can adjust the contrast balance of the high-pass filtered image, so as to redistribute the highlights and shadows in the high-pass filtered image, so as to achieve a balance between the highlights and the shadows. The contrast processing is for example but not limited to global contrast enhancement (Global Enhancement Methods), histogram equalization contrast enhancement (Histogram Equalization), adaptive histogram equalization contrast enhancement (Adaptive Histogram Equalization), limited contrast adaptive histogram equalization ( Contrast Limited Adaptive Histogram Equalization), etc.

合併參照圖6，圖6係為本發明一實施例之區分為多個區塊61的待測影像的示意圖。在取得待測影像後，將待測影像區分為多個區塊61（步驟S207）。具體來說，影像處理裝置200將待測影像以多個邊界63區分為分別包含多個像素的多個區塊61，每一區塊61所包含的像素數量可為一致的，且每一區塊61定義在待測影像中的不同的位置並且互相不重疊。換言之，相鄰的複數個像素組合成為一區塊61。在一些實施例中，每一區塊61包含由複數個像素組成的一像素矩陣（例如3*3的像素矩陣，亦即9個像素組合成的像素矩陣）。Referring to FIG. 6 together, FIG. 6 is a schematic diagram of an image to be tested divided into a plurality of blocks 61 according to an embodiment of the present invention. After the image to be tested is obtained, the image to be tested is divided into a plurality of blocks 61 (step S207 ). Specifically, the image processing device 200 divides the image to be tested into a plurality of blocks 61 respectively containing a plurality of pixels by a plurality of boundaries 63, the number of pixels included in each block 61 may be the same, and each region The blocks 61 are defined at different positions in the image to be tested and do not overlap with each other. In other words, a plurality of adjacent pixels are combined into a block 61 . In some embodiments, each block 61 includes a pixel matrix composed of a plurality of pixels (for example, a 3*3 pixel matrix, that is, a pixel matrix composed of 9 pixels).

接著，對每一區塊61中的複數像素的亮度值計算一亮度平均值（步驟S209）。具體來說，待測影像中的每一像素具有一亮度值，亮度值的範圍可以依據影像處理裝置200所設定的色彩解析度來調整，其中色彩解析度為影像處理裝置200是使用幾個位元來記錄色彩。例如若影像處理裝置200使用八位元來記錄色彩，則亮度值的範圍為0~255。影像處理裝置200將每一區塊61中的每一像素的亮度值相加後除以每一區塊61中的像素數量，以計算得每一區塊61的亮度平均值，如式3所示，其中I _B為每一區塊61的亮度平均值，n為每一區塊61中像素的數量，I _i為每一區塊61中每一像素的亮度值。 Next, a brightness average value is calculated for the brightness values of the plurality of pixels in each block 61 (step S209 ). Specifically, each pixel in the image to be tested has a brightness value, and the range of the brightness value can be adjusted according to the color resolution set by the image processing device 200, wherein the color resolution is how many bits are used by the image processing device 200 element to record the color. For example, if the image processing device 200 uses eight bits to record colors, the range of brightness values is 0-255. The image processing device 200 adds the brightness value of each pixel in each block 61 and divides it by the number of pixels in each block 61 to calculate the average brightness value of each block 61, as shown in Equation 3 , where I _B is the average brightness of each block 61, n is the number of pixels in each block 61, and _Ii is the brightness value of each pixel in each block 61.

……（式3）

... (Formula 3)

在計算得每一區塊61的亮度平均值後，對於每一區塊61計算其與相鄰的多個區塊61的一亮度梯度（步驟S211）。具體來說，影像處理裝置200對每一區塊61與其相鄰的多個區塊61分別進行偏微分以計算得亮度梯度（例如，若與當前欲計算的區塊61相鄰的多個區塊61的數量是8個，則將當前區塊61分別一對一計算其與相鄰的區塊61的亮度梯度，亦即計算得分別對應每一相鄰區塊61的8個亮度梯度），亦即計算得每一區塊61其與相鄰的多個區塊61之間的亮度值差異，（例如，計算得每一區塊61其與相鄰的多個區塊61之間的亮度值的變化率），換言之，影像處理裝置200是根據每一區塊61的亮度平均值其與相鄰的多個區塊61的亮度平均值之間的變化程度來計算亮度梯度。After calculating the average brightness of each block 61 , for each block 61 , calculate a brightness gradient between it and a plurality of adjacent blocks 61 (step S211 ). Specifically, the image processing device 200 performs partial differentiation on each block 61 and its adjacent blocks 61 to calculate the brightness gradient (for example, if the multiple areas adjacent to the current block 61 to be calculated If the number of blocks 61 is 8, the brightness gradients between the current block 61 and the adjacent blocks 61 are calculated one-to-one, that is, 8 brightness gradients corresponding to each adjacent block 61 are calculated) , that is, calculate the brightness value difference between each block 61 and the adjacent multiple blocks 61, (for example, calculate the brightness value difference between each block 61 and the adjacent multiple blocks 61 Change rate of luminance value), in other words, the image processing device 200 calculates the luminance gradient according to the degree of change between the average luminance of each block 61 and the average luminance of a plurality of adjacent blocks 61 .

合併參照圖7，圖7係為本發明一實施例之髒污偵測結果。在計算得每一區塊61的亮度梯度後，影像處理裝置200判斷區塊61的亮度梯度是否大於一亮度門檻值（步驟S213）。在區塊61的亮度梯度大於亮度門檻值時，影像處理裝置200判斷亮度梯度大於亮度門檻值的區塊61為一髒污10（步驟S215）。換言之，當區塊61的亮度梯度大於亮度門檻值時，表示當前區塊61其與相鄰的多個區塊61之間的亮度值差異過大，此時當前區塊61可能為鏡頭髒污的邊緣，因而影像處理裝置200判斷當前的區塊61為髒污。在區塊61的亮度梯度不大於亮度門檻值時，影像處理裝置200判斷亮度梯度不大於亮度門檻值的區塊為一非髒污（步驟S217）。影像處理裝置200藉由組合判斷為髒污的區塊61，以獲得鏡頭的每個髒污10的位置及大小，並產生一髒污結果，並於髒污結果中標記出髒污10的位置及大小。例如，如圖7所示，影像處理裝置200於影像中標記出判斷為髒污10的區塊61，並將判斷為髒污10的相鄰的區塊61以一標示線串連而形成一髒污標記60，以產生具有髒污10的位置與髒污10的大小的髒污標記60的一髒污偵測結果。藉此，可精準的判斷出鏡頭是否有髒污10、髒污10的位置及大小，以快速判斷相機模組100是否有瑕疵。Combined with reference to FIG. 7 , FIG. 7 is a dirt detection result according to an embodiment of the present invention. After calculating the brightness gradient of each block 61 , the image processing device 200 determines whether the brightness gradient of the block 61 is greater than a brightness threshold (step S213 ). When the brightness gradient of the block 61 is greater than the brightness threshold, the image processing device 200 determines that the block 61 whose brightness gradient is greater than the brightness threshold is a dirty 10 (step S215 ). In other words, when the luminance gradient of the block 61 is greater than the luminance threshold value, it means that the luminance value difference between the current block 61 and the adjacent multiple blocks 61 is too large, and at this time the current block 61 may be lens dirty. Therefore, the image processing device 200 judges that the current block 61 is dirty. When the luminance gradient of the block 61 is not greater than the luminance threshold, the image processing device 200 determines that the block whose luminance gradient is not greater than the luminance threshold is a non-dirty block (step S217 ). The image processing device 200 obtains the position and size of each stain 10 on the lens by combining the blocks 61 judged to be dirty, and generates a dirty result, and marks the position of the dirty 10 in the dirty result and size. For example, as shown in FIG. 7 , the image processing device 200 marks the block 61 judged to be dirty 10 in the image, and connects the adjacent blocks 61 judged to be dirty 10 in series with a marking line to form a The dirt mark 60 is used to generate a dirt detection result of the dirt mark 60 having the position of the dirt 10 and the size of the dirt 10 . In this way, it is possible to accurately determine whether the lens is dirty 10 , the location and size of the dirt 10 , so as to quickly determine whether the camera module 100 is flawed.

參照圖8，圖8係為本發明一實施例之另一待測影像的示意圖。區塊A5與區塊A1~A4、A6~A9相鄰，區塊A5位於髒污10的邊緣。在此實施例中，可見區塊A5的亮度平均值小於與其相鄰的區塊A1、A4、A7、A8，亦即區塊A5與其相鄰的區塊A1、A4、A7、A8之間的亮度差異是較大的，換言之計算得的區塊A5的亮度梯度大於亮度門檻值，而影像處理裝置200將會判斷區塊A5為髒污10。另一方面，區塊B5與區塊B1~B4、B6~B9相鄰，區塊B5未位於髒污10的邊緣，且區塊B5的亮度平均值與其相鄰的區塊B1~B4、B6~B9相同，因而計算得的區塊B5的亮度梯度小於亮度門檻值，此時影像處理裝置200將判斷區塊B5為非髒污。Referring to FIG. 8 , FIG. 8 is a schematic diagram of another image to be tested according to an embodiment of the present invention. Block A5 is adjacent to blocks A1~A4, A6~A9, and block A5 is located at the edge of dirt 10. In this embodiment, the average brightness of the visible block A5 is smaller than that of its adjacent blocks A1, A4, A7, and A8, that is, the average brightness between the block A5 and its adjacent blocks A1, A4, A7, and A8 The luminance difference is large, in other words, the calculated luminance gradient of the block A5 is greater than the luminance threshold, and the image processing device 200 will judge the block A5 as dirty 10 . On the other hand, block B5 is adjacent to blocks B1~B4, B6~B9, block B5 is not located at the edge of dirty 10, and the average brightness of block B5 is adjacent to blocks B1~B4, B6 ~ B9 are the same, so the calculated luminance gradient of the block B5 is smaller than the luminance threshold value, and the image processing device 200 will determine that the block B5 is not dirty.

在一些實施例中，在步驟S207中，影像處理裝置200根據待測影像的一影像解析度決定待測影像被區分成區塊61的數量。例如，若待測影像的影像解析度較高時（亦即相機模組100所使用的鏡頭的解析度較高時），則每一區塊61所包含的像素數量可以大於在較低影像解析度區分出的每一區塊61所包含的像素數量，以使對於解析度高的鏡頭及解析度低的鏡頭皆可以快速的偵測髒污10。在一些實施例中，亮度門檻值可以根據待測影像的影像解析度而作調整。例如，若待測影像的影像解析度較高時，此時每一區塊61包含較大的像素數量，而致使每一區塊61其與相鄰的多個區塊61的亮度梯度值可能較小（亦即，二區塊61之間的亮度值差異可能較小），因此，為了精準的判斷區塊61是否為髒污10，影像處理裝置200可調降亮度門檻值。In some embodiments, in step S207 , the image processing device 200 determines the number of blocks 61 in which the image to be tested is divided into according to an image resolution of the image to be tested. For example, if the image resolution of the image to be tested is relatively high (that is, when the resolution of the lens used by the camera module 100 is relatively high), the number of pixels included in each block 61 may be larger than that of the image with a lower resolution. The number of pixels contained in each block 61 distinguished by the high-resolution, so that the dirt 10 can be quickly detected for both the high-resolution lens and the low-resolution lens. In some embodiments, the brightness threshold can be adjusted according to the image resolution of the image to be tested. For example, if the image resolution of the image to be tested is relatively high, each block 61 contains a large number of pixels, so that the brightness gradient values of each block 61 and adjacent blocks 61 may be Therefore, in order to accurately determine whether the block 61 is dirty 10 , the image processing device 200 can lower the brightness threshold.

參照圖9，圖9係為本發明一實施例之相機模組100的鏡頭髒污偵測方法的流程圖。在一些實施例中，在步驟S205中，選用反銳化遮掩處理作為高通濾波處理。反銳化遮掩處理為使用模糊（blurred）、非銳化（unsharp）或負影像（negative）來對原影像進行遮罩（mask），以產生較原影像清晰（銳化或不模糊）的新影像。在一些實施例中，反銳化遮掩處理可為以線性高頻濾波器或非線性高頻率波器來放大影像中高頻的分量。Referring to FIG. 9 , FIG. 9 is a flowchart of a method for detecting lens contamination of the camera module 100 according to an embodiment of the present invention. In some embodiments, in step S205, the unsharp masking process is selected as the high-pass filtering process. Unsharp masking is the use of blurred, unsharp, or negative images to mask the original image to produce a new image that is clearer (sharp or unblurred) than the original image. image. In some embodiments, the unsharp masking process may use a linear high-frequency filter or a nonlinear high-frequency filter to amplify high-frequency components in the image.

於此說明反銳化遮掩處理的程序。在取得灰階影像後，對灰階影像執行模糊處理而取得一模糊影像（步驟S2051）。具體來說，影像處理裝置200以一低通濾波器提取灰階影像中的低頻分量，以產生模糊影像。所述低通濾波器可為框濾波器（box filter）、高斯模糊濾波器（Gaussian blur filter）、中值濾波器（median filter）、雙邊濾波器（bilateral filter）等。舉例來說，以高斯模糊濾波器為例，影像處理裝置200將灰階影像與灰階影像的常態分佈進行卷積（Convolution），而產生降低雜訊與影像細節層次的模糊影像。在一些實施例中，低通濾波器在進行卷積以提取灰階影像中的低頻分量時，所使用的卷積核（kernel）的尺寸可以根據灰階影像的影像解析度（或鏡頭的解析度）來設定，例如若灰階影像的影像解析度較高時，可以將卷積核設定為較大的尺寸。The procedure of unsharp masking is described here. After the grayscale image is obtained, a blurring process is performed on the grayscale image to obtain a blurred image (step S2051 ). Specifically, the image processing device 200 uses a low-pass filter to extract low-frequency components in the grayscale image to generate a blurred image. The low-pass filter may be a box filter, a Gaussian blur filter, a median filter, a bilateral filter, and the like. For example, taking the Gaussian blur filter as an example, the image processing device 200 performs convolution on the grayscale image and the normal distribution of the grayscale image to generate a blurred image with reduced noise and image detail level. In some embodiments, when the low-pass filter performs convolution to extract low-frequency components in the gray-scale image, the size of the convolution kernel (kernel) used can be based on the image resolution of the gray-scale image (or the resolution of the lens) degree), for example, if the image resolution of the grayscale image is higher, the convolution kernel can be set to a larger size.

在取得模糊影像後，對模糊影像執行一亮度反向處理而取得一負影像（步驟S2053）。具體來說，影像處理裝置200將模糊影像中的每個像素的亮度值反轉（inverse），而產生負影像。亦即，影像處理裝置200將模糊影像中的原明亮處變暗，並將原暗處變亮。舉例來說，以影像處理裝置200使用八位元來記錄色彩，而每個像素的亮度值的範圍為0~255為例，當影像處理裝置200執行亮度反向處理時，以亮度值範圍的最大值（例如255）減去每一像素的原亮度值而計算出每一像素的新亮度值，以產生具有新的亮度值的負影像。After the blurred image is obtained, a brightness inversion process is performed on the blurred image to obtain a negative image (step S2053 ). Specifically, the image processing device 200 inverts the brightness value of each pixel in the blurred image to generate a negative image. That is, the image processing device 200 darkens the original bright areas in the blurred image, and brightens the original dark areas. For example, taking the image processing device 200 using eight bits to record colors, and the range of the brightness value of each pixel is 0~255 as an example, when the image processing device 200 performs brightness inverse processing, the brightness value range is The original brightness value of each pixel is subtracted from the maximum value (eg, 255) to calculate a new brightness value of each pixel, so as to generate a negative image with the new brightness value.

在取得灰階影像及負影像後，依據負影像及灰階影像，執行一線性光處理（Linear light）而取得一線性光影像（步驟S2055）。具體來說，影像處理裝置200將灰階影像及負影像進行透明合成（Alpha blending）後產生線性光影像。更具體地來說，影像處理裝置200將灰階影像中的每一像素的原亮度值根據式4計算出新的亮度值而產生具有新的亮度值的線性光影像。在一些實施例中，線性光處理可為結合線性加深處理（Linear Burn）與線性減淡處理（Linear Dodge），且線性光處理的效果可為類似於亮光處理（Vivid light）的效果。其中，線性加深處理為根據式5進行影像處理，線性減淡處理為根據式6進行影像處理，亮光處理為根據式7進行影像處理。其中式4的L表示線性光影像中每一像素的亮度值，式5的L表示經線性加深處理後產生的影像中每一像素的亮度值，式6的L表示經線性減淡處理後產生的影像中每一像素的亮度值，式7的L表示經亮光處理後的影像中每一像素的亮度值。其中，式4~式7的O表示灰階影像中每一像素的亮度值，N表示負影像中每一像素的亮度值，Max表示亮度值範圍的最大值。After the grayscale image and the negative image are obtained, a linear light processing (Linear light) is performed according to the negative image and the grayscale image to obtain a linear light image (step S2055 ). Specifically, the image processing device 200 performs alpha blending on the grayscale image and the negative image to generate a linear light image. More specifically, the image processing device 200 calculates a new brightness value from the original brightness value of each pixel in the grayscale image according to Equation 4 to generate a linear light image with a new brightness value. In some embodiments, the linear light processing can be a combination of linear burn processing (Linear Burn) and linear dodge processing (Linear Dodge), and the effect of the linear light processing can be similar to that of the bright light processing (Vivid light). Wherein, the linear deepening process is to perform image processing according to Formula 5, the linear lightening process is to perform image processing according to Formula 6, and the highlight processing is to perform image processing according to Formula 7. Among them, L in formula 4 represents the brightness value of each pixel in the linear light image, L in formula 5 represents the brightness value of each pixel in the image generated after linear deepening processing, and L in formula 6 represents the brightness value generated after linear lightening processing The luminance value of each pixel in the image of , and L in Equation 7 represents the luminance value of each pixel in the image after bright light processing. Among them, O in Equation 4 to Equation 7 represents the brightness value of each pixel in the grayscale image, N represents the brightness value of each pixel in the negative image, and Max represents the maximum value of the brightness value range.

……（式4）

... (Formula 4)

……（式5）

... (Formula 5)

……（式6）

... (Formula 6)

……（式7）

... (Formula 7)

在取得灰階影像及線性光影像後，合併灰階影像與線性光影像以產生一高通濾波影像（步驟S2057）。具體來說，影像處理裝置200將灰階影像及線性光影像進行透明合成後產生放大了灰階影像中的高頻分量的高通濾波影像。更具體地來說，影像處理裝置200將灰階影像中的每一像素的原亮度值根據式8計算出新的亮度值而產生具有新的亮度值的高通濾波影像。其中式8的L表示線性光影像中每一像素的亮度值，O表示灰階影像中每一像素的亮度值，H表示高通濾波影像中每一像素的亮度值。After the grayscale image and the linear light image are obtained, the grayscale image and the linear light image are combined to generate a high-pass filtered image (step S2057 ). Specifically, the image processing device 200 transparently synthesizes the grayscale image and the linear light image to generate a high-pass filter image that amplifies high frequency components in the grayscale image. More specifically, the image processing device 200 calculates a new brightness value from the original brightness value of each pixel in the grayscale image according to Formula 8 to generate a high-pass filtered image with a new brightness value. L in Formula 8 represents the brightness value of each pixel in the linear light image, O represents the brightness value of each pixel in the grayscale image, and H represents the brightness value of each pixel in the high-pass filter image.

……（式8）

... (Formula 8)

在取得高通濾波影像後，可選用自適應直方圖均衡化處理作為對比處理，以取得待測影像（步驟S2059）。舉例來說，影像處理裝置200可以設定一直方圖亮度閾值，並將高通濾波影像根據直方圖亮度閾值而區分為像素的亮度值小於直方圖亮度閾值的一區域及像素的亮度值大於直方圖亮度閾值的另一區域。影像處理裝置200分別對該二區域進行直方圖均衡化處理，以重新分布高通濾波影像的像素的亮度值，而改進高通濾波影像的局部對比度和增強高通濾波影像中的邊緣來獲得更多細節。例如影像處理裝置200可以根據式9與式10來分別對該二區域進行直方圖均衡化處理。其中，式9與式10的

表示為待測影像的像素的亮度值，

表示為待測影像的該些像素中所具有的最小亮度值，

表示為直方圖亮度閾值，

表示為待測影像的該些像素中所具有的最大亮度值。在式9中，

表示為高通濾波影像的小於直方圖亮度閾值的像素的累積分布函數，

表示為高通濾波影像的小於直方圖亮度閾值的該些像素中所具有的最小累積分布函數，

表示為高通濾波影像的小於直方圖亮度閾值的該些像素中所具有的最大累積分布函數。在式10中，

表示為高通濾波影像的大於直方圖亮度閾值的像素的累積分布函數，

表示為高通濾波影像的大於直方圖亮度閾值的該些像素中所具有的最小累積分布函數，

表示為高通濾波影像的大於直方圖亮度閾值的該些像素中所具有的最大累積分布函數。 After obtaining the high-pass filtered image, an adaptive histogram equalization process can be selected as a comparison process to obtain the image to be tested (step S2059 ). For example, the image processing device 200 can set a histogram brightness threshold, and divide the high-pass filtered image into an area with a pixel brightness value smaller than the histogram brightness threshold value and a pixel brightness value greater than the histogram brightness value according to the histogram brightness threshold value. Another region of the threshold. The image processing device 200 respectively performs histogram equalization processing on the two regions to redistribute the brightness values of the pixels of the high-pass filtered image, improve the local contrast of the high-pass filtered image and enhance the edges in the high-pass filtered image to obtain more details. For example, the image processing device 200 may perform histogram equalization processing on the two regions according to Equation 9 and Equation 10, respectively. Among them, formula 9 and formula 10

Expressed as the brightness value of the pixel of the image to be tested,

Expressed as the minimum brightness value of the pixels in the image to be tested,

Expressed as a histogram brightness threshold,

It is expressed as the maximum brightness value of the pixels in the image to be tested. In Equation 9,

Expressed as the cumulative distribution function of pixels less than the histogram brightness threshold of the high-pass filtered image,

Expressed as the minimum cumulative distribution function of those pixels of the high-pass filtered image that are less than the histogram brightness threshold,

Expressed as the maximum cumulative distribution function of those pixels of the high-pass filtered image that are less than the histogram brightness threshold. In Equation 10,

Expressed as the cumulative distribution function of pixels above the histogram brightness threshold of the high-pass filtered image,

Expressed as the minimum cumulative distribution function of those pixels of the high-pass filtered image that are greater than the histogram brightness threshold,

Expressed as the maximum cumulative distribution function of those pixels of the high-pass filtered image that are greater than the histogram brightness threshold.

……（式9）

... (Formula 9)

……（式10）

... (Formula 10)

在一些實施例中，影像處理裝置200還可以決定待測影像的暗邊區（步驟S219），以協助判斷區塊61是否為髒污10。具體來說，影像處理裝置200判斷區塊61是否位於暗邊區（步驟S221），若區塊61位於暗邊區中，則影像處理裝置200判斷區塊61是非髒污（步驟S217）；若區塊61位於暗邊區之外，則影像處理裝置200判斷該區塊61可能為髒污10（步驟S215），或是進一步執行其他協助判斷區塊61是否為髒污10的步驟。由於相機模組100的設計及結構因素，一般所拍攝出的影像四周會具有暗邊區，而暗邊區中的像素其亮度值為小的，而若髒污10為位於拍攝出的影像的暗邊區時，並不會影響影像的成像結果。因此，影像處理裝置200在對鏡頭進行髒污偵測時，可以排除位於暗邊區中的區塊61而僅對位於暗邊區以外的區塊61進行髒污偵測。In some embodiments, the image processing device 200 can also determine the dark border area of the image to be tested (step S219 ), so as to help determine whether the block 61 is dirty 10 . Specifically, the image processing device 200 judges whether the block 61 is located in the dark border area (step S221), if the block 61 is located in the dark border area, the image processing device 200 judges that the block 61 is not dirty (step S217); if the block If 61 is located outside the dark border area, the image processing device 200 determines that the block 61 may be dirty 10 (step S215 ), or further performs other steps to help determine whether the block 61 is dirty 10 . Due to the design and structural factors of the camera module 100, generally there will be a dark border area around the captured image, and the brightness value of the pixel in the dark border area is small, and if the dirt 10 is located in the dark border area of the captured image , it will not affect the imaging result of the image. Therefore, when the image processing device 200 detects dirt on the lens, it can exclude the block 61 located in the dark border area and only perform dirty detection on the block 61 located outside the dark border area.

參照圖10，圖10係為本發明一實施例之決定出暗邊區20的待測影像的示意圖。在一些實施例中，暗邊區20是待測影像的一影像邊緣21向內位移一距離23所界定的一暗邊邊緣25和影像邊緣21之間的區域。例如，影像邊緣21往待測影像的中心軸C位移一距離23以界定暗邊邊緣25。在一些實施例中，影像邊緣21向內位移的距離23可以根據暗邊邊緣25對應至中心軸C的間距27與影像邊緣21對應至中心軸C的間距29之間的比例關係來設定。例如，暗邊邊緣25對應至中心軸C的間距27可為影像邊緣21對應至中心軸C的間距29之98.5%，因此距離23可為影像邊緣21對應至中心軸C的間距29之1.5%。Referring to FIG. 10 , FIG. 10 is a schematic diagram of an image to be tested for determining a dark border area 20 according to an embodiment of the present invention. In some embodiments, the dark edge area 20 is an area between a dark edge 25 and the image edge 21 defined by an image edge 21 of the image to be tested inwardly displaced by a distance 23 . For example, the image edge 21 is displaced by a distance 23 to the central axis C of the image to be tested to define the dark edge 25 . In some embodiments, the inward displacement distance 23 of the image edge 21 can be set according to the proportional relationship between the distance 27 corresponding to the central axis C from the dark edge 25 and the distance 29 corresponding to the central axis C from the image edge 21 . For example, the distance 27 corresponding to the central axis C from the dark edge 25 can be 98.5% of the distance 29 from the image edge 21 to the central axis C, so the distance 23 can be 1.5% of the distance 29 from the image edge 21 to the central axis C .

在一些實施例中，影像處理裝置200還可以決定待測影像的暗角區（步驟S223），以協助判斷區塊61是否為髒污10。具體來說，影像處理裝置200判斷區塊61是否位於暗角區（步驟S225），若區塊61位於暗角區中，則影像處理裝置200判斷區塊61是非髒污（步驟S217）；若區塊61位於暗角區之外，則影像處理裝置200判斷該區塊61可能為髒污10（步驟S215），或是進一步執行其他協助判斷區塊61是否為髒污10的步驟。由於相機模組100的設計及結構因素，一般所拍攝出的影像四角落會具有暗角區，而暗角區中的像素其亮度值為小的，而若髒污10為位於拍攝出的影像的暗角區時，並不會影響影像的成像結果。因此，影像處理裝置200在對鏡頭進行髒污偵測時，可以排除位於暗角區中的區塊61而僅對位於暗角區以外的區塊61進行髒污偵測。In some embodiments, the image processing device 200 can also determine the shading area of the image to be tested (step S223 ), so as to help determine whether the block 61 is dirty 10 . Specifically, the image processing device 200 judges whether the block 61 is located in the dark corner area (step S225), if the block 61 is located in the dark corner area, the image processing device 200 judges that the block 61 is not dirty (step S217); if If the block 61 is located outside the dark corner area, the image processing device 200 determines that the block 61 may be dirty 10 (step S215 ), or further performs other steps to help determine whether the block 61 is dirty 10 . Due to the design and structural factors of the camera module 100, generally the four corners of the captured image will have vignetting areas, and the brightness value of the pixels in the vignetting area is small, and if the dirt 10 is located in the captured image It will not affect the imaging result of the image when the vignetting area is displayed. Therefore, when the image processing device 200 detects dirt on the lens, it can exclude the blocks 61 located in the vignetting area and only perform dirty detection on the blocks 61 located outside the vignetting area.

參照圖11，圖11係為本發明一實施例之決定出暗角區30的待測影像的示意圖。在一些實施例中，暗角區30是待測影像的一影像邊角31向內位移一距離35所界定的一暗邊邊角33和影像邊角31之間的區域。例如，影像邊角31往待測影像的中心點cp位移一距離35以界定暗邊邊角33。在一些實施例中，影像邊角31向內位移的距離35可以根據暗邊邊角33對應至中心點cp的間距37與影像邊角31對應至中心點cp的間距39之間的比例關係來設定。例如，暗邊邊角33對應至中心點cp的間距37可為影像邊角31對應至中心點cp間距39之85%，因此距離35可為影像邊角31對應至中心點cp間距39之15%。Referring to FIG. 11 , FIG. 11 is a schematic diagram of an image to be tested for determining the vignetting region 30 according to an embodiment of the present invention. In some embodiments, the dark corner region 30 is an area between a dark corner 33 and the image corner 31 defined by an image corner 31 of the image to be tested inwardly displaced by a distance 35 . For example, the image corner 31 is displaced by a distance 35 to the center point cp of the image to be tested to define the dark edge corner 33 . In some embodiments, the inward displacement distance 35 of the image corner 31 can be determined according to the proportional relationship between the distance 37 corresponding to the center point cp from the dark edge corner 33 and the distance 39 corresponding to the center point cp from the image corner 31 set up. For example, the distance 37 corresponding to the center point cp from the dark edge corner 33 can be 85% of the distance 39 from the image corner 31 to the center point cp, so the distance 35 can be 15% of the distance 39 from the image corner 31 to the center point cp %.

在一些實施例中，步驟S213、步驟S221及步驟S225的順序可以相互對調，換言之，可以先判斷區塊61是否位於暗邊區20或暗角區30後，始判斷區塊61的亮度梯度是否大於亮度門檻值；或是先判斷區塊61的亮度梯度是否大於亮度門檻值後，始判斷區塊61是否位於暗邊區20或暗角區30。在一些實施例中，步驟S219及步驟S223可以在步驟S201~S213之間執行，需注意的是步驟S221需在步驟S219之後執行，步驟S225需在步驟S223之後執行。In some embodiments, the order of step S213, step S221 and step S225 can be reversed. In other words, it is possible to judge whether the block 61 is located in the dark edge area 20 or the dark corner area 30 first, and then determine whether the brightness gradient of the block 61 is greater than Brightness threshold; or first judge whether the brightness gradient of the block 61 is greater than the brightness threshold, and then judge whether the block 61 is located in the dark edge area 20 or the dark corner area 30 . In some embodiments, step S219 and step S223 may be performed between steps S201-S213, and it should be noted that step S221 must be performed after step S219, and step S225 must be performed after step S223.

參照圖12，圖12係為本發明一實施例之相機模組100的鏡頭髒污偵測方法的流程圖。在一些實施例中，在某些情形下，由於以包含複數個像素的多個區塊61進行髒污偵測可能會漏偵測髒污10。例如，當前欲偵測的區塊61與其相鄰的區塊61皆位於髒污10的邊緣時，可能會導致判斷當前的區塊61為非髒污而漏偵測到此區塊61，亦即發生髒污偵測誤判。因此為了提升髒污偵測的精準度，在判斷完每一區塊61是否為髒污10後（步驟S215及S217），影像處理裝置200將區分為多個區塊61（於後稱為第一區塊）的多個邊界63（於後稱為第一邊界）沿一方向移動一步幅（stride）而根據移動後的多個邊界63（於後稱為第二邊界）界定另一多個區塊61（於後稱為第二區塊）（步驟S227）。Referring to FIG. 12 , FIG. 12 is a flowchart of a method for detecting lens contamination of the camera module 100 according to an embodiment of the present invention. In some embodiments, in some cases, the dirt 10 may be missed due to the dirt detection performed with multiple blocks 61 including a plurality of pixels. For example, when the current block 61 to be detected and its adjacent blocks 61 are all located at the edge of the dirty 10, it may cause the current block 61 to be judged to be non-dirty and this block 61 may be missed. That is, a false detection of dirt occurs. Therefore, in order to improve the accuracy of dirt detection, after determining whether each block 61 is dirty 10 (steps S215 and S217), the image processing device 200 will be divided into a plurality of blocks 61 (hereinafter referred to as the first A plurality of boundaries 63 (hereinafter referred to as the first boundary) of a block) move a step (stride) in one direction and define another plurality of boundaries 63 (hereinafter referred to as the second boundary) according to the moved block 61 (hereinafter referred to as the second block) (step S227 ).

參照圖13，圖13係為本發明一實施例之將區分的第一區塊40移動第一邊界41後，形成的第二邊界51所界定的第二區塊50的待測影像的示意圖。在一些實施例中，影像處理裝置200可以將第一區塊40的第一邊界41沿著待測影像的一維度方向（例如水平維度或垂直維度）移動一步輻後，形成第二邊界51而以第二邊界51界定第二區塊50。在一些實施例中，步輻的大小可以根據第一區塊40的像素所組合成的像素矩陣來設定，例如步輻的大小可設定成一半的像素矩陣。舉例來說，若第一區塊40的像素矩陣為3*3的矩陣（亦即第一區塊40包含9個像素），則影像處理裝置200可以將第一區塊40的第一邊界41由左至右移動1.5個像素距離後，形成第二邊界51以界定第二區塊50。Referring to FIG. 13 , FIG. 13 is a schematic diagram of an image to be tested of the second block 50 defined by the second boundary 51 formed after the distinguished first block 40 is moved by the first boundary 41 according to an embodiment of the present invention. In some embodiments, the image processing device 200 may move the first boundary 41 of the first block 40 along the one-dimensional direction (such as the horizontal dimension or the vertical dimension) of the image to be tested by one step to form the second boundary 51. The second block 50 is defined by a second boundary 51 . In some embodiments, the size of the spokes can be set according to the pixel matrix formed by the pixels of the first block 40 , for example, the size of the spokes can be set as half of the pixel matrix. For example, if the pixel matrix of the first block 40 is a 3*3 matrix (that is, the first block 40 includes 9 pixels), the image processing device 200 can make the first boundary 41 of the first block 40 After moving 1.5 pixels from left to right, a second boundary 51 is formed to define the second block 50 .

在界定出第二區塊50後，對每一第二區塊50中的多個像素計算亮度平均值，亦即執行步驟S209。在計算得每一第二區塊50的亮度平均值後，計算每一第二區塊50其與相鄰的多個第二區塊50的亮度梯度，亦即執行步驟S211，並接續後續步驟（步驟S213~步驟S225），以判斷第二區塊50是否為髒污10。藉此，可提升髒污偵測的準確率。在判斷完每一第二區塊50是否為髒污後（步驟S215及S217），可再回去執行步驟S227而將第二區塊50的第二邊界51沿著前一次移動的方向（亦即第一邊界41所移動的方向）或沿著另一方向（例如前一次為沿著水平維度移動，此次為沿著垂直維度移動；例如前一次為由左至右移動，此次為由上至下移動），而形成新的邊界以界定新的區塊61，並重複後續判斷髒污10的步驟，以完善的偵測相機模組100經由鏡頭所拍攝的影像是否具有髒污10。After the second block 50 is defined, the average brightness of the pixels in each second block 50 is calculated, that is, step S209 is executed. After calculating the average brightness of each second block 50, calculate the brightness gradient between each second block 50 and a plurality of adjacent second blocks 50, that is, execute step S211, and continue with subsequent steps (step S213 ~ step S225 ), to determine whether the second block 50 is dirty 10 . In this way, the accuracy of dirt detection can be improved. After judging whether each second block 50 is dirty (steps S215 and S217), step S227 can be executed again to move the second boundary 51 of the second block 50 along the previous moving direction (that is, The direction in which the first border 41 moves) or along another direction (for example, the previous time moved along the horizontal dimension, this time it moved along the vertical dimension; for example, the previous time moved from left to right, this time it moved from top to bottom move down), to form a new boundary to define a new block 61 , and repeat the subsequent step of determining the dirt 10 , so as to perfectly detect whether the image captured by the camera module 100 through the lens has the dirt 10 .

在一些實施例中，髒污偵測也可以由物體偵測（Object Detection）來實現。例如使用YOLOv3的神經網路分析方式來實現。In some embodiments, dirt detection can also be implemented by object detection. For example, use YOLOv3's neural network analysis method to achieve.

100:相機模組 200:影像處理裝置 S201~S227:步驟 10:髒污 60:髒污標記 61,A1~A9,B1~B9:區塊 63:邊界 20:暗邊區 21:影像邊緣 23:距離 25:暗邊邊緣 27:間距 29:間距 C:中心軸 30:暗角區 31:影像邊角 33:暗邊邊角 35:距離 37:間距 39:間距 cp:中心點 40:第一區塊 41:第一邊界 50:第二區塊 51:第二邊界 100: Camera module 200: image processing device S201~S227: steps 10: Dirty 60: Dirty mark 61, A1~A9, B1~B9: blocks 63: Boundary 20: Dark Frontier 21: Image edge 23: Distance 25: dark edge edge 27: Spacing 29: Spacing C: central axis 30: dark corner area 31: Image corner 33: dark edge corner 35: Distance 37: Spacing 39: Spacing cp: center point 40: First block 41: First Boundary 50:Second block 51:Second Boundary

[圖1]係為本發明一實施例之鏡頭髒污偵測的系統的方塊示意圖。 [圖2]係為本發明一實施例之相機模組的鏡頭髒污偵測方法的流程圖。 [圖3]係為本發明一實施例之相機模組拍攝白色背景模板所產生的影像的示意圖。 [圖4]係為本發明一實施例之灰階影像的示意圖。 [圖5]係為本發明一實施例之待測影像的示意圖。 [圖6]係為本發明一實施例之區分為多個區塊的待測影像的示意圖。 [圖7]係為本發明一實施例之髒污偵測結果的示意圖。 [圖8]係為本發明一實施例之另一待測影像的示意圖。 [圖9]係為本發明一實施例之相機模組的鏡頭髒污偵測方法的流程圖。 [圖10]係為本發明一實施例之決定出暗邊區的待測影像的示意圖。 [圖11]係為本發明一實施例之決定出暗角區的待測影像的示意圖。 [圖12]係為本發明一實施例之相機模組的鏡頭髒污偵測方法的流程圖。 [圖13]係為本發明一實施例之將區分的第一區塊移動第一邊界後，形成的第二邊界所界定的第二區塊的待測影像的示意圖。 [ FIG. 1 ] is a schematic block diagram of a system for detecting lens contamination according to an embodiment of the present invention. [ FIG. 2 ] is a flow chart of a method for detecting lens contamination of a camera module according to an embodiment of the present invention. [ FIG. 3 ] is a schematic diagram of an image generated by a camera module shooting a white background template according to an embodiment of the present invention. [FIG. 4] is a schematic diagram of a grayscale image according to an embodiment of the present invention. [ FIG. 5 ] is a schematic diagram of an image to be tested according to an embodiment of the present invention. [ FIG. 6 ] is a schematic diagram of an image to be tested divided into multiple blocks according to an embodiment of the present invention. [ FIG. 7 ] is a schematic diagram of the dirt detection result of an embodiment of the present invention. [ FIG. 8 ] is a schematic diagram of another image to be tested according to an embodiment of the present invention. [ FIG. 9 ] is a flow chart of a method for detecting lens contamination of a camera module according to an embodiment of the present invention. [ FIG. 10 ] is a schematic diagram of an image to be tested with a dark border area determined according to an embodiment of the present invention. [ FIG. 11 ] is a schematic diagram of an image to be tested for determining a vignetting area according to an embodiment of the present invention. [ FIG. 12 ] is a flowchart of a method for detecting lens contamination of a camera module according to an embodiment of the present invention. [ FIG. 13 ] is a schematic diagram of the image to be tested of the second block defined by the second boundary formed after the first distinguished first block is moved to the first boundary according to an embodiment of the present invention.

S201~S217:步驟 S201~S217: steps

Claims

一種相機模組的鏡頭髒污偵測方法，包含：取得該相機模組拍攝的一影像；對該影像執行一灰階處理以取得一灰階影像；對該灰階影像執行一高通濾波處理及一對比處理以取得一待測影像；將該待測影像區分為多個區塊；對每一該區塊中的複數像素的亮度值計算一亮度平均值；對於各該區塊計算其與相鄰的該些區塊的一亮度梯度；將區分為該些區塊的多個邊界沿一方向移動一步幅而根據移動後的該些邊界界定另一該些區塊；對移動該些邊界後所界定的每一該區塊中的該些像素的亮度值計算該亮度平均值；對於移動該些邊界後所界定的各該區塊計算其與相鄰的該些區塊的該亮度梯度；以及當該亮度梯度大於一亮度門檻值時，判斷對應的該區塊為一髒污。 A method for detecting lens contamination of a camera module, comprising: obtaining an image captured by the camera module; performing a grayscale process on the image to obtain a grayscale image; performing a high-pass filter process on the grayscale image and A comparison process to obtain an image to be tested; the image to be tested is divided into a plurality of blocks; a brightness average value is calculated for the brightness values of the complex pixels in each block; A luminance gradient of the adjacent blocks; move the boundaries that are divided into these blocks by one step along a direction and define other blocks according to the moved boundaries; after moving the boundaries calculating the brightness average value of the brightness values of the pixels in each of the defined blocks; calculating the brightness gradient between the blocks and the adjacent blocks for each of the blocks defined after moving the boundaries; And when the brightness gradient is greater than a brightness threshold, it is determined that the corresponding block is dirty.

如請求項1所述之相機模組的鏡頭髒污偵測方法，更包含：決定該待測影像的一暗邊區；其中，判斷該區塊為該髒污的步驟是對於在該待測影像中位於該暗邊區之外的該區塊執行。 The lens dirty detection method of the camera module as described in claim 1 further includes: determining a dark border area of the image to be tested; wherein, the step of judging that the block is the dirty is for the image to be tested Execute the block outside the dark border area.

如請求項2所述之相機模組的鏡頭髒污偵測方法，其中該暗邊區是該待測影像的一影像邊緣向內位移一距離所界定的一暗邊邊緣和該影像邊緣之間的區域。 The method for detecting lens contamination of a camera module as described in claim 2, wherein the dark border area is an area between a dark border border defined by a distance inwardly shifting an image border of the image to be tested and the border of the image. area.

如請求項1所述之相機模組的鏡頭髒污偵測方法，更包含：決定該待測影像的一暗角區；其中，判斷該區塊為該髒污的步驟是對於在該待測影像中位於該暗角區之外的該區塊執行。 The lens dirty detection method of the camera module as described in claim item 1 further includes: determining a vignetting area of the image to be tested; wherein, the step of judging that the block is the dirty is for the area to be tested Execute the block outside the vignetting area in the image.

如請求項4所述之相機模組的鏡頭髒污偵測方法，其中該暗角區是該待測影像的一影像邊角向內位移一距離所界定的一暗邊邊角和該影像邊角之間的區域。 The lens dirty detection method of the camera module as described in claim 4, wherein the vignetting area is a dark edge corner and the image edge defined by an image corner of the image to be tested inwardly displaced by a distance the area between the corners.

如請求項1所述之相機模組的鏡頭髒污偵測方法，其中該高通濾波處理包含：對該灰階影像執行一模糊處理而取得一模糊影像；對該模糊影像執行一亮度反向處理而取得一負影像；依據該負影像及該灰階影像，執行一線性光處理而取得一線性光影像；以及合併該灰階影像與該線性光影像以產生一高通濾波影像，以供執行該對比處理而取得該待測影像。 The method for detecting lens contamination of a camera module according to claim 1, wherein the high-pass filter processing includes: performing a blurring process on the grayscale image to obtain a blurred image; performing a brightness inversion process on the blurred image and obtain a negative image; according to the negative image and the grayscale image, perform a linear light processing to obtain a linear light image; and combine the grayscale image and the linear light image to generate a high-pass filtered image for performing the The image to be tested is obtained through comparison processing.

如請求項1所述之相機模組的鏡頭髒污偵測方法，其中對於各該區塊計算其與相鄰的該些區塊的該亮度梯度的步驟，是根據各該區塊的該亮度平均值與其相鄰的該些區塊的該些亮度平均值之間的變化程度來計算該亮度梯度。 The lens dirty detection method of the camera module as described in claim 1, wherein the step of calculating the brightness gradient between each block and the adjacent blocks is based on each block The brightness gradient is calculated by the degree of change between the brightness average of the block and the brightness averages of the adjacent blocks.

如請求項1所述之相機模組的鏡頭髒污偵測方法，其中將該待測影像區分為多個區塊的步驟，是根據該待測影像的一影像解析度，而將該待測影像區分為該些區塊。 The method for detecting lens contamination of a camera module as described in claim 1, wherein the step of dividing the image to be tested into a plurality of blocks is to divide the image to be tested according to an image resolution of the image to be tested The image area is divided into these blocks.

如請求項1所述之相機模組的鏡頭髒污偵測方法，其中該對比處理為一自適應直方圖均衡化處理。The method for detecting lens contamination of a camera module according to Claim 1, wherein the comparison process is an adaptive histogram equalization process.