TW202111601A - Maritime image filter method and processing device - Google Patents

Abstract

A method for filtering a maritime image noise at least includes an image processing device and a photoelectric sensor. The image processing device is implemented by a processing unit. The method includes at least the following steps: (A) performing a dark channel spatial domain estimation on an original input image to obtain a dark channel two-dimensional matrix of the image; (B) performing a photometric spatial domain estimation on the original input image to obtain a photometric two-dimensional matrix of the image; (C) performing a convolution operation on the dark channel two-dimensional matrix and the photometric two-dimensional matrix to calculate an estimated value of a credibility of an object on the water and obtaining a two-dimensional credibility matrix. The two-dimensional credibility matrix can be used in the image to filter interferences and noises associated with water surface phenomenon to achieve the filtering effect of the maritime image.

Description

水上影像雜訊濾波方法及其影像處理裝置 Water image noise filtering method and image processing device

本發明是有關於一種影像資料處理，特別是指一種用於水面特性之水上影像雜訊濾波方法及其處理裝置。 The present invention relates to an image data processing, in particular to a water image noise filtering method and its processing device for water surface characteristics.

習知通過使用光學(Electro-optical Sensors)感測器來擷取周遭環境之光學數據，藉以記錄、儲存為影像或視訊資料，廣泛應用於視覺偵測或監控系統等應用。 It is known to use Electro-optical Sensors to capture optical data of the surrounding environment, and to record and store it as image or video data, which is widely used in applications such as visual detection or surveillance systems.

雖然數位影像處理與計算機視覺技術發展多年，並已成功開發出許多影像處理技術及其裝置，然而，目前鮮少係針對水上影像或視訊資料之濾波(Filter)之技術，因為在出現水面之環境中，水之特殊物理特性所產生動態背景(Dynamic Background)、波浪(Waves)、尾跡(Wakes)、泡沫(Foam)與反光(Glints)等多樣化現象，因此，習知的影像處理技術難以建立模型來降低其現象對於影像或視訊資料之影響，導致物件在水上影像或視訊資料中過濾、偵測效果並無法令人滿意，也進一步直接降低後續視覺偵測或監控系統之如識別、追蹤、行為分析與應用的結果。 Although digital image processing and computer vision technology have been developed for many years, and many image processing technologies and devices have been successfully developed, however, currently there are few technologies for filtering (Filter) of water images or video data, because in the environment where the water surface appears Among them, the special physical properties of water produce diverse phenomena such as Dynamic Background, Waves, Wakes, Foam, and Glints. Therefore, it is difficult to establish conventional image processing techniques. The model is used to reduce the effect of its phenomenon on the image or video data, resulting in unsatisfactory filtering and detection effects of objects in the water image or video data, and further directly reducing the subsequent visual detection or monitoring system such as recognition, tracking, Results of behavior analysis and application.

雖然數位影像處理與計算機視覺技術發展多年，並已成功開發出許多影像處理技術及其裝置，但運用該習知的影像或視覺處理技術對於海上過濾效果，其準確性仍有待加強。 Although digital image processing and computer vision technologies have been developed for many years, and many image processing technologies and devices have been successfully developed, the accuracy of the maritime filtering effect using the conventional image or vision processing technology still needs to be enhanced.

因此，習知的影像雜訊濾波處理技術對於海上過濾效果並無 法令人滿意。 Therefore, the conventional image noise filtering processing technology has no effect on maritime filtering. The law is satisfactory.

有鑑於此，本發明特別是用於水上之影像雜訊濾波方法及其影像處理裝置，用以過濾有水環境對於光學感測器所擷取之影像或視訊資料，改善影像或視訊資料所造成的不清晰現象，進一步降低視覺系統之錯誤偵測結果，為其主要目的。 In view of this, the present invention is particularly used for a water image noise filtering method and an image processing device, which is used to filter the image or video data captured by the optical sensor in the water environment, and improve the image or video data caused by The main purpose is to further reduce the false detection results of the vision system.

為達上揭目的，本發明之一種水上特性之影像雜訊濾波方法，由一處理單元來實施，其至少包含以下步驟：(A)將原始輸入影像進行暗通道空間域估測，而獲得影像之暗通道二維矩陣；(B)將原始輸入影像進行光度空間域估測，而獲得影像之光度二維矩陣；(C)將暗通道二維矩陣與光度二維矩陣進行卷積(Convolution)運算，計算水上物件可信度估計值，並獲得可信度二維矩陣圖。 In order to achieve the above-mentioned purpose, an image noise filtering method with water characteristics of the present invention is implemented by a processing unit, which at least includes the following steps: (A) Perform dark channel spatial domain estimation on the original input image to obtain the image The dark channel two-dimensional matrix; (B) the original input image is estimated in the photometric space domain to obtain the image's two-dimensional photometric matrix; (C) the dark channel two-dimensional matrix and the photometric two-dimensional matrix are convolved (Convolution) Calculate the credibility estimates of the objects on the water, and obtain a two-dimensional matrix of credibility.

為達上揭目的，本發明之一種水上特性之影像雜訊濾波影像處理裝置，至少包含一影像處理裝置100，及一光電感測器200。 To achieve the above-mentioned purpose, a water-based image noise filtering image processing device of the present invention at least includes an image processing device 100 and a photoelectric sensor 200.

100‧‧‧影像前處理模組 100‧‧‧Image pre-processing module

110‧‧‧水上影像雜訊濾波處理模組 110‧‧‧Water image noise filter processing module

120‧‧‧影像後處理模組 120‧‧‧Image post-processing module

130‧‧‧二值化運算模組 130‧‧‧Binary operation module

140‧‧‧物件偵測、識別與追蹤模組 140‧‧‧Object detection, identification and tracking module

200‧‧‧光電感測器 200‧‧‧Photoelectric Sensor

111~113‧‧‧水上影像雜訊濾波處理之步驟 111~113‧‧‧Steps of noise filtering processing on water images

圖1是一方塊圖，示例地繪示一用來實施本發明水上影像雜訊濾波方法之一實施例的影像處理裝置；圖2是一流程圖，說明本發明的實施例；圖3係為本發明於視覺監控系統之使用狀態圖；圖4係為本發明於影像之使用狀態圖；圖5係為本發明於水面影像與視訊資料之使用圖 Fig. 1 is a block diagram illustrating an example of an image processing device used to implement an embodiment of the method for filtering water image noise according to the present invention; Fig. 2 is a flowchart illustrating an embodiment of the present invention; Fig. 3 is The use state diagram of the present invention in the visual monitoring system; Figure 4 is the use state diagram of the present invention in the image; Figure 5 is the use diagram of the present invention in the water surface image and video data

為利 貴審查員了解本創作之技術特徵、內容與優點及其所能達成之功效，茲將本創作配合附圖，並以實施例之表達形式詳細說明如下，參閱圖1，說明用來實施本發明水上環境之一實施例的一影像處理裝置100，及一光電感測器200。 In order to help your examiners understand the technical features, content and advantages of this creation and the effects that can be achieved, this creation is combined with the drawings and described in detail in the form of embodiments as follows. Refer to Figure 1 to illustrate the implementation of this creation. An image processing device 100 and a photoelectric sensor 200 are invented as an embodiment of the marine environment.

參閱圖2水上影像雜訊濾波步驟：首先，一種水上特性之影像雜訊濾波方法，由一處理單元來實施，其至少包含以下步驟：(A)將原始輸入影像進行暗通道空間域估測，而獲得影像之暗通道二維矩陣；(B)將原始輸入影像進行光度空間域估測，而獲得影像之光度二維矩陣；(C)將暗通道二維矩陣與光度二維矩陣進行卷積(Convolution)運算，計算水上物件可信度估計值，並獲得可信度二維矩陣圖。 Refer to Figure 2 for filtering steps of water image noise: First, a method for filtering water image noise with characteristics of water is implemented by a processing unit, which at least includes the following steps: (A) Perform dark channel spatial domain estimation on the original input image, The dark channel two-dimensional matrix of the image is obtained; (B) the original input image is estimated in the photometric space domain to obtain the photometric two-dimensional matrix of the image; (C) the dark channel two-dimensional matrix and the photometric two-dimensional matrix are convolved (Convolution) operation to calculate the estimated value of the credibility of the object on the water, and obtain a two-dimensional matrix of the credibility.

請參照圖2，在步驟111中，依據下式計算暗通道空間域估計：

其中，D為暗通道二維矩陣、

為以像素點位置

為中心、大小為n_d×n_d之鄰近點、σ_d為正值參數、η( )函數計算如下式：

其中，I^C為光電感測器所獲得的原始輸入影像(亦即輸入影像SIMG)、R、G、B分別表示紅、綠、藍三種色彩空間。 Please refer to Figure 2. In step 111, the dark channel spatial domain estimate is calculated according to the following formula:

Among them, D is a two-dimensional matrix of dark channels,

In pixels

Is the center, the adjacent point _{with size n d} ×n _{d , σ d} is a positive parameter, and the η() function is calculated as follows:

Wherein, I ^C is the original input image obtained by the photoelectric sensor (i.e., input image SIMG), R, G, B represent the red, green, blue color space.

其次，在步驟112中，依據下式計算光度空間域估計：

其中，R為光度二維矩陣、

為以像素點位置

為中心、大小為n_r×n_r之鄰近點、σ_r為一正值參數、δ( )函數計算如下式：

其中，

為以像素點位置

為中心、大小為n_r×n_r之鄰近點。 Secondly, in step 112, the photometric spatial domain estimate is calculated according to the following formula:

Among them, R is the two-dimensional photometric matrix,

In pixels

Is the center, the adjacent point _{with size n r} ×n _{r , σ r} is a positive parameter, and the δ() function is calculated as follows:

among them,

In pixels

It is the adjacent point with the center and the size of n _r ×n _r.

接下來，在步驟113中，依據下式計算可信度估計：

Next, in step 113, the reliability estimate is calculated according to the following formula:

其中，K為一正規化常數、0[

]

[0,1]，獲得可信度二維矩陣圖O，該可信度二維矩陣圖用以達到水面雜訊去除功效之示意圖如圖5所示。 Among them, K is a normalization constant, 0[

]

[0,1], obtain a credibility two-dimensional matrix map O, and a schematic diagram of the credibility two-dimensional matrix map for achieving the effect of removing water surface noise is shown in FIG. 5.

進一步，如第3圖所示，本發明應用於視覺監控系統使用圖，由使用者針對該水上影像雜訊濾波方法所計算之可信度二維矩陣圖，設定一門檻值參數 k *，或利用大津演算法(Otsu’s Algorithm)透過最大化類別間變異量(Maximizing between-class Variances)，來自動計算出全域最佳之二值化門檻值(Thresholding Value) k *如下式：

Further, as shown in Fig. 3, the present invention is applied to the use diagram of the visual monitoring system, and the user sets a threshold parameter k * for the two-dimensional matrix diagram of the reliability calculated by the water image noise filtering method, or Using Otsu's Algorithm to automatically calculate the global best binarization threshold k * by maximizing between-class Variances (Maximizing between-class Variances), k * is as follows:

接下來，根據參數 k *與可信度二維矩陣圖，進行二值化運算如下式所示：

其中，σ²表示類別間之變異量(Variance)經由門檻值參數與二值化運算可獲得去除雜訊之二值化影像。 Next, according to the parameter k * and the two-dimensional matrix of credibility, the binarization operation is performed as shown in the following formula:

Among them, σ ² represents the variation between categories (Variance) through threshold parameters and binarization operations to obtain a binarized image with noise removed.

進一步，如第4圖所示，本發明應用於影像中，過濾水面現象干擾與雜訊之使用圖，達到水面濾波功效之示意圖如圖5所示。 Furthermore, as shown in FIG. 4, the application diagram of the present invention applied to the image to filter the interference and noise of the water surface phenomenon, and the schematic diagram of achieving the water surface filtering effect is shown in FIG. 5.

而關於實施例中的計算式(1)~(8)，除可以透過電路進行運算來實現外，也可以依據各式(1)~(8)中，各參數間的關係來透過建立查找表的方式來完成。 Regarding the calculation formulas (1)~(8) in the embodiment, in addition to the calculation through the circuit, it can also be based on the relationship between the various parameters in the formulas (1)~(8) to establish a look-up table Way to complete.

如上述的說明，本發明之一種水上特性之影像雜訊濾波影像處理裝置，至少包含一影像處理裝置100，及一光電感測器200，第1圖、第2圖、第3圖、第4圖中的這些模組與步驟可以由硬體(例如，電路)或者具有軟體的硬體(例如，具有程式的處理器)來實現，該圖示中的這些模組也可以被組合成更少模組或者被分成更多的模組。 As described above, an image noise filtering image processing device with water characteristics of the present invention includes at least an image processing device 100 and a photoelectric sensor 200, as shown in Figure 1, Figure 2, Figure 3, and Figure 4. The modules and steps in the figure can be implemented by hardware (for example, circuits) or hardware with software (for example, processors with programs), and the modules in the figure can also be combined into fewer Modules may be divided into more modules.

綜上所述，本發明提供影像與視訊之水面現象過濾動作，來使輸入之原始影像可以更為優質化，並藉此於輸出影像產生更為清晰的之水上物件。另外，本發明實施例並提出暗通道空間域估測與光度空間域估測根據影像中光影條件，以動態的調整濾波強度，如此一來，對應影像處理裝置所處的環境的變化，其濾波動作可以更具有適應性，提供優質的輸出影像。 In summary, the present invention provides water surface phenomenon filtering actions for images and videos, so that the input original image can be more high-quality, and thereby produce clearer water objects in the output image. In addition, the embodiment of the present invention also proposes dark channel spatial domain estimation and photometric spatial domain estimation to dynamically adjust the filter intensity according to the light and shadow conditions in the image. In this way, the filter is filtered in response to changes in the environment in which the image processing device is located. Actions can be more adaptable and provide high-quality output images.

雖然本發明已以實施例揭露如上，然其並非用以限定本發明，任何所屬技術領域中具有通常知識者，在不脫離本發明的精神和範圍內，當可作些許的更動與潤飾，故本發明的保護範圍當視後附的申請專利範圍所界定者為準。 Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

本發明的其他的特徵及功效，將於參照圖式的實施方式中清楚地呈現，其中：圖1是一方塊圖，示例地，示例地繪示一用來實施本發明 水上影像雜訊濾波方法之一實施例的影像處理裝置；圖2是一流程圖，說明本發明的實施例；圖3係為本發明於水上視訊監控系統之使用狀態圖；圖4係為本發明於水上影像之使用狀態圖；圖5係為本發明於水面影像與視訊資料之使用圖。 Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, in which: FIG. 1 is a block diagram, exemplarily showing a method for implementing the present invention Figure 2 is a flowchart illustrating an embodiment of the present invention; Figure 3 is a diagram showing the state of use of the present invention in a maritime video monitoring system; Figure 4 is a diagram showing the state of use of the present invention in an on-water video monitoring system Figure 5 shows the usage diagram of the invention on the water surface image and video data.

綜上所述，本發明提供一較佳可行之海上影像處理裝置，爰依法提呈發明專利之申請；本發明之技術內容及技術特點巳揭示如上，然而熟悉本項技術之人士仍可能基於本發明之揭示而作各種不背離本案創作精神之替換及修飾。因此，本創作之保護範圍應不限於實施例所揭示者，而應包括各種不背離本發明之替換及修飾，並為以下之申請專利範圍所涵蓋。 In summary, the present invention provides a better and feasible marine image processing device, and an application for a patent for invention is filed in accordance with the law; the technical content and technical features of the present invention are disclosed above, but those familiar with this technology may still be based on this The disclosure of the invention makes various substitutions and modifications that do not depart from the creative spirit of the case. Therefore, the protection scope of this creation should not be limited to those disclosed in the embodiments, but should include various replacements and modifications that do not deviate from the present invention, and are covered by the following patent applications.

【本案指定代表圖】：第(1)圖。 【Designated Representative Picture of this Case】: Picture (1).

【本代表圖之符號簡單說明】： [A brief description of the symbols of this representative picture]:

100‧‧‧影像前處理模組 100‧‧‧Image pre-processing module

110‧‧‧水上影像雜訊濾波處理模組 110‧‧‧Water image noise filter processing module

120‧‧‧影像後處理模組 120‧‧‧Image post-processing module

200‧‧‧光電感測器 200‧‧‧Photoelectric Sensor

【本案指定代表圖】：第(2)圖。 【Designated Representative Picture of this Case】: Picture (2).

【本代表圖之符號簡單說明】： [A brief description of the symbols of this representative picture]:

111~113‧‧‧水上影像雜訊濾波處理之步驟 111~113‧‧‧Steps of noise filtering processing on water images

Claims (9)

一種水上特性之影像雜訊濾波方法，由一處理單元來實施，包含以下步驟：(A)將原始輸入影像進行暗通道空間域估測，而獲得影像之暗通道二維矩陣；(B)將原始輸入影像進行光度空間域估測，而獲得影像之光度二維矩陣；(C)運用可信度估計，對於暗通道二維矩陣與光度二維矩陣進行計算，並獲得可信度二維矩陣圖；及(D)全域二值化，對可信度二維矩陣圖計算類間最大變異量，以獲得輸出之二值化影像。 An image noise filtering method with water characteristics, implemented by a processing unit, including the following steps: (A) Perform dark channel spatial domain estimation on the original input image to obtain a two-dimensional dark channel matrix of the image; (B) The original input image is estimated in the photometric space domain to obtain the two-dimensional photometric matrix of the image; (C) Using the credibility estimation, the two-dimensional matrix of the dark channel and the two-dimensional photometric matrix are calculated, and the two-dimensional matrix of credibility is obtained Figure; and (D) global binarization, calculate the maximum variation between classes on the reliability two-dimensional matrix graph to obtain the output binarized image. 如請求項1所述的水上特性之影像雜訊濾波方法，其中，在步驟(A)中，暗通道空間域估測值為該輸入影像的像素與其鄰近區塊(Patch)內空間域濾波運算值，以獲得暗通道二維矩陣。 The method for filtering image noise with maritime characteristics according to claim 1, wherein, in step (A), the dark channel spatial domain estimation value is a spatial domain filtering operation in the pixels of the input image and its neighboring patches (Patch) Value to obtain a two-dimensional matrix of dark channels. 如請求項1所述的水上特性之影像雜訊濾波方法，其中，在步驟(B)中，該光度空間域估測值為該輸入影像的像素與其鄰近區塊(Patch)內空間域濾波運算值，以獲得光度二維矩陣。 The method for filtering image noise with water characteristics according to claim 1, wherein, in step (B), the luminosity spatial domain estimation value is a spatial domain filtering operation in the pixels of the input image and its neighboring patches (Patch) Value to obtain a two-dimensional matrix of luminosity. 如請求項1所述的水上特性之影像雜訊濾波方法，其中，在步驟(C)中，可信度估計值為該暗通道二維矩陣與該光度二維矩陣的元素(Elements)進行卷積(Convolution)運算之結果，以獲得可信度二維矩陣圖。 The method for filtering image noise with water characteristics according to claim 1, wherein, in step (C), the credibility estimated value is the dark channel two-dimensional matrix and the elements of the luminosity two-dimensional matrix. Calculate the result of the Convolution operation to obtain a two-dimensional matrix of credibility. 如請求項1所述的水上特性之影像雜訊濾波方法，其中，在步驟(D)中，全域二值化針對該一可信度二維矩陣圖計算出類間最大變異量，作為全域最佳化之二值化參數，進一步，用該二值化參數對可信度二維矩陣圖進行二值化運算，而獲得輸出之二值化影像。 The method for filtering image noise with maritime characteristics according to claim 1, wherein, in step (D), global binarization calculates the maximum inter-class variation for the two-dimensional matrix graph of reliability, which is regarded as the global maximum To optimize the binarization parameter, further, use the binarization parameter to perform a binarization operation on the reliability two-dimensional matrix graph to obtain the output binarized image. 如請求項1所述的水上特性之影像雜訊濾波方法，其中在輸入影像在水面像素點位置之強度值與該暗通道二維矩陣值成反比，並取該些數值中的 最大者來產生該暗通道二維矩陣值。 The image noise filtering method for water characteristics according to claim 1, wherein the intensity value of the pixel position of the input image on the water surface is inversely proportional to the two-dimensional matrix value of the dark channel, and one of these values is taken The largest one is used to generate the two-dimensional matrix value of the dark channel. 如請求項1所述的水上特性之影像雜訊濾波方法，其中在輸入影像在水面像素點位置之強度值與該光度二維矩陣值成正比，並取該些數值中的最大者與最小者之差值來產生該光度二維矩陣值。 The image noise filtering method for water characteristics according to claim 1, wherein the intensity value of the pixel point on the water surface of the input image is proportional to the value of the luminosity two-dimensional matrix, and the maximum and minimum of these values are taken The difference value is used to generate the two-dimensional matrix value of the luminosity. 如申請專利範圍第1項所述的水上影像濾波方法，其中該可信度估計計算各該暗通道平均動態範圍寬度與該光度動態範圍寬度的卷積運算值，以獲得影像中像素點位置可信度值，該值與水面影像中屬於非水物件之機率成正比。 For example, the water image filtering method described in item 1 of the scope of patent application, wherein the credibility estimation calculates the convolution operation value of the average dynamic range width of each dark channel and the photometric dynamic range width to obtain the position of the pixel in the image. Reliability value, which is proportional to the probability of non-aqueous objects in the water surface image. 一種水上特性之影像雜訊濾波影像處理裝置，至少包含：如申請專利範圍第1項所述的一種水上特性之影像雜訊濾波方法及一影像處理裝置100，及一光電感測器200，方法中的模組與步驟可以由硬體(例如，電路)或者具有軟體的硬體(例如，具有程式的處理器)來實現，該模組與步驟也可以被組合成更少模組或者被分成更多的模組 An image processing device for filtering image noise with water characteristics, at least including: a method for filtering image noise with water characteristics as described in item 1 of the patent application, an image processing device 100, and a photoelectric sensor 200. The modules and steps in can be implemented by hardware (for example, circuits) or hardware with software (for example, processors with programs). The modules and steps can also be combined into fewer modules or divided into More modules

Images