TWI673685B - Electronic image defogging system and method thereof - Google Patents
Electronic image defogging system and method thereof Download PDFInfo
- Publication number
- TWI673685B TWI673685B TW107120602A TW107120602A TWI673685B TW I673685 B TWI673685 B TW I673685B TW 107120602 A TW107120602 A TW 107120602A TW 107120602 A TW107120602 A TW 107120602A TW I673685 B TWI673685 B TW I673685B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- value
- pixels
- air light
- generate
- Prior art date
Links
Landscapes
- Image Processing (AREA)
Abstract
本發明係關於一種電子影像除霧系統，包括：用以對至少一影像進行非連續的數值抽樣而產生一空氣光值的一空氣光估算模組；可不倚賴或等待空氣光值而同時動作且用以對該至少一影像進行數值分析而產生一透射率修正值的一透射率修正模組；以及用以接收至少一影像、空氣光值以及透射率修正值以根據空氣光值以及透射率修正值修正至少一影像而輸出至少一除霧影像的一影像還原模組。由於空氣光估算模組與透射率修正模組可同時分別運作，使得本發明可增強影像除霧效率且降低影像除霧硬體與功耗成本。本發明更提出一種電子影像除霧方法。The present invention relates to an electronic image defogging system, including: an air light estimation module for discontinuous numerical sampling of at least one image to generate an air light value; the air light value can be operated simultaneously without relying on or waiting for the air light value and A transmittance correction module for numerically analyzing the at least one image to generate a transmittance correction value; and receiving at least one image, air light value, and transmittance correction value to correct according to the air light value and transmittance An image restoration module that corrects at least one image and outputs at least one defog image. Since the air light estimation module and the transmittance correction module can operate separately at the same time, the present invention can enhance the image defogging efficiency and reduce the image defogging hardware and power consumption costs. The invention further provides a method for defogging an electronic image.
Description
本發明係關於一種電子影像除霧系統及其方法，尤其是一種可高效率進行的電子影像除霧系統及其方法。The invention relates to an electronic image defogging system and a method thereof, in particular to an electronic image defogging system and a method thereof that can be performed with high efficiency.
隨著科技日新月異，網路平台的高速發展，車聯網這個概念漸漸地從理論普及到市場上，IC Insights指出「車用市場2015 年至2020 年複合成長率將達4.9%」由這句話顯示出車用IC 電子方面也逐漸成長，加上基於安全的原因，消費者對於行車器錄器加裝率逐年升高，加上目前的車輛量銷售導向一直注重提倡安全駕駛的概念與標語，車輛的安全機制主要可以分為被動式防護機制和主動式防護機制，其中被動式防護機制，如煞車系統及安全氣囊等，主要目的都是在行車事故發生時，確保將傷害降到最低的安全機制。而主動式防護機制是在行車事故尚未發生時，就能夠預先做出適當的處置行為，避免車輛發生行車事故，當然這也間接避免被動式防護機制的啟動。這技術發展需要相關的感知技術配合，例如紅外線、雷達及行車影像等，其中行車影像資料再搭配後續的先進駕駛輔助系統(Advanced-driver Assistance Systems, ADAS)，將可以完成如車道偏移警示系統（Lane Departure Warning System, LDWS）、車前碰撞警示系統（Forward Collision Warning System, FCWS）、盲點偵測警示系統（Blind Spot Detection System, BSDS）、360 度環景影像輔助系統（360 Surrounded View）及車道維持輔助系統（Lane Keeping Assistance, LKA），行車安全大幅提升，若以預防勝於治療的觀點來看，主動式防護機制扮演著車輛安全的第一道關卡，因此吸引各大車廠致力於主動式防護機制的技術發展。With the rapid development of technology and the rapid development of the Internet platform, the Internet concept has gradually spread from the market to the market. IC Insights points out that "the market for compounding will grow by 4.9% in 2015" to 2020. " The use of ICs for electronics has also gradually grown. In addition, due to safety reasons, consumers have increased their equipment installations, and the current sales guidance has always focused on the principles and slogans that promote safe driving. The safety mechanism can be divided into passive protection mechanism and active protection mechanism. Among them, the passive protection mechanism, such as brake system and airbag, is mainly used to ensure that the injury will be minimized to the safety mechanism when an accident occurs. The active protection mechanism is able to make appropriate actions in advance before the accident has occurred, so as to avoid accidents in the vehicle. Of course, this also indirectly prevents the activation of the passive protection mechanism. The development of this technology requires the cooperation of relevant sensing technologies, such as infrared, radar, and high-definition images. Among them, the high-definition imagery and advanced advanced driver assistance systems (ADAS) can be used to complete such roadway warning systems. (Lane Departure Warning System (LDWS)), Forward Collision Warning System (FCWS), Blind Spot Detection System (BSDS), 360 度 Surround View View (360 Surrounded View), and The Lane Keeping Assistance (LKA) system has greatly improved safety. From the perspective of prevention rather than governance, the active protection mechanism plays the first level of vehicle safety, so it attracts major manufacturers to take the initiative. Technical development of the protection mechanism.
由上述可知，行車影像資訊在ADAS中扮演重要的角色，因此要如何維持行車影像資訊的清晰度，對於後續ADAS而言，將是一項極為重要的技術議題。因為車輛在大部分時間都行駛於外部開放空間，所以行車影像資訊會直接受到氣候方面的影響，而導致行車影像的清晰度不夠，進而影響後續ADAS的正確性，一般行車環境下，最容易遇到的天氣狀況以下雨和霧氣較為常見，以下雨天氣來說，一般車輛的擋風玻璃上都會配置雨刷，藉由雨刷運作，再搭配擋風玻璃防水劑，就可以讓車內行車的攝像鏡頭得到較為清晰的畫面，讓後續的ADAS能夠正常判讀和動作，但如果是起霧的天氣，這對於行車安全是一大威脅，在天氣視線受到影響時，交通事故會比正常天氣狀態下增加75%，即便是車輛有ADAS防護機制，那也需要有清晰的行車影像資料，才能夠發會其正常功能。如果針對行車攝像鏡頭配置一項能夠對於行車影像的除霧技術，則所有受到起霧影響的行車畫面，都可以透過此技術將霧化的行車影像還原成清晰的行車影像，提供給後續ADAS防護機制進行判讀，對整體車輛做出相關的安全處置作為，以確保整體車輛在起霧天氣的行車安全性。在車用影像方面，當影像為低解析度時，辨識系統能夠判斷精確度的資訊相對就會不足夠，使得行車輔助系統的安全性降低；當每秒所接收的幀數(Frames Per Second, FPS)較少，影像就會不流暢，若車輛行駛於高速狀態中發生狀況時，辨識系統就會呈現非即時性判斷，讓判斷結果延遲，所以要提供辨識系統乾淨清晰及穩定的輸入源，就需要高畫質即時影像除霧，能讓後端行車輔助系統更精確地辨識、判斷並且執行正確的提升安全的輔助功能。From the above, it can be known that image information plays an important role in ADAS, so how to maintain the clarity of image information will be an extremely important technical issue for subsequent ADAS. Because most vehicles drive outside in open space, the image information will be directly affected by the climate, which will cause the image to be clear and inadequate, which will affect the accuracy of subsequent ADAS. In general, the environment is the most tolerant. In the weather conditions, rain and fog are more common. In rainy weather, generally, the windshield of a car will be equipped with a wiper. The windshield wiper works with the windshield waterproofing agent to make the camera inside the camera. Obtain a clearer picture, so that subsequent ADAS can judge and act normally, but if it is foggy weather, this is a big threat to the safety of traffic. When the weather sight is affected, traffic accidents will increase by 75 compared to normal weather. %, That is, a car has ADAS protection mechanism, it also needs to have clear image data to be able to develop its normal function. If a defogging technology is available for the 行 車 camera lens, all 行 車 images affected by fogging can be restored to a clear 行 車 image by this technology, providing subsequent ADAS protection. The mechanism makes judgments and makes relevant safety measures for the entire vehicle to ensure the overall safety of the vehicle in foggy weather. In terms of the use of images, when the image is low-resolution, the discrimination system can determine that relatively accurate information is relatively insufficient, which makes the security of the auxiliary system low; when the frames received per second (Frames Per Second, (FPS) is less, the image will be smooth, when a car is driving in a high-speed state, the discrimination system will present a non-immediate judgment, delaying the judgment result, so it is necessary to provide a clean, clear and stable input source for the discrimination system. It needs high-definition real-time image defogging, which can allow the back-end system to identify, judge, and perform the correct auxiliary functions to improve safety.
然而，隨著解析度的提昇以及2K、4K甚至8K等規格的標準化，傳統的影像除霧法勢必將會面臨硬體效能瓶頸的障礙，一般在硬體實現方面在單一像素值所需要的運算單元基本包含加法器、減法器、乘法器、除法器以及指數運算，其中，無論是針對多張影像除霧或單張影像除霧，若使用多個除法器與指數運算單元，必須採用龐大演算機制或多次疊代運算，會使得整體電路的運算速度降低，難以在硬體上實現到更好的處理結果而很可能會在既有的硬體上更增添其他成本。However, with the improvement of resolution and the standardization of 2K, 4K, and even 8K specifications, the traditional image defogging method will inevitably face the obstacles of hardware performance bottlenecks. Generally, the calculation required for a single pixel value in hardware implementation The unit basically includes an adder, a subtracter, a multiplier, a divider, and a finger operation. Among them, whether it is defogging multiple images or a single image, using multiple dividers and finger arithmetic units requires a huge calculation. The mechanism or multiple iterations will make the overall computing speed of the circuit low, it is difficult to achieve better processing results on the hardware and it is likely to add additional costs to the existing hardware.
因此，要必要提出一種高效率影像除霧的系統及方法，以解決現有技術效率不彰以及硬體與功耗成本無法降低的技術問題。Therefore, it is necessary to propose a system and method for high-efficiency image defogging, in order to solve the technical problems of inefficiency of the existing technology and the cost of hardware and power consumption cannot be reduced.
鑒於前述之習知技術的缺點，本發明之主要目的係提供一種電子影像除霧系統，其具備可高效率進行影像除霧的功能且免除了為多次疊代運算而添購硬體成本的必要性，而可以達到增強影像除霧效率且降低影像除霧硬體與功耗成本的目的。In view of the shortcomings of the aforementioned conventional technology, the main object of the present invention is to provide an electronic image defogging system, which has the function of efficiently performing image defogging and eliminates the need to purchase hardware costs for multiple iteration operations Necessary, and can achieve the purpose of enhancing image defogging efficiency and reducing image defogging hardware and power consumption costs.
為了達到前述目的以及其他目的，本發明提供了一種電子影像除霧系統，包括：一空氣光估算模組，係用以對至少一影像進行非連續的數值抽樣而產生一空氣光值；其中，該空氣光估算模組包括一RGB暗通道、一第一數值抽樣元件以及一第二數值抽樣元件；該RGB暗通道係用以對該至少一影像之多數個像素進行RGB色彩模型分析，以產生對應該多數個像素各別的R、G、B通道值，並對該多數個像數的R、G、B通道值分別進行暗通道處理，而產生多數個第一最小值；該第一數值抽樣單元係用以接收該多數個第一最小值，並對該多數個像素所形成非連續的多數個第一矩陣的每一個進行最小值抽樣而產生多數個第二最小值，且該第一數值抽樣單元同時對其中的相鄰四個第一矩陣進行一邊緣均一判斷，若該相鄰四個第一矩陣的該邊緣均一判斷的結果為非均一時，則該第一數值抽樣單元對該相鄰四個第一矩陣之間的該多數個像素所形成的一第二矩陣進行最小值抽樣而產生一第三最小值；該第二數值抽樣元件係用以依據大小排序該多數個第二最小值以及多數個該第三最小值，並擷取其中五個最大數值者以及其對應於該至少一影像上的五個像素之後，依據該五個像素於該至少一影像上的平均中央位置為中心對周圍的十個像素進行取樣並取平均而獲得該空氣光值；一透射率修正模組，用以對該至少一影像進行數值分析而產生一透射率修正值；其中，該透射率修正模組包括一分析單元、一深度轉換器、一銳化濾波器、一低通濾波器以及一修正單元；該分析單元用以將該至少一影像之多數個像素的R、G、B通道值作HSV色彩模型轉換，而獲得該至少一影像之每一該多數個像素的一飽和度、一明度；該深度轉換器擷取每一該多數個像素的該飽和度以及該明度之後輸出對應於每一該多數個像素的一深度值；該銳化濾波器擷取多數個該深度值之後，分別針對每一該多數個像素進行該邊緣均一判斷，當其中一該多數個像素被判斷為邊緣時，該銳化濾波器將其所對應的該深度值做為一第一值；當其中一該多數個像素被判斷為非邊緣時，該低通濾波器將其所對應的該深度值轉換為一第二值；該修正單元擷取該第一值以及該第二值之後輸出該透射率修正值；以及一影像還原模組，係分別連接該空氣光估算模組以及該透射率修正模組，且用以接收該至少一影像、該空氣光值以及該透射率修正值，以根據該空氣光值以及該透射率修正值修正該至少一影像，而輸出至少一除霧影像。In order to achieve the foregoing objectives and other objectives, the present invention provides an electronic image defogging system including: an air light estimation module for discontinuous numerical sampling of at least one image to generate an air light value; wherein, The air light estimation module includes an RGB dark channel, a first numerical sampling element and a second numerical sampling element. The RGB dark channel is used to perform RGB color model analysis on a plurality of pixels of the at least one image to generate Corresponding to the respective R, G, and B channel values of the plurality of pixels, and performing dark channel processing on the R, G, and B channel values of the plurality of pixels, respectively, to generate a plurality of first minimum values; the first value The sampling unit is configured to receive the plurality of first minimum values, and perform minimum sampling on each of the discontinuous plurality of first matrices formed by the plurality of pixels to generate a plurality of second minimum values, and the first The numerical sampling unit performs an edge uniformity judgment on four adjacent first matrices at the same time. If the edge uniformity judgment result of the four adjacent first matrices is non-uniform, the A numerical sampling unit performs minimum sampling on a second matrix formed by the plurality of pixels between adjacent four first matrices to generate a third minimum; the second numerical sampling element is used to After sorting the plurality of second minimum values and the plurality of third minimum values, and extracting the five maximum values and corresponding five pixels on the at least one image, the five minimum values are applied to the at least one according to the five pixels. The average central position on the image is taken as the center, and the surrounding ten pixels are sampled and averaged to obtain the air light value. A transmittance correction module is used to numerically analyze the at least one image to generate a transmittance correction value. ; Wherein the transmittance correction module includes an analysis unit, a depth converter, a sharpening filter, a low-pass filter, and a correction unit; the analysis unit is configured to convert the at least one pixel of the at least one image The R, G, and B channel values are converted by the HSV color model to obtain a saturation and a brightness of each of the plurality of pixels of the at least one image; the depth converter captures each of the plurality of pixels. After the saturation and brightness of the pixels are output, a depth value corresponding to each of the plurality of pixels is output; after the sharpening filter captures the plurality of the depth values, the edge is uniformized for each of the plurality of pixels, respectively. Judgment, when one of the plurality of pixels is judged as an edge, the sharpening filter takes the corresponding depth value as a first value; when one of the plurality of pixels is judged as non-edge, the The low-pass filter converts the corresponding depth value into a second value; the correction unit extracts the first value and the second value and outputs the transmittance correction value; and an image restoration module, respectively The air light estimation module and the transmittance correction module are connected, and are used to receive the at least one image, the air light value, and the transmittance correction value, so as to modify the at least one light source according to the air light value and the transmittance correction value. An image, and output at least one defog image.
開始使用本發明之電子影像除霧系統時，一方面該空氣光估算模組通過了該RGB暗通道、該第一數值抽樣元件以及該第二數值抽樣元件並以非連續的方式對該至少一影像進行抽樣而獲得該空氣光值，其中，該RGB暗通道對該多數個像數的R、G、B通道值分別進行暗通道處理而產生該多數個第一最小值，接著該第一數值抽樣單元以非連續像素的方式進行最小值抽樣與該邊緣均一判斷而產生了該多數個第二最小值以及該多數個第三最小值，之後再以數值大小排列並擷取其中五個最大數值者以及其對應於該至少一影像上的五個像素之後，依據該五個像素於該至少一影像上的平均中央位置為中心對周圍的十個像素進行取樣並取平均而獲得該空氣光值；另一方面，該透射率修正模組可同時並行地對該至少一影像進行數值分析而產生該透射率修正值，其中，是通過獲得該至少一影像之每一該多數個像素的該飽和度、該明度以及該深度值之後，配合對該多數個像素進行該邊緣均一判斷而輸出該透射率修正值；最後，根據該空氣光值以及該透射率修正值修正該至少一影像而輸出該至少一除霧影像。When starting to use the electronic image defogging system of the present invention, on the one hand, the air light estimation module passed the RGB dark channel, the first numerical sampling element and the second numerical sampling element and discontinuously processed the at least one The image is sampled to obtain the air light value, wherein the RGB dark channel performs dark channel processing on the R, G, and B channel values of the plurality of image numbers to generate the plurality of first minimum values, and then the first value. The sampling unit performs discontinuous pixel sampling and the edge uniformity judgment to generate the plurality of second minimum values and the plurality of third minimum values, and then arranges and extracts five of the maximum values in numerical value. And corresponding to five pixels on the at least one image, the surrounding ten pixels are sampled and averaged according to the average central position of the five pixels on the at least one image to obtain the air light value On the other hand, the transmittance correction module can simultaneously and concurrently perform numerical analysis on the at least one image to generate the transmittance correction value. After the saturation, the lightness, and the depth value of each of the plurality of pixels of the at least one image, the transmittance correction value is output in cooperation with the edge uniform determination of the plurality of pixels; finally, according to the air light value And the transmittance correction value corrects the at least one image and outputs the at least one defogging image.
即是，本發明的電子影像除霧系統根據該至少一影像中的該多數個像數的R、G、B通道值可分別且同時並行地進行處理而獲得該空氣光值以及該透射率修正值，可避免傳統除霧方法中必須採用龐大演算機制或多次疊代運算而無法提升效率且降低硬體成本的問題，因此本發明所述的電子影像除霧系統可以達到提升電子除霧效率且降低影像除霧硬體與功耗成本的目的。That is, the electronic image defogging system of the present invention can obtain the air light value and the transmittance correction according to the R, G, and B channel values of the plurality of image numbers in the at least one image, which can be processed separately and simultaneously. Value, which can avoid the problem that the traditional defogging method must use a huge calculation mechanism or multiple iterations to improve efficiency and reduce hardware costs. Therefore, the electronic image defogging system according to the present invention can achieve improved electronic defogging efficiency. And the purpose of reducing image defogging hardware and power consumption costs.
進一步而言，所述的電子影像除霧系統更包括用於行車記錄、公路監控、空拍記錄、醫療內視鏡之其中至少一者的一顯示器，其連接該影像還原模組，且用以接收並顯示該至少一除霧影像。Further, the electronic image defogging system further includes a display for at least one of driving records, highway monitoring, aerial photography records, and medical endoscopes, which is connected to the image restoration module and used for Receiving and displaying the at least one defogging image.
進一步而言，所述的電子影像除霧系統更包括用於行車記錄、公路監控、空拍記錄、醫療內視鏡之其中至少一者的至少一影像擷取單元，每一該至少一影像擷取單元係分別連接至該空氣光估算模組以及該透射率修正模組，且用以輸出該至少一影像至該空氣光估算模組以及該透射率修正模組。Further, the electronic image defogging system further includes at least one image capturing unit for at least one of driving records, highway monitoring, aerial recordings, and medical endoscopes, each of the at least one image capturing The fetching unit is respectively connected to the air light estimation module and the transmittance correction module, and is used to output the at least one image to the air light estimation module and the transmittance correction module.
進一步而言，該第一矩陣係為該至少一影像中的其中九個像素所構成的一方陣。Further, the first matrix is a square matrix composed of nine pixels in the at least one image.
進一步而言，該第二矩陣係為該至少一影像中的其中九個像素所構成的一方陣。Further, the second matrix is a square matrix composed of nine pixels in the at least one image.
鑒於前述之習知技術的缺點，本發明之另一目的係提供一種電子影像除霧方法，包括下列步驟：對至少一影像進行非連續的數值抽樣而產生一空氣光值；其中，包括下列步驟：對該至少一影像之多數個像素進行RGB色彩模型分析，以產生對應該多數個像素各別的R、G、B通道值，並對該多數個像數的R、G、B通道值分別進行暗通道處理，而產生多數個第一最小值；接收該多數個第一最小值，並對該多數個像素所形成非連續的多數個第一矩陣的每一個進行最小值抽樣而產生多數個第二最小值，且該第一數值抽樣單元同時對其中的相鄰四個第一矩陣進行一邊緣均一判斷，若該相鄰四個第一矩陣的該邊緣均一判斷的結果為非均一時，則該第一數值抽樣單元對該相鄰四個第一矩陣之間的該多數個像素所形成的一第二矩陣進行最小值抽樣而產生一第三最小值；依據大小排序該多數個第二最小值以及多數個該第三最小值，並擷取其中五個最大數值者以及其對應於該至少一影像上的五個像素之後，依據該五個像素於該至少一影像上的平均中央位置為中心對周圍的十個像素進行取樣並取平均而獲得該空氣光值；對該至少一影像進行數值分析而產生一透射率修正值；其中，包括下列步驟：將該至少一影像之多數個像素的R、G、B通道值作HSV色彩模型轉換，而獲得該至少一影像之每一該多數個像素的一飽和度、一明度；擷取每一該多數個像素的該飽和度以及該明度之後輸出對應於每一該多數個像素的一深度值；擷取多數個該深度值之後，分別針對每一該多數個像素進行該邊緣均一判斷，當其中一該多數個像素被判斷為邊緣時，該銳化濾波器將其所對應的該深度值做為一第一值；當其中一該多數個像素被判斷為非邊緣時，將其所對應的該深度值轉換為一第二值；擷取該第一值以及該第二值之後輸出該透射率修正值；以及接收該至少一影像、該空氣光值以及該透射率修正值，以根據該空氣光值以及該透射率修正值修正該至少一影像，而輸出至少一除霧影像。In view of the shortcomings of the foregoing conventional technology, another object of the present invention is to provide an electronic image defogging method, which includes the following steps: discontinuous numerical sampling of at least one image to generate an air light value; including the following steps: : Perform RGB color model analysis on a plurality of pixels of the at least one image to generate R, G, and B channel values corresponding to the plurality of pixels, and respectively set the R, G, and B channel values of the plurality of pixels. Dark channel processing is performed to generate a plurality of first minimum values; the plurality of first minimum values are received, and each of a plurality of discontinuous first matrices formed by the plurality of pixels is sampled to generate a plurality of first minimum values; The second minimum value, and the first numerical sampling unit performs an edge uniformity judgment on the four adjacent first matrices at the same time; if the edge uniformity judgment result of the four adjacent first matrices is non-uniform, Then, the first numerical sampling unit performs minimum sampling on a second matrix formed by the plurality of pixels between adjacent four first matrices to generate a third minimum; based on After sorting the plurality of second minimum values and the plurality of third minimum values, and extracting five of the maximum values and corresponding to five pixels on the at least one image, the five minimum values are applied to the at least one An average central position on an image is taken as a center, and the surrounding ten pixels are sampled and averaged to obtain the air light value. The at least one image is numerically analyzed to generate a transmittance correction value. The method includes the following steps: The R, G, and B channel values of the plurality of pixels of the at least one image are converted by the HSV color model to obtain a saturation and a brightness of each of the plurality of pixels of the at least one image; capturing each of the plurality After the saturation of the pixel and the brightness, a depth value corresponding to each of the plurality of pixels is output; after capturing the plurality of the depth values, the edge uniformity judgment is performed for each of the plurality of pixels, and when one of the When a plurality of pixels are judged as edges, the sharpening filter takes the corresponding depth value as a first value; when one of the plurality of pixels is judged as non-edges Converting the corresponding depth value into a second value; outputting the transmittance correction value after capturing the first value and the second value; and receiving the at least one image, the air light value, and the transmittance correction Value to correct the at least one image according to the air light value and the transmittance correction value, and output at least one defogging image.
進一步而言，所述的電子影像除霧方法更包括一步驟：用於行車記錄、公路監控、空拍記錄、醫療內視鏡之其中至少一者的一顯示器擷取並顯示該至少一除霧影像。Further, the electronic image defogging method further includes a step of: capturing and displaying the at least one defogging by a display for at least one of driving records, highway monitoring, aerial photography records, and medical endoscopes. image.
進一步而言，所述的電子影像除霧方法更包括一步驟：用於行車記錄、公路監控、空拍記錄、醫療內視鏡之其中至少一者的至少一影像擷取單元輸出該至少一影像。Further, the electronic image defogging method further includes a step: at least one image capturing unit for at least one of driving records, highway monitoring, aerial recording, and medical endoscopes to output the at least one image .
進一步而言，該第一矩陣係為該至少一影像中的其中九個像素所構成的一方陣。Further, the first matrix is a square matrix composed of nine pixels in the at least one image.
進一步而言，該第二矩陣係為該至少一影像中的其中九個像素所構成的一方陣。Further, the second matrix is a square matrix composed of nine pixels in the at least one image.
以下係藉由特定的具體實施例說明本發明之實施方式，熟悉此技術之人士可由本說明書所揭示之內容輕易地瞭解本發明之其他優點及功效。本發明亦可藉由其他不同的具體實例加以施行或應用，本發明說明書中的各項細節亦可基於不同觀點與應用在不悖離本發明之精神下進行各種修飾與變更。The following is a description of specific embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and various details in the description of the present invention can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention.
須知，本說明書所附圖式繪示之結構、比例、大小、元件數量等，均僅用以配合說明書所揭示之內容，以供熟悉此技術之人士瞭解與閱讀，並非用以限定本發明可實施之限定條件，故不具技術上之實質意義，任何結構之修飾、比例關係之改變或大小之調整，在不影響本發明所能產生之功效及所能達成之目的下，均應落在本發明所揭示之技術內容得能涵蓋之範圍內。It should be noted that the structures, proportions, sizes, component numbers, etc. shown in the drawings of this specification are only used to match the content disclosed in the description for those familiar with this technology to understand and read, not to limit the invention. The implementation of the conditions, so it does not have technical significance, any modification of the structure, the change of the proportional relationship or the adjustment of the size, without affecting the efficacy and the purpose that can be achieved by the present invention, should fall into this The technical content disclosed by the invention can be covered.
請參閱圖1至圖4所示，其中，圖1為本發明之一實施例的電子影像除霧系統的架構示意圖；圖2為本發明之該實施例的電子影像除霧系統之空氣光估算模組的架構示意圖；圖3為本發明之該實施例的電子影像除霧系統之空氣光估算模組的操作示意圖；圖4為本發明之該實施例的電子影像除霧系統之透射率修正模組的架構示意圖。Please refer to FIG. 1 to FIG. 4, wherein FIG. 1 is a schematic structural diagram of an electronic image defogging system according to an embodiment of the present invention; and FIG. 2 is an air light estimation of the electronic image defogging system according to this embodiment of the present invention Schematic diagram of the module; Figure 3 is a schematic diagram of the operation of the air light estimation module of the electronic image defogging system of the embodiment of the present invention; Figure 4 is the transmittance correction of the electronic image defogging system of the embodiment of the present invention Schematic diagram of the module.
以下依據本發明之第一實施例，描述電子影像除霧系統1，包括：一空氣光估算模組10、一透射率修正模組20以及一影像還原模組30。The following describes an electronic image defogging system 1 according to a first embodiment of the present invention, which includes: an air light estimation module 10, a transmittance correction module 20, and an image restoration module 30.
其中，空氣光估算模組10係用以對影像100進行非連續的數值抽樣而產生空氣光值200。其中空氣光估算模組10包括一RGB暗通道11、一第一數值抽樣元件12以及一第二數值抽樣元件13。The air light estimation module 10 is configured to perform discontinuous numerical sampling on the image 100 to generate an air light value 200. The air light estimation module 10 includes an RGB dark channel 11, a first numerical sampling element 12 and a second numerical sampling element 13.
RGB暗通道11係用以對影像100之多數個像素(圖中未示)進行RGB色彩模型分析，以產生對應多數個像素各別的R、G、B通道值(如圖3所示的R、G、B)，並對多數個像數的R、G、B通道值分別進行暗通道處理，而產生多數個第一最小值110。在本發明的所述實施例中，暗通道處理屬於一種簡單且有效的影像先驗方式，主要是針對單一影像觀察，於一般彩色影像中，在大部分非天空區域部分，RGB 通道中至少有一個顏色通道的像素值是較低的(如灰階/明度0~255)，若要符合R、G、B三通道取最小值的最後其中一通道值會被留下，其餘因為各通道值皆存在零所以會消失，此方法也可以用下述(式1)表示： (式1) 其中， 為影像，即原始彩色霧圖；Ω(x)為以像素x 為中心點的範圍； 表示在Ω(x)範圍內的最小值，稱之為暗通道值。 The RGB dark channel 11 is used to perform RGB color model analysis on a plurality of pixels (not shown) of the image 100 to generate R, G, and B channel values corresponding to the plurality of pixels (as shown in R in FIG. 3). , G, B), and perform dark channel processing on the R, G, and B channel values of the plurality of image numbers, respectively, to generate a plurality of first minimum values 110. In the embodiment of the present invention, dark channel processing belongs to a simple and effective image prior method, which is mainly for single image observation. In general color images, in most non-sky region parts, there are at least RGB channels. The pixel value of a color channel is low (such as grayscale / brightness 0 ~ 255). It must meet the minimum value of the three channels of R, G, and B. The last one of the channel values will be captured. The rest is due to the value of each channel. All of them exist, so they disappear. This method can also be expressed by the following (Equation 1): (Equation 1) where Is the image, that is, the original color fog map; Ω (x) is the range with the pixel x as the center point; The minimum value in the range of Ω (x) is called the dark channel value.
第一數值抽樣單元12係用以接收多數個第一最小值110，並對多數個像素所形成非連續的多數個第一矩陣(如圖3中的A以及B)的每一個進行最小值抽樣而產生多數個第二最小值121，且第一數值抽樣單元12同時對其中的相鄰四個第一矩陣(可包括A或B或A及B)進行一邊緣均一判斷，若相鄰四個第一矩陣的邊緣均一判斷的結果為非均一時(如圖3所示，即是包括A或B或A及B所構成的四個第一矩陣中，若四個的一矩陣非均為A或非均為B的狀況下時)，則第一數值抽樣單元12對相鄰四個第一矩陣之間的多數個像素所形成的一第二矩陣C進行最小值抽樣而產生一第三最小值122。在本發明之所述實施例中，每一個第一矩陣以及每一個第二矩陣個別為影像100中的其中九個像素(3x3)所構成的一方陣。The first numerical sampling unit 12 is configured to receive a plurality of first minimum values 110 and perform a minimum value sampling on each of a plurality of discontinuous first matrices (such as A and B in FIG. 3). A plurality of second minimum values 121 are generated, and the first numerical sampling unit 12 performs a uniform edge judgment on four adjacent first matrices (which may include A or B or A and B) at the same time. When the result of the edge uniformity judgment of the first matrix is non-uniform (as shown in FIG. 3, that is, among the four first matrices including A or B or A and B, if four of the first matrices are not all A (When not all are B), the first numerical sampling unit 12 performs minimum sampling on a second matrix C formed by a plurality of pixels between four adjacent first matrices to generate a third minimum Value 122. In the embodiment of the present invention, each of the first matrix and each of the second matrix is a square matrix composed of nine pixels (3 × 3) in the image 100.
第二數值抽樣元件13係用以依據大小排序多數個第二最小值121以及多數個第三最小值122，並擷取其中五個最大數值者以及其對應於影像100上的五個像素之後，依據所述五個像素於影像100上的平均中央位置為中心對周圍的十個像素進行取樣並取平均而獲得空氣光值200。The second numerical sampling element 13 is used for sorting the plurality of second minimum values 121 and the plurality of third minimum values 122 according to the size, and after extracting the five maximum values and their corresponding five pixels on the image 100, The surrounding ten pixels are sampled and averaged according to the average central position of the five pixels on the image 100 to obtain an air light value of 200.
透射率修正模組20用以對影像100進行數值分析而產生透射率修正值300。其中透射率修正模組20包括一分析單元21、一深度轉換器22、一銳化濾波器23、一低通濾波器24以及一修正單元25；分析單元21用以將影像100之多數個像素(圖中未示)的R、G、B通道值作HSV色彩模型轉換，而獲得影像100之多數個像素的飽和度211、明度212；深度轉換器22擷取多數個像素的飽和度211以及明度212之後輸出對應於每一個像素的一深度值221；銳化濾波器23擷取多數個深度值221之後，分別針對每一個像素進行邊緣均一判斷，當其中一個像素被判斷為邊緣時，銳化濾波器23將其所對應的深度值221做為一第一值231；當其中一個像素被判斷為非邊緣時，低通濾波器24將其所對應的深度值221轉換為一第二值241；修正單元25擷取第一值231以及第二值241之後輸出透射率修正值300。在本發明所述的電子影像除霧系統1中，透射率估算是不需要等待空氣光值200來進行之後動作，在實際的數值運算上是利用色彩差距的模型，定義出每個像素色彩之間的變化與景物深度的關係式，先將RGB色彩模型經由下述(式2)轉化成HSV色彩模型： (式2) 前述h、s、v分別代表HSV色彩模型的色相、飽和度211、明度212。 The transmittance correction module 20 is configured to perform a numerical analysis on the image 100 to generate a transmittance correction value 300. The transmittance correction module 20 includes an analysis unit 21, a depth converter 22, a sharpening filter 23, a low-pass filter 24, and a correction unit 25. The analysis unit 21 is used to convert most pixels of the image 100. (Not shown) The R, G, and B channel values are converted by the HSV color model to obtain the saturation 211 and brightness 212 of the majority pixels of the image 100; the depth converter 22 captures the saturation 211 of the majority pixels and After the lightness 212, a depth value 221 corresponding to each pixel is output; after the sharpening filter 23 captures a plurality of depth values 221, the edge is judged uniformly for each pixel. When one of the pixels is judged as an edge, the sharpness The filter 23 uses the corresponding depth value 221 as a first value 231; when one of the pixels is judged as a non-edge, the low-pass filter 24 converts the corresponding depth value 221 into a second value 241; The correction unit 25 extracts the first value 231 and the second value 241 and outputs a transmittance correction value 300. In the electronic image defogging system 1 of the present invention, the transmittance estimation does not need to wait for the air light value of 200 to perform subsequent operations. In actual numerical calculation, a model of color difference is used to define the color of each pixel. The relationship between the change between the scene and the depth of the scene is first converted to the HSV color model through the following (Equation 2): (Eq. 2) The foregoing h, s, and v respectively represent hue, saturation 211, and lightness 212 of the HSV color model.
接著藉由將飽和度211與明度212代入(式3)得出像素的深度值221，再經由本發明所研究開發的修正透射率模型，其為使用銳化濾波器23作為遮罩來計算出當下像素是否為邊緣值，若該像素所計算出來的值大於邊界閥值則視為邊緣像素，並保留原像素的數值(即為第一值231)，反之，則會使用低通濾波器24來做影像平滑化(即為第二值241)。 (式3) 其中d(x)代表深度值221。 Then, by substituting the saturated chirp 211 and the bright chirp 212 into (Equation 3), the depth value 221 of the pixel is obtained, and then the modified transmission chirp model researched and developed by the present invention is calculated by using the sharpening chirp 23 as a mask. Whether the current pixel is an edge value. If the value of 來 calculated by this pixel is greater than the boundary threshold, it is regarded as an edge pixel, and the original pixel value (that is, the first value 231) is preserved. Otherwise, a low-pass wave filter 24 is used. Do image flattening (that is, the second value 241). (Equation 3) where d (x) represents a depth value 221.
影像還原模組30係分別連接空氣光估算模組10以及透射率修正模組20，且用以接收影像100、空氣光值200以及透射率修正值300，以根據空氣光值200以及透射率修正值300修正影像100，而輸出至少一除霧影像400。The image restoration module 30 is respectively connected to the air light estimation module 10 and the transmittance correction module 20, and is used to receive the image 100, the air light value 200, and the transmittance correction value 300, so as to correct the air light value 200 and the transmittance. A value of 300 modifies the image 100 and outputs at least one defogging image 400.
在本發明之所述實施例中的電子影像除霧系統1更包括用於行車記錄、公路監控、空拍記錄、醫療內視鏡之其中至少一者的一顯示器(圖中未示)以及至少一影像擷取單元(圖中未示)；其中，顯示器連接影像還原模組30且用以接收並顯示至少一除霧影像400；每一影像擷取單元係分別連接至空氣光估算模組10以及透射率修正模組20，且用以輸出100影像至空氣光估算模組10以及透射率修正模組20。The electronic image defogging system 1 in the embodiment of the present invention further includes a display (not shown) for at least one of driving records, highway monitoring, aerial recording, medical endoscopes, and at least An image capture unit (not shown); wherein the display is connected to the image restoration module 30 and used to receive and display at least one defogging image 400; each image capture unit is separately connected to the air light estimation module 10 And a transmittance correction module 20 for outputting 100 images to the air light estimation module 10 and the transmittance correction module 20.
請參閱圖5及前述內容，其中圖5為本發明之所述實施例的電子影像除霧方法的流程圖。Please refer to FIG. 5 and the foregoing, where FIG. 5 is a flowchart of an electronic image defogging method according to the embodiment of the present invention.
開始使用本發明之電子影像除霧系統1時，一方面空氣光估算模組10通過了RGB暗通道11、第一數值抽樣元件12以及第二數值抽樣元件13並以非連續的方式對影像100進行抽樣而獲得空氣光值200(步驟S01)，其中，RGB暗通道11對多數個像數的R、G、B通道值分別進行暗通道處理而產生多數個第一最小值110，接著第一數值抽樣單元12以非連續像素的方式進行最小值抽樣與邊緣均一判斷而產生了多數個第二最小值121以及多數個第三最小值122，之後再以數值大小排列並擷取其中五個最大數值者以及其對應於影像100上的五個像素之後，依據五個像素於影像100上的平均中央位置為中心對周圍的十個像素進行取樣並取平均而獲得空氣光值200；另一方面，透射率修正模組20可同時並行地對影像100進行數值分析而產生透射率修正值300(步驟S02)，其中，是通過獲得影像100之每一多數個像素的飽和度211、明度212以及深度值221之後，配合對多數個像素進行邊緣均一判斷而輸出透射率修正值300；最後，根據空氣光值200以及透射率修正值300修正影像100而輸出至少一除霧影像400(步驟S03)。When the electronic image defogging system 1 of the present invention is started, on the one hand, the air light estimation module 10 passes the RGB dark channel 11, the first numerical sampling element 12, and the second numerical sampling element 13 and discontinuously processes the image 100. Sampling is performed to obtain an air light value of 200 (step S01), wherein the RGB dark channel 11 performs dark channel processing on the R, G, and B channel values of a plurality of image numbers to generate a plurality of first minimum values 110, and then a first The numerical sampling unit 12 performs minimum sampling and edge uniformity judgment in the form of discontinuous pixels to generate a plurality of second minimums 121 and a plurality of third minimums 122, and then arranges and extracts five of the maximums After the value person and its corresponding five pixels on the image 100, the surrounding ten pixels are sampled and averaged according to the average central position of the five pixels on the image 100 to obtain an air light value of 200; The transmittance correction module 20 can perform a numerical analysis on the image 100 in parallel to generate a transmittance correction value 300 (step S02), wherein each majority of the image 100 is obtained by After the pixel saturation 211, lightness 212, and depth value 221, the transmittance correction value 300 is output in accordance with the edge uniformity determination of a plurality of pixels; finally, the image 100 is corrected based on the air light value 200 and the transmittance correction value 300 to output at least A defogging image 400 (step S03).
即是，本發明的電子影像除霧系統根據至少一影像中的多數個像數的R、G、B通道值可分別且同時並行地進行處理而獲得空氣光值200以及透射率修正值300，可避免傳統除霧方法中必須採用龐大演算機制或多次疊代運算而無法提升效率且降低硬體成本的問題，因此本發明所述的電子影像除霧系統可以達到提升電子除霧效率且降低影像除霧硬體與功耗成本的目的。That is, the electronic image defogging system of the present invention can obtain air light value 200 and transmittance correction value 300 according to the R, G, and B channel values of a plurality of image numbers in at least one image, which can be processed separately and simultaneously. It can avoid the problem that the traditional defogging method must use a huge calculation mechanism or multiple iterations to improve efficiency and reduce hardware costs. Therefore, the electronic image defogging system according to the present invention can improve electronic defogging efficiency and reduce Purpose of image defogging hardware and power cost.
儘管已參考本申請的許多說明性實施例描述了實施方式，但應瞭解的是，本領域技術人員能夠想到多種其他改變及實施例，這些改變及實施例將落入本公開原理的精神與範圍內。尤其是，在本公開、圖式以及所附申請專利範圍的範圍內，對主題結合設置的組成部分及/或設置可作出各種變化與修飾。除對組成部分及/或設置做出的變化與修飾之外，可替代的用途對本領域技術人員而言將是顯而易見的。Although the embodiments have been described with reference to many illustrative embodiments of the present application, it should be understood that those skilled in the art can think of many other changes and embodiments that will fall within the spirit and scope of the principles of the present disclosure Inside. In particular, various changes and modifications can be made to the components and / or settings of the subject combination setting within the scope of the present disclosure, the drawings, and the scope of the attached patent application. In addition to variations and modifications in the component parts and / or arrangements, alternative uses will be apparent to those skilled in the art.
1‧‧‧電子影像除霧系統1‧‧‧Electronic image defogging system
10‧‧‧空氣光估算模組 10‧‧‧Air Light Estimation Module
11‧‧‧RGB暗通道 11‧‧‧RGB dark channel
12‧‧‧第一數值抽樣元件 12‧‧‧The first numerical sampling element
13‧‧‧第二數值抽樣元件 13‧‧‧Second numerical sampling element
20‧‧‧透射率修正模組 20‧‧‧Transmittance correction module
21‧‧‧分析單元 21‧‧‧analysis unit
22‧‧‧深度轉換器 22‧‧‧ Depth Converter
23‧‧‧銳化濾波器 23‧‧‧Sharpening filter
24‧‧‧低通濾波器 24‧‧‧ Low Pass Filter
25‧‧‧修正單元 25‧‧‧ correction unit
30‧‧‧影像還原模組 30‧‧‧Image Recovery Module
100‧‧‧影像 100‧‧‧ video
110‧‧‧第一最小值 110‧‧‧ the first minimum
121‧‧‧第二最小值 121‧‧‧ second minimum
122‧‧‧第三最小值 122‧‧‧ third minimum
A、B‧‧‧第一矩陣 A, B‧‧‧ First Matrix
C‧‧‧第二矩陣 C‧‧‧Second Matrix
200‧‧‧空氣光值 200‧‧‧air light value
211‧‧‧飽和度 211‧‧‧Saturation
212‧‧‧明度 212‧‧‧lightness
221‧‧‧深度值 221‧‧‧ Depth
231‧‧‧第一值 231‧‧‧first value
241‧‧‧第二值 241‧‧‧Second Value
300‧‧‧透射率修正值 300‧‧‧ transmittance correction value
400‧‧‧除霧影像 400‧‧‧ Defog image
S01~S03‧‧‧步驟 S01 ~ S03‧‧‧step
圖1為本發明之一實施例的電子影像除霧系統的架構示意圖； 圖2為本發明之該實施例的電子影像除霧系統之空氣光估算模組的架構示意圖； 圖3為本發明之該實施例的電子影像除霧系統之空氣光估算模組的操作示意圖； 圖4為本發明之該實施例的電子影像除霧系統之透射率修正模組的架構示意圖；以及 圖5為本發明之該實施例的電子影像除霧方法的流程圖。FIG. 1 is a schematic diagram of an electronic image defogging system according to an embodiment of the present invention; FIG. 2 is a schematic diagram of an air light estimation module of the electronic image defogging system according to the embodiment of the present invention; Operation diagram of the air light estimation module of the electronic image defogging system of the embodiment; FIG. 4 is a schematic diagram of the structure of the transmittance correction module of the electronic image defogging system of the embodiment of the present invention; and FIG. 5 is the present invention The flowchart of the electronic image defogging method of this embodiment.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW107120602A TWI673685B (en) | 2018-06-14 | 2018-06-14 | Electronic image defogging system and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW107120602A TWI673685B (en) | 2018-06-14 | 2018-06-14 | Electronic image defogging system and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TWI673685B true TWI673685B (en) | 2019-10-01 |
| TW202001800A TW202001800A (en) | 2020-01-01 |
Family
ID=69023893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW107120602A TWI673685B (en) | 2018-06-14 | 2018-06-14 | Electronic image defogging system and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI673685B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI423166B (en) * | 2009-12-04 | 2014-01-11 | Huper Lab Co Ltd | Method for determining if an input image is a foggy image, method for determining a foggy level of an input image and cleaning method for foggy images |
| CN104616258A (en) * | 2015-01-26 | 2015-05-13 | 中南大学 | Rapid defogging method for road image |
| CN105303532A (en) * | 2015-10-21 | 2016-02-03 | 北京工业大学 | Wavelet domain Retinex image defogging method |
| TWI542212B (en) * | 2013-03-19 | 2016-07-11 | Univ Chaoyang Technology | Photographic system with visibility enhancement |
-
2018
- 2018-06-14 TW TW107120602A patent/TWI673685B/en active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI423166B (en) * | 2009-12-04 | 2014-01-11 | Huper Lab Co Ltd | Method for determining if an input image is a foggy image, method for determining a foggy level of an input image and cleaning method for foggy images |
| TWI542212B (en) * | 2013-03-19 | 2016-07-11 | Univ Chaoyang Technology | Photographic system with visibility enhancement |
| CN104616258A (en) * | 2015-01-26 | 2015-05-13 | 中南大学 | Rapid defogging method for road image |
| CN105303532A (en) * | 2015-10-21 | 2016-02-03 | 北京工业大学 | Wavelet domain Retinex image defogging method |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202001800A (en) | 2020-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9384532B2 (en) | Apparatus for improving fogged image using user-controllable root operator | |
| US8823798B2 (en) | Obscuring identification information in an image of a vehicle | |
| US10290081B2 (en) | System for image dehazing by modifying lower bound of transmittance and method therefor | |
| US9418402B2 (en) | System for improving foggy luminance image using fog reduction estimation model | |
| CN111754440B (en) | License plate image enhancement method, system, equipment and storage medium | |
| US8319854B2 (en) | Shadow removal in an image captured by a vehicle based camera using a non-linear illumination-invariant kernel | |
| Ko et al. | Artifact-free low-light video enhancement using temporal similarity and guide map | |
| CN111079669A (en) | Image processing method, device and storage medium | |
| CN113673584A (en) | Image detection method and related device | |
| JP2004310475A (en) | Image processor, cellular phone for performing image processing, and image processing program | |
| CN103778605A (en) | Greasy weather image enhancement method | |
| KR101470833B1 (en) | Apparatus for enhancing the foggy luminance image using lower bound of transmission rate | |
| CN113139419B (en) | Unmanned aerial vehicle detection method and device | |
| CN112241982A (en) | Image processing method and device and machine-readable storage medium | |
| TWI673685B (en) | Electronic image defogging system and method thereof | |
| CN117274107A (en) | End-to-end color and detail enhancement method, device and equipment under low-illumination scene | |
| KR101535630B1 (en) | Apparatus for enhancing the brightness of night image using brightness conversion model | |
| KR102118783B1 (en) | Preprocessing method for color filtering robust against illumination environment and the system thereof | |
| CN111062272A (en) | Image processing and pedestrian identification method and device based on color recovery and readable storage medium | |
| TWI630818B (en) | Dynamic image feature enhancement method and system | |
| JP5822739B2 (en) | Image processing apparatus, method, and program | |
| KR101625158B1 (en) | Apparatus and Method for Processing photographed image of Blackbox in vehicle | |
| US20200130581A1 (en) | Reverse image display method | |
| EP3389257A1 (en) | Method for adapting brightness of image data, image capturing system and advanced driver assistance system | |
| Kumari et al. | Fast and efficient contrast enhancement for real time video dehazing and defogging |