TWI749714B - Method for defect detection, method for defect classification and system thereof - Google Patents
Method for defect detection, method for defect classification and system thereof Download PDFInfo
- Publication number
- TWI749714B TWI749714B TW109127911A TW109127911A TWI749714B TW I749714 B TWI749714 B TW I749714B TW 109127911 A TW109127911 A TW 109127911A TW 109127911 A TW109127911 A TW 109127911A TW I749714 B TWI749714 B TW I749714B
- Authority
- TW
- Taiwan
- Prior art keywords
- defect
- images
- image
- good product
- good
- Prior art date
Links
Images
Landscapes
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
Description
說明書公開一種瑕疵檢測方法,特別是一種從良品學習出的影像特徵進行瑕疵檢測與瑕疵分類的方法與系統。The manual discloses a flaw detection method, especially a method and system for flaw detection and flaw classification based on image features learned from good products.
在工業製造產業中,除了通過人手或人眼判斷物品瑕疵外,隨著電腦與學習演算法的技術發展,利用學習影像中瑕疵的技術因應而生,利用軟體方法能更有效率地執行瑕疵判斷。然而,以目前技術來看,以影像識別技術來檢測產品的瑕疵,除了需要收集大量的影像外,還有無法收集到完整的瑕疵圖像而導致機器學習效果不彰的問題。In the industrial manufacturing industry, in addition to judging defects by human hands or eyes, with the development of computers and learning algorithms, the technology of learning defects in images is developed, and software methods can be used to perform defect judgments more efficiently. . However, judging from the current technology, using image recognition technology to detect product defects requires the collection of a large number of images, as well as the inability to collect complete images of defects, resulting in poor machine learning effects.
另外,在機器學習的技術中,利用深度學習演算法可以從大量取得的樣品影像中學習得出瑕疵判斷的模型,然而,要得出有效判斷瑕疵的模型,需要收集到各種種類的瑕疵影像,例如,學習過程中須標記每個瑕疵位置,再執行軟體學習,操作繁雜且費時,仍需要投入大量的學習與時間成本,進一步地,使用深度學習演算法,也有硬體成本高的問題,若學習結果不如預期,還需要重新學習。In addition, in machine learning technology, deep learning algorithms can be used to learn from a large number of sample images to obtain a defect judgment model. However, to obtain an effective defect judgment model, it is necessary to collect various types of defect images, such as In the learning process, you must mark each defect location, and then perform software learning. The operation is complicated and time-consuming. It still requires a lot of learning and time costs. Furthermore, the use of deep learning algorithms also has the problem of high hardware costs. If the learning results Not as expected, but need to learn again.
說明書公開一種瑕疵檢測方法、瑕疵分類方法及其系統,不同於一般針對瑕疵品學習瑕疵影像特徵以判斷瑕疵品的方式,根據一實施例,所提出的瑕疵檢測系統包括一或多個影像擷取裝置,用以取得多個良品或一待測物的影像,設有一計算裝置,包括處理電路與介面電路,可通過介面電路接收一或多個影像擷取裝置拍攝多個良品或待測物所得出的影像,通過處理電路針對這些一或多張影像執行一瑕疵檢測方法。The specification discloses a defect detection method, a defect classification method and a system, which are different from the general method of learning defect image characteristics for defect products to determine the defect. According to an embodiment, the proposed defect detection system includes one or more image captures. The device is used to obtain images of multiple good products or an object under test. It is equipped with a computing device, including a processing circuit and an interface circuit. It can receive one or more image capture devices through the interface circuit to capture multiple good products or objects under test. The output image is processed by the processing circuit to perform a defect detection method for the one or more images.
在瑕疵檢測方法中,對所取得的多個良品的多張影像執行一機器學習法,以學習多個良品中的影像特徵,建立用以描述良品影像特徵的一良品模型,因此可以應用此良品模型套用在待測物的影像,以取得待測物影像中的瑕疵數據,之後根據檢測參數,對照待測物影像中的瑕疵數據判斷待測物是否為一瑕疵品。In the defect detection method, a machine learning method is performed on the multiple images of the multiple good products obtained to learn the image characteristics of the multiple good products, and a good product model to describe the characteristics of the good product images is established, so this good product can be applied The model is applied to the image of the object to be tested to obtain the defect data in the image of the object to be tested, and then according to the detection parameters, it is judged whether the object to be tested is a defective product by comparing the defect data in the image of the object to be tested.
進一步地,待測物的瑕疵數據根據待測物的屬性可以包括一或多個瑕疵的位置、面積、顏色、亮度以及形狀的其中之一或任意組合。Further, the defect data of the object to be measured may include one or any combination of the position, area, color, brightness, and shape of one or more defects according to the properties of the object to be measured.
如此,待測物的各項瑕疵數據對照檢測參數中對於各項瑕疵數據設定的門檻,可以判斷待測物是否為一瑕疵品,所判斷的瑕疵類型至少包括髒污、缺漏、破損、顏色變化以面積變化。In this way, the various defect data of the object under test can be compared with the threshold set for each defect data in the detection parameters to determine whether the object under test is a defective product. The type of defect to be judged includes at least dirt, omission, damage, and color change. Change by area.
在一實施例中,當系統取得良品影像後,可切割為多個區塊影像,再以機器學習法學習各區塊影像中良品影像特徵,形成描述良品影像特徵的良品模型成為各區塊正常分佈標準表示,各區塊正常分佈標準包括各畫素的強度、面積與顏色的分布。In one embodiment, after the system obtains the good product image, it can be cut into multiple block images, and then machine learning method is used to learn the good product image characteristics in each block image to form a good product model describing the good product image characteristics and become normal for each block. The distribution standard means that the normal distribution standard of each block includes the intensity, area and color distribution of each pixel.
根據揭露書所提出的瑕疵分類方法實施例,同樣先取得多個良品的多張影像,經執行機器學習法學習多個良品中的影像特徵,建立用以描述良品影像特徵的良品模型,之後拍攝多個待測物以取得多個待測物的多張影像,可應用良品模型從這些待測物取得具有瑕疵數據的多張圖檔,根據事先定義的瑕疵類別,經執行一深度學習法,可分類影像中的瑕疵,以產生瑕疵類別,建立各分類瑕疵項目。According to the defect classification method embodiment proposed in the disclosure book, multiple images of multiple good products are also obtained first, and the image features of the multiple good products are learned by the machine learning method to establish a good product model to describe the image characteristics of the good products, and then shoot Multiple DUTs to obtain multiple images of multiple DUTs. The good product model can be used to obtain multiple images with defect data from these DUTs. According to the pre-defined defect categories, a deep learning method can be implemented. Defects in the image can be classified to generate defect categories, and create each classified defect item.
進一步地,於拍攝待測物影像的步驟中,可以連續拍攝取得待測物多張影像,並能動態定位被測物的位置,精準定位多張影像中的同一個檢測區塊,以標示其中多個瑕疵位置與特徵資訊。並且,於定位檢測區塊時,還能選擇固定或任意形狀框選檢測範圍。Furthermore, in the step of shooting images of the object to be measured, multiple images of the object to be measured can be continuously shot, and the position of the object to be measured can be dynamically located, and the same detection area in the multiple images can be accurately positioned to mark many of them. Information on the location and characteristics of each defect. In addition, when positioning the detection block, you can also select a fixed or arbitrary shape to select the detection range.
進一步地,當取得一或多個瑕疵位置後,可以裁切影像的方式得出多個瑕疵位置區塊,使得在後續進行一深度學習法時,可以大幅提升深度學習效率。Further, after obtaining one or more defect locations, the image can be cropped to obtain multiple defect location blocks, so that when a deep learning method is subsequently performed, the efficiency of deep learning can be greatly improved.
之後,取得再一待測物的一或多張影像,應用良品模型取得待測物的瑕疵數據後,可根據瑕疵類別來分類待測物之瑕疵。After that, one or more images of another object to be tested are obtained, and after the defect data of the object to be tested is obtained using the good product model, the defects of the object to be tested can be classified according to the defect category.
為使能更進一步瞭解本發明的特徵及技術內容,請參閱以下有關本發明的詳細說明與圖式,然而所提供的圖式僅用於提供參考與說明,並非用來對本發明加以限制。In order to further understand the features and technical content of the present invention, please refer to the following detailed description and drawings about the present invention. However, the provided drawings are only for reference and description, and are not used to limit the present invention.
以下是通過特定的具體實施例來說明本發明的實施方式,本領域技術人員可由本說明書所公開的內容瞭解本發明的優點與效果。本發明可通過其他不同的具體實施例加以施行或應用,本說明書中的各項細節也可基於不同觀點與應用,在不悖離本發明的構思下進行各種修改與變更。另外,本發明的附圖僅為簡單示意說明,並非依實際尺寸的描繪,事先聲明。以下的實施方式將進一步詳細說明本發明的相關技術內容,但所公開的內容並非用以限制本發明的保護範圍。The following are specific specific examples to illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be based on different viewpoints and applications, and various modifications and changes can be made without departing from the concept of the present invention. In addition, the drawings of the present invention are merely schematic illustrations, and are not drawn according to actual size, and are stated in advance. The following embodiments will further describe the related technical content of the present invention in detail, but the disclosed content is not intended to limit the protection scope of the present invention.
應當可以理解的是,雖然本文中可能會使用到“第一”、“第二”、“第三”等術語來描述各種元件或者信號,但這些元件或者信號不應受這些術語的限制。這些術語主要是用以區分一元件與另一元件,或者一信號與另一信號。另外,本文中所使用的術語“或”,應視實際情況可能包括相關聯的列出項目中的任一個或者多個的組合。It should be understood that although terms such as “first”, “second”, and “third” may be used herein to describe various elements or signals, these elements or signals should not be limited by these terms. These terms are mainly used to distinguish one element from another, or one signal from another signal. In addition, the term "or" used in this document may include any one or a combination of more of the associated listed items depending on the actual situation.
說明書公開一種瑕疵檢測方法、瑕疵分類方法以及系統,有別於一般通過學習瑕疵品影像特徵建立檢測瑕疵的模型的方式,揭露書所提出的瑕疵檢測方法係以良品(non-defective unit)影像為基礎,學習良品的影像特徵,作為瑕疵檢驗的依據。The manual discloses a defect detection method, defect classification method and system, which is different from the general method of establishing a defect detection model by learning the image characteristics of defective products. The defect detection method proposed in the disclosure book is based on non-defective unit images. Basic, learn the image characteristics of good products, as the basis for defect inspection.
相對於需要取得具有各種瑕疵樣式的瑕疵品而言,取得良品樣品相對容易多了,根據瑕疵檢測流程的技術概念,第一階段關於品質檢測,需要採樣多個良品,但良品樣品收集相對容易,之後利用機器學習法(machine-learning method)學習良品的影像特徵,目的是找出定義良品的影像特徵與各種特徵之間的關聯性,這方面比學習瑕疵影像特徵需要的硬體算力相對較低,且效率更好。通過機器學習法篩選出瑕疵影像,之後還可經過人工或機器判斷其準確度,經過參數調整與多次訓練後,可以精進所得出的模型。之後,根據事先定義的瑕疵類別,再進行第二階段以深度學習法(deep-learning method)建立瑕疵分類,用於指出各種瑕疵的類別。其中優勢之一是,在第一階段篩選出可疑瑕疵後,可大幅縮小需分類的瑕疵圖像的範圍,讓深度學習分類可更快速運算。Compared with the need to obtain defective products with various defect patterns, it is easier to obtain good samples. According to the technical concept of the defect detection process, the first stage of quality inspection requires sampling of multiple good products, but the collection of good samples is relatively easy. After that, the machine-learning method is used to learn the image features of the good product, the purpose is to find the correlation between the image features that define the good product and the various features, which is relatively more powerful than the hardware computing power required to learn the defect image characteristics Low, and better efficiency. The flawed image is screened out by machine learning, and then the accuracy can be judged manually or by machine. After parameter adjustment and multiple trainings, the resulting model can be refined. After that, according to the pre-defined defect categories, proceed to the second stage to establish the defect classification using the deep-learning method, which is used to point out the categories of various defects. One of the advantages is that after the suspicious flaws are screened out in the first stage, the range of flawed images to be classified can be greatly reduced, so that the deep learning classification can be calculated more quickly.
實現所述瑕疵檢測方法的系統實施例可參考圖1所示利用攝影機拍攝樣品的系統示意圖。For an embodiment of the system for realizing the defect detection method, refer to the schematic diagram of the system shown in FIG. 1 for taking a sample with a camera.
瑕疵檢測系統主要元件包括有一或多個以各種形式的光學設備實現的影像擷取裝置,實現自動光學檢測的目標,如圖中顯示有攝影機111, 112, 113,用以取得良品或待測物的影像,如示意圖顯示在一光源14下拍攝樣品12,當系統要建立良品模型時,即以多個良品替換作為樣品12,以攝影機111, 112, 113從不同角度拍攝樣品12,取得良品的影像。瑕疵檢測系統設有一計算裝置10,其中主要元件包括處理電路101、儲存裝置103與介面電路105,通過介面電路105可接收一或多個攝影機111, 112, 113拍攝良品或待測物所得出的一或多張影像,通過處理電路101針對所取得的一或多張影像執行瑕疵檢測方法。儲存裝置103用以儲存影像數據,以及經過分析得出的數據等。根據一實施例,瑕疵檢測系統可設有控制電路107,控制電路107為計算裝置10用以控制攝影機111, 112, 113運作的驅動電路,讓使用者可以通過控制電路107操作攝影機的運作與拍攝。The main components of the defect detection system include one or more image capture devices implemented by various forms of optical equipment to achieve the goal of automatic optical inspection. As shown in the figure, there are
根據以上描述的瑕疵檢測系統的實施例, 即以計算裝置10中的處理電路101執行瑕疵檢測方法與瑕疵分類方法,特別是其中應用的機器學習法與深度學習法。常見機器學習的方法例如(但不限於此)線性回歸(linear regression)、邏輯回歸(logistic regression)、支援向量機(support vector machine)、分類和回歸樹(classification and regression tree)等。According to the embodiment of the flaw detection system described above, the
上述機器學習法等相關學習演算法可以從大量收集的影像數據中自動分析獲得影像中特定目標(如良品、瑕疵品)的影像特徵,也包括學習到未知的特徵,目標是能夠得出各種數據的分佈(主要針對良品),以及數據之間的關係。然而,為了確保機器學習得出符合需求的模型,還需要在學習訓練數據的過程提供大量的樣品(良品、瑕疵品)影像數據,以揭露書所揭示的方法為例,先取得生產線上良品的影像數據,作為學習演算法訓練的數據。The above-mentioned machine learning method and other related learning algorithms can automatically analyze the image characteristics of specific targets (such as good products and defective products) in the image from a large amount of collected image data, including learning unknown features, and the goal is to be able to obtain various data The distribution (mainly for good products), and the relationship between the data. However, in order to ensure that machine learning can obtain a model that meets the requirements, it is necessary to provide a large number of samples (good and defective) image data in the process of learning training data. Take the method disclosed in the disclosure book as an example, first obtain the good product on the production line. The image data is used as the training data of the learning algorithm.
根據實施例,其中機器學習法用以學習在生產線上特定產品的影像中的特徵,其中主要是從生產線上得出良品,從良品的影像學習得到良品的影像特徵,例如面積的特徵、顏色的特徵、亮度的特徵以及形狀的特徵等,將這些良品影像特徵建立一良品模型,反過來用來檢測出不符這些特徵的瑕疵品。舉例來說,可通過良品模型檢測的瑕疵類型如氧化、髒污、刮傷、印刷不良與破損等。According to the embodiment, the machine learning method is used to learn the features in the image of a specific product on the production line, where the good product is mainly obtained from the production line, and the image characteristics of the good product are learned from the image of the good product, such as area characteristics and color characteristics. Features, brightness features, shape features, etc., build a good product model from these good product image characteristics, which in turn is used to detect defective products that do not match these characteristics. For example, the types of defects that can be detected by the good product model are oxidation, dirt, scratches, poor printing, and breakage.
圖2顯示瑕疵檢測方法中學習良品影像特徵建立模型的實施例流程圖。FIG. 2 shows a flowchart of an embodiment of a model for learning image characteristics of a good product in a defect detection method.
首先,在所應用的生產線上取得多個良品的多個影像(步驟S201),接著以機器學習法學習良品影像特徵(步驟S203),學習影像特徵的過程中,根據一實施例,針對取得的灰階或彩色影像進行影像分析,通過機器學習法得出特定產品(良品)影像中可識別圖像的位置、面積、顏色、亮度以及形狀等描述良品的特徵,從影像特徵中得出描述良品的資訊,取得影像畫素中涉及良品的規則,建立良品模型(步驟S205)。舉例來說,良品模型描述的是良品的影像特性,可以特徵向量、影像屬性標籤與各畫素值等方式表示。First, obtain multiple images of multiple good products on the applied production line (step S201), and then learn the image features of the good products by machine learning (step S203). In the process of learning image features, according to an embodiment, the Perform image analysis on grayscale or color images, and use machine learning to obtain the location, area, color, brightness, and shape of the recognizable image of a specific product (good product). Obtain the good product rules in the image pixels, and establish a good product model (step S205). For example, the good product model describes the image characteristics of the good product, which can be represented by feature vectors, image attribute tags, and pixel values.
為了能夠根據良品模型檢測瑕疵,即決定檢測參數,其中關於判斷瑕疵品的各種門檻(步驟S207)。根據實施例之一,進行瑕疵檢測時,可先根據待測物的屬性(如待測物為電子產品、電路板、印刷品、塑膠商品、金屬物品…等)決定瑕疵檢測參數為基於其中一或多個瑕疵的位置、面積、顏色、亮度以及形狀的其中之一或任意組合,如此,在檢測瑕疵時,可根據這些檢測參數,對照待測物的一或多張影像中的瑕疵數據,判斷待測物是否為一瑕疵品。In order to be able to detect defects based on the good product model, the detection parameters are determined, including various thresholds for judging defective products (step S207). According to one of the embodiments, when performing defect detection, the defect detection parameters can be determined based on one or the other according to the properties of the object to be tested (for example, the object to be tested is an electronic product, a circuit board, a printed product, a plastic product, a metal product, etc.) One or any combination of the position, area, color, brightness, and shape of multiple flaws. In this way, when detecting flaws, you can compare the flaw data in one or more images of the object to be tested based on these detection parameters. Whether the object to be tested is a defective product.
依據圖2描述的流程建立的良品模型,圖3接著顯示瑕疵檢測方法的實施例之一流程圖。According to the good product model established according to the process described in FIG. 2, FIG. 3 then shows a flow chart of an embodiment of the defect detection method.
利用系統提供的攝影機拍攝一待測物(步驟S301),取得待測物的一或多張影像(步驟S303),接著應用良品模型,根據待測物的屬性取得待測物影像中不符良品模型中描述的良品影像特徵的一或多張影像中的瑕疵數據(步驟S305),取得疑似瑕疵的位置以及相關瑕疵數據,如面積、顏色、亮度以及形狀的其中之一或任意組合(步驟S307),接著,可根據待測物的屬性取得根據檢測參數,即將判斷為疑似瑕疵的影像比對檢測參數(步驟S309),其中會依據系統針對瑕疵設定的判斷條件(強度、容許度,並可依據需求調整)篩選出確定瑕疵的部份,篩選時,可以待測物的各項瑕疵數據(面積、顏色、亮度以及形狀的其中之一或任意組合)對照檢測參數中對於各項瑕疵數據設定的門檻,以判斷待測物是否為一瑕疵品(步驟S311)。Use the camera provided by the system to shoot an object under test (step S301), obtain one or more images of the object under test (step S303), and then apply the good product model to obtain the non-good product model in the image of the test object according to the properties of the test object Defect data in one or more images of the good image characteristics described in (step S305), obtain the position of the suspected defect and related defect data, such as one or any combination of area, color, brightness, and shape (step S307) , Then, according to the attributes of the object to be tested, the detection parameters, namely the image comparison detection parameters that are judged as suspected flaws, can be obtained according to the attributes of the test object (step S309), which will be based on the judgment conditions (strength, tolerance, and Need to adjust) to filter out the part that determines the defect. When screening, the defect data (one or any combination of area, color, brightness and shape) of the object to be tested can be compared with the setting for each defect data in the detection parameters Threshold to determine whether the object to be tested is a defective product (step S311).
在此一提的是,以上方法以良品影像作為訓練模型的材料,過程中並未判斷與標註瑕疵的需求,因此可以省下大量的運算時間,建立良品模型可以判斷出待測物上的瑕疵,接著再套用瑕疵判斷條件。It is mentioned here that the above method uses good product images as the training model material, and there is no need to judge and mark defects in the process, so a lot of computing time can be saved, and the establishment of a good product model can determine the defects on the object under test. , And then apply the defect judgment condition.
在判斷是否有瑕疵的方法中,還可將影像分割為m x n區塊,各區塊根據學習結果得出良品的影像特徵,例如可以是各區塊中各畫素的強度、面積與顏色(影像特徵)的分布,因此,描述良品影像特徵的良品模型則可以是各區塊正常分佈標準,相關實施例可參考圖4所示瑕疵檢測方法中建立瑕疵判斷標準的實施例流程圖。In the method of judging whether there is a defect, the image can also be divided into mxn blocks, and each block can obtain the image characteristics of the good product according to the learning result, such as the intensity, area and color of each pixel in each block (image Therefore, the good product model describing the characteristics of the good product image can be the normal distribution standard of each block. For related embodiments, please refer to the flowchart of the embodiment of establishing the defect judgment standard in the defect detection method shown in FIG. 4.
先取得一或多張良品影像(步驟S401),並將各影像切割為多個區塊影像(步驟S403),接著可參考以上實施例的描述,以機器學習法學習各區塊影像中良品影像特徵(步驟S405),如此可以建立各區塊正常分佈標準(步驟S407)。也就是說,利用機器學習法建立的良品模型可以各區塊正常分佈標準表示,各區塊正常分佈標準包括各畫素的強度、面積與顏色的分布。因此,之後可以此分佈標準(良品模型)取得待測物的一或多張影像中的瑕疵數據,配合依照待測物的屬性設定的檢測參數,以比對待測物影像中各區塊的畫素影像值,可判斷各區是否具有瑕疵,再以整體來看,判斷待測物是否屬於瑕疵品。First obtain one or more good images (step S401), and cut each image into multiple block images (step S403), and then refer to the description of the above embodiment to learn the good images in each block image by machine learning Features (step S405), so that the normal distribution standard of each block can be established (step S407). In other words, the good product model established by the machine learning method can be represented by the normal distribution standard of each block, and the normal distribution standard of each block includes the distribution of the intensity, area and color of each pixel. Therefore, the distribution standard (good product model) can be used to obtain the defect data in one or more images of the object to be tested, and the detection parameters set according to the attributes of the object to be tested can be used to compare the images of each block in the image of the object to be tested. Based on the element image value, it can be judged whether each area has a defect, and then as a whole, it can be judged whether the object to be tested is a defective product.
根據一實施例,當完成學習良品影像特徵後,系統可產生多種(如3種)不同強度的篩檢率數值,使得使用者可在系統提供的使用者介面中,根據系統建議參數值,透過照片呈現不同強度的篩選效果,直觀式挑選適合的數值。According to one embodiment, after learning the image characteristics of good products, the system can generate multiple (for example, 3) screening rate values with different intensities, so that the user can use the system's suggested parameter values in the user interface provided by the system to pass The photos show different intensities of screening effects, and the appropriate values can be selected intuitively.
以上實施例所述各區塊影像特徵的分佈可參考圖5所示瑕疵檢測方法中某區塊正常分佈標準的示意圖。For the distribution of the image features of each block in the above embodiment, refer to the schematic diagram of the normal distribution standard of a block in the defect detection method shown in FIG. 5.
圖中顯示良品或待測物的影像中某區塊的影像特徵分佈,其中橫軸為畫素的強度值,縱軸為某一檢測參數,例如為良品參數頻率值,顯示的曲線可以是各畫素的強度、面積與顏色的分布,其中有兩條垂直虛線,為決定是否屬於正常範圍的門檻一501與門檻二502,門檻一501與門檻二502即用於篩選瑕疵影像,並可依照實際需求調整。The figure shows the image feature distribution of a certain block in the image of the good product or the object to be tested. The horizontal axis is the intensity value of the pixel, and the vertical axis is a certain detection parameter, such as the frequency value of the good product parameter. The displayed curve can be various The intensity, area, and color distribution of the pixels. There are two vertical dotted lines. Threshold 1 501 and Threshold 2 502 are used to determine whether they belong to the normal range. Threshold 1 501 and Threshold 2 502 are used to screen defect images. Actual demand adjustment.
針對良品而言,經過機器學習法可以得出整張影像的正常分佈標準,如圖6所示。接著,後續比對的待測物也同樣地將影像切割為多個區塊,通過每個區塊的分佈判斷是否為疑似瑕疵,作為初步篩選的方法,因此可以有效降低瑕疵判斷的計算需求。For good products, the normal distribution standard of the entire image can be obtained through the machine learning method, as shown in Figure 6. Then, the subsequent comparison of the test object also cuts the image into multiple blocks, and judges whether it is a suspected defect based on the distribution of each block, as a preliminary screening method, thus effectively reducing the calculation requirements for defect judgment.
其中,利用良品的各影像區塊的正常分佈標準建立的良品模型,可執行瑕疵檢測,如圖7所示瑕疵檢測方法的另一實施例流程圖。Among them, the good product model established by the normal distribution standard of each image block of the good product can perform defect detection, as shown in FIG. 7 as a flowchart of another embodiment of the defect detection method.
先拍攝取得待測物影像(步驟S701),在將待測物影像切割為多個區塊影像(步驟S703),並取得各區塊影像特徵,如強度、其中圖案面積與顏色的特徵分佈等(步驟S705),同樣可得到各區塊的影像特徵分佈,這時,經比對各區塊正常分佈標準(步驟S707),配合上述實施例描述的門檻範圍,可以判斷各區塊是否屬於瑕疵,並再判斷整體是否符合瑕疵品的條件(步驟S709)。First, take the image of the object to be measured (step S701), and then divide the image of the object to be measured into multiple block images (step S703), and obtain the image characteristics of each block, such as the intensity, the pattern area and the color feature distribution, etc. (Step S705), the image feature distribution of each block can also be obtained. At this time, after comparing the normal distribution standard of each block (step S707), in accordance with the threshold range described in the above embodiment, it can be judged whether each block is a defect. And then judge whether the whole meets the condition of the defective product (step S709).
值得一提的是,說明書所提出的瑕疵檢測方法可以連續拍攝取得多張待測物影像後,系統可動態定位被測物的位置,且針對多個區塊影像執行影像處理程序中,能高速變換不同的影像(圖檔),還可精準定位到同一個檢測區塊,通過並一次標示多個瑕疵位置與特徵資訊。根據上述圖4描述的實施例,通過機器學習法學習良品影像特徵後,可定義出多重瑕疵,如此,只要檢驗一次,即可標示出待測物影像中由不同成因而造成的多個瑕疵。It is worth mentioning that the defect detection method proposed in the manual can continuously capture multiple images of the object under test, and the system can dynamically locate the position of the object under test, and the image processing program for multiple block images can be high-speed Different images (image files) can be changed, and the same inspection area can be accurately positioned, and multiple defect locations and characteristic information can be marked at one time. According to the embodiment described in FIG. 4, multiple flaws can be defined after learning the characteristics of a good product image through a machine learning method. In this way, with only one inspection, multiple flaws caused by different causes in the image of the object to be tested can be marked.
再者,由於待測物型態多變,在定位檢測區塊時,可以任意形狀框選檢測範圍,且能獨立設置不檢測區塊,並可透過一次的框選固定或任意形狀,若可框選任意形狀,可增加使用者操作軟體方便度,此外,透過設置不檢測區塊之功能,可加速人員判斷瑕疵範圍並只聚焦於檢測區塊上。Furthermore, because the types of objects to be tested are changeable, when locating the detection area, the detection range can be selected in any shape, and the non-detection area can be set independently, and the fixed or arbitrary shape can be selected through a single frame. Frame selection of any shape can increase the user's convenience in operating the software. In addition, by setting the function of not detecting the area, it can speed up the personnel to judge the scope of the defect and focus only on the inspection area.
舉例來說,其中各區影像特徵(依照待測物屬性與需求)包括由待測物各區域影像特徵所計算得出的強度改變(亮度)、面積變化、顏色變化,比對由系統根據需求建立的判斷瑕疵的條件,包括瑕疵涵蓋範圍(如門檻一501與門檻二502),若判斷其中之一區塊符合瑕疵條件,但仍可能不算為瑕疵,進一步設定的檢測參數可以是符合瑕疵條件的區塊數量或位置,因此仍需要全部或部份區塊判斷待測物是否為瑕疵品。For example, the image characteristics of each area (according to the attributes and requirements of the object to be measured) include the intensity change (brightness), area change, and color change calculated from the image characteristics of each area of the object to be measured. The comparison is performed by the system according to the requirements. The established conditions for judging defects, including the scope of defects (such as threshold one 501 and threshold two 502), if one of the blocks is judged to meet the defect conditions, but it may still not be considered as a defect, the further detection parameters can be set to meet the defect The number or location of the condition of the block, so all or part of the block still needs to be judged whether the object under test is a defective product.
以上各實施例所描述的瑕疵檢測方法在整個系統運作上為第一階段,當第一階段利用機器學習法建立良品模型可篩選出待測物的可疑瑕疵後,對照事先定義的瑕疵類別,還可在第二階段以深度學習法建立瑕疵分類,用於指出各種瑕疵的類別,且第一階段的結果可大幅縮小需分類的瑕疵圖像的範圍,讓深度學習分類可更快速運算。The flaw detection methods described in the above embodiments are the first stage in the operation of the entire system. When the first stage uses machine learning to establish a good product model to screen out the suspicious flaws of the object to be tested, compare the pre-defined flaw categories and return In the second stage, the deep learning method can be used to establish the defect classification to point out the categories of various defects, and the results of the first stage can greatly reduce the range of the defect images to be classified, so that the deep learning classification can be calculated more quickly.
值得一提的是,當通過深度學習得出瑕疵分類後,還可經過調整再次以深度學習法,通過多次訓練改善瑕疵分類的判斷。It is worth mentioning that after the defect classification is obtained through deep learning, it can be adjusted again to use the deep learning method to improve the judgment of the defect classification through multiple training.
圖8顯示瑕疵分類方法的實施例流程圖,系統可先根據需求,包括先前收集得到的瑕疵圖案,可以指定瑕疵的項目,作為瑕疵分類的依據。在流程中,先取得待測物影像(步驟S801),較佳地,拍攝多個待測物以取得多個待測物的多張影像,再套用良品模型(步驟S803),應用根據待測物屬性選擇的良品模型從多個待測物取得具有瑕疵數據的多張圖檔。其中,可以對整張待測物影像得出具有瑕疵的位置與其形式,如面積、顏色、亮度以及形狀等,或是經過切割為多個影像區塊後比對其中色彩或亮度等的正常分佈標準,可取得瑕疵區塊(步驟S805)。Fig. 8 shows a flowchart of an embodiment of a defect classification method. The system can firstly specify defect items according to requirements, including previously collected defect patterns, as a basis for defect classification. In the process, the image of the object to be tested is first obtained (step S801). Preferably, multiple images of the object to be tested are taken to obtain multiple images of the plurality of objects to be tested, and then the good product model is applied (step S803). The good product model selected by object attributes obtains multiple images with defect data from multiple objects to be tested. Among them, the position and form of flaws, such as area, color, brightness, and shape, can be obtained for the entire image of the object under test, or the normal distribution of color or brightness can be compared after being cut into multiple image blocks. Standard, the defective block can be obtained (step S805).
在此一提的是,在所述瑕疵分類方法中,在進行分類之前,可於取得一或多個瑕疵位置後,以一裁切影像的方式得出多個瑕疵位置區塊,使得方法在後續進行一深度學習法時,可以大幅提升深度學習效率。It is mentioned here that in the defect classification method, before the classification is performed, after obtaining one or more defect positions, a plurality of defect position blocks can be obtained by a cropped image, so that the method is When a deep learning method is subsequently performed, the efficiency of deep learning can be greatly improved.
接著即以深度學習法學習各種瑕疵特徵,以分類瑕疵,其中採用的方法例如(但不限於此)邏輯迴歸模型(logistic regression model)與k-次交叉驗證(k-fold cross validation),將輸入的瑕疵影像通過學習得到描述各種瑕疵類別的函數,建立各分類瑕疵項目,包括可建立瑕疵檢測模型(步驟S807)。之後,根據這個瑕疵檢測模型,可以針對生產線上的產品執行瑕疵檢測與分類(步驟S809),其中,同樣輸入待測物的影像,除判斷是否有瑕疵外,更可針對瑕疵分類,以利後續修正系統參數。Next, learn various defect features by deep learning method to classify defects. Methods such as (but not limited to) logistic regression model and k-fold cross validation are used, and input Through learning, the defect image can obtain a function describing various defect categories, and establish each classified defect item, including the establishment of a defect detection model (step S807). After that, according to this flaw detection model, flaw detection and classification can be performed on the products on the production line (step S809), in which the image of the object to be tested is also input, in addition to judging whether there is a flaw, it can also be classified for the flaw to facilitate subsequent follow-up Correct the system parameters.
圖9A至9E顯示幾種瑕疵分類的範例示意圖,但瑕疵分類並不以此例為限。圖中顯示之範例為以一印刷品上的字母為例,經拍攝此印刷品得到影像後,通過良品模型判斷出瑕疵的部份,再以瑕疵檢測模型進行分類。舉例來說,得出如圖9A示意表示其中有髒污的瑕疵;如圖9B有因為印刷不良造成的缺漏問題;如圖9C因為印刷過程中被機械刮傷產生了破損;如圖9D顯示因為印刷顏料不良或是過程中的缺失產生了顏色變化的問題;以及圖9E所示因為印刷問題形成了面積變化等。Figures 9A to 9E show schematic diagrams of several examples of defect classification, but the defect classification is not limited to this example. The example shown in the figure is a letter on a printed matter as an example. After shooting the printed matter to obtain an image, the defective part is judged by the good product model, and then the defect detection model is used for classification. For example, as shown in Figure 9A, it is shown that there is a dirty defect; Figure 9B has a defect caused by poor printing; Figure 9C is damaged due to mechanical scratches during the printing process; Figure 9D shows that because The problem of color change caused by poor printing pigments or missing in the process; and the area change caused by printing problems as shown in Figure 9E.
綜上所述,以現有技術而言,如在製造業中,產品檢驗大多仍仰賴人工檢測的方式,機器檢測技術為輔助人工為主,因此有耗費人力、缺乏效率,以及不容易執行更精密的檢測,甚至容易發生疏失,因此,根據以上實施例所描述的瑕疵檢測與分類方法,所提出的瑕疵檢測系統利用良品影像執行機器學習法,學習良品的影像特徵後建立用以描述良品影像特徵的良品模型,之後的應用僅需設定瑕疵強度與瑕疵大小容許度,即可依照實際需求調整檢測嚴謹度,根據所設定的檢測參數檢測物品的瑕疵,還可利用深度學習分類瑕疵,強化檢測精度,以此方法因應現行發生在生產線上的檢測需求,實現強大卻又簡易的檢測能力,更可大幅提升產品良率。To sum up, in terms of existing technology, for example, in manufacturing, most product inspections still rely on manual inspection. Machine inspection technology is mainly assisted by manual labor. Therefore, it is labor intensive, inefficient, and difficult to perform more precise. The detection is even prone to errors. Therefore, according to the defect detection and classification method described in the above embodiment, the proposed defect detection system uses the good product image to perform the machine learning method, learns the image feature of the good product, and establishes it to describe the good product image feature The next application only needs to set the flaw strength and flaw size tolerance, and then adjust the inspection rigor according to actual needs. According to the set inspection parameters, it can detect the defects of the item. It can also use deep learning to classify the defects and strengthen the detection accuracy. In this way, in response to the current testing needs that occur on the production line, a powerful but simple testing capability can be achieved, and the product yield rate can be greatly improved.
以上所公開的內容僅為本發明的優選可行實施例,並非因此侷限本發明的申請專利範圍,所以凡是運用本發明說明書及圖式內容所做的等效技術變化,均包含於本發明的申請專利範圍內。The content disclosed above is only a preferred and feasible embodiment of the present invention, and does not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the description and schematic content of the present invention are included in the application of the present invention. Within the scope of the patent.
10:計算裝置10: Computing device
101:處理電路101: processing circuit
103:儲存裝置103: storage device
105:介面電路105: Interface circuit
107:控制電路107: Control circuit
111,112,113:攝影機111, 112, 113: camera
12:樣品12: sample
14:光源14: light source
501:門檻一501: Threshold One
502:門檻二502: Threshold Two
步驟S201~S207:建立良品模型的步驟Steps S201~S207: Steps to build a good product model
步驟S301~S311:瑕疵檢測步驟Steps S301~S311: Defect detection step
步驟S401~S407:建立區塊正常分佈標準的步驟Steps S401~S407: Steps to establish the normal distribution standard of blocks
步驟S701~S709:瑕疵檢測步驟Steps S701~S709: Defect detection step
步驟S801~S809:瑕疵分類步驟Steps S801~S809: Defect classification step
圖1顯示利用攝影機拍攝樣品的系統示意圖;Figure 1 shows a schematic diagram of a system that uses a camera to photograph samples;
圖2顯示在瑕疵檢測方法中學習良品影像特徵建立模型的實施例流程圖;FIG. 2 shows a flowchart of an embodiment of learning image features of good products in a defect detection method to establish a model;
圖3顯示瑕疵檢測方法的實施例之一流程圖;Figure 3 shows a flow chart of an embodiment of the defect detection method;
圖4顯示瑕疵檢測方法中建立瑕疵判斷標準的實施例流程圖;Figure 4 shows a flowchart of an embodiment of establishing a defect judgment standard in a defect detection method;
圖5顯示瑕疵檢測方法中某區塊正常分佈標準的示意圖;Figure 5 shows a schematic diagram of the normal distribution standard of a certain block in the defect detection method;
圖6顯示瑕疵檢測方法中每張影像中多個區塊正常分佈標準的示意圖;Figure 6 shows a schematic diagram of the normal distribution standard of multiple blocks in each image in the flaw detection method;
圖7顯示瑕疵檢測方法的實施例之二流程圖;FIG. 7 shows a flowchart of the second embodiment of the defect detection method;
圖8顯示瑕疵分類方法的實施例流程圖;以及FIG. 8 shows a flowchart of an embodiment of a defect classification method; and
圖9A至9E顯示幾種瑕疵分類的範例示意圖。Figures 9A to 9E show schematic diagrams of several types of defects.
S301:拍攝待測物 S301: Shoot the object under test
S303:取得待測物影像 S303: Obtain the image of the object under test
S305:套用良品模型 S305: Apply good product model
S307:取得瑕疵位置 S307: Get defect location
S309:比對檢測參數 S309: Compare detection parameters
S311:判斷是否為瑕疵品 S311: Determine whether it is a defective product
Claims (14)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109127911A TWI749714B (en) | 2020-08-17 | 2020-08-17 | Method for defect detection, method for defect classification and system thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109127911A TWI749714B (en) | 2020-08-17 | 2020-08-17 | Method for defect detection, method for defect classification and system thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI749714B true TWI749714B (en) | 2021-12-11 |
TW202209173A TW202209173A (en) | 2022-03-01 |
Family
ID=80681024
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109127911A TWI749714B (en) | 2020-08-17 | 2020-08-17 | Method for defect detection, method for defect classification and system thereof |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI749714B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI826108B (en) * | 2022-11-10 | 2023-12-11 | 州巧科技股份有限公司 | Method for establishing defect-detection model using fake defect images and system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101866427A (en) * | 2010-07-06 | 2010-10-20 | 西安电子科技大学 | Method for detecting and classifying fabric defects |
CN103473770A (en) * | 2013-09-05 | 2013-12-25 | 东华大学 | Linear fabric flaw detection method based on rectangular window projection and singular value decomposition |
TW202018600A (en) * | 2018-10-29 | 2020-05-16 | 財團法人工業技術研究院 | Neural-network-based classification device and classification method |
US20200226744A1 (en) * | 2019-01-16 | 2020-07-16 | Elad Cohen | Method of defect detection on a specimen and system thereof |
-
2020
- 2020-08-17 TW TW109127911A patent/TWI749714B/en active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101866427A (en) * | 2010-07-06 | 2010-10-20 | 西安电子科技大学 | Method for detecting and classifying fabric defects |
CN103473770A (en) * | 2013-09-05 | 2013-12-25 | 东华大学 | Linear fabric flaw detection method based on rectangular window projection and singular value decomposition |
TW202018600A (en) * | 2018-10-29 | 2020-05-16 | 財團法人工業技術研究院 | Neural-network-based classification device and classification method |
US20200226744A1 (en) * | 2019-01-16 | 2020-07-16 | Elad Cohen | Method of defect detection on a specimen and system thereof |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI826108B (en) * | 2022-11-10 | 2023-12-11 | 州巧科技股份有限公司 | Method for establishing defect-detection model using fake defect images and system |
Also Published As
Publication number | Publication date |
---|---|
TW202209173A (en) | 2022-03-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111161246A (en) | Product defect detection method, device and system | |
CN110618134A (en) | Steel plate surface quality defect detection and rating system and method | |
CN104198497A (en) | Surface defect detection method based on visual saliency map and support vector machine | |
JP2011158373A (en) | Method for creation of teacher data for use in automatic defect classification, and method and apparatus for automatic defect classification | |
CN113706490B (en) | Wafer defect detection method | |
CN111179250A (en) | Industrial product defect detection system based on multitask learning | |
TWI747686B (en) | A defect detection method and a defect detection device | |
WO2024055796A1 (en) | Defect detection system, method and apparatus, electronic device, and storage medium | |
CN106501272A (en) | Machine vision scolding tin position detecting system | |
CN110096980A (en) | Character machining identifying system | |
TW202127014A (en) | Intelligent Production Line Monitoring System and Implementation Method Thereof | |
CN106651802A (en) | Machine vision tin soldering location detection method | |
CN110689524A (en) | No-reference online image definition evaluation method and system | |
CN112614105A (en) | Depth network-based 3D point cloud welding spot defect detection method | |
KR102666787B1 (en) | Method, apparatus and program for inspecting defect | |
CN116071315A (en) | Product visual defect detection method and system based on machine vision | |
TWI749714B (en) | Method for defect detection, method for defect classification and system thereof | |
CN112419261A (en) | Visual acquisition method and device with abnormal point removing function | |
CN114226262A (en) | Flaw detection method, flaw classification method and flaw detection system | |
TWM606740U (en) | Defect detection system | |
CN117689646A (en) | High-precision defect detection method, system and medium for positive and negative sample fusion | |
CN117351472A (en) | Tobacco leaf information detection method and device and electronic equipment | |
CN116091506B (en) | Machine vision defect quality inspection method based on YOLOV5 | |
CN111563869B (en) | Stain test method for quality inspection of camera module | |
CN115947066A (en) | Belt tearing detection method, device and system |