TWI718573B - Apparatus and method for inspecting for defects - Google Patents

Apparatus and method for inspecting for defects Download PDF

Info

Publication number
TWI718573B
TWI718573B TW108122230A TW108122230A TWI718573B TW I718573 B TWI718573 B TW I718573B TW 108122230 A TW108122230 A TW 108122230A TW 108122230 A TW108122230 A TW 108122230A TW I718573 B TWI718573 B TW I718573B
Authority
TW
Taiwan
Prior art keywords
image
defect
defect detection
learning
color image
Prior art date
Application number
TW108122230A
Other languages
Chinese (zh)
Other versions
TW202014985A (en
Inventor
宋德王
李在旻
秋淵學
李錫中
Original Assignee
南韓商樂人股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南韓商樂人股份有限公司 filed Critical 南韓商樂人股份有限公司
Publication of TW202014985A publication Critical patent/TW202014985A/en
Application granted granted Critical
Publication of TWI718573B publication Critical patent/TWI718573B/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G01N21/95607—Inspecting patterns on the surface of objects using a comparative method
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

本發明提出缺陷檢測裝置及方法，根據一實施例，缺陷檢測裝置包含：儲存部，儲存預定的一個標準影像以及拍攝檢驗對象獲得的至少一個拍攝影像；以及控制部，運算標準影像以及各拍攝影像的差異，產生對應各個拍攝影像的差分影像，將標準影像、各拍攝影像以及各差分影像合成，產生對應各拍攝影像的至少一個彩色影像，利用產生的彩色影像實施缺陷檢測模型的學習或缺陷檢測中至少一個。 The present invention provides a defect detection device and method. According to an embodiment, the defect detection device includes: a storage unit storing a predetermined standard image and at least one shot image obtained by shooting an inspection object; and a control unit, calculating the standard image and each shot image Differential images corresponding to each shot image are generated, the standard image, each shot image, and each difference image are synthesized to generate at least one color image corresponding to each shot image, and the generated color images are used to implement defect detection model learning or defect detection At least one of them.

Description

缺陷檢測裝置及方法 Defect detection device and method

本發明說明書中公開的多個實施例涉及缺陷檢測裝置及方法，具體是，利用檢驗對象的拍攝影像檢測對象中包含的缺陷時，利用可提升缺陷檢測性能的影像處理方法進行缺陷訓練以及實施檢測的缺陷檢測裝置及方法。 A number of embodiments disclosed in the specification of the present invention relate to defect detection devices and methods. Specifically, when a defect contained in an object is detected by using a photographed image of an inspection object, an image processing method that can improve defect detection performance is used for defect training and detection. Defect detection device and method.

對品質統一的量產產品的不良與否進行檢測的方式有多種。如包裝紙或標簽紙等印刷品或印刷電路板或已組裝完的電路等電子產品生產製程上的中間或最終生產產品等，對實際樣子或形狀相同的生產產品積極應用影像分析技術，以減少檢測不良所需的時間以及勞力的同時提升檢測品質。 There are many ways to detect the defects of mass-produced products with uniform quality. Printed products such as packaging paper or label paper, or intermediate or final products in the production process of electronic products such as printed circuit boards or assembled circuits. Actively apply image analysis technology to production products with the same actual appearance or shape to reduce detection It takes time and labor to improve the quality of inspection.

目前是利用人工智慧的影像分析技術檢測產品不良，從而獲得降低誤檢或過檢率等的技術成果。 At present, artificial intelligence image analysis technology is used to detect product defects, so as to obtain technical results such as reducing false detection or over-detection rate.

如上所述，利用人工智慧的影像分析技術檢測產品缺陷的通常方法是，利用充分數量的正常及/或不良產品的影像學習判定缺陷所需的模型以後，利用學習模型判定檢驗對象的影像正常或不良與否。但利用現有的這種缺陷訓練以及檢測方式無法將各種缺陷全部識別出來，進而導致整體缺陷檢測性能不良的問題。 As mentioned above, the usual method of using artificial intelligence image analysis technology to detect product defects is to use a sufficient number of images of normal and/or defective products to learn the models required to determine the defects, and then use the learning model to determine whether the images of the inspection object are normal or Bad or not. However, it is impossible to identify all kinds of defects by using the existing defect training and detection methods, which leads to the problem of poor overall defect detection performance.

例如，根據現有的印刷電路板的缺陷檢測方法，如第1圖所示利用將生產的印刷電路板拍攝獲取的拍攝影像10學習模型，標記有缺陷11的位置，對有缺陷的區域以及背景區域進行訓練，進而缺陷檢測模型學習無缺陷的背景區域以及有缺陷的區域，之後輸入檢驗對象的拍攝影像時，可以輸出該拍攝影像包含缺陷與否。 For example, according to the existing defect detection method for printed circuit boards, as shown in Figure 1, the model is learned from the photographed image 10 obtained by photographing the produced printed circuit board, and the position of the defect 11 is marked, and the defect area and the background area After training, the defect detection model learns the defect-free background area and the defective area, and then when the photographed image of the inspection object is input, it can output whether the photographed image contains defects or not.

但這種現有對印刷電路板的缺陷檢測方法是，對斷開(OPEN；電路模式應連接著的區域連接斷開而發生的缺陷)或短路(SHORT；不應該相互連接的電路模式相互連接而發生的缺陷)等結構的缺陷檢出率不高，應該有孔(HOLE)的位置漏孔的缺陷或電路模式以及模式之間的間隔保持得不充分而發生的空間(SPACE)未達缺陷等無法檢測不到。 However, this existing defect detection method for printed circuit boards is to detect disconnection (OPEN; the defect caused by the disconnection of the area where the circuit pattern should be connected) or short circuit (SHORT; the circuit pattern that should not be connected to each other. The defect detection rate of the structure such as the occurrence of defects) is not high, and there should be holes (HOLE). The defect of the hole or the circuit pattern and the interval between the patterns are not maintained sufficiently and the space that occurs (SPACE) does not reach the defect, etc. Undetectable.

利用人工智慧的影像分析技術檢測產品缺陷的另一種方法是，利用充分數量的正常及/或不良產品的拍攝影像以及預先設定的標準影像產生差分(Difference)影像，利用差分影像使模型學習以後，利用學習的模型判定檢驗對象的影像正常或不良與否。但即便藉由這樣的方式，結構缺陷的檢測率仍然未能得到充分提升，因差分影像的一部分區域顏色或亮度與背景或邊緣部分相似而缺陷檢測率降低。 Another method of using artificial intelligence image analysis technology to detect product defects is to use a sufficient number of normal and/or defective product images and pre-set standard images to generate a difference image. After the model is learned using the difference image, Use the learned model to determine whether the image of the test subject is normal or bad. However, even in this way, the detection rate of structural defects has not been fully improved, and the defect detection rate is reduced because the color or brightness of a part of the differential image is similar to the background or edge portion.

相關的習知技術文獻即韓國注册專利第10-1128322號涉及印刷電路板的光學檢驗裝置及方法，對於將標準PCB模式影像以及拍攝PCB模式影像比較判定不良與否的技術進行敘述。但習知技術僅僅是將標準影像以及拍攝的影像亮度模式與數值比較後判定正常或不良與否，難以檢測出細小的缺陷，也不能辨別缺陷的種類或正確位置。因此需要一種可以解决上述問題的技術。 The related prior art document, Korean Registered Patent No. 10-1128322, relates to an optical inspection device and method for printed circuit boards, and describes a technique for comparing standard PCB pattern images and shooting PCB pattern images to determine whether they are defective. However, the conventional technology only compares the standard image and the brightness mode of the shot image with the value to determine whether it is normal or defective. It is difficult to detect small defects, nor can it distinguish the type or correct position of the defect. Therefore, a technology that can solve the above-mentioned problems is needed.

上述背景技術是發明人為了獲得本發明而擁有或者在獲得本發明的過程中掌握的技術訊息，並非是申請本發明之間已向普通公衆公開的習知技術。 The above-mentioned background art is technical information possessed by the inventor or mastered in the process of obtaining the present invention in order to obtain the present invention, and is not a conventional technique that has been disclosed to the general public before the present invention is applied for.

本說明書中公開的實施例的目的在於提供一種缺陷檢測裝置及方法。 The purpose of the embodiments disclosed in this specification is to provide a defect detection device and method.

本說明書中公開的實施例的目的在於利用3通道的彩色影像提升檢驗對象的缺陷檢測性能。 The purpose of the embodiments disclosed in this specification is to use 3-channel color images to improve the defect detection performance of the inspection object.

本說明書中公開的實施例的目的在於藉由對檢測出的缺陷類型的學習，判定缺陷的類型。 The purpose of the embodiments disclosed in this specification is to determine the type of the defect by learning the type of the detected defect.

本發明中公開的實施例的目的在於，利用根據缺陷位置分別學習的缺陷檢測模型而提升缺陷檢測以及分類的準確性。 The purpose of the embodiments disclosed in the present invention is to improve the accuracy of defect detection and classification by using defect detection models that are separately learned according to defect positions.

為解决所述問題，本發明所採用的技術方案是，根據一實施例作為檢測檢驗對象缺陷的裝置，包含：儲存部，儲存預定的一個標準影像以及拍攝檢驗對象獲得的至少一個拍攝影像；以及控制部，運算所述標準影像以及各拍攝影像的差異，產生對應各拍攝影像的差分影像，將所述標準影像、各拍攝影像以及各差分影像合成，產生對應各拍攝影像的至少一個彩色影像，利用產生的彩色影像實施缺陷檢測模型的學習或缺陷檢測中至少一個。 To solve the problem, the technical solution adopted by the present invention is, according to an embodiment, as a device for detecting defects of an inspection object, including: a storage unit storing a predetermined standard image and at least one photographed image obtained by photographing the inspection object; and The control unit calculates the difference between the standard image and each shot image, generates a difference image corresponding to each shot image, combines the standard image, each shot image, and each difference image to generate at least one color image corresponding to each shot image, At least one of learning of a defect detection model or defect detection is implemented using the generated color image.

為解决所述技術問題，本發明所採用的技術方案是，根據一實施例作為藉由缺陷檢測裝置實施的檢測檢驗對象缺陷的方法，包含：獲得拍攝檢 驗對象產生的至少一個拍攝影像的步驟；利用各拍攝影像以及預定的一個標準影像之間的差異產生對應各個拍攝影像的差分影像的步驟；將所述標準影像、各個拍攝影像以及各個差分影像合成而產生對應各個拍攝影像的至少一個彩色影像的步驟；以及利用產生的彩色影像實施缺陷檢測模型的學習或缺陷檢測中至少一個的步驟。 In order to solve the technical problem, the technical solution adopted by the present invention is, according to an embodiment, as a method for detecting defects of an inspection object implemented by a defect detection device, including: obtaining a photographic inspection The step of examining at least one photographed image generated by the subject; the step of using the difference between each photographed image and a predetermined standard image to generate a differential image corresponding to each photographed image; combining the standard image, each photographed image, and each differential image The step of generating at least one color image corresponding to each captured image; and using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

進一步為解决所述技術問題，本發明所採用的技術方案是，根據另一實施例，記錄實施缺陷檢測方法的程式的電腦可讀記錄媒體中所述缺陷檢測方法包含：獲得拍攝檢驗對象產生的至少一個拍攝影像的步驟；利用各拍攝影像以及預定的一個標準影像之間的差異產生對應各個拍攝影像的差分影像的步驟；將所述標準影像、各個拍攝影像以及各個差分影像合成而產生對應各個拍攝影像的至少一個彩色影像的步驟；以及利用產生的彩色影像實施缺陷檢測模型的學習或缺陷檢測中至少一個的步驟。 To further solve the technical problem, the technical solution adopted by the present invention is that, according to another embodiment, the defect detection method in the computer-readable recording medium that records the program for implementing the defect detection method includes: At least one step of photographing images; a step of generating a differential image corresponding to each photographed image by using the difference between each photographed image and a predetermined standard image; combining the standard image, each photographed image, and each differential image to generate a corresponding The step of capturing at least one color image of the image; and the step of using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

為解决所述技術問題，本發明所採用的技術方案是，根據另一實施例，藉由缺陷檢測裝置實施並為實施缺陷檢測方法而儲存在媒體中的電腦程式中，所述缺陷檢測方法包含：獲得拍攝檢驗對象產生的至少一個拍攝影像的步驟；利用各拍攝影像以及預定的一個標準影像之間的差異產生對應各個拍攝影像的差分影像的步驟；將所述標準影像、各個拍攝影像以及各個差分影像合成而產生對應各個拍攝影像的至少一個彩色影像的步驟；以及利用產生的彩色影像實施缺陷檢測模型的學習或缺陷檢測中至少一個的步驟。 In order to solve the technical problem, the technical solution adopted by the present invention is, according to another embodiment, implemented by a defect detection device and stored in a computer program in a medium for implementing the defect detection method, the defect detection method includes : The step of obtaining at least one shot image generated by the shooting test object; the step of using the difference between each shot image and a predetermined standard image to generate a difference image corresponding to each shot image; combining the standard image, each shot image, and each shot image The step of synthesizing the difference images to generate at least one color image corresponding to each captured image; and using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

根據上述的技術方案之一，可以提出缺陷檢測裝置以及方法。 According to one of the above technical solutions, a defect detection device and method can be proposed.

根據上述的技術方案之一，利用3通道彩色影像而提升檢驗對象的缺陷檢測性能。 According to one of the above technical solutions, 3-channel color images are used to improve the defect detection performance of the inspection object.

根據上述的技術方案一，可以提供學習檢測的缺陷類型而判定缺陷類型的缺陷檢測裝置及方法。 According to the first technical solution mentioned above, it is possible to provide a defect detection device and method that learns the defect type to be detected and determines the defect type.

根據上述的技術方案之一，利用根據缺陷位置分別學習的缺陷檢測模型而提升缺陷檢測及分類的準確性。 According to one of the above technical solutions, the defect detection model learned separately according to the defect position is used to improve the accuracy of defect detection and classification.

藉由公開實施例可獲得的效果並不限於以上效果，根據以下記載本領域具有通常知識者可以清楚地瞭解到未涉及的其它效果。 The effects that can be obtained by the disclosed embodiments are not limited to the above effects. According to the following description, those with ordinary knowledge in the art can clearly understand other effects that are not involved.

a:標準影像 a: Standard image

b、10:拍攝影像 b, 10: Shooting images

c:差分影像 c: Differential image

CLS:缺陷類型選擇按鈕 CLS: Defect type selection button

d:彩色影像 d: color image

e:裁剪影像 e: crop image

S610、S620、S630、S640、S650、S710、S720、S730、S740:缺陷檢測方法步驟 S610, S620, S630, S640, S650, S710, S720, S730, S740: Defect detection method steps

11:缺陷 11: Defects

100:缺陷檢測裝置 100: Defect detection device

110:輸入輸出部 110: Input and output department

120:儲存部 120: Storage Department

130:通訊部 130: Ministry of Communications

140:控制部 140: Control Department

200:拍攝裝置 200: Camera

第1圖是繪示標記缺陷位置的印刷電路板的拍攝影像一例的例示圖；第2圖是說明一實施例的缺陷檢測裝置結構的框圖；第3圖是繪示一實施例的缺陷檢測裝置中用於缺陷檢測的基準影像、拍攝影像以及差分影像的一例的例示圖；第4圖是繪示一實施例的檢測裝置中利用基準影像以及拍攝影像以及差分影像產生的彩色影像的一例的例示圖；第5圖是說明一實施例的對缺陷檢測裝置中檢測的缺陷類型進行學習的過程的例示圖；第6圖及第7圖是繪示一實施例的缺陷檢測方法的步驟流程圖。 Fig. 1 is an exemplary diagram showing an example of a photographed image of a printed circuit board marking the position of a defect; Fig. 2 is a block diagram illustrating the structure of a defect detection device according to an embodiment; Fig. 3 is a diagram showing a defect detection according to an embodiment An illustration of an example of a reference image, a captured image, and a differential image used for defect detection in the device; Figure 4 shows an example of a color image generated using the reference image, the captured image, and the differential image in the detection device of an embodiment Illustrative diagram; FIG. 5 is an illustrative diagram illustrating the process of learning the type of defects detected in the defect detection device according to an embodiment; FIG. 6 and FIG. 7 are flowcharts illustrating the steps of a defect detection method according to an embodiment .

下面將結合附圖詳細描述各種實施例。以下說明的實施例是也可以修改成其它不同形態實現。為了更加明確說明，對於以下實施例所屬技術領 域的具有通常知識者已廣泛知曉的內容不再詳細說明。圖中省略與實施例說明無關的部分，對說明書中類似的部分使用了類似的圖號。 Various embodiments will be described in detail below with reference to the accompanying drawings. The embodiments described below can also be modified into other different forms for implementation. In order to explain more clearly, the technical field of the following embodiments The content of the domain that has been widely known by ordinary knowledgeable persons will not be explained in detail. In the figure, parts irrelevant to the description of the embodiments are omitted, and similar numbers are used for similar parts in the specification.

說明書中描述某個元件「連接」於其它元件是，不僅表示直接連接，中間還會存在其它元件。說明書中描述某個構成「包含」某個元件時，在沒有特別相反的描述的前提下，並不是指排除其它元件，而是說明還可以包含其它元件。 The description in the specification that a certain element is "connected" to other elements not only means that it is directly connected, but there may be other elements in between. When a component is described in the specification as "comprising" a certain element, provided that there is no description to the contrary, it does not mean that other elements are excluded, but it is stated that other elements may also be included.

下面結合附圖詳述多個實施例。 A number of embodiments will be described in detail below with reference to the drawings.

說明之前先定義下面使用的用語。 Before explaining, define the terms used below.

「拍攝影像」是將檢驗對象直接拍攝獲得的影像。以下以印刷電路板作為「檢驗對象」進行說明，但只要是如電子產品的中間或最終產物或包裝紙或標簽紙等印刷品等，生產時已具有預定的一定模式的產品皆可。拍攝影像並不限於簡單的利用可見光拍攝的影像，也可以是根據檢測對象的種類或性質利用X射線或紅外線等各種光學拍攝方式獲得的影像。 "Shooting image" is an image obtained by directly shooting the test object. The following uses printed circuit boards as the "test object" for description, but as long as it is an intermediate or final product of an electronic product, or printed matter such as packaging paper or label paper, it can be a product that has a predetermined pattern at the time of production. The captured image is not limited to a simple image captured with visible light, but may also be an image obtained using various optical imaging methods such as X-rays or infrared rays according to the type or nature of the object to be detected.

「標準影像」是繪示檢驗對象需具備的理想模式的影像，是各個檢驗對象拍攝影像的實際標準，任何檢驗對象的拍攝影像與標準影像一起用來產生後述的差分影像。 The "standard image" is an image that shows the ideal mode that the test object needs to have, and is the actual standard for shooting images of each test object. Any image of the test object and the standard image are used together to generate the differential image described later.

「差分影像」是利用上述的拍攝影像以及標準影像產生的影像，利用拍攝影像以及標準影像之差運算。例如，對於二元影像，差分影像可以僅藉由兩個影像的各像素值之差來產生，對於非二元影像，可以直接利用各像素值之差或者利用與閾值比較的結果獲得。在此像素值是影像中包含的各像素的亮度值，可以具有8位元值即0至255中一個值。差分影像不僅可以藉由上述的簡單減法運算獲得，也可以按需給拍攝影像或標準影像的值賦予加權值，或者將各像素值之差利用兩個以上的閾值轉換成多個級別值而獲得。如上所述，一實施例中，差分影像只要是可以適當彌補運算方式中檢驗對象種類或性質的，藉 由任何運算方式的皆可。進一步為了獲得差分影像，可以經過至少對拍攝影像以及標準影像之一的預處理過程。 "Differential image" is an image generated using the above-mentioned shooting image and standard image, and calculating the difference between the shooting image and the standard image. For example, for a binary image, the differential image can be generated only by the difference between the pixel values of the two images. For a non-binary image, the difference between the pixel values can be used directly or the result of comparison with a threshold value can be used. The pixel value here is the brightness value of each pixel included in the image, and can have an 8-bit value, that is, a value from 0 to 255. The difference image can not only be obtained by the above-mentioned simple subtraction operation, but also can be obtained by assigning a weighted value to the value of the captured image or standard image as needed, or converting the difference between each pixel value into multiple level values using two or more thresholds. As mentioned above, in one embodiment, as long as the differential image can appropriately compensate for the type or nature of the test object in the calculation method, Any calculation method can be used. Further, in order to obtain a differential image, a preprocessing process of at least one of the captured image and the standard image may be carried out.

拍攝影像為多個時，差分影像會產生與各拍攝影像對應的數量。 When there are multiple shot images, the number of difference images corresponding to each shot image will be generated.

下面「彩色影像」是將拍攝影像、標準影像以及差分影像分別作為彩色通道的影像。為了形成彩色影像，形成各影像通道的拍攝影像、標準影像以及差分影像可以分別以8位元灰度影像形成。為此，根據需求，後述的缺陷檢測裝置是拍攝影像、標準影像以及差分影像不是8位元灰度影像時，可以轉換成8位元灰度影像。此時8位元灰度影像是指各像素值具有8位元的值即0至255中一個亮度值的1通道影像。就是說，沒有彩色通道，是以具有0至255中一個亮度值的像素構成的影像。 The following "color image" is an image that uses the captured image, standard image, and differential image as the color channels. In order to form color images, the captured images, standard images, and differential images of each image channel can be formed as 8-bit grayscale images, respectively. For this reason, according to requirements, the defect detection device described later can convert into 8-bit gray-scale images when shooting images, standard images, and differential images are not 8-bit gray-scale images. At this time, an 8-bit grayscale image refers to a 1-channel image in which each pixel value has an 8-bit value, that is, a brightness value from 0 to 255. In other words, there is no color channel, and an image composed of pixels with a brightness value from 0 to 255.

另外彩色影像可以以將由上述的8位元灰度影像形成的拍攝影像、標準影像以及差分影像分別作為不同彩色通道的3通道24位元影像形成。例如，標準影像可以形成紅色(RED)通道、拍攝影像可以形成綠色(GREEN)通道，差分影像可以形成藍色(BLUE)通道，各通道的各像素值可以顯示彩色影像各像素的相應顏色的亮度值。此時拍攝影像是多個，故彩色影像可以與各個拍攝影像以及差分影像對應地產生。 In addition, the color image can be formed by using the captured image, the standard image, and the differential image formed by the aforementioned 8-bit grayscale image as 3-channel 24-bit images with different color channels, respectively. For example, a standard image can form a red (RED) channel, a captured image can form a green (GREEN) channel, and a differential image can form a blue (BLUE) channel. Each pixel value of each channel can display the brightness of the corresponding color of each pixel of the color image. value. At this time, there are multiple shot images, so color images can be generated corresponding to each shot image and difference image.

此時彩色影像用於一實施例的缺陷檢測裝置以及方法中缺陷檢測模型的學習。此外利用檢測對象的拍攝影像最終產生的彩色影像還可以用於利用學習的缺陷檢測模型檢測實際檢驗對象所具有缺陷。 At this time, the color image is used for learning the defect detection model in the defect detection device and method of an embodiment. In addition, the color image finally generated by the shooting image of the inspection object can also be used to detect the defects of the actual inspection object by using the learned defect detection model.

除了以上定義的用語之外其它需要說明的用語是在下面分別進行說明。 In addition to the terms defined above, other terms that need to be explained are separately explained below.

第2圖是說明一實施例的缺陷檢測裝置的結構的框圖。 Fig. 2 is a block diagram illustrating the structure of a defect detection device according to an embodiment.

缺陷檢測裝置100是用於檢測檢驗對象缺陷所需學習以及利用學習的模型檢測缺陷的裝置。 The defect detection device 100 is a device for learning required to detect defects of an inspection object and detecting defects using a learned model.

所述缺陷檢測裝置100可以由電子終端實現或者伺服器-客戶端系統(或雲端系統)實現，所述系統可以包含安裝與使用者的交互所需的網上服務應用程式的電子終端。 The defect detection apparatus 100 may be implemented by an electronic terminal or a server-client system (or cloud system), and the system may include an electronic terminal for installing online service applications required for interaction with users.

該電子終端10是可以以藉由網絡(N)連接遠程伺服器，或者可與其它終端以及伺服器連接的電腦，或攜帶型終端、電視機、可穿戴設備(Wearable Device)等實現。在此電腦例如包含裝載網絡瀏覽器(WEB Browser)的筆記本電腦、桌上型電腦(desktop)、攜帶型電腦(laptop)等，攜帶型終端是如可以保證攜帶性以及移動性的無線通訊裝置，可以包含PCS(Personal Communication System)、PDC(Personal Digital Cellular)、PHS(Personal Handyphone System)、PDA(Personal Digital Assistant)、GSM(Global System for Mobile communications)、IMT(International Mobile Telecommunication)-2000、CDMA(Code Division Multiple Access)-2000、W-CDMA(W-Code Division Multiple Access)、Wibro(Wireless Broadband Internet)、智慧手機(Smart Phone)、移動WiMAX(Mobile Worldwide Interoperability for Microwave Access)等所有種類的基於手持式(Handheld)的無線通訊裝置。電視機是可以包含IPTV(Internet Protocol Television)、網絡TV(Internet Television)、無線電視、有線電視等。進一步可穿戴設備是如錶、眼鏡、飾品、服裝、鞋等人體可以直接穿戴的訊息處理裝置，直接或藉由其它訊息處理裝置經過網絡連接到遠程伺服器或連接其它終端。 The electronic terminal 10 can be realized by connecting a remote server via a network (N), or a computer that can be connected to other terminals and servers, or a portable terminal, a television, a wearable device, and the like. Computers here include, for example, laptops, desktops, and laptops with a web browser (WEB Browser). Portable terminals are, for example, wireless communication devices that can ensure portability and mobility. It can include PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication)-2000, CDMA ( Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), smart phone (Smart Phone), mobile WiMAX (Mobile Worldwide Interoperability for Microwave Access) and other types of handheld-based Handheld wireless communication device. The television may include IPTV (Internet Protocol Television), Internet TV (Internet Television), wireless television, cable television, and so on. Further, wearable devices are information processing devices that can be directly worn by the human body such as watches, glasses, accessories, clothing, shoes, etc., directly or through other information processing devices connected to a remote server or other terminals via a network.

一實施例的缺陷檢測裝置100包含輸入輸出部110、儲存部120、通訊部130以及控制部140中至少一部分。 The defect detection apparatus 100 of an embodiment includes at least a part of an input/output unit 110, a storage unit 120, a communication unit 130, and a control unit 140.

輸入輸出部110可以包含用來獲得檢測檢驗對象缺陷所需的學習過程或缺陷檢測過程中需求的使用者輸入或者被選擇所需文件而需要的輸入部以及用來顯示操作執行結果或缺陷檢測裝置100狀態等訊息的輸出部。例如，輸 入輸出部110可以包含接收使用者輸入的操作面板(operation panel)以及顯示畫面的顯示面板(display panel)等。 The input and output unit 110 may include a user input required to obtain the learning process required for detecting the defect of the inspection object or the user input required in the defect detection process or an input unit required to select a required file, and an input unit used to display the operation execution result or a defect detection device 100 status and other information output unit. For example, lose The input/output unit 110 may include an operation panel (operation panel) that receives user input, a display panel (display panel) that displays a screen, and the like.

具體是，輸入部可以包含如鍵盤、物理按鈕、觸控螢幕、攝影機或麥克風等各種形態的接收使用者輸入的裝置。輸出部可以包含顯示面板或喇叭等。但並不限於此，輸入輸出部110可以包含各種支援輸入輸出的結構。 Specifically, the input unit may include various types of devices that receive user input, such as a keyboard, physical buttons, touch screen, camera, or microphone. The output unit may include a display panel or a speaker. However, it is not limited to this, and the input and output unit 110 may include various structures that support input and output.

缺陷檢測裝置100可以包含儲存部120。儲存部120可以暫存學習缺陷檢測模型所需的多個拍攝影像以及標準影像，至少可以暫存利用這些產生的差分影像。進一步，至少可以暫存拍攝影像以及標準影像以及利用差分影像獲得的彩色影像。儲存部120利用彩色影像學習缺陷檢測模型的過程中，將已學習的模型在每次被執行學習時更新儲存。儲存部120可以將各缺陷檢測模型分別區分成獨立的檢測文件儲存。 The defect detection apparatus 100 may include a storage unit 120. The storage unit 120 can temporarily store a plurality of photographed images and standard images required for learning the defect detection model, and at least can temporarily store the difference images generated using these. Furthermore, at least the shot image and the standard image and the color image obtained by using the differential image can be temporarily stored. In the process of learning the defect detection model by using the color image, the storage unit 120 updates and stores the learned model each time learning is performed. The storage unit 120 can separate each defect inspection model into independent inspection files for storage.

儲存部120儲存的彩色影像中會標記出缺陷位置，用以學習過程。 The defect location is marked in the color image stored in the storage unit 120 for the learning process.

缺陷檢測裝置100還可以包含通訊部130。 The defect detection device 100 may further include a communication unit 130.

通訊部130可以與其它裝置或網絡實施有無線通訊。為此，通訊部130可以包含支援各種有無線通訊方法中至少一個的通訊模組。例如，通訊模組可以以晶片組(chipset)的形態實現。 The communication unit 130 may implement wireless communication with other devices or networks. To this end, the communication unit 130 may include a communication module that supports at least one of various wireless communication methods. For example, the communication module can be implemented in the form of a chipset.

通訊部130支援的無線通訊例如是Wi-Fi(Wireless Fidelity)、Wi-Fi Direct、藍牙(Bluetooth)、UWB(Ultra Wide Band)或者NFC(Near Field Communication)等。通訊部130支援的有線通訊例如是USB或者HDMI(High Definition Multimedia Interface)等。 The wireless communication supported by the communication unit 130 is, for example, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth (Bluetooth), UWB (Ultra Wide Band), or NFC (Near Field Communication). The wired communication supported by the communication unit 130 is, for example, USB or HDMI (High Definition Multimedia Interface).

尤其通訊部130可以與拍攝裝置200通訊從拍攝裝置200接收拍攝影像。此時拍攝裝置200例如是具備攝影機等光學儀器的裝置。此時拍攝裝置200可以是只可拍攝可見光光譜內波長的光的裝置，也可以是可以拍攝紅外線或X射線等其它波長的光的裝置。 In particular, the communication unit 130 can communicate with the camera 200 to receive the shot image from the camera 200. In this case, the imaging device 200 is, for example, a device provided with optical equipment such as a camera. At this time, the imaging device 200 may be a device that can only capture light of wavelengths in the visible light spectrum, or a device that can capture light of other wavelengths such as infrared rays or X-rays.

控制部140控制缺陷檢測裝置100的整體動作，可以包含CPU等處理器。控制部140例如為了執行與藉由輸入輸出部110接收的使用者輸入對應的動作，可以控制缺陷檢測裝置100中包含的其它結構。 The control unit 140 controls the overall operation of the defect detection apparatus 100, and may include a processor such as a CPU. The control unit 140 may control other structures included in the defect detection apparatus 100 in order to perform an action corresponding to the user input received by the input/output unit 110, for example.

例如，控制部140可以使儲存部120中儲存的程式運行，或者讀取儲存於儲存部120的文件，或者將新檔案存到儲存部120。 For example, the control unit 140 can run a program stored in the storage unit 120, or read a file stored in the storage unit 120, or store a new file in the storage unit 120.

根據本說明書中記載的一實施例，控制部140可以獲得拍攝影像。拍攝影像如上所述是拍攝檢驗對象獲得的影像，是藉由上述的拍攝裝置200產生並傳遞到缺陷檢測裝置100的。此時拍攝影像可以包含將多個檢驗對象分別拍攝的多個影像。 According to an embodiment described in this specification, the control unit 140 can obtain a captured image. As described above, the captured image is the image obtained by shooting the inspection object, which is generated by the above-mentioned imaging device 200 and transmitted to the defect detection device 100. At this time, the captured image may include multiple images respectively captured by multiple inspection objects.

控制部140如第3圖所示，可以利用標準影像a以及各個拍攝影像b產生對應各個拍攝影像的差分影像c。在此差分影像c如上所述，是由控制部140利用拍攝影像b以及標準影像a之差運算而算出。 As shown in FIG. 3, the control unit 140 can use the standard image a and each captured image b to generate a differential image c corresponding to each captured image. Here, the difference image c is calculated by the control unit 140 using the difference between the captured image b and the standard image a as described above.

例如，標準影像a可以在儲存部120預先儲存為8位元的灰度影像，控制部140可以將各個拍攝影像b轉換成8位元的灰度影像。控制部140從標準影像a的各個像素值中減掉各個拍攝影像b的各像素值，將以算出的各值作為像素值的8位元灰度影像產生為差分影像c。 For example, the standard image a can be pre-stored as an 8-bit grayscale image in the storage unit 120, and the control unit 140 can convert each captured image b into an 8-bit grayscale image. The control unit 140 subtracts each pixel value of each captured image b from each pixel value of the standard image a, and generates an 8-bit grayscale image with each calculated value as the pixel value as a differential image c.

控制部140藉由減法運算以上的追加運算，例如實施對標準影像a或拍攝影像b的像素值賦予已設定的加權值，或者將用減法運算算出的值與一個以上的閾值比較將各像素值與多個已設定的級別值對應等運算而產生差分影像c。 The control unit 140 performs additional operations such as subtraction or more, for example, to assign a set weight to the pixel values of the standard image a or the captured image b, or to compare the value calculated by the subtraction with one or more thresholds to obtain each pixel value A difference image c is generated by performing operations corresponding to a plurality of set level values.

進而控制部140如第4圖所示，可以利用經過上述過程產生的標準影像a、各個拍攝影像b以及對應各拍攝影像b的差分影像c獲得彩色影像d。此時彩色影像d也可以按與各拍攝影像b對應的數量產生。 Furthermore, as shown in FIG. 4, the control unit 140 can obtain a color image d using the standard image a, each shot image b, and the difference image c corresponding to each shot image b generated through the above process. At this time, the color image d can also be generated in a quantity corresponding to each shot image b.

具體是，控制部140可以產生將標準影像a、各拍攝影像b以及對應各個拍攝影像b的差分影像c作為彩色通道的3通道的彩色影像d。此時各個拍攝影像b、標準影像a以及對應各個拍攝影像的差分影像c如上所述，可以以8位元的灰度影像形成，其中一部分不符合8位元的灰度影像時，控制部140轉換成8位元灰度影像進行彩色影像d的產生準備。 Specifically, the control unit 140 may generate a 3-channel color image d with a standard image a, each captured image b, and a differential image c corresponding to each captured image b as color channels. At this time, each shot image b, standard image a, and the difference image c corresponding to each shot image can be formed as 8-bit grayscale images as described above, and when some of them do not conform to the 8-bit grayscale image, the control unit 140 Converted into 8-bit grayscale image to prepare for the generation of color image d.

然後控制部140產生將各個8位元灰度影像分別作為紅、綠、藍通道的3通道24位元彩色影像d。進而產生的彩色影像d可以成為將各拍攝影像b、標準影像a以及對應各拍攝影像b的差分影像c的各像素值作為彩色影像d各像素的紅、綠、藍色亮度值的影像。根據實施例，包含標準影像a、各拍攝影像b以及對應各拍攝影像b的差分影像c的三個影像可以以與上述的紅、綠、藍不同順序分別形成相互不同的彩色通道。 Then the control unit 140 generates a 3-channel 24-bit color image d with each 8-bit grayscale image as the red, green, and blue channels. Furthermore, the generated color image d can be an image in which each pixel value of each captured image b, the standard image a, and the difference image c corresponding to each captured image b is used as the red, green, and blue brightness value of each pixel of the color image d. According to the embodiment, the three images including the standard image a, each shot image b, and the difference image c corresponding to each shot image b may respectively form mutually different color channels in a different order from the above red, green, and blue.

控制部140利用如上所述產生的一個以上的彩色影像d實施缺陷檢測模型的學習。此時控制部140首先為了能從彩色影像d中一次性檢測缺陷可疑位置，實施缺陷檢測模型的學習。 The control unit 140 uses the one or more color images d generated as described above to implement the learning of the defect detection model. At this time, the control unit 140 first implements the learning of the defect detection model in order to detect the suspicious position of the defect from the color image d at one time.

此時控制部140為實施缺陷檢測模型的學習而利用的學習對象彩色影像d可以分別被標記有缺陷的缺陷位置。為此，缺陷檢測裝置100可以將學習對象即各個彩色影像d中包含的缺陷位置訊息從使用者藉由輸入輸出部110輸入而接收，或者藉由其它裝置藉由通訊部130接收已標記缺陷位置的彩色影像d。 At this time, the learning target color image d used by the control unit 140 to implement the learning of the defect detection model may be respectively marked with defective defect positions. To this end, the defect detection device 100 can receive the defect location information contained in each color image d, which is the learning object, from the user via the input and output unit 110, or receive the marked defect location via the communication unit 130 via other devices. Color image d.

控制部140可以實施學習標記缺陷位置的學習對象彩色影像d而一次性辨別可疑缺陷位置的學習。具體是，控制部140可以實施用於辨別缺陷可疑位置的缺陷檢測模型的學習，例如，控制部140可以利用機器技術進一步例如利用深度學習技術實施由人工神經網路構成的缺陷檢測模型的學習。控制部140將3通道24位元的彩色影像d的各像素值作為輸入變量，藉由將缺陷可疑位置的訊息作為結果值求出缺陷檢測模型常數值的過程實施缺陷檢測模型的學習。 The control unit 140 can perform learning to identify the suspicious defect position by learning the learning target color image d of the marked defect position. Specifically, the control unit 140 may implement learning of a defect detection model for identifying suspicious locations of defects. For example, the control unit 140 may use machine technology to further implement learning of a defect detection model composed of artificial neural networks, such as deep learning technology. The control unit 140 takes each pixel value of the 3-channel 24-bit color image d as an input variable, and implements the learning of the defect detection model through the process of obtaining the defect detection model constant value by using the information of the suspected defect location as the result value.

進而缺陷檢測模型是可在彩色影像d中找到缺陷可疑位置地被學習。 Furthermore, the defect detection model can be learned to find the suspicious position of the defect in the color image d.

此時藉由控制部140學習的缺陷檢測模型可以具有如根據RGB值限定在特定範圍內的相鄰像素形成的樣子、面積、與周邊像素值的相對關係等檢測缺陷可疑位置的特性。例如，第4圖中繪示的例示中，將彩色影像d中包含的黃色點分別標記為缺陷位置以後，利用這些影像實施缺陷檢測模型的學習。此時黃色點例如可以顯示印刷電路板中漏孔缺陷，缺陷檢測模型是藉由學習可以學習這些漏孔缺陷的共同顏色、大小、樣子、位置特性等。進一步，除了漏孔缺陷之外，學習包含其它類型缺陷的多個彩色影像d時，缺陷檢測模型學習到各種類型的缺陷而檢測出缺陷可疑位置。 At this time, the defect detection model learned by the control unit 140 may have characteristics such as the appearance, area, and relative relationship with surrounding pixel values of adjacent pixels defined in a specific range based on the RGB values to detect the suspicious location of the defect. For example, in the example shown in Figure 4, after the yellow dots included in the color image d are marked as defect locations, these images are used to implement the learning of the defect detection model. At this time, the yellow dots can display leak defects in the printed circuit board, for example, and the defect detection model can learn the common color, size, appearance, and location characteristics of these leak defects through learning. Further, in addition to leak defects, when learning multiple color images d containing other types of defects, the defect detection model learns various types of defects and detects suspicious locations of the defects.

控制部140利用以缺陷可疑位置為中心將一定區域裁剪製作的裁剪影像追加實施缺陷檢測模型的學習。 The control unit 140 additionally implements learning of a defect detection model using a cropped image produced by cropping a certain area with the suspicious defect position as the center.

例如，如第5圖所示，控制部140可以將彩色影像d的一定區域裁剪後產生裁剪圖片e。此時裁剪影像e例如是以缺陷可疑位置為中心的已設定大小的區域影像。 For example, as shown in FIG. 5, the control unit 140 may crop a certain area of the color image d to generate a cropped picture e. At this time, the cropped image e is, for example, an area image with a set size centered on the suspected defect position.

控制部140從包含缺陷的學習對象彩色影像d中以缺陷位置為中心獲得裁剪影像e以後，利用該裁剪影像e實施缺陷檢測模型的學習。此時裁剪影像e上可以標出缺陷存在與否及/或缺陷類型。具體是，如第5圖所示，該裁剪影像e的缺陷為過檢的非缺陷時，使使用者選擇「0.通過」，是缺陷則根據該缺陷類型選擇「1.RC」、「2.凸塊」、「3.短路」等多個缺陷類型選擇按鈕(CLS)中的一個，使裁剪影像e的缺陷存在與否或缺陷類型被標記。利用如上所述被標記缺陷存在與否或缺陷類型的裁剪影像e可以進一步實施缺陷檢測模型的學習。 After obtaining the cropped image e from the learning target color image d containing the defect with the defect position as the center, the control unit 140 implements the learning of the defect detection model using the cropped image e. At this time, the cropped image e can indicate the presence or absence of defects and/or the type of defects. Specifically, as shown in Figure 5, when the defect of the cropped image e is a non-defect that has passed inspection, the user is allowed to select "0. Pass", and if it is a defect, select "1.RC" and "2" according to the defect type. One of multiple defect type selection buttons (CLS) such as "bump", "3. Short circuit", etc., enables the presence or absence of defects in the cropped image e or the defect type to be marked. Using the cropped image e of the presence or absence of the marked defect or the defect type as described above, the learning of the defect detection model can be further implemented.

進而缺陷檢測模型不僅學習得可以更準確地判定缺陷存在與否，還可以辨別缺陷類型。 Furthermore, the defect detection model can not only learn to determine the existence of defects more accurately, but also distinguish the types of defects.

第5圖中繪示的例中，裁剪影像e中包含的缺陷是漏孔缺陷，故使用者可以選擇「9.漏孔」按鈕，進而控制部140將該裁剪影像e標記成漏孔而實施缺陷檢測模型的學習。 In the example shown in Figure 5, the defect contained in the cropped image e is a leak defect, so the user can select the "9. Leak" button, and the control unit 140 marks the cropped image e as a leak for implementation Learning of defect detection models.

此時為檢測缺陷可疑位置而學習彩色影像的缺陷檢測模型以及為檢測缺陷存在與否及/或缺陷類型而學習裁剪影像的缺陷檢測模型是可以分別以不同的檢測模型構成。 At this time, the defect detection model for learning color images to detect the suspicious location of the defect and the defect detection model for learning cropped images to detect the presence or absence of defects and/or defect types can be composed of different detection models.

根據一實施例的缺陷檢測裝置100可以根據缺陷可疑位置具有不同標準地實施缺陷檢測模型的學習。例如，檢測印刷電路板的缺陷時，對於印刷電路板的外廓容許某種程度的缺陷，但對電路區域按嚴格標準辨別缺陷。 The defect detection apparatus 100 according to an embodiment can implement the learning of the defect detection model according to different standards of the suspicious defect position. For example, when detecting the defects of printed circuit boards, a certain degree of defects are allowed for the outline of the printed circuit board, but the defects are identified according to strict standards for the circuit area.

控制部140根據對應彩色影像d的印刷電路板的實際結構，將彩色影像d分成多個區域。例如，將彩色影像d區域預先分成外廓、網目(Mesh)、電路、壓焊、焊墊等多個區域以後，可以辨別檢測的缺陷可疑位置對應哪個區域。此時控制部140為了按區分的各區域將缺陷檢測標準不同地設置，可以按區域分別實施相互不同的檢測模型的學習，各區域的不同檢測模型是以相互不同的檢測檔案形態存到儲存部120。 The control unit 140 divides the color image d into a plurality of regions according to the actual structure of the printed circuit board corresponding to the color image d. For example, after pre-dividing the color image d area into multiple areas such as outline, mesh, circuit, bonding, pad, etc., it is possible to identify which area the suspicious location of the detected defect corresponds to. At this time, in order to set the defect detection standards differently for each area, the control unit 140 can implement the learning of different detection models for each area, and the different detection models in each area are stored in the storage unit in the form of mutually different detection files. 120.

控制部140利用缺陷可疑位置的裁剪影像e實施與將該缺陷可疑位置包含的區域對應的缺陷檢測模型的學習。 The control unit 140 uses the cropped image e of the suspicious defect position to perform learning of the defect detection model corresponding to the area included in the suspicious defect position.

由此根據包含缺陷可疑位置的區域對應哪個區域學習不同的缺陷檢測模型，因此可以根據缺陷可疑位置設置不同的檢測標準。 In this way, different defect detection models can be learned according to which area the area containing the suspicious location of the defect corresponds to, so different detection standards can be set according to the suspicious location of the defect.

另外，控制部140利用如上所述學習的缺陷檢測模型實施缺陷檢測。為此缺陷檢測裝置100可以獲得為了辨別實際缺陷存在與否而拍攝對象的拍攝影像b。缺陷檢測裝置100重新如第3圖所示，利用所獲得的拍攝影像b以及標準影像a的差異，產生差分影像c。 In addition, the control unit 140 performs defect detection using the defect detection model learned as described above. For this reason, the defect detection apparatus 100 can obtain a photographed image b of the photographed object in order to distinguish whether the actual defect exists or not. As shown in FIG. 3 again, the defect detection device 100 uses the obtained difference between the captured image b and the standard image a to generate a differential image c.

缺陷檢測裝置100如第4圖所示，可以產生將拍攝影像b、各標準影像a以及差分影像c作為RGB通道的3通道24位元的彩色影像d。 As shown in FIG. 4, the defect detection device 100 can generate a 3-channel 24-bit color image d using the captured image b, each standard image a, and the differential image c as RGB channels.

然後控制部140利用已學習的缺陷檢測模型辨別彩色影像d中是否存在缺陷以及缺陷類型。此時控制部140首先可從彩色影像d中提取缺陷可疑位置。為了檢測缺陷可疑位置，可以利用學習彩色影像的缺陷檢測模型。 Then, the control unit 140 uses the learned defect detection model to discriminate whether there are defects in the color image d and the defect types. At this time, the control unit 140 can first extract the suspected defect location from the color image d. In order to detect the suspicious location of the defect, a defect detection model that learns color images can be used.

然後控制部140檢測到缺陷檢測位置時，以該缺陷檢測位置為中心獲得差分影像e。此外控制部140利用缺陷檢測模型辨別差分影像e的缺陷存在與否及/或缺陷類型。此時為了辨別缺陷存在與否及/或缺陷類型，可以利用學習差分影像的缺陷檢測模型。 Then, when the control unit 140 detects the defect detection position, it obtains the differential image e with the defect detection position as the center. In addition, the control unit 140 uses the defect detection model to identify the presence or absence of a defect in the differential image e and/or the defect type. At this time, in order to distinguish the existence of defects and/or defect types, the defect detection model of learning differential images can be used.

進一步，根據包含缺陷可疑位置的區域分別學習的多個檢測模型作為各個檢測檔案被分別儲存時，控制部140利用與檢測的缺陷可疑位置對應的檢測模型讀取差分影像e。 Furthermore, when a plurality of detection models learned separately from the area including the suspicious defect location are stored as respective detection files, the control unit 140 reads the difference image e using the detection model corresponding to the suspicious defect location detected.

隨之正確辨別有無缺陷或缺陷類型，根據位置上的相對標準精細分類檢測對象有無缺陷。 Then correctly distinguish whether there is a defect or defect type, and finely classify whether the inspection object has a defect according to the relative standard of the position.

第6圖及第7圖是說明一實施例的缺陷檢測方法的順序圖。 Fig. 6 and Fig. 7 are sequence diagrams for explaining a defect detection method according to an embodiment.

第6圖及第7圖中繪示的實施例的缺陷檢測方法包含第1圖或第2圖中繪示的缺陷檢測裝置100中按時序列處理的步驟。下面即便是省略的內容，但以上關於第1圖或第2圖中繪示的缺陷檢測裝置100敘述的內容也可以適用到第6圖及第7圖中繪示的實施例的缺陷檢測方法。 The defect detection method of the embodiment shown in FIG. 6 and FIG. 7 includes the steps of processing in a time sequence in the defect detection apparatus 100 shown in FIG. 1 or FIG. 2. Even if the content is omitted below, the content described above with respect to the defect detection device 100 shown in Fig. 1 or Fig. 2 can also be applied to the defect detection method of the embodiment shown in Fig. 6 and Fig. 7.

如第6圖所示，一實施列的缺陷檢測方法中缺陷檢測裝置100可以獲得直接拍攝檢驗對象而獲取的拍攝影像。缺陷檢測裝置100直接包含拍攝手段而獲得拍攝影像，或者從包含拍攝手段的其它拍攝裝置200接收拍攝影像(步驟S610)。 As shown in FIG. 6, in one embodiment of the defect detection method, the defect detection device 100 can obtain the photographed image obtained by directly photographing the inspection object. The defect detection device 100 directly includes a photographing means to obtain a photographed image, or receives a photographed image from another photographing device 200 including a photographing means (step S610).

缺陷檢測裝置100算出標準影像以及拍攝影像的差異而獲得差分影像(步驟S620)。 The defect detection device 100 calculates the difference between the standard image and the captured image to obtain a difference image (step S620).

然後缺陷檢測裝置100將標準影像以及各個拍攝影像以及對應各拍攝影像的差分影像分別轉換成8位元灰度影像(步驟S630)。標準影像、拍攝影像、差分影像中至少一部分已產生或轉換成8位元灰度影像時，對於該影像可以不實施轉換程式。例如，標準影像可能已儲存為8位元灰度影像，拍攝影像也是在拍攝裝置200中已轉換成8位元灰度影像被傳遞。 Then, the defect detection device 100 converts the standard image, each shot image, and the difference image corresponding to each shot image into 8-bit grayscale images (step S630). When at least a part of the standard image, the captured image, and the differential image has been generated or converted into an 8-bit grayscale image, the conversion program may not be implemented for the image. For example, the standard image may have been stored as an 8-bit gray-scale image, and the captured image is also converted into an 8-bit gray-scale image in the shooting device 200 to be transmitted.

進一步標準影像以及拍攝影像已經是8位元灰度影像的狀態下進行運算，進而差分影像也可以產生為8位元灰度影像。 Further, the standard image and the captured image are already 8-bit grayscale images, and the differential image can also be generated as an 8-bit grayscale image.

然後缺陷檢測裝置100可以產生將各個灰度影像作為紅、綠、藍通道的3通道24位元的彩色影像(步驟S640)。例如，拍攝影像可以形成彩色影像紅色通道，標準影像可以形成綠色通道，差分影像可以形成藍色通道，進而彩色影像可以形成對於各像素均具備紅、綠、藍亮度值的影像。 Then, the defect detection apparatus 100 can generate a 3-channel 24-bit color image with each grayscale image as the red, green, and blue channels (step S640). For example, a captured image can form a color image red channel, a standard image can form a green channel, a differential image can form a blue channel, and a color image can form an image with red, green, and blue brightness values for each pixel.

缺陷檢測裝置100利用如此產生的彩色影像實施缺陷檢測模型的學習，或者對已學習的缺陷檢測模型輸入該彩色影像而檢測缺陷存在與否(步驟S650)。 The defect detection apparatus 100 implements learning of a defect detection model using the color image generated in this way, or inputs the color image to the learned defect detection model to detect the presence or absence of a defect (step S650).

如上所述，以利用紅、綠、藍3通道的彩色影像判定缺陷存在與否而藉由灰度影像判定缺陷存在與否，缺陷以及背景之間的亮度差不大，因此解决無法正確讀取的問題。 As mentioned above, the color image of red, green, and blue is used to determine whether the defect exists or not, and the gray image is used to determine whether the defect exists or not. The brightness difference between the defect and the background is not large, so it is solved that it cannot be read correctly. The problem.

另一方面，在上述的缺陷檢測方法中，作為可以進一步提升缺陷檢測性能的具體學習或缺陷檢測方法，根據其它實施例的缺陷檢測方法中缺陷檢測裝置100首先可以確認彩色影像中缺陷可疑位置(步驟S710)。為此各個學習對象彩色影像中已標記缺陷可疑位置，可以利用彩色影像使缺陷檢測模型檢測出缺陷可疑位置地學習。 On the other hand, in the above-mentioned defect detection method, as a specific learning or defect detection method that can further improve the defect detection performance, the defect detection device 100 in the defect detection method according to other embodiments can first confirm the suspicious location of the defect in the color image ( Step S710). For this reason, the suspicious location of the marked defect in the color image of each learning object can be learned by using the color image to make the defect detection model detect the suspicious location of the defect.

缺陷檢測裝置100可以獲得對應缺陷可疑位置的裁剪影像(步驟S720)。實施缺陷檢測模型的學習時，裁剪影像可以被設定為在彩色影像中被使用者選擇的區域。或者利用預先可檢測缺陷可疑位置地學習的缺陷檢測模型先檢測出缺陷可疑位置以後，以缺陷可疑位置為中心獲得裁剪影像。 The defect detection device 100 can obtain a cropped image corresponding to the suspicious location of the defect (step S720). When implementing the learning of the defect detection model, the cropped image can be set as the area selected by the user in the color image. Or a defect detection model that is learned in advance to detect the suspicious location of the defect is used to first detect the suspicious location of the defect, and then obtain a cropped image with the suspicious location as the center.

隨之缺陷檢測裝置100利用獲得的裁剪影像學習缺陷存在與否或類型，或者檢測出缺陷。 Subsequently, the defect detection device 100 uses the obtained cropped image to learn the existence or type of the defect, or to detect the defect.

為此根據實施例，缺陷檢測裝置100可以選擇對應缺陷位置的檢測文件(步驟S730)。按區域利用不同的檢測模型時，缺陷檢測裝置100調出與缺陷位置所屬區域對應的檢測模型被儲存的檢測檔案，利用調出的檢測檔案的檢測模型學習裁剪影像或者分析裁剪影像而判定缺陷(步驟S740)。 To this end, according to an embodiment, the defect inspection apparatus 100 may select an inspection file corresponding to the defect position (step S730). When using different inspection models by area, the defect inspection device 100 calls up the inspection file in which the inspection model corresponding to the area of the defect location belongs, and uses the inspection model of the called inspection file to learn or analyze the cut image to determine the defect ( Step S740).

實施缺陷檢測模型的學習時，缺陷檢測裝置100利用對應缺陷可疑位置的檢測模型學習對應缺陷可疑位置的裁剪影像。此時各裁剪對象中可以標記缺陷存在與否及/或缺陷類型，基此於，與該缺陷可疑位置對應的檢測模型可以學習缺陷存在與否或類型。 When implementing the learning of the defect detection model, the defect detection device 100 uses the detection model corresponding to the suspicious location of the defect to learn the cropped image corresponding to the suspicious location of the defect. At this time, the presence or absence of defects and/or defect types can be marked in each cropped object. Based on this, the detection model corresponding to the suspicious location of the defect can learn the presence or absence or type of the defect.

利用缺陷檢測模型判定檢驗對象的缺陷時，缺陷檢測裝置100調出對應缺陷可疑位置的檢測檔案判定缺陷存在與否，並將以缺陷可疑位置為中心獲得的裁剪影像輸入到缺陷檢測模型而判定缺陷存在與否或缺陷類型。 When the defect inspection model is used to determine the defect of the inspection object, the defect inspection device 100 calls the inspection file corresponding to the suspicious location of the defect to determine whether the defect exists or not, and inputs the cropped image obtained from the suspicious location of the defect as the center to the defect inspection model to determine the defect Existence or defect type.

如上所述，缺陷檢測裝置100可以利用彩色影像實施檢測缺陷可疑位置的學習，利用以缺陷可疑位置為中心獲得的裁剪影像實施學習具體缺陷存在與否或類型的兩步驟學習。檢測缺陷時，缺陷檢測裝置100也可以實施先利用彩色影像提取缺陷可疑位置以後，以提取的缺陷可疑位置為中心獲得裁剪影像後判定具體缺陷存在與否或類型的兩步驟檢測過程。 As described above, the defect detection apparatus 100 can use color images to implement learning to detect the suspicious location of the defect, and use cropped images obtained centering on the suspicious defect location to implement a two-step learning of the existence or type of a specific defect. When detecting a defect, the defect detection device 100 may also implement a two-step detection process of first extracting the suspicious location of the defect by using the color image, and then obtaining a cropped image with the extracted suspicious location as the center, and then determining the existence or type of the specific defect.

進一步，缺陷檢測裝置100實施根據缺陷位置不同的檢測模型的學習，進而根據印刷電路板等檢驗對象的特性，對每個缺陷可疑位置建立不同的判定標準，用此靈活且合理地實施缺陷檢測。 Furthermore, the defect detection device 100 implements learning of different detection models according to different defect positions, and then establishes different judgment standards for each suspicious position of defects according to the characteristics of inspection objects such as printed circuit boards, and uses this to implement defect detection flexibly and reasonably.

以上實施例中使用的所謂「~部」的用語是指軟體或FPGA(field Programmable gate array)或者ASIC等硬體構成要素，「~部」執行某種角色。但「~部」的意義並不限於軟體或硬體。「~部」可以在可處理的儲存媒體上配置，也可以使一個或其以上的多個處理器再生地配置。作為一例，「~部」包含軟體構成要素、面向對象軟體構成要素、類構成要素以及任務構成要素等多個構成要素、多個處理器、多個函數、多個屬性、多個程式、多個子程式、程式專利代碼的多個段、多個驅動器、韌體、微指令、電路、資料、資料庫、多個資料結構、多個表格、多個陣列及多個變量。 The term "~ part" used in the above embodiments refers to hardware components such as software or FPGA (field Programmable gate array) or ASIC, and the "~ part" performs a certain role. But the meaning of "~部" is not limited to software or hardware. The "~ part" can be placed on a storage medium that can be processed, or one or more processors can be reproduced. As an example, "~" includes software components, object-oriented software components, class components, and task components, multiple components, multiple processors, multiple functions, multiple attributes, multiple programs, multiple sub-elements, etc. Programs, multiple segments of program patent codes, multiple drivers, firmware, micro-commands, circuits, data, databases, multiple data structures, multiple tables, multiple arrays, and multiple variables.

多個構成要素以及多個「~部」中提供的功能是，可與更少數的構成要素以及多個「~部」結合，或者與追加的多個構成要素以及「~部」分離。 The functions provided in the multiple constituent elements and multiple "~parts" can be combined with a smaller number of constituent elements and multiple "~parts", or separated from the added multiple constituent elements and "~parts".

不僅如此，多個構成要素以及多個「~部」是可以使設備或安全多媒體卡內的一個或其以上的多個CPU再生地實現。 Not only that, but multiple constituent elements and multiple "~parts" can be realized by reproducing one or more CPUs in the device or the secure multimedia card.

根據第6圖及第7圖說明的實施例的缺陷檢測方法是可以以儲存可在電腦上運行的命令的資料並且可以用電腦讀取的媒體形態實現。命令以及資料可以儲存為程式碼形態，藉由處理器處理時產生既定的程度模組而執行既定的運行。電腦可讀媒體是電腦可以訪問的任意可用媒體，包含揮發性以及非揮發性媒體、分離型以及非分離型媒體。電腦可讀媒體可以包含電腦記錄媒體。電腦記錄媒體包含電腦可讀命令、資料結構、程式模組或者其它以資料等訊息儲存為目的的任意方法或技術實現的揮發性以及非揮發性、分離型以及非分離型媒體。例如，電腦記錄媒體可能是HDD以及SSD等磁性儲存媒體，CD、DVD 以及藍光光碟等光學記錄媒體，或者藉由網絡可以訪問的伺服器中包含的儲存器。 The defect detection method according to the embodiment illustrated in Figs. 6 and 7 can be implemented in the form of a medium that can store data of commands that can be run on a computer and can be read by a computer. Commands and data can be stored in the form of program code, and a predetermined degree module is generated when the processor is processed to execute a predetermined operation. Computer-readable media is any available media that the computer can access, including volatile and non-volatile media, separate and non-separable media. The computer-readable medium may include a computer recording medium. Computer recording media include computer-readable commands, data structures, program modules, or other volatile, non-volatile, discrete, and non-separable media realized by any method or technology for the purpose of storing information such as data. For example, computer recording media may be magnetic storage media such as HDD and SSD, CD, DVD And optical recording media such as Blu-ray Discs, or storage included in servers accessible via the Internet.

藉由第6圖及第7圖說明的實施例的缺陷檢測方法可以藉由包含可用電腦運行的命令的電腦程式(或者電腦程式產品)實現。電腦程式包含藉由處理器處理的可編程的命令，可以用高級編程語言(High-level Programming Language)、面向對象編程語言(Object-oriented Programming Language)、匯編語言或機械語言等實現。電腦程式可以記錄在電腦可讀記錄媒體(例如儲存器、硬碟、磁/光媒體或SSD(Solid-State Drive))上。 The defect detection method of the embodiment described in FIGS. 6 and 7 can be implemented by a computer program (or a computer program product) that includes commands that can be run by a computer. Computer programs include programmable commands processed by the processor, and can be implemented in high-level programming language, object-oriented programming language, assembly language, or mechanical language. The computer program can be recorded on a computer-readable recording medium (such as a memory, a hard disk, a magnetic/optical medium, or an SSD (Solid-State Drive)).

進而藉由第6圖及第7圖說明的實施例的缺陷檢測方法可以藉由如上所述的電腦程式在計算設備上運行而實現。計算設備包含處理器、儲存器、儲存裝置、與儲存器以及高速擴充埠連接的高速連接埠、與低速匯流排以及儲存裝置連接的低速連接埠中的至少一部分。所述成分分別利用各種匯流排相互連接，可以安裝在通用主機板或者用其它適當方式裝配。 Furthermore, the defect detection method of the embodiment illustrated in FIGS. 6 and 7 can be implemented by running the computer program as described above on a computing device. The computing device includes at least a part of a processor, a storage, a storage device, a high-speed port connected to the storage and a high-speed expansion port, and a low-speed port connected to a low-speed bus and the storage device. The components are connected to each other by various bus bars, and can be installed on a general-purpose motherboard or assembled in other suitable ways.

處理器可在計算設備內處理命令，所述命令例如是，如連接高速連接埠上的顯示器，為了顯示在外部輸入輸出設備上提供GUI(Graphic User Interface)所需的圖形訊息而儲存在儲存器或儲存裝置上的命令。另一實施例是，多個處理器及(或者)多個總線可以適當地與多個儲存器以及儲存形態一起被使用。處理器可以用由包含獨立的多個模擬及(或)數字處理器的多個晶片組成的晶片組實現。 The processor can process commands in the computing device. The commands are, for example, connected to a display on a high-speed port, and are stored in the memory in order to display the graphic information required by the GUI (Graphic User Interface) on the external input and output device. Or a command on the storage device. In another embodiment, multiple processors and/or multiple buses can be used with multiple memories and storage forms as appropriate. The processor can be implemented by a chipset consisting of multiple chips including independent multiple analog and/or digital processors.

儲存器是在計算設備內儲存訊息。一例是，儲存器可以由揮發性儲存單元或其集合組成。另一例是，儲存器可以由非揮發性儲存單元或其集合組成。儲存器也可以是磁碟或光碟等其它形態的電腦可讀媒體。 The storage is to store information in the computing device. For one example, the storage may consist of volatile storage units or a collection thereof. Another example is that the storage can be composed of non-volatile storage units or a collection thereof. The storage may also be other forms of computer readable media such as floppy disks or optical discs.

儲存裝置可以給計算設備提供大容量儲存空間。儲存裝置可以是電腦可讀媒體，或者包含這些媒體組成。例如，可以包含SAN(Storage Area Network)內的多個裝置或其它組件，可以是硬碟驅動器、硬碟裝置、光碟裝置或磁帶裝置、快閃、與其類似的其它半導體儲存設備或設備陣列。 The storage device can provide a large-capacity storage space for the computing device. The storage device may be a computer-readable medium or a composition containing these media. For example, it can contain SAN (Storage Area Multiple devices or other components in the Network) can be hard disk drives, hard disk devices, optical disk devices or tape devices, flash, and other similar semiconductor storage devices or device arrays.

以上實施例僅用以說明本發明的技術方案，而非對其限制；儘管參照前述實施例對本發明進行了詳細的說明，本領域的具有通常知識者應當理解：其依然可以對前述各實施例所述的技術方案進行修改，而這些修改，並不使相應技術方案的本質脫離本發明各實施例所述技術方案的範圍。例如，描述單一型的各個構成要素可以分散實施，同樣描述為分散的構成要素也可以以結合形態實現。 The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still compare the foregoing embodiments. The described technical solutions are modified, and these modifications do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions described in the embodiments of the present invention. For example, various constituent elements described as a single type can be implemented in a dispersed manner, and constituent elements described as dispersed can also be implemented in a combined form.

本發明的專利範圍應根據下述的發明申請專利範圍進行解釋，而且在其同等範圍內的任何變更或修改等應都屬本發明的專利範圍。 The patent scope of the present invention should be interpreted according to the following patent application scope of the invention, and any change or modification within its equivalent scope shall belong to the patent scope of the present invention.

100:缺陷檢測裝置 100: Defect detection device

110:輸入輸出部 110: Input and output department

120:儲存部 120: Storage Department

130:通訊部 130: Ministry of Communications

140:控制部 140: Control Department

200:拍攝裝置 200: Camera

Claims (14)

一種缺陷檢測裝置，係作為檢測檢驗對象的缺陷的裝置，其包含：一儲存部，儲存預定的一個標準影像以及拍攝檢驗對象獲得的至少一個拍攝影像；以及一控制部，運算該標準影像以及各該拍攝影像的差異，產生對應各該拍攝影像的一差分影像，將該標準影像、各該拍攝影像以及各該差分影像合成，產生對應各該拍攝影像的至少一個彩色影像，以及在該彩色影像中標記出缺陷的位置，藉此實施缺陷檢測模型的學習，或將該彩色影像輸入至已學習的缺陷檢測模型而在該彩色影像中提取缺陷可疑位置，藉此實施缺陷的檢測。 A defect detection device, as a device for detecting defects of an inspection object, comprises: a storage part storing a predetermined standard image and at least one shot image obtained by shooting the inspection object; and a control part calculating the standard image and each The difference of the shot images generates a difference image corresponding to each shot image, the standard image, each shot image, and each difference image are synthesized to generate at least one color image corresponding to each shot image, and the color image Mark the position of the defect in, thereby implementing the learning of the defect detection model, or input the color image to the learned defect detection model and extract the suspicious location of the defect from the color image, thereby implementing the defect detection. 如申請專利範圍第1項所述的缺陷檢測裝置，其中該控制部將該標準影像、各該拍攝影像、以及對應各該拍攝影像產生的該差分影像形成為8位元灰度影像。 According to the defect detection device described in item 1 of the scope of patent application, the control unit forms the standard image, each of the photographed images, and the differential image generated corresponding to each of the photographed images into 8-bit grayscale images. 如申請專利範圍第2項所述的缺陷檢測裝置，其中該控制部產生將以8位元灰度影像形成的該標準影像、各該拍攝影像以及對應各該拍攝影像產生的該差分影像作為相互不同的3個彩色通道的3通道24位元彩色影像。 For the defect detection device described in item 2 of the scope of patent application, wherein the control unit generates the standard image formed with 8-bit grayscale images, each of the shot images, and the difference image generated corresponding to each of the shot images as mutual 3-channel 24-bit color image with 3 different color channels. 如申請專利範圍第1項所述的缺陷檢測裝置，其中該控制部利用該彩色影像實施缺陷檢測模型的學習時，該控制部具有兩個對缺陷檢測模型的學習步驟，第一步驟係在該彩色影像中可檢測出缺陷可疑位置，第二步驟係在缺陷可疑位置的一定區域的一裁剪影像中，標記缺陷存在與否或缺陷類型。 For the defect detection device described in item 1 of the scope of patent application, when the control unit uses the color image to implement the learning of the defect detection model, the control unit has two learning steps for the defect detection model, and the first step is in the The suspicious location of the defect can be detected in the color image. The second step is to mark the existence of the defect or the type of the defect in a cropped image of a certain area of the suspicious location of the defect. 如申請專利範圍第4項所述的缺陷檢測裝置，其中該儲存部係儲存使學習根據位置而不同的缺陷檢測模型所需的各個檢測檔案；該控 制部係利用與該彩色影像的缺陷可疑位置對應的該檢測檔案實施該裁剪影像的學習。 For example, the defect detection device described in item 4 of the scope of patent application, wherein the storage unit stores various detection files required for learning different defect detection models according to the location; the control The manufacturing department uses the detection file corresponding to the suspicious location of the color image to implement the learning of the cropped image. 如申請專利範圍第1項所述的缺陷檢測裝置，其中該控制部利用該彩色影像實施缺陷檢測時，係從該彩色影像中檢測缺陷可疑位置，利用將檢測出的缺陷可疑位置的一定區域影像裁剪而獲得一裁剪影像實施缺陷存在與否或缺陷類型的判定。 For example, the defect detection device described in item 1 of the scope of patent application, wherein when the control unit uses the color image to implement defect detection, it detects the suspicious position of the defect from the color image, and uses the image of a certain area of the detected suspicious position of the defect Crop to obtain a cropped image to determine whether the defect exists or not or the type of the defect. 如申請專利範圍第6項所述的缺陷檢測裝置，其中該儲存部係儲存利用根據位置而不同的缺陷檢測模型實施缺陷檢測的各個檢測檔案；該控制部係利用對應檢測出的缺陷可疑位置的該檢測檔案判定該裁剪影像的缺陷存在與否或缺陷類型。 For example, the defect detection device described in item 6 of the scope of patent application, wherein the storage unit stores various inspection files for defect detection using different defect detection models according to the location; the control unit uses the information corresponding to the suspicious location of the detected defect The inspection file determines whether the defect of the cropped image exists or not or the defect type. 一種缺陷檢測方法，係作為藉由缺陷檢測裝置檢測檢驗對象的缺陷的方法，其包含：獲得拍攝檢驗對象產生的至少一個拍攝影像的步驟；利用各該拍攝影像以及預定的一個標準影像之間的差異產生對應各該拍攝影像的一差分影像的步驟；將該標準影像、各該拍攝影像以及各該差分影像合成而產生對應各該拍攝影像的至少一個彩色影像的步驟；以及在該彩色影像中標記出有缺陷的位置，藉此實施缺陷檢測模型學習的步驟，或將該彩色影像輸入至已學習的缺陷檢測模型而在該彩色影像中提取缺陷可疑位置，藉此實施缺陷檢測的步驟。 A defect detection method, as a method for detecting defects of an inspection object by a defect detection device, includes: obtaining at least one photographed image generated by photographing the inspection object; using the difference between each of the photographed images and a predetermined standard image The step of generating a difference image corresponding to each shot image by difference; the step of synthesizing the standard image, each shot image, and each difference image to generate at least one color image corresponding to each shot image; and in the color image Mark the defect location, thereby implementing the step of defect detection model learning, or input the color image into the learned defect detection model and extract the suspicious location of the defect from the color image, thereby implementing the step of defect detection. 如申請專利範圍第8項所述的缺陷檢測方法，其中該彩色影像的產生步驟包含：將該標準影像、各該拍攝影像以及對應各該拍攝影像產生的該差分影像形成為各8位元灰度影像的步驟；以及 產生將以8位元灰度影像形成的該標準影像、各該拍攝影像以及對應各該拍攝影像產生的該差分影像作為相互不同的3個彩色通道的3通道24位元彩色影像的步驟。 According to the defect detection method described in item 8 of the scope of patent application, the step of generating the color image includes: forming the standard image, each of the shot images, and the difference image generated corresponding to each of the shot images into 8-bit grays The steps to obtain an image; and The step of generating the standard image formed by the 8-bit grayscale image, each of the shot images, and the difference image generated corresponding to each of the shot images as a 3-channel 24-bit color image of three different color channels. 如申請專利範圍第9項所述的缺陷檢測方法，其中利用產生的該彩色影像實施缺陷檢測模型的學習或缺陷檢測包含：從該彩色影像中檢測缺陷可疑位置的步驟；將檢測的缺陷可疑位置的一定區域影像裁剪而獲得一裁剪影像的步驟；以及在該裁剪影像中標記出缺陷存在與否或缺陷的類型，藉此實施缺陷檢測模型學習的步驟，或將該裁剪影像輸入至已學習的缺陷檢測模型而在該裁剪影像中判斷缺陷存在與否或缺陷的類型，藉此實施缺陷判定的步驟。 For example, the defect detection method described in item 9 of the scope of patent application, wherein the use of the generated color image to implement the learning of the defect detection model or the defect detection includes: the step of detecting the suspicious position of the defect from the color image; and the step of detecting the suspicious position of the defect The step of cropping a certain area of the image to obtain a cropped image; and marking the presence or absence of the defect or the type of the defect in the cropped image, thereby implementing the step of learning the defect detection model, or inputting the cropped image to the learned The defect detection model is used to determine the presence or absence of the defect or the type of the defect in the cropped image, thereby implementing the step of defect determination. 如申請專利範圍第10項所述的缺陷檢測方法，其中利用產生的該彩色影像實施缺陷檢測模型的學習或缺陷檢測包含：實施利用該彩色影像檢測缺陷可疑位置所需學習的步驟。 For the defect detection method described in item 10 of the scope of patent application, the use of the generated color image to implement the learning of the defect detection model or the defect detection includes: implementing the step of learning required to use the color image to detect the suspicious location of the defect. 如申請專利範圍第10項所述的缺陷檢測方法，其中該缺陷檢測方法係進一步包含：保持根據位置而採用不同的缺陷檢測模型的步驟；在該裁剪影像中標記出缺陷存在與否或缺陷的類型，藉此實施缺陷檢測模型學習的步驟，或將該裁剪影像輸入至已學習的缺陷檢測模型而在該裁剪影像中判斷缺陷存在與否或缺陷的類型，藉此實施缺陷判定的步驟進一步包含：利用與檢測的缺陷可疑位置對應的缺陷檢測模型實施利用該裁剪影像的缺陷存在與否或缺陷類型的學習或判定的步驟。 For example, the defect detection method described in item 10 of the scope of patent application, wherein the defect detection method further includes the step of maintaining different defect detection models according to the position; marking the presence or absence of defects or defects in the cropped image Type, by which the step of learning the defect detection model is implemented, or the cropped image is input to the learned defect detection model and the presence or absence of the defect or the type of the defect is determined in the cropped image, whereby the step of implementing the defect determination further includes : Use the defect detection model corresponding to the suspicious position of the detected defect to implement the step of learning or judging whether the defect exists or not or the defect type of the cropped image. 一種記錄如申請專利範圍第8項所述方法的實施程式的電腦可讀記錄媒體。 A computer-readable recording medium that records the implementation program of the method described in item 8 of the scope of patent application. 一種藉由缺陷檢測裝置實施且為實施如申請專利範圍第8項所述方法而儲存在媒體中的電腦程式。 A computer program implemented by a defect detection device and stored in a medium for implementing the method described in item 8 of the scope of the patent application.
TW108122230A 2018-10-11 2019-06-25 Apparatus and method for inspecting for defects TWI718573B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020180120853A KR102175286B1 (en) 2018-10-11 2018-10-11 Apparatus and method for inspecting for defects
KR10-2018-0120853 2018-10-11

Publications (2)

Publication Number Publication Date
TW202014985A TW202014985A (en) 2020-04-16
TWI718573B true TWI718573B (en) 2021-02-11

Family

ID=66959925

Family Applications (1)

Application Number Title Priority Date Filing Date
TW108122230A TWI718573B (en) 2018-10-11 2019-06-25 Apparatus and method for inspecting for defects

Country Status (3)

Country Link
KR (1) KR102175286B1 (en)
CN (1) CN109916906B (en)
TW (1) TWI718573B (en)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10977509B2 (en) 2017-03-27 2021-04-13 Samsung Electronics Co., Ltd. Image processing method and apparatus for object detection
KR102252592B1 (en) * 2019-08-16 2021-05-17 라온피플 주식회사 Apparatus and method for inspecting substrate defect
CN110487803A (en) * 2019-08-20 2019-11-22 Oppo（重庆）智能科技有限公司 The defect inspection method and device of infrared light-emitting component
CN110763699B (en) * 2019-10-12 2022-12-20 广州兴森快捷电路科技有限公司 Analysis method for inner layer interconnection defects of circuit board and circuit board
CN110726724A (en) * 2019-10-22 2020-01-24 北京百度网讯科技有限公司 Defect detection method, system and device
WO2021090395A1 (en) * 2019-11-06 2021-05-14 株式会社Ｆｕｊｉ Image processing device, component mounting system, and image processing method
CN111008961B (en) * 2019-11-25 2021-10-19 深圳供电局有限公司 Transmission line equipment defect detection method and system, equipment and medium thereof
JP2021089219A (en) * 2019-12-05 2021-06-10 東洋製罐グループホールディングス株式会社 Image inspection system and image inspection method
CN113516612B (en) * 2020-04-13 2022-06-14 阿里巴巴集团控股有限公司 Data processing method, device, equipment and storage medium
CN111507975B (en) * 2020-04-23 2022-02-08 西南交通大学 Method for detecting abnormity of outdoor insulator of traction substation
CN111754456B (en) * 2020-05-15 2022-10-14 清华大学 Two-dimensional PCB appearance defect real-time automatic detection technology based on deep learning
US11423577B2 (en) * 2020-07-08 2022-08-23 International Business Machines Corporation Printed circuit board assembly defect detection
TWI770561B (en) * 2020-07-22 2022-07-11 鴻海精密工業股份有限公司 Product defect detection method, computer device and storage medium
CN112085722B (en) * 2020-09-07 2024-04-09 凌云光技术股份有限公司 Training sample image acquisition method and device
CN112184689B (en) * 2020-10-12 2024-06-25 罗建华 Semiconductor device detection method and device, intelligent terminal and storage medium
KR102505484B1 (en) * 2020-12-18 2023-03-03 한국공학대학교산학협력단 Method and apparatus for inspecting tire appearance based on deep learning
CN113111828B (en) * 2021-04-23 2022-03-18 中国科学院宁波材料技术与工程研究所 Three-dimensional defect detection method and system for bearing
KR102394093B1 (en) * 2021-09-13 2022-05-04 오병후 Method of reading defects using X-ray images of helicopter blades
KR102471441B1 (en) 2021-12-20 2022-11-28 주식회사 아이코어 Vision inspection system for detecting failure based on deep learning
CN114466183A (en) * 2022-02-21 2022-05-10 江东电子材料有限公司 Copper foil flaw detection method and device based on characteristic spectrum and electronic equipment
CN114862856B (en) * 2022-07-07 2022-10-11 成都数之联科技股份有限公司 Panel defect area identification method and device, electronic equipment and medium
CN115115625B (en) * 2022-08-26 2022-11-04 聊城市正晟电缆有限公司 Cable production abnormity detection method based on image processing
WO2024049201A1 (en) * 2022-08-31 2024-03-07 주식회사 엘지에너지솔루션 Device and method for inspecting battery electrode
KR102610783B1 (en) * 2022-11-03 2023-12-07 시냅스이미징(주) Deep learning inspection system implementing image preprocessing of CAD image

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200414488A (en) * 2002-05-17 2004-08-01 Gsi Lumonics Corp Method and system for marking a workpiece such as a semiconductor wafer and laser marker for use therein
US20070050712A1 (en) * 2005-08-23 2007-03-01 Hull Jonathan J Visibly-Perceptible Hot Spots in Documents
CN103234476A (en) * 2013-04-01 2013-08-07 廖怀宝 Method for identifying object two-dimensional outlines
CN104363843A (en) * 2012-04-09 2015-02-18 伊西康内外科公司 Rotatable electrical connection for ultrasonic surgical instruments
CN106530284A (en) * 2016-10-21 2017-03-22 广州视源电子科技股份有限公司 Welding spot type detection and device based on image recognition

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3344338B2 (en) * 1997-12-25 2002-11-11 日本電気株式会社 Image defect detection apparatus and defect detection method
US8223327B2 (en) * 2009-01-26 2012-07-17 Kla-Tencor Corp. Systems and methods for detecting defects on a wafer
JP5846100B2 (en) * 2012-11-01 2016-01-20 三菱電機株式会社 Display device defect inspection method
JP6395481B2 (en) * 2014-07-11 2018-09-26 キヤノン株式会社 Image recognition apparatus, method, and program
JP2016212539A (en) 2015-05-01 2016-12-15 富士通株式会社 Information processor, defective part determination program, and defective part determination method
CN105894514B (en) * 2016-04-06 2019-01-11 广东工业大学 A kind of printed matter defect inspection method and system based on GPU concurrent operation
CN108896550B (en) * 2018-03-30 2021-10-22 湖北工程学院 Facial mask printing quality detection method and system
CN108596892B (en) * 2018-04-23 2020-08-18 西安交通大学 Weld defect identification method based on improved LeNet-5 model
CN109084957B (en) * 2018-08-31 2024-03-19 华南理工大学 Defect detection and color sorting method and system for photovoltaic solar crystalline silicon cell

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200414488A (en) * 2002-05-17 2004-08-01 Gsi Lumonics Corp Method and system for marking a workpiece such as a semiconductor wafer and laser marker for use therein
US20070050712A1 (en) * 2005-08-23 2007-03-01 Hull Jonathan J Visibly-Perceptible Hot Spots in Documents
CN104363843A (en) * 2012-04-09 2015-02-18 伊西康内外科公司 Rotatable electrical connection for ultrasonic surgical instruments
CN103234476A (en) * 2013-04-01 2013-08-07 廖怀宝 Method for identifying object two-dimensional outlines
CN106530284A (en) * 2016-10-21 2017-03-22 广州视源电子科技股份有限公司 Welding spot type detection and device based on image recognition

Also Published As

Publication number Publication date
CN109916906B (en) 2022-10-14
KR20200046137A (en) 2020-05-07
KR102175286B1 (en) 2020-11-06
CN109916906A (en) 2019-06-21
TW202014985A (en) 2020-04-16

Similar Documents

Publication Publication Date Title
TWI718573B (en) Apparatus and method for inspecting for defects
CN107810505B (en) Machine learning of real-time image capture parameters
US9697416B2 (en) Object detection using cascaded convolutional neural networks
US10740912B2 (en) Detection of humans in images using depth information
TWI590650B (en) Object tracking in encoded video streams
US10430694B2 (en) Fast and accurate skin detection using online discriminative modeling
US9070044B2 (en) Image adjustment
WO2021115345A1 (en) Image processing method and apparatus, computer device, and storage medium
JP2012032370A (en) Defect detection method, defect detection apparatus, learning method, program, and recording medium
US20150039637A1 (en) Systems Apparatus and Methods for Determining Computer Apparatus Usage Via Processed Visual Indicia
US10068616B2 (en) Thumbnail generation for video
TWI743837B (en) Training data increment method, electronic apparatus and computer-readable medium
US20200204864A1 (en) Classification of images based on static components
CN110858277A (en) Method and device for obtaining attitude classification model
CN114461986A (en) Method for training identification model and method and device for image identification
KR102375547B1 (en) Apparatus for Semi-supervised Learning Artificial Intelligence and Driving Method Thereof
CN111581679A (en) Method for preventing screen from shooting based on deep network
US20210012511A1 (en) Visual search method, computer device, and storage medium
US20170091572A1 (en) System And Method For Text Detection In An Image
JP2011186595A (en) Information generation device, object detection device, information generation method, object detection method, and program
CN112752110B (en) Video presentation method and device, computing device and storage medium
CN113269730A (en) Image processing method, image processing device, computer equipment and storage medium
KR102252590B1 (en) Method and apparatus for setting test environment based on machine learning environment
US20230386056A1 (en) Systems and techniques for depth estimation
CN113117341B (en) Picture processing method and device, computer readable storage medium and electronic equipment