WO2021192627A1 - Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method - Google Patents

Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method Download PDF

Info

Publication number
WO2021192627A1
WO2021192627A1 PCT/JP2021/003830 JP2021003830W WO2021192627A1 WO 2021192627 A1 WO2021192627 A1 WO 2021192627A1 JP 2021003830 W JP2021003830 W JP 2021003830W WO 2021192627 A1 WO2021192627 A1 WO 2021192627A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
learning
inspection
detection target
learning data
Prior art date
Application number
PCT/JP2021/003830
Other languages
French (fr)
Japanese (ja)
Inventor
学 梅田
Original Assignee
株式会社Ｌｉｘｉｌ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2020051136A external-priority patent/JP7449739B2/en
Priority claimed from JP2020051135A external-priority patent/JP2021148719A/en
Application filed by 株式会社Ｌｉｘｉｌ filed Critical 株式会社Ｌｉｘｉｌ
Publication of WO2021192627A1 publication Critical patent/WO2021192627A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present disclosure relates to a technique for inspecting an inspection target, and in particular, an inspection system for inspecting an inspection target, a learning device available for the inspection system, a learning program, a learning method, and an inspection device available for the inspection system. Regarding inspection programs and inspection methods.
  • Patent Document 1 In order to automatically detect such a defect, a technique for determining the surface state from the captured image of the inspection object has been proposed (see, for example, Patent Document 1).
  • the present disclosure is made in view of such a problem, and the purpose thereof is to improve the accuracy of inspecting the inspection target.
  • the inspection system of a certain aspect of the present disclosure uses a learning device that learns a detector for detecting a detection target site to be inspected, and a detector learned by the learning device. It is provided with an inspection device for detecting a detection target site to be inspected.
  • the learning device acquires learning data from a learning image acquisition unit that acquires an image of the inspection target including the detection target part and an image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit. It includes a learning data generation unit to be generated, and a learning unit to learn a detector using the learning data generated by the learning data generation unit.
  • the learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, generates learning data from each of the divided images of the detection target parts, and the inspection device uses the captured image of the inspection target. It is provided with an inspection image acquisition unit for acquiring the data, and a detection unit for detecting a detection target portion from the captured image acquired by the inspection image acquisition unit using a detector.
  • Another aspect of the present disclosure is a learning device.
  • This device acquires learning data from the image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit and the learning image acquisition unit that acquires the captured image of the inspection target including the detection target part. It includes a learning data generation unit to be generated, and a learning unit to learn a detector for detecting a detection target site to be inspected by using the learning data generated by the learning data generation unit.
  • the learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, and generates learning data from each of the divided images of the detection target parts.
  • Yet another aspect of the disclosure is a learning program.
  • This program uses a computer from a learning image acquisition unit that acquires an image of an inspection target including a detection target part and an image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit. It functions as a learning data generation unit that generates training data and a learning unit that learns a detector for detecting a detection target part to be inspected by using the learning data generated by the training data generation unit.
  • the learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, and generates learning data from each of the divided images of the detection target parts.
  • Yet another aspect of the disclosure is a learning method.
  • This method is generated by a computer with a step of acquiring an image of the inspection target including the detection target part and a step of generating learning data from the image of the detection target part included in the acquired image of the inspection target.
  • the step of learning the detector for detecting the detection target part to be inspected and the step of learning are executed.
  • the learning step at least a part of the images of the plurality of detection target parts is divided into a plurality of images, and learning data is generated from each of the divided images of the detection target parts.
  • Yet another aspect of the present disclosure is an inspection device.
  • This device was trained using an inspection image acquisition unit that acquires an image to be inspected and learning data generated by dividing an image of a detection target part included in the image to be inspected into a plurality of images. It includes a detection unit that detects a detection target site from an captured image acquired by an inspection image acquisition unit using a detector.
  • Yet another aspect of the disclosure is an inspection program.
  • This program uses a computer with an inspection image acquisition unit that acquires an image to be inspected, and learning data generated by dividing an image of a detection target part included in the image to be inspected into a plurality of images. Using the learned detector, it functions as a detection unit that detects a detection target site from the captured image acquired by the inspection image acquisition unit.
  • Yet another aspect of the present disclosure is an inspection method.
  • the detection is learned by using the step of acquiring the captured image of the inspection target and the learning data generated by dividing the image of the detection target part included in the captured image of the inspection target into a plurality of images on a computer.
  • the step of detecting the detection target site from the acquired captured image is executed.
  • a learning device that learns a detector for detecting a detection target site to be inspected and a detector learned by the learning device are used to detect the detection target site to be inspected. It is equipped with an inspection device for detecting.
  • the learning device converts the image to be captured before generating learning data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the learning image acquisition unit.
  • An image processing unit that performs image processing, a learning data generation unit that generates learning data from the image of the detection target part processed by the image processing unit, and a detector using the training data generated by the learning data generation unit. It has a learning department for learning.
  • the inspection device includes an inspection image acquisition unit that acquires a captured image of an inspection target, and a detection unit that detects a detection target portion from the captured image acquired by the inspection image acquisition unit using a detector.
  • Yet another aspect of the present disclosure is a learning device.
  • This device converts the image to be captured before generating learning data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the image acquisition unit for learning.
  • An inspection target using an image processing unit that performs image processing, a learning data generation unit that generates training data from an image of a detection target part processed by the image processing unit, and training data generated by the training data generation unit. It is provided with a learning unit for learning a detector for detecting a detection target portion of the above.
  • Yet another aspect of the disclosure is a learning program.
  • This program uses the computer to generate training data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the image acquisition unit for learning.
  • a learning data generation unit that generates learning data from the image of the detection target part processed by the image processing unit, and learning data generated by the training data generation unit.
  • It functions as a learning unit that learns a detector for detecting a detection target part to be inspected.
  • Yet another aspect of the disclosure is a learning method.
  • This method includes a step of acquiring a captured image of an inspection target including a detection target site on a computer, and a step of performing image processing on the captured image before generating learning data from the acquired captured image of the inspection target.
  • a step of generating training data from the processed image of the detection target part and a step of learning a detector for detecting the detection target part of the inspection target using the generated training data are executed.
  • Yet another aspect of the present disclosure is an inspection device.
  • This device performs image processing on the captured image before detecting the detection target portion from the inspection image acquisition unit that acquires the captured image of the inspection target and the captured image of the inspection target acquired by the inspection image acquisition unit. Detected from the captured image acquired by the inspection image acquisition unit using the image processing unit and the detector learned using the learning data generated from the image of the detection target part included in the captured image to be inspected. It is provided with a detection unit for detecting a target site.
  • Yet another aspect of the disclosure is an inspection program.
  • This program displays an image on the captured image before detecting the detection target part from the inspection image acquisition unit that acquires the captured image of the inspection target and the captured image of the inspection target acquired by the inspection image acquisition unit.
  • Yet another aspect of the present disclosure is an inspection method.
  • This method includes a step of acquiring an image to be inspected by a computer, a step of performing image processing on the image to be detected before detecting a part to be detected from the acquired image of the object to be inspected, and an image of the image to be inspected.
  • the step of detecting the detection target site from the acquired captured image is executed.
  • the learning device learns a defect detector for detecting defects such as cracks generated on the surface of the tank.
  • the inspection device detects defects from the captured image of the tank by using the defect detector learned by the learning device.
  • the efficiency and accuracy of the inspection can be improved, so that the burden on the inspector can be significantly reduced.
  • the ratio of determining the accepted product as the rejected product can be reduced, the time and effort for reviewing the rejected product can be reduced.
  • accurate inspections can be carried out regardless of the inspector's experience and skill, it is possible to eliminate the personalization of inspections, reduce the shortage of inspectors, and improve the quality of products. It can be stabilized well.
  • FIG. 1 shows the configuration of the inspection system according to the embodiment.
  • the inspection system 1 includes a tank 2 to be inspected, an imaging device 3 for imaging the appearance of the tank 2, and a learning device 100 for learning a defect detector for detecting defects generated on the surface of the tank 2. It is provided with an inspection device 200 for inspecting the surface of the tank 2 using a defect detector, and an Internet 4 which is an example of a communication network connecting these devices.
  • FIG. 2 shows an example of a captured image of the surface of the tank 2.
  • FIG. 2A shows an image of a crack formed on the surface of the tank 2.
  • the captured image 10 includes images of four cracks 11a, 11b, 11c, and 11d.
  • the learning device 100 attaches an annotation (teacher label) indicating that the crack is a crack to the crack included in the captured image 10, and learns the defect detector by supervised learning.
  • one defect image is divided into a plurality of parts, and learning data is generated from each of the divided defect images. As a result, a large amount of training data can be efficiently generated to learn the defect detector, so that the generalization performance and accuracy of the defect detector can be improved.
  • the learning device 100 may add annotations 12a, 12b, 12c, 12d to the images of the four cracks 11a, 11b, 11c, and 11d, respectively.
  • the three images of 11a, 11b, and 11c are each divided into a plurality of images, and the divided images of the plurality of cracks are obtained.
  • the image of the crack 11a is divided into three parts, and annotations 12a1, 12a2, and 12a3 are added to each of them to generate three learning data.
  • the number of annotated training data can be increased, so that the generalization performance and accuracy of the defect detector can be improved.
  • the learning device 100 may accept the designation of the annotation from the person in charge, or may automatically or semi-automatically add the annotation.
  • the learning device 100 receives the designation of the position, size, and type of the defect included in the captured image 10 from the person in charge, and divides the image of the defect according to the size, type, number, and the like of the designated defect. Whether to generate the training data from the data or to generate the training data without dividing it, and in the case of dividing, the number of divisions and the size and shape of each divided region may be determined. For example, since the crack 11d shown in FIG. 2A has a monotonous shape regardless of the position, the annotation 12d may be added to the entire crack without dividing it. Since the cracks 11a, 11b, and 11c have various shapes, they may be divided into a number according to the size and shape of the cracks and annotated with each of them.
  • FIG. 3 shows an example of a captured image of the surface of the tank 2.
  • FIG. 3A shows an image of a crack formed on the surface of the tank 2.
  • the captured image of the surface of the tank 2 may include the contrast of brightness caused by defects such as cracks, as well as the contrast of brightness caused by the unevenness of the surface of the tank 2 and the ambient light at the time of imaging. If the defect detector mislearns such features of the background image that should not be detected, the detection accuracy may decrease.
  • the captured image is subjected to image processing before the learning data is generated from the captured image to be inspected.
  • This preprocessing may be an image process capable of reducing the contrast of an image of a portion other than the detection target portion such as a defect to smooth the brightness.
  • the blur filter is, for example, a box-shaped filter that averages the pixel values within the range of the kernel, a Gaussian filter that changes the weight according to the distance to the pixel of interest, and a median that adopts the median value of all pixels within the range of the kernel. It may be a filter, a bilateral filter which is a Gaussian filter with a weight of a normal distribution, or the like.
  • the preprocessing may be image processing that lowers the resolution of the image. As a result, the learning efficiency can be improved by simple image processing, so that a highly accurate defect detector can be generated in a short period of time.
  • FIG. 3B shows an image obtained by applying a blur filter to the image shown in FIG. 3A. Since the background is blurred and the contrast is reduced, it is possible to reduce erroneous learning of background features. The contrast of the defects is also slightly reduced, but the characteristics are preserved so that the defect detector can learn the characteristics of the defects.
  • the image processing may be performed under conditions such that the contrast characteristics in the detection target portion are not lost and the contrast in the non-detection target portion is sufficiently smoothed.
  • the learning device 100 acquires the luminance distribution in the detection target portion and the luminance distribution in the background other than the detection target portion, and determines conditions such as the type of image processing and the kernel size based on the respective luminance distributions. You may. When the learning device 100 inspects a plurality of captured images obtained by capturing the same type of inspection target in the same environment, the learning device 100 may perform image processing under the same conditions.
  • the imaging may be performed in an imaging environment in which the contrast of the non-defect portion is sufficiently low.
  • the tank 2 may be irradiated with light with an illuminance corresponding to the color and reflectance of the surface of the tank 2. Further, the focal length, resolution, aperture and the like of the image pickup apparatus 3 may be adjusted.
  • FIG. 4 is a flowchart showing the procedure of the learning method according to the embodiment.
  • the learning device 100 acquires a captured image of an inspection target having a defect (S10), and performs preprocessing such as blurring on the acquired captured image (S12).
  • the learning device 100 divides at least a part of the defect image included in the captured image into a plurality of parts (S14), adds annotations to each of the divided plurality of defect images, and generates learning data (S16).
  • the learning device 100 may add different annotations for each type of defect. As a result, the characteristics can be learned by the defect detector for each type of defect, so that the defect can be detected from the captured image to be inspected and the type of defect can be determined at the same time.
  • the learning device 100 learns the defect detector by supervised learning using the generated learning data.
  • the learning device 100 may apply any known learning algorithm when learning the defect detector.
  • FIG. 5 is a flowchart showing the procedure of the inspection method according to the embodiment.
  • the inspection device 200 acquires the captured image to be inspected (S30), and performs the same image processing on the captured image as that performed when the learning device 100 generates the learning data (S32).
  • the inspection device 200 detects a defect from the captured image using the learned defect detector (S34), and outputs the detection result (S36).
  • the present inventor learns the defect detector by using the captured images of 212 rejected products of the tank 2 including defects by the above method, and uses the learned defect detector to learn the passed product 500 of the tank 2. Inspection of points and 23 points of rejected products was carried out. The number of cases in which a rejected product was erroneously judged as a passed product was 0. As a result, it was shown that the technique of the present embodiment can minimize the possibility of erroneously shipping the rejected product. In addition, the number of cases in which the accepted product was erroneously determined as the rejected product was 7, and the correct answer rate was 98.6%. As a result, it was shown that the time and effort required for the inspector to confirm the rejected product can be significantly reduced, and the rate at which the accepted product cannot be shipped and is lost can be significantly reduced.
  • FIG. 6 shows the configuration of the learning device 100 according to the embodiment.
  • the learning device 100 includes a display device 112, an input device 113, a communication device 114, a processing device 120, and a storage device 130.
  • the learning device 100 may be a server device, a device such as a personal computer, or a mobile terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.
  • the display device 112 displays the screen generated by the processing device 120.
  • the display device 112 may be a liquid crystal display device, an organic EL display device, or the like.
  • the input device 113 transmits the instruction input by the user of the learning device 100 to the processing device 120.
  • the input device 113 may be a mouse, a keyboard, a touch pad, or the like.
  • the display device 112 and the input device 113 may be mounted as a touch panel.
  • the communication device 114 controls communication with other devices.
  • the communication device 114 may perform communication by any wired or wireless communication method.
  • the communication device 114 communicates with the image pickup device 3 and the inspection device 200 via the Internet 4.
  • the storage device 130 stores programs, data, and the like used by the processing device 120.
  • the storage device 130 may be a semiconductor memory, a hard disk, or the like.
  • the storage device 130 stores a learning image holding unit 131, a learning data holding unit 132, and a defect detector 133.
  • the processing device 120 includes a learning image acquisition unit 121, an image processing unit 122, a learning data generation unit 123, a learning unit 124, and a defect detector providing unit 125. These configurations are realized by the CPU, memory, and other LSIs of any computer in terms of hardware, and are realized by programs loaded in memory in terms of software. It depicts a functional block realized by. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms such as hardware alone or a combination of hardware and software.
  • the learning image acquisition unit 121 acquires an captured image of an inspection target having a defect from the image pickup device 3 and stores it in the learning image holding unit 131.
  • the image processing unit 122 performs image processing such as blurring on the learning image stored in the learning image holding unit 131.
  • the learning data generation unit 123 annotates defects included in the learning image.
  • the learning data generation unit 123 divides at least a part of the defective images into a plurality of parts, and annotates each of the divided defective images.
  • the learning data generation unit 123 may display the learning image on the display device 112 and accept the designation of the annotation from the person in charge via the input device 113.
  • the learning data generation unit 123 may automatically add annotations to defects.
  • the learning data generation unit 123 generates learning data and stores it in the learning data holding unit 132.
  • the learning unit 124 learns the defect detector 133 using the learning data stored in the learning data holding unit 132.
  • the defect detector providing unit 125 provides the trained defect detector 133 to the inspection device 200.
  • FIG. 7 shows the configuration of the inspection device 200 according to the embodiment.
  • the inspection device 200 includes a display device 212, an input device 213, a communication device 214, a processing device 220, and a storage device 230.
  • the inspection device 200 may be a server device, a device such as a personal computer, or a mobile terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.
  • the display device 212 displays the screen generated by the processing device 220.
  • the display device 212 may be a liquid crystal display device, an organic EL display device, or the like.
  • the input device 213 transmits an instruction input by the user of the inspection device 200 to the processing device 220.
  • the input device 213 may be a mouse, a keyboard, a touch pad, or the like.
  • the display device 212 and the input device 213 may be mounted as a touch panel.
  • Communication device 214 controls communication with other devices.
  • the communication device 214 may perform communication by any wired or wireless communication method.
  • the communication device 214 communicates with the image pickup device 3 and the learning device 100 via the Internet 4.
  • the storage device 230 stores programs, data, etc. used by the processing device 220.
  • the storage device 230 may be a semiconductor memory, a hard disk, or the like.
  • the storage device 230 stores an inspection image holding unit 231, a detection result holding unit 232, and a defect detector 233.
  • the processing device 220 includes an inspection image acquisition unit 221, an image processing unit 222, a defect detection unit 223, a detection result generation unit 224, and a detection result output unit 225. These configurations can also be realized in various forms such as hardware alone or a combination of hardware and software.
  • the inspection image acquisition unit 221 acquires the image of the inspection target captured by the image pickup device 3 from the image pickup device 3 and stores it in the inspection image holding unit 231.
  • the image processing unit 222 performs the same image processing on the inspection image as that performed on the learning image by the learning device 100.
  • the defect detection unit 223 detects defects from the inspection image stored in the inspection image holding unit 231 by using the learned defect detector 233 acquired from the learning device 100, and detects the detection result in the detection result holding unit 232. Store in.
  • the detection result generation unit 224 generates information on the detected defects. When a plurality of defects detected by the defect detection unit 223 are close to each other, the detection result generation unit 224 combines the defects into one defect.
  • the detection result generation unit 224 generates information such as the position, size, number, and type of the detected defects.
  • the detection result output unit 225 outputs the detection result generated by the detection result generation unit 224 to the display device 212 or the like.
  • the technique of detecting cracks generated on the surface of the sanitary ware has been mainly described, but the technique of the present embodiment has another kind of defect occurring on the surface of the sanitary ware, for example, iron. It can also be applied to detect the adhesion of copper, substrate, foreign matter, etc., abnormalities such as glaze and color, and the occurrence of bubbles, cracks, chips, etc. Further, any product other than sanitary ware may be the inspection target, or any detection target site other than the defect may be the detection target.
  • the present disclosure relates to a technique for inspecting an inspection target, and in particular, an inspection system for inspecting an inspection target, a learning device available for the inspection system, a learning program, a learning method, and an inspection device available for the inspection system. Regarding inspection programs and inspection methods.
  • 1 inspection system 2 tanks, 3 imaging devices, 4 internet, 10 captured images, 11 cracks, 12 annotations, 100 learning devices, 121 learning image acquisition units, 122 image processing units, 123 learning data generation units, 124 learning units, 125 Defect detector providing unit, 131 Learning image holding unit, 132 Learning data holding unit, 133 Defect detector, 200 Inspection device, 221 Inspection image acquisition unit, 222 Image processing unit, 223 Defect detection unit, 224 Detection result generation 225 Detection result output unit, 231 Inspection image holding unit, 232 Detection result holding unit, 233 Defect detector.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

An inspection system 1 is provided with: a learning device 100 that performs learning of a detector for detecting a detection target portion of a tank 2, which is an inspection target; and an inspection device 200 that detects the detection target portion of the tank 2 by using the detector that has undergone the learning by the learning device 100. The learning device 100 is provided with: a learning image acquisition unit that acquires an image taken of the inspection target including the detection target portion; a learning data generation unit that generates learning data from the image of the detection target portion included in the image taken of the inspection target acquired by the learning image acquisition unit; and a learning unit that performs learning of the detector by using the learning data generated by the learning data generation unit. The learning data generation unit divides at least some of a plurality of images of the detection target portion into a plurality of images, and generates learning data from each of the divided images of the detection target portion.

Description

検査システム、学習装置、学習プログラム、学習方法、検査装置、検査プログラム、検査方法Inspection system, learning device, learning program, learning method, inspection device, inspection program, inspection method
　本開示は、検査対象を検査するための技術に関し、とくに、検査対象を検査するための検査システム、検査システムに利用可能な学習装置、学習プログラム、学習方法、検査システムに利用可能な検査装置、検査プログラム、及び検査方法に関する。 The present disclosure relates to a technique for inspecting an inspection target, and in particular, an inspection system for inspecting an inspection target, a learning device available for the inspection system, a learning program, a learning method, and an inspection device available for the inspection system. Regarding inspection programs and inspection methods.
　便器装置の貯水タンクなどの衛生陶器を製造する際に、施釉や焼成などの工程において、釉薬のはげや亀裂（クラック）などの不具合が生じうる。従来は、熟練した検査員が目視で外観を検査していたが、人材を確保するのが困難で、教育負荷も高いため、属人化の解消が大きな課題となっていた。 When manufacturing sanitary ware such as a water storage tank for a toilet bowl, problems such as glaze baldness and cracks may occur in processes such as glaze and firing. In the past, skilled inspectors visually inspected the appearance, but it was difficult to secure human resources and the educational burden was high, so eliminating personalization was a major issue.
　このような不具合を自動的に検出するために、検査対象物の撮像画像から表面状態を判定する技術が提案されている（例えば、特許文献１参照）。 In order to automatically detect such a defect, a technique for determining the surface state from the captured image of the inspection object has been proposed (see, for example, Patent Document 1).
特開２００６－０４７０９８号公報Japanese Unexamined Patent Publication No. 2006-047098
　不具合を検出できずに、不具合のあるままで製品を出荷してしまうと、回収、代品出荷、再加工などの手間が生じるとともに、製品の信頼性が低下してしまう。したがって、不具合を自動的に検出する精度を更に高める必要がある。 If a defect cannot be detected and the product is shipped with the defect, it will take time and effort for collection, substitute shipment, reprocessing, etc., and the reliability of the product will decrease. Therefore, it is necessary to further improve the accuracy of automatically detecting defects.
　本開示は、このような課題に鑑みてなされ、その目的は、検査対象を検査する精度を向上させることにある。 The present disclosure is made in view of such a problem, and the purpose thereof is to improve the accuracy of inspecting the inspection target.
　上記課題を解決するために、本開示のある態様の検査システムは、検査対象の検出対象部位を検出するための検出器を学習する学習装置と、学習装置により学習された検出器を用いて、検査対象の検出対象部位を検出する検査装置と、を備える。学習装置は、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して検出器を学習する学習部と、を備える。学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成し、検査装置は、検査対象の撮像画像を取得する検査用画像取得部と、検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、を備える。 In order to solve the above problems, the inspection system of a certain aspect of the present disclosure uses a learning device that learns a detector for detecting a detection target site to be inspected, and a detector learned by the learning device. It is provided with an inspection device for detecting a detection target site to be inspected. The learning device acquires learning data from a learning image acquisition unit that acquires an image of the inspection target including the detection target part and an image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit. It includes a learning data generation unit to be generated, and a learning unit to learn a detector using the learning data generated by the learning data generation unit. The learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, generates learning data from each of the divided images of the detection target parts, and the inspection device uses the captured image of the inspection target. It is provided with an inspection image acquisition unit for acquiring the data, and a detection unit for detecting a detection target portion from the captured image acquired by the inspection image acquisition unit using a detector.
　本開示の別の態様は、学習装置である。この装置は、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、を備える。学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する。 Another aspect of the present disclosure is a learning device. This device acquires learning data from the image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit and the learning image acquisition unit that acquires the captured image of the inspection target including the detection target part. It includes a learning data generation unit to be generated, and a learning unit to learn a detector for detecting a detection target site to be inspected by using the learning data generated by the learning data generation unit. The learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, and generates learning data from each of the divided images of the detection target parts.
　本開示のさらに別の態様は、学習プログラムである。このプログラムは、コンピュータを、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、として機能させ、学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する。 Yet another aspect of the disclosure is a learning program. This program uses a computer from a learning image acquisition unit that acquires an image of an inspection target including a detection target part and an image of the detection target part included in the image of the inspection target acquired by the learning image acquisition unit. It functions as a learning data generation unit that generates training data and a learning unit that learns a detector for detecting a detection target part to be inspected by using the learning data generated by the training data generation unit. The learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, and generates learning data from each of the divided images of the detection target parts.
　本開示のさらに別の態様は、学習方法である。この方法は、コンピュータに、検出対象部位を含む検査対象の撮像画像を取得するステップと、取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成するステップと、生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習するステップと、を実行させる。学習するステップにおいて、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する。 Yet another aspect of the disclosure is a learning method. This method is generated by a computer with a step of acquiring an image of the inspection target including the detection target part and a step of generating learning data from the image of the detection target part included in the acquired image of the inspection target. Using the learned data, the step of learning the detector for detecting the detection target part to be inspected and the step of learning are executed. In the learning step, at least a part of the images of the plurality of detection target parts is divided into a plurality of images, and learning data is generated from each of the divided images of the detection target parts.
　本開示のさらに別の態様は、検査装置である。この装置は、検査対象の撮像画像を取得する検査用画像取得部と、検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、を備える。 Yet another aspect of the present disclosure is an inspection device. This device was trained using an inspection image acquisition unit that acquires an image to be inspected and learning data generated by dividing an image of a detection target part included in the image to be inspected into a plurality of images. It includes a detection unit that detects a detection target site from an captured image acquired by an inspection image acquisition unit using a detector.
　本開示のさらに別の態様は、検査プログラムである。このプログラムは、コンピュータを、検査対象の撮像画像を取得する検査用画像取得部と、検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、として機能させる。 Yet another aspect of the disclosure is an inspection program. This program uses a computer with an inspection image acquisition unit that acquires an image to be inspected, and learning data generated by dividing an image of a detection target part included in the image to be inspected into a plurality of images. Using the learned detector, it functions as a detection unit that detects a detection target site from the captured image acquired by the inspection image acquisition unit.
　本開示のさらに別の態様は、検査方法である。この方法は、コンピュータに、検査対象の撮像画像を取得するステップと、検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、取得された撮像画像から検出対象部位を検出するステップと、を実行させる。 Yet another aspect of the present disclosure is an inspection method. In this method, the detection is learned by using the step of acquiring the captured image of the inspection target and the learning data generated by dividing the image of the detection target part included in the captured image of the inspection target into a plurality of images on a computer. Using a device, the step of detecting the detection target site from the acquired captured image is executed.
　本開示のさらに別の態様の検査システムは、検査対象の検出対象部位を検出するための検出器を学習する学習装置と、学習装置により学習された検出器を用いて、検査対象の検出対象部位を検出する検査装置と、を備える。学習装置は、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、撮像画像に画像処理を施す画像処理部と、画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して検出器を学習する学習部と、を備える。検査装置は、検査対象の撮像画像を取得する検査用画像取得部と、検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、を備える。 In yet another aspect of the inspection system of the present disclosure, a learning device that learns a detector for detecting a detection target site to be inspected and a detector learned by the learning device are used to detect the detection target site to be inspected. It is equipped with an inspection device for detecting. The learning device converts the image to be captured before generating learning data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the learning image acquisition unit. An image processing unit that performs image processing, a learning data generation unit that generates learning data from the image of the detection target part processed by the image processing unit, and a detector using the training data generated by the learning data generation unit. It has a learning department for learning. The inspection device includes an inspection image acquisition unit that acquires a captured image of an inspection target, and a detection unit that detects a detection target portion from the captured image acquired by the inspection image acquisition unit using a detector.
　本開示のさらに別の態様は、学習装置である。この装置は、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、撮像画像に画像処理を施す画像処理部と、画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、を備える。 Yet another aspect of the present disclosure is a learning device. This device converts the image to be captured before generating learning data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the image acquisition unit for learning. An inspection target using an image processing unit that performs image processing, a learning data generation unit that generates training data from an image of a detection target part processed by the image processing unit, and training data generated by the training data generation unit. It is provided with a learning unit for learning a detector for detecting a detection target portion of the above.
　本開示のさらに別の態様は、学習プログラムである。このプログラムは、コンピュータを、検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、撮像画像に画像処理を施す画像処理部と、画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、として機能させる。 Yet another aspect of the disclosure is a learning program. This program uses the computer to generate training data from the image acquisition unit for learning that acquires the captured image of the inspection target including the detection target site and the captured image of the inspection target acquired by the image acquisition unit for learning. Using an image processing unit that performs image processing on the captured image, a learning data generation unit that generates learning data from the image of the detection target part processed by the image processing unit, and learning data generated by the training data generation unit. , It functions as a learning unit that learns a detector for detecting a detection target part to be inspected.
　本開示のさらに別の態様は、学習方法である。この方法は、コンピュータに、検出対象部位を含む検査対象の撮像画像を取得するステップと、取得された検査対象の撮像画像から学習データを生成する前に、撮像画像に画像処理を施すステップと、処理された検出対象部位の画像から学習データを生成するステップと、生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習するステップと、を実行させる。 Yet another aspect of the disclosure is a learning method. This method includes a step of acquiring a captured image of an inspection target including a detection target site on a computer, and a step of performing image processing on the captured image before generating learning data from the acquired captured image of the inspection target. A step of generating training data from the processed image of the detection target part and a step of learning a detector for detecting the detection target part of the inspection target using the generated training data are executed.
　本開示のさらに別の態様は、検査装置である。この装置は、検査対象の撮像画像を取得する検査用画像取得部と、検査用画像取得部により取得された検査対象の撮像画像から検出対象部位を検出する前に、撮像画像に画像処理を施す画像処理部と、検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、を備える。 Yet another aspect of the present disclosure is an inspection device. This device performs image processing on the captured image before detecting the detection target portion from the inspection image acquisition unit that acquires the captured image of the inspection target and the captured image of the inspection target acquired by the inspection image acquisition unit. Detected from the captured image acquired by the inspection image acquisition unit using the image processing unit and the detector learned using the learning data generated from the image of the detection target part included in the captured image to be inspected. It is provided with a detection unit for detecting a target site.
　本開示のさらに別の態様は、検査プログラムである。このプログラムは、コンピュータを、検査対象の撮像画像を取得する検査用画像取得部と、検査用画像取得部により取得された検査対象の撮像画像から検出対象部位を検出する前に、撮像画像に画像処理を施す画像処理部と、検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、として機能させる。 Yet another aspect of the disclosure is an inspection program. This program displays an image on the captured image before detecting the detection target part from the inspection image acquisition unit that acquires the captured image of the inspection target and the captured image of the inspection target acquired by the inspection image acquisition unit. Imaging acquired by the inspection image acquisition unit using the image processing unit to be processed and the detector learned using the learning data generated from the image of the detection target part included in the image to be inspected. It functions as a detection unit that detects the detection target part from the image.
　本開示のさらに別の態様は、検査方法である。この方法は、コンピュータに、検査対象の撮像画像を取得するステップと、取得された検査対象の撮像画像から検出対象部位を検出する前に、撮像画像に画像処理を施すステップと、検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、取得された撮像画像から検出対象部位を検出するステップと、を実行させる。 Yet another aspect of the present disclosure is an inspection method. This method includes a step of acquiring an image to be inspected by a computer, a step of performing image processing on the image to be detected before detecting a part to be detected from the acquired image of the object to be inspected, and an image of the image to be inspected. Using a detector learned using the learning data generated from the image of the detection target site included in the image, the step of detecting the detection target site from the acquired captured image is executed.
　なお、以上の構成要素の任意の組合せ、本発明の表現を方法、装置、システム、記録媒体、コンピュータプログラムなどの間で変換したものもまた、本発明の態様として有効である。 It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of the present invention.
実施の形態に係る検査システムの構成を示す図である。It is a figure which shows the structure of the inspection system which concerns on embodiment. タンクの表面の撮像画像の例を示す図である。It is a figure which shows the example of the captured image of the surface of a tank. タンクの表面の撮像画像の例を示す図である。It is a figure which shows the example of the captured image of the surface of a tank. 実施の形態に係る学習方法の手順を示すフローチャートである。It is a flowchart which shows the procedure of the learning method which concerns on embodiment. 実施の形態に係る検査方法の手順を示すフローチャートである。It is a flowchart which shows the procedure of the inspection method which concerns on embodiment. 実施の形態に係る学習装置の構成を示す図である。It is a figure which shows the structure of the learning apparatus which concerns on embodiment. 実施の形態に係る検査装置の構成を示す図である。It is a figure which shows the structure of the inspection apparatus which concerns on embodiment.
　本開示の実施の形態として、衛生陶器の一例であるタンクの外観を検査する技術について説明する。実施の形態に係る学習装置は、タンクの表面に生じたクラックなどの欠陥を検出するための欠陥検出器を学習する。実施の形態に係る検査装置は、学習装置により学習された欠陥検出器を使用してタンクの撮像画像から欠陥を検出する。これにより、検査の効率及び精度を向上させることができるので、検査員の負担を大幅に軽減させることができる。また、合格品を不合格品と判定する割合を低減させることができるので、不合格品の見直しなどの手間を軽減させることができる。また、検査員の経験や技量などによらず、精確な検査を実施することができるので、検査の属人化を解消し、検査員の人員不足を軽減させることができるとともに、製品の品質を良好に安定させることができる。 As an embodiment of the present disclosure, a technique for inspecting the appearance of a tank, which is an example of sanitary ware, will be described. The learning device according to the embodiment learns a defect detector for detecting defects such as cracks generated on the surface of the tank. The inspection device according to the embodiment detects defects from the captured image of the tank by using the defect detector learned by the learning device. As a result, the efficiency and accuracy of the inspection can be improved, so that the burden on the inspector can be significantly reduced. Further, since the ratio of determining the accepted product as the rejected product can be reduced, the time and effort for reviewing the rejected product can be reduced. In addition, since accurate inspections can be carried out regardless of the inspector's experience and skill, it is possible to eliminate the personalization of inspections, reduce the shortage of inspectors, and improve the quality of products. It can be stabilized well.
　図１は、実施の形態に係る検査システムの構成を示す。検査システム１は、検査対象であるタンク２と、タンク２の外観を撮像するための撮像装置３と、タンク２の表面に生じた欠陥を検出するための欠陥検出器を学習する学習装置１００と、欠陥検出器を用いてタンク２の表面を検査する検査装置２００と、それらの装置を接続する通信網の一例であるインターネット４とを備える。 FIG. 1 shows the configuration of the inspection system according to the embodiment. The inspection system 1 includes a tank 2 to be inspected, an imaging device 3 for imaging the appearance of the tank 2, and a learning device 100 for learning a defect detector for detecting defects generated on the surface of the tank 2. It is provided with an inspection device 200 for inspecting the surface of the tank 2 using a defect detector, and an Internet 4 which is an example of a communication network connecting these devices.
　図２は、タンク２の表面の撮像画像の例を示す。図２（ａ）は、タンク２の表面に生じたクラックの画像を示す。撮像画像１０には、４つのクラック１１ａ、１１ｂ、１１ｃ、１１ｄの画像が含まれている。学習装置１００は、撮像画像１０に含まれるクラックに対して、クラックであることを示すアノテーション（教師ラベル）を付与し、教師あり学習によって欠陥検出器を学習する。 FIG. 2 shows an example of a captured image of the surface of the tank 2. FIG. 2A shows an image of a crack formed on the surface of the tank 2. The captured image 10 includes images of four cracks 11a, 11b, 11c, and 11d. The learning device 100 attaches an annotation (teacher label) indicating that the crack is a crack to the crack included in the captured image 10, and learns the defect detector by supervised learning.
　欠陥検出器の汎化性能（未知データに対する検出性能）を向上せるためには、多種多様な欠陥の画像を学習データとして欠陥検出器を学習する必要があるが、タンク２を製造する際にタンク２の表面に欠陥が発生する割合は非常に低いので、実際にタンク２の表面に欠陥が生じている画像を大量に収集して学習データを生成するのは困難である。このような課題を解決するために、本実施の形態では、１つの欠陥の画像を複数に分割し、分割された欠陥の画像のそれぞれから学習データを生成する。これにより、大量の学習データを効率良く生成して欠陥検出器を学習することができるので、欠陥検出器の汎化性能及び精度を向上させることができる。このようにして学習された欠陥検出器により欠陥を検出する場合、１つの欠陥が存在する場合であっても、細分化された欠陥の一部ずつが複数個に分けて検出される可能性があるが、検出後に近接している欠陥同士をまとめればよい。この手法は、クラックなど、１つの欠陥においても位置によって形状などの特徴が異なりうる欠陥の検出器を学習する際にとくに効果的である。 In order to improve the generalization performance (detection performance for unknown data) of the defect detector, it is necessary to learn the defect detector using images of various defects as training data. Since the rate at which defects occur on the surface of tank 2 is very low, it is difficult to collect a large number of images in which defects actually occur on the surface of tank 2 and generate learning data. In order to solve such a problem, in the present embodiment, one defect image is divided into a plurality of parts, and learning data is generated from each of the divided defect images. As a result, a large amount of training data can be efficiently generated to learn the defect detector, so that the generalization performance and accuracy of the defect detector can be improved. When a defect is detected by the defect detector learned in this way, even if one defect is present, there is a possibility that a part of each of the subdivided defects is detected in a plurality of parts. However, it is sufficient to combine defects that are close to each other after detection. This method is particularly effective when learning a defect detector, such as a crack, in which features such as shape may differ depending on the position of even one defect.
　学習装置１００は、図２（ａ）に示すように、４つのクラック１１ａ、１１ｂ、１１ｃ、１１ｄの画像のそれぞれにアノテーション１２ａ、１２ｂ、１２ｃ、１２ｄを付与してもよいが、本実施の形態では、図２（ｂ）に示すように、４つのクラック１１ａ、１１ｂ、１１ｃ、１１ｄの画像のうち１１ａ、１１ｂ、１１ｃの３つの画像をそれぞれ複数に分割し、分割された複数のクラックの画像のそれぞれにアノテーションを付与して、別々の学習データを生成する。例えば、クラック１１ａの画像を３つに分割し、それぞれにアノテーション１２ａ１、１２ａ２、１２ａ３を付与して、３つの学習データを生成する。これにより、アノテーションが付与された学習データの数を増やすことができるので、欠陥検出器の汎化性能及び精度を向上させることができる。 As shown in FIG. 2A, the learning device 100 may add annotations 12a, 12b, 12c, 12d to the images of the four cracks 11a, 11b, 11c, and 11d, respectively. Then, as shown in FIG. 2B, out of the images of the four cracks 11a, 11b, 11c, and 11d, the three images of 11a, 11b, and 11c are each divided into a plurality of images, and the divided images of the plurality of cracks are obtained. Annotate each of them to generate separate training data. For example, the image of the crack 11a is divided into three parts, and annotations 12a1, 12a2, and 12a3 are added to each of them to generate three learning data. As a result, the number of annotated training data can be increased, so that the generalization performance and accuracy of the defect detector can be improved.
　学習装置１００は、担当者からアノテーションの指定を受け付けてもよいし、自動的又は半自動的にアノテーションを付与してもよい。学習装置１００は、撮像画像１０に含まれる欠陥の位置、大きさ、種類の指定を担当者から受け付け、指定された欠陥の大きさ、種類、数などに応じて、欠陥の画像を分割してから学習データを生成するのか分割せずに学習データを生成するのかや、分割する場合は分割数や分割したそれぞれの領域の大きさや形状などを決定してもよい。例えば、図２（ａ）に示したクラック１１ｄは、位置によらず単調な形状を有しているので、分割せずに全体にアノテーション１２ｄを付与してもよい。クラック１１ａ、１１ｂ、１１ｃは、変化に富んだ形状を有しているので、クラックの大きさや形状の変化などに応じた数に分割して、それぞれにアノテーションを付与してもよい。 The learning device 100 may accept the designation of the annotation from the person in charge, or may automatically or semi-automatically add the annotation. The learning device 100 receives the designation of the position, size, and type of the defect included in the captured image 10 from the person in charge, and divides the image of the defect according to the size, type, number, and the like of the designated defect. Whether to generate the training data from the data or to generate the training data without dividing it, and in the case of dividing, the number of divisions and the size and shape of each divided region may be determined. For example, since the crack 11d shown in FIG. 2A has a monotonous shape regardless of the position, the annotation 12d may be added to the entire crack without dividing it. Since the cracks 11a, 11b, and 11c have various shapes, they may be divided into a number according to the size and shape of the cracks and annotated with each of them.
　図３は、タンク２の表面の撮像画像の例を示す。図３（ａ）は、タンク２の表面に生じたクラックの画像を示す。タンク２の表面の撮像画像には、クラックなどの欠陥に起因する輝度のコントラストの他に、タンク２の表面の凹凸や撮像時の環境光などに起因する輝度のコントラストが含まれうる。このような検出すべきでない背景画像の特徴を欠陥検出器が誤学習すると、検出精度が低下しうる。このような課題を解決するために、本実施の形態では、検査対象の撮像画像から学習データを生成する前に、撮像画像に画像処理を施す。この前処理は、欠陥などの検出対象部位ではない部位の画像のコントラストを低下させて輝度を平滑化することが可能な画像処理であってもよい。例えば、画像全体にぼかし（ブラー）フィルタをかけてもよい。ぼかしフィルタは、例えば、カーネルの範囲内の画素値の平均を取る箱形フィルタ、注目画素との距離に応じて重みを変えるガウシアンフィルタ、カーネルの範囲内の全画素の中央値を採用する中央値フィルタ、正規分布の重みを付けたガウシアンフィルタであるバイラテラルフィルタなどであってもよい。前処理は、画像の解像度を下げる画像処理であってもよい。これにより、簡易な画像処理によって学習効率を向上させることができるので、高精度な欠陥検出器を短期間で生成することができる。 FIG. 3 shows an example of a captured image of the surface of the tank 2. FIG. 3A shows an image of a crack formed on the surface of the tank 2. The captured image of the surface of the tank 2 may include the contrast of brightness caused by defects such as cracks, as well as the contrast of brightness caused by the unevenness of the surface of the tank 2 and the ambient light at the time of imaging. If the defect detector mislearns such features of the background image that should not be detected, the detection accuracy may decrease. In order to solve such a problem, in the present embodiment, the captured image is subjected to image processing before the learning data is generated from the captured image to be inspected. This preprocessing may be an image process capable of reducing the contrast of an image of a portion other than the detection target portion such as a defect to smooth the brightness. For example, you may apply a blur filter to the entire image. The blur filter is, for example, a box-shaped filter that averages the pixel values within the range of the kernel, a Gaussian filter that changes the weight according to the distance to the pixel of interest, and a median that adopts the median value of all pixels within the range of the kernel. It may be a filter, a bilateral filter which is a Gaussian filter with a weight of a normal distribution, or the like. The preprocessing may be image processing that lowers the resolution of the image. As a result, the learning efficiency can be improved by simple image processing, so that a highly accurate defect detector can be generated in a short period of time.
　図３（ｂ）は、図３（ａ）に示した画像にぼかしフィルターをかけた画像を示す。背景はぼかされてコントラストが低下しているので、背景の特徴を誤学習するのを低減させることができる。欠陥のコントラストも若干低下するが、特徴は保たれているので、欠陥の特徴を欠陥検出器に学習させることができる。 FIG. 3B shows an image obtained by applying a blur filter to the image shown in FIG. 3A. Since the background is blurred and the contrast is reduced, it is possible to reduce erroneous learning of background features. The contrast of the defects is also slightly reduced, but the characteristics are preserved so that the defect detector can learn the characteristics of the defects.
　画像処理は、検出対象部位におけるコントラストの特徴が失われず、検出対象部位ではない部位におけるコントラストが十分に平滑化されるような条件で実行されればよい。学習装置１００は、検出対象部位における輝度の分布と、検出対象部位ではない背景における輝度の分布を取得し、それぞれの輝度の分布に基づいて、画像処理の種類やカーネルサイズなどの条件を決定してもよい。学習装置１００は、同種の検査対象を同様の環境で撮像した複数の撮像画像を検査する場合は、同じ条件で画像処理を施してもよい。 The image processing may be performed under conditions such that the contrast characteristics in the detection target portion are not lost and the contrast in the non-detection target portion is sufficiently smoothed. The learning device 100 acquires the luminance distribution in the detection target portion and the luminance distribution in the background other than the detection target portion, and determines conditions such as the type of image processing and the kernel size based on the respective luminance distributions. You may. When the learning device 100 inspects a plurality of captured images obtained by capturing the same type of inspection target in the same environment, the learning device 100 may perform image processing under the same conditions.
　撮像装置３により検査対象のタンク２の表面を撮像する際に、欠陥ではない部位のコントラストが十分に低くなるような撮像環境で撮像を実行してもよい。例えば、タンク２の表面の色や反射率などに応じた照度でタンク２に光を照射してもよい。また、撮像装置３の焦点距離、解像度、絞りなどを調整してもよい。 When the surface of the tank 2 to be inspected is imaged by the imaging device 3, the imaging may be performed in an imaging environment in which the contrast of the non-defect portion is sufficiently low. For example, the tank 2 may be irradiated with light with an illuminance corresponding to the color and reflectance of the surface of the tank 2. Further, the focal length, resolution, aperture and the like of the image pickup apparatus 3 may be adjusted.
　図４は、実施の形態に係る学習方法の手順を示すフローチャートである。学習装置１００は、欠陥を有する検査対象の撮像画像を取得し（Ｓ１０）、取得した撮像画像に対してぼかしなどの前処理を実施する（Ｓ１２）。学習装置１００は、撮像画像に含まれる欠陥の画像の少なくとも一部を複数に分割し（Ｓ１４）、分割した複数の欠陥画像のそれぞれにアノテーションを付与して学習データを生成する（Ｓ１６）。学習装置１００は、欠陥の種類ごとに異なるアノテーションを付与してもよい。これにより、欠陥の種類ごとに特徴を欠陥検出器に学習させることができるので、欠陥検出器を用いて検査対象の撮像画像から欠陥を検出すると同時に欠陥の種類を判定することができる。学習装置１００は、生成した学習データを使用して教師あり学習により欠陥検出器を学習する。学習装置１００は、欠陥検出器を学習する際に、既知の任意の学習アルゴリズムを適用してもよい。 FIG. 4 is a flowchart showing the procedure of the learning method according to the embodiment. The learning device 100 acquires a captured image of an inspection target having a defect (S10), and performs preprocessing such as blurring on the acquired captured image (S12). The learning device 100 divides at least a part of the defect image included in the captured image into a plurality of parts (S14), adds annotations to each of the divided plurality of defect images, and generates learning data (S16). The learning device 100 may add different annotations for each type of defect. As a result, the characteristics can be learned by the defect detector for each type of defect, so that the defect can be detected from the captured image to be inspected and the type of defect can be determined at the same time. The learning device 100 learns the defect detector by supervised learning using the generated learning data. The learning device 100 may apply any known learning algorithm when learning the defect detector.
　図５は、実施の形態に係る検査方法の手順を示すフローチャートである。検査装置２００は、検査対象の撮像画像を取得し（Ｓ３０）、学習装置１００が学習データを生成する際に実施したのと同様の画像処理を撮像画像に対して実施する（Ｓ３２）。検査装置２００は、学習済みの欠陥検出器を用いて撮像画像から欠陥を検出し（Ｓ３４）、検出結果を出力する（Ｓ３６）。 FIG. 5 is a flowchart showing the procedure of the inspection method according to the embodiment. The inspection device 200 acquires the captured image to be inspected (S30), and performs the same image processing on the captured image as that performed when the learning device 100 generates the learning data (S32). The inspection device 200 detects a defect from the captured image using the learned defect detector (S34), and outputs the detection result (S36).
　本発明者は、上記の方法により、欠陥を含むタンク２の不合格品２１２点の撮像画像を使用して欠陥検出器を学習し、学習した欠陥検出器を使用してタンク２の合格品５００点と不合格品２３点の検査を実施した。不合格品を合格品と誤判定した件数は０件であった。これにより、本実施の形態の技術によって、不合格品を誤って出荷してしまう可能性を最小限に抑えることができることが示された。また、合格品を不合格品と誤判定した件数は７件であり、正解率は９８．６％であった。これにより、不合格品を検査員が確認する手間を大幅に軽減させることができるとともに、合格品を出荷できずにロスしてしまう割合を大幅に低減させることができることが示された。 The present inventor learns the defect detector by using the captured images of 212 rejected products of the tank 2 including defects by the above method, and uses the learned defect detector to learn the passed product 500 of the tank 2. Inspection of points and 23 points of rejected products was carried out. The number of cases in which a rejected product was erroneously judged as a passed product was 0. As a result, it was shown that the technique of the present embodiment can minimize the possibility of erroneously shipping the rejected product. In addition, the number of cases in which the accepted product was erroneously determined as the rejected product was 7, and the correct answer rate was 98.6%. As a result, it was shown that the time and effort required for the inspector to confirm the rejected product can be significantly reduced, and the rate at which the accepted product cannot be shipped and is lost can be significantly reduced.
　図６は、実施の形態に係る学習装置１００の構成を示す。学習装置１００は、表示装置１１２、入力装置１１３、通信装置１１４、処理装置１２０、及び記憶装置１３０を備える。学習装置１００は、サーバ装置であってもよいし、パーソナルコンピューターなどの装置であってもよいし、携帯電話端末、スマートフォン、タブレット端末などの携帯端末であってもよい。 FIG. 6 shows the configuration of the learning device 100 according to the embodiment. The learning device 100 includes a display device 112, an input device 113, a communication device 114, a processing device 120, and a storage device 130. The learning device 100 may be a server device, a device such as a personal computer, or a mobile terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.
　表示装置１１２は、処理装置１２０により生成される画面を表示する。表示装置１１２は、液晶表示装置、有機ＥＬ表示装置などであってもよい。入力装置１１３は、学習装置１００の使用者による指示入力を処理装置１２０に伝達する。入力装置１１３は、マウス、キーボード、タッチパッドなどであってもよい。表示装置１１２及び入力装置１１３は、タッチパネルとして実装されてもよい。 The display device 112 displays the screen generated by the processing device 120. The display device 112 may be a liquid crystal display device, an organic EL display device, or the like. The input device 113 transmits the instruction input by the user of the learning device 100 to the processing device 120. The input device 113 may be a mouse, a keyboard, a touch pad, or the like. The display device 112 and the input device 113 may be mounted as a touch panel.
　通信装置１１４は、他の装置との間の通信を制御する。通信装置１１４は、有線又は無線の任意の通信方式により通信を行ってもよい。通信装置１１４は、インターネット４を介して撮像装置３及び検査装置２００との間で通信を行う。 The communication device 114 controls communication with other devices. The communication device 114 may perform communication by any wired or wireless communication method. The communication device 114 communicates with the image pickup device 3 and the inspection device 200 via the Internet 4.
　記憶装置１３０は、処理装置１２０により使用されるプログラム、データなどを記憶する。記憶装置１３０は、半導体メモリ、ハードディスクなどであってもよい。記憶装置１３０には、学習用画像保持部１３１、学習データ保持部１３２、及び欠陥検出器１３３が格納される。 The storage device 130 stores programs, data, and the like used by the processing device 120. The storage device 130 may be a semiconductor memory, a hard disk, or the like. The storage device 130 stores a learning image holding unit 131, a learning data holding unit 132, and a defect detector 133.
　処理装置１２０は、学習用画像取得部１２１、画像処理部１２２、学習データ生成部１２３、学習部１２４、及び欠陥検出器提供部１２５を備える。これらの構成は、ハードウエア的には、任意のコンピュータのＣＰＵ、メモリ、その他のＬＳＩなどにより実現され、ソフトウエア的にはメモリにロードされたプログラムなどによって実現されるが、ここではそれらの連携によって実現される機能ブロックを描いている。したがって、これらの機能ブロックがハードウエアのみ、またはハードウエアとソフトウエアの組合せなど、いろいろな形で実現できることは、当業者には理解されるところである。 The processing device 120 includes a learning image acquisition unit 121, an image processing unit 122, a learning data generation unit 123, a learning unit 124, and a defect detector providing unit 125. These configurations are realized by the CPU, memory, and other LSIs of any computer in terms of hardware, and are realized by programs loaded in memory in terms of software. It depicts a functional block realized by. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms such as hardware alone or a combination of hardware and software.
　学習用画像取得部１２１は、欠陥を有する検査対象の撮像画像を撮像装置３から取得して学習用画像保持部１３１に格納する。画像処理部１２２は、学習用画像保持部１３１に格納された学習用画像に対して、ぼかしなどの画像処理を施す。学習データ生成部１２３は、学習用画像に含まれる欠陥にアノテーションを付与する。学習データ生成部１２３は、少なくとも一部の欠陥画像を複数に分割し、分割されたそれぞれの欠陥画像にアノテーションを付与する。学習データ生成部１２３は、学習用画像を表示装置１１２に表示し、入力装置１１３を介して担当者からアノテーションの指定を受け付けてもよい。学習データ生成部１２３は、欠陥にアノテーションを自動的に付与してもよい。学習データ生成部１２３は、学習データを生成して学習データ保持部１３２に格納する。 The learning image acquisition unit 121 acquires an captured image of an inspection target having a defect from the image pickup device 3 and stores it in the learning image holding unit 131. The image processing unit 122 performs image processing such as blurring on the learning image stored in the learning image holding unit 131. The learning data generation unit 123 annotates defects included in the learning image. The learning data generation unit 123 divides at least a part of the defective images into a plurality of parts, and annotates each of the divided defective images. The learning data generation unit 123 may display the learning image on the display device 112 and accept the designation of the annotation from the person in charge via the input device 113. The learning data generation unit 123 may automatically add annotations to defects. The learning data generation unit 123 generates learning data and stores it in the learning data holding unit 132.
　学習部１２４は、学習データ保持部１３２に格納された学習データを使用して、欠陥検出器１３３を学習する。欠陥検出器提供部１２５は、学習済みの欠陥検出器１３３を検査装置２００に提供する。 The learning unit 124 learns the defect detector 133 using the learning data stored in the learning data holding unit 132. The defect detector providing unit 125 provides the trained defect detector 133 to the inspection device 200.
　図７は、実施の形態に係る検査装置２００の構成を示す。検査装置２００は、表示装置２１２、入力装置２１３、通信装置２１４、処理装置２２０、及び記憶装置２３０を備える。検査装置２００は、サーバ装置であってもよいし、パーソナルコンピューターなどの装置であってもよいし、携帯電話端末、スマートフォン、タブレット端末などの携帯端末であってもよい。 FIG. 7 shows the configuration of the inspection device 200 according to the embodiment. The inspection device 200 includes a display device 212, an input device 213, a communication device 214, a processing device 220, and a storage device 230. The inspection device 200 may be a server device, a device such as a personal computer, or a mobile terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.
　表示装置２１２は、処理装置２２０により生成される画面を表示する。表示装置２１２は、液晶表示装置、有機ＥＬ表示装置などであってもよい。入力装置２１３は、検査装置２００の使用者による指示入力を処理装置２２０に伝達する。入力装置２１３は、マウス、キーボード、タッチパッドなどであってもよい。表示装置２１２及び入力装置２１３は、タッチパネルとして実装されてもよい。 The display device 212 displays the screen generated by the processing device 220. The display device 212 may be a liquid crystal display device, an organic EL display device, or the like. The input device 213 transmits an instruction input by the user of the inspection device 200 to the processing device 220. The input device 213 may be a mouse, a keyboard, a touch pad, or the like. The display device 212 and the input device 213 may be mounted as a touch panel.
　通信装置２１４は、他の装置との間の通信を制御する。通信装置２１４は、有線又は無線の任意の通信方式により通信を行ってもよい。通信装置２１４は、インターネット４を介して撮像装置３及び学習装置１００との間で通信を行う。 Communication device 214 controls communication with other devices. The communication device 214 may perform communication by any wired or wireless communication method. The communication device 214 communicates with the image pickup device 3 and the learning device 100 via the Internet 4.
　記憶装置２３０は、処理装置２２０により使用されるプログラム、データなどを記憶する。記憶装置２３０は、半導体メモリ、ハードディスクなどであってもよい。記憶装置２３０には、検査用画像保持部２３１、検出結果保持部２３２、及び欠陥検出器２３３が格納される。 The storage device 230 stores programs, data, etc. used by the processing device 220. The storage device 230 may be a semiconductor memory, a hard disk, or the like. The storage device 230 stores an inspection image holding unit 231, a detection result holding unit 232, and a defect detector 233.
　処理装置２２０は、検査用画像取得部２２１、画像処理部２２２、欠陥検出部２２３、検出結果生成部２２４、及び検出結果出力部２２５を備える。これらの構成も、ハードウエアのみ、またはハードウエアとソフトウエアの組合せなど、いろいろな形で実現できる。 The processing device 220 includes an inspection image acquisition unit 221, an image processing unit 222, a defect detection unit 223, a detection result generation unit 224, and a detection result output unit 225. These configurations can also be realized in various forms such as hardware alone or a combination of hardware and software.
　検査用画像取得部２２１は、撮像装置３により撮像された検査対象の画像を撮像装置３から取得して検査用画像保持部２３１に格納する。画像処理部２２２は、学習装置１００が学習用画像に対して実施したのと同じ画像処理を検査用画像に対して実施する。 The inspection image acquisition unit 221 acquires the image of the inspection target captured by the image pickup device 3 from the image pickup device 3 and stores it in the inspection image holding unit 231. The image processing unit 222 performs the same image processing on the inspection image as that performed on the learning image by the learning device 100.
　欠陥検出部２２３は、学習装置１００から取得した学習済みの欠陥検出器２３３を用いて、検査用画像保持部２３１に格納された検査用画像から欠陥を検出し、検出結果を検出結果保持部２３２に格納する。検出結果生成部２２４は、検出された欠陥に関する情報を生成する。検出結果生成部２２４は、欠陥検出部２２３により検出された複数の欠陥が近接している場合、それらの欠陥をまとめて１つの欠陥とする。検出結果生成部２２４は、検出された欠陥の位置、大きさ、数、種類などの情報を生成する。検出結果出力部２２５は、検出結果生成部２２４により生成された検出結果を表示装置２１２などに出力する。 The defect detection unit 223 detects defects from the inspection image stored in the inspection image holding unit 231 by using the learned defect detector 233 acquired from the learning device 100, and detects the detection result in the detection result holding unit 232. Store in. The detection result generation unit 224 generates information on the detected defects. When a plurality of defects detected by the defect detection unit 223 are close to each other, the detection result generation unit 224 combines the defects into one defect. The detection result generation unit 224 generates information such as the position, size, number, and type of the detected defects. The detection result output unit 225 outputs the detection result generated by the detection result generation unit 224 to the display device 212 or the like.
　以上、実施の形態に基づき本発明を説明したが、実施の形態は、本発明の原理、応用を示すにすぎない。また、実施の形態には、請求の範囲に規定された本発明の思想を逸脱しない範囲において、多くの変形例や配置の変更が可能である。 The present invention has been described above based on the embodiments, but the embodiments merely show the principles and applications of the present invention. Further, in the embodiment, many modifications and arrangements can be changed without departing from the idea of the present invention defined in the claims.
　上記の実施の形態では、主に衛生陶器の表面に生じたクラックを検出する技術について説明したが、本実施の形態の技術は、衛生陶器の表面に生じた別の種類の不具合、例えば、鉄、銅、素地、異物などの付着、釉薬や色などの異常、気泡、割れ、欠けなどの発生などを検出する場合にも適用可能である。また、衛生陶器以外の任意の製品を検査対象としてもよいし、欠陥以外の任意の検出対象部位を検出対象としてもよい。 In the above embodiment, the technique of detecting cracks generated on the surface of the sanitary ware has been mainly described, but the technique of the present embodiment has another kind of defect occurring on the surface of the sanitary ware, for example, iron. It can also be applied to detect the adhesion of copper, substrate, foreign matter, etc., abnormalities such as glaze and color, and the occurrence of bubbles, cracks, chips, etc. Further, any product other than sanitary ware may be the inspection target, or any detection target site other than the defect may be the detection target.
　本開示は、検査対象を検査するための技術に関し、とくに、検査対象を検査するための検査システム、検査システムに利用可能な学習装置、学習プログラム、学習方法、検査システムに利用可能な検査装置、検査プログラム、及び検査方法に関する。 The present disclosure relates to a technique for inspecting an inspection target, and in particular, an inspection system for inspecting an inspection target, a learning device available for the inspection system, a learning program, a learning method, and an inspection device available for the inspection system. Regarding inspection programs and inspection methods.
　１　検査システム、２　タンク、３　撮像装置、４　インターネット、１０　撮像画像、１１　クラック、１２　アノテーション、１００　学習装置、１２１　学習用画像取得部、１２２　画像処理部、１２３　学習データ生成部、１２４　学習部、１２５　欠陥検出器提供部、１３１　学習用画像保持部、１３２　学習データ保持部、１３３　欠陥検出器、２００　検査装置、２２１　検査用画像取得部、２２２　画像処理部、２２３　欠陥検出部、２２４　検出結果生成部、２２５　検出結果出力部、２３１　検査用画像保持部、２３２　検出結果保持部、２３３　欠陥検出器。 1 inspection system, 2 tanks, 3 imaging devices, 4 internet, 10 captured images, 11 cracks, 12 annotations, 100 learning devices, 121 learning image acquisition units, 122 image processing units, 123 learning data generation units, 124 learning units, 125 Defect detector providing unit, 131 Learning image holding unit, 132 Learning data holding unit, 133 Defect detector, 200 Inspection device, 221 Inspection image acquisition unit, 222 Image processing unit, 223 Defect detection unit, 224 Detection result generation 225 Detection result output unit, 231 Inspection image holding unit, 232 Detection result holding unit, 233 Defect detector.

Claims (35)

  1. 　検査対象の検出対象部位を検出するための検出器を学習する学習装置と、
    　前記学習装置により学習された前記検出器を用いて、検査対象の検出対象部位を検出する検査装置と、
    を備え、
    　前記学習装置は、
    　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して前記検出器を学習する学習部と、
    を備え、
    　前記学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成し、
    　前記検査装置は、
    　検査対象の撮像画像を取得する検査用画像取得部と、
    　前記検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    を備える検査システム。     A learning device that learns a detector for detecting the detection target part of the inspection target,
    An inspection device that detects a detection target site to be inspected by using the detector learned by the learning device, and an inspection device.
    With
    The learning device is
    A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    A learning data generation unit that generates learning data from an image of a detection target part included in a captured image of an inspection target acquired by the learning image acquisition unit, and a learning data generation unit.
    A learning unit that learns the detector using the learning data generated by the learning data generation unit, and
    With
    The learning data generation unit divides at least a part of the images of the plurality of detection target parts into a plurality of images, and generates learning data from each of the divided images of the detection target parts.
    The inspection device is
    An inspection image acquisition unit that acquires an image to be inspected,
    Using the detector, a detection unit that detects a detection target site from the captured image acquired by the inspection image acquisition unit, and a detection unit.
    Inspection system equipped with.
  2. 　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、
    を備え、
    　前記学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する学習装置。     A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    A learning data generation unit that generates learning data from an image of a detection target part included in a captured image of an inspection target acquired by the learning image acquisition unit, and a learning data generation unit.
    Using the learning data generated by the learning data generation unit, a learning unit that learns a detector for detecting a detection target part to be inspected, and a learning unit.
    With
    The learning data generation unit is a learning device that divides at least a part of an image of a plurality of detection target parts into a plurality of images and generates learning data from each of the divided images of the detection target part.
  3. 　前記学習データ生成部は、前記検査対象の撮像画像に含まれる検出対象部位の画像に教師ラベルを付して学習データを生成し、検査対象部位の画像が複数に分割された場合は、複数に分割された検査対象部位の画像のそれぞれに教師ラベルを付して学習データを生成する請求項２に記載の学習装置。 The learning data generation unit attaches a teacher label to the image of the detection target part included in the captured image of the inspection target to generate learning data, and when the image of the inspection target part is divided into a plurality of parts, the learning data generation unit is divided into a plurality of parts. The learning device according to claim 2, wherein a teacher label is attached to each of the divided images of the inspection target portion to generate learning data.
  4. 　前記学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、前記撮像画像に画像処理を施す画像処理部を更に備える請求項２又は３に記載の学習装置。 The learning device according to claim 2 or 3, further comprising an image processing unit that performs image processing on the captured image before generating learning data from the captured image of the inspection target acquired by the learning image acquisition unit.
  5. 　前記画像処理は、前記検出対象部位ではない部位における解像度又は画素値のコントラストを下げる処理である請求項４に記載の学習装置。 The learning device according to claim 4, wherein the image processing is a process of lowering the contrast of the resolution or the pixel value in a portion other than the detection target portion.
  6. 　前記画像処理は、前記撮像画像の全体にぼかしフィルタをかける処理である請求項５に記載の学習装置。 The learning device according to claim 5, wherein the image processing is a process of applying a blur filter to the entire captured image.
  7. 　前記検査対象は衛生陶器であり、前記検出対象部位は前記衛生陶器の表面に生じた欠陥である請求項２から６のいずれかに記載の学習装置。 The learning device according to any one of claims 2 to 6, wherein the inspection target is sanitary ware, and the detection target portion is a defect generated on the surface of the sanitary ware.
  8. 　前記欠陥はクラックである請求項７に記載の学習装置。 The learning device according to claim 7, wherein the defect is a crack.
  9. 　コンピュータを、
    　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、
    として機能させ、
    　前記学習データ生成部は、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する学習プログラム。     Computer,
    A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    A learning data generation unit that generates learning data from an image of a detection target part included in a captured image of an inspection target acquired by the learning image acquisition unit, and a learning data generation unit.
    Using the learning data generated by the learning data generation unit, a learning unit that learns a detector for detecting a detection target part to be inspected, and a learning unit.
    To function as
    The learning data generation unit is a learning program that divides at least a part of an image of a plurality of detection target parts into a plurality of images and generates learning data from each of the divided images of the detection target part.
  10. 　コンピュータに、
    　検出対象部位を含む検査対象の撮像画像を取得するステップと、
    　取得された検査対象の撮像画像に含まれる検出対象部位の画像から学習データを生成するステップと、
    　生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習するステップと、
    を実行させ、
    　前記学習するステップにおいて、複数の検出対象部位の画像のうち少なくとも一部を複数に分割し、分割された検出対象部位の画像のそれぞれから学習データを生成する学習方法。     On the computer
    The step of acquiring the captured image of the inspection target including the detection target site,
    The step of generating learning data from the image of the detection target part included in the acquired image of the inspection target, and
    Using the generated training data, the step of learning the detector to detect the detection target part to be inspected, and
    To run,
    A learning method in which at least a part of an image of a plurality of detection target parts is divided into a plurality of images in the learning step, and learning data is generated from each of the divided images of the detection target part.
  11. 　検査対象の撮像画像を取得する検査用画像取得部と、
    　検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    を備える検査装置。     An inspection image acquisition unit that acquires an image to be inspected,
    From the captured image acquired by the inspection image acquisition unit using a detector learned using the learning data generated by dividing the image of the detection target part included in the captured image of the inspection target into a plurality of parts. A detection unit that detects the detection target site and
    Inspection device equipped with.
  12. 　前記検出部により検出された複数の検出対象部位が近接している場合、それらの検出対象部位をまとめて１つの検出対象部位として検出する請求項１１に記載の検査装置。 The inspection device according to claim 11, wherein when a plurality of detection target sites detected by the detection unit are close to each other, those detection target sites are collectively detected as one detection target site.
  13. 　前記検査用画像取得部により取得された検査対象の撮像画像から検出対象部位を検出する前に、前記撮像画像に画像処理を施す画像処理部を更に備える請求項１１又は１２に記載の検査装置。 The inspection device according to claim 11 or 12, further comprising an image processing unit that performs image processing on the captured image before detecting a detection target portion from the captured image of the inspection target acquired by the inspection image acquisition unit.
  14. 　前記画像処理は、前記検出対象部位ではない部位における解像度又は画素値のコントラストを下げる処理である請求項１３に記載の検査装置。 The inspection device according to claim 13, wherein the image processing is a process for lowering the contrast of a resolution or a pixel value in a portion other than the detection target portion.
  15. 　前記画像処理は、前記撮像画像の全体にぼかしフィルタをかける処理である請求項１４に記載の検査装置。 The inspection device according to claim 14, wherein the image processing is a process of applying a blur filter to the entire captured image.
  16. 　前記検査対象は衛生陶器であり、前記検出対象部位は前記衛生陶器の表面に生じた欠陥である請求項１１から１５のいずれかに記載の検査装置。 The inspection device according to any one of claims 11 to 15, wherein the inspection target is a sanitary ware, and the detection target portion is a defect generated on the surface of the sanitary ware.
  17. 　前記欠陥はクラックである請求項１６に記載の検査装置。 The inspection device according to claim 16, wherein the defect is a crack.
  18. 　コンピュータを、
    　検査対象の撮像画像を取得する検査用画像取得部と、
    　検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    として機能させるための検査プログラム。     Computer,
    An inspection image acquisition unit that acquires an image to be inspected,
    From the captured image acquired by the inspection image acquisition unit using a detector learned using the learning data generated by dividing the image of the detection target part included in the captured image of the inspection target into a plurality of parts. A detection unit that detects the detection target site and
    Inspection program to function as.
  19. 　コンピュータに、
    　検査対象の撮像画像を取得するステップと、
    　検査対象の撮像画像に含まれる検出対象部位の画像を複数に分割して生成された学習データを使用して学習された検出器を用いて、取得された撮像画像から検出対象部位を検出するステップと、
    を実行させる検査方法。     On the computer
    Steps to acquire the captured image of the inspection target and
    A step of detecting a detection target part from an acquired captured image using a detector learned using learning data generated by dividing an image of a detection target part included in an image to be inspected into a plurality of parts. When,
    Inspection method to execute.
  20. 　検査対象の検出対象部位を検出するための検出器を学習する学習装置と、
    　前記学習装置により学習された前記検出器を用いて、検査対象の検出対象部位を検出する検査装置と、
    を備え、
    　前記学習装置は、
    　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、前記撮像画像に画像処理を施す画像処理部と、
    　前記画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して前記検出器を学習する学習部と、
    を備え、
    　前記検査装置は、
    　検査対象の撮像画像を取得する検査用画像取得部と、
    　前記検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    を備える検査システム。     A learning device that learns a detector for detecting the detection target part of the inspection target,
    An inspection device that detects a detection target site to be inspected by using the detector learned by the learning device, and an inspection device.
    With
    The learning device is
    A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    An image processing unit that performs image processing on the captured image before generating learning data from the captured image of the inspection target acquired by the learning image acquisition unit.
    A learning data generation unit that generates learning data from an image of a detection target portion processed by the image processing unit, and a learning data generation unit.
    A learning unit that learns the detector using the learning data generated by the learning data generation unit, and
    With
    The inspection device is
    An inspection image acquisition unit that acquires an image to be inspected,
    Using the detector, a detection unit that detects a detection target site from the captured image acquired by the inspection image acquisition unit, and a detection unit.
    Inspection system equipped with.
  21. 　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、前記撮像画像に画像処理を施す画像処理部と、
    　前記画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、
    を備える学習装置。     A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    An image processing unit that performs image processing on the captured image before generating learning data from the captured image of the inspection target acquired by the learning image acquisition unit.
    A learning data generation unit that generates learning data from an image of a detection target portion processed by the image processing unit, and a learning data generation unit.
    Using the learning data generated by the learning data generation unit, a learning unit that learns a detector for detecting a detection target part to be inspected, and a learning unit.
    A learning device equipped with.
  22. 　前記画像処理は、前記検出対象部位ではない部位における解像度又は画素値のコントラストを下げる処理である請求項２１に記載の学習装置。 The learning device according to claim 21, wherein the image processing is a process of lowering the contrast of a resolution or a pixel value in a portion other than the detection target portion.
  23. 　前記画像処理は、前記撮像画像の全体にぼかしフィルタをかける処理である請求項２２に記載の学習装置。 The learning device according to claim 22, wherein the image processing is a process of applying a blur filter to the entire captured image.
  24. 　前記検査対象は衛生陶器であり、前記検出対象部位は前記衛生陶器の表面に生じた欠陥である請求項２１から２３のいずれかに記載の学習装置。 The learning device according to any one of claims 21 to 23, wherein the inspection target is sanitary ware, and the detection target portion is a defect generated on the surface of the sanitary ware.
  25. 　前記欠陥はクラックである請求項２４に記載の学習装置。 The learning device according to claim 24, wherein the defect is a crack.
  26. 　コンピュータを、
    　検出対象部位を含む検査対象の撮像画像を取得する学習用画像取得部と、
    　前記学習用画像取得部により取得された検査対象の撮像画像から学習データを生成する前に、前記撮像画像に画像処理を施す画像処理部と、
    　前記画像処理部により処理された検出対象部位の画像から学習データを生成する学習データ生成部と、
    　前記学習データ生成部により生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習する学習部と、
    として機能させるための学習プログラム。     Computer,
    A learning image acquisition unit that acquires an image of the inspection target including the detection target site,
    An image processing unit that performs image processing on the captured image before generating learning data from the captured image of the inspection target acquired by the learning image acquisition unit.
    A learning data generation unit that generates learning data from an image of a detection target portion processed by the image processing unit, and a learning data generation unit.
    Using the learning data generated by the learning data generation unit, a learning unit that learns a detector for detecting a detection target part to be inspected, and a learning unit.
    A learning program to function as.
  27. 　コンピュータに、
    　検出対象部位を含む検査対象の撮像画像を取得するステップと、
    　取得された検査対象の撮像画像から学習データを生成する前に、前記撮像画像に画像処理を施すステップと、
    　処理された検出対象部位の画像から学習データを生成するステップと、
    　生成された学習データを使用して、検査対象の検出対象部位を検出するための検出器を学習するステップと、
    を実行させる学習方法。     On the computer
    The step of acquiring the captured image of the inspection target including the detection target site,
    A step of performing image processing on the captured image before generating learning data from the acquired captured image of the inspection target, and
    Steps to generate training data from the processed image of the detection target part,
    Using the generated training data, the step of learning the detector to detect the detection target part to be inspected, and
    Learning method to execute.
  28. 　検査対象の撮像画像を取得する検査用画像取得部と、
    　前記検査用画像取得部により取得された検査対象の撮像画像から検出対象部位を検出する前に、前記撮像画像に画像処理を施す画像処理部と、
    　検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    を備える検査装置。     An inspection image acquisition unit that acquires an image to be inspected,
    An image processing unit that performs image processing on the captured image before detecting the detection target portion from the captured image of the inspection target acquired by the inspection image acquisition unit.
    The detection target part is detected from the captured image acquired by the inspection image acquisition unit using a detector learned using the learning data generated from the image of the detection target part included in the image to be inspected. With the detector
    Inspection device equipped with.
  29. 　前記検出部により検出された複数の検出対象部位が近接している場合、それらの検出対象部位をまとめて１つの検出対象部位として検出する請求項２８に記載の検査装置。 The inspection device according to claim 28, wherein when a plurality of detection target sites detected by the detection unit are close to each other, those detection target sites are collectively detected as one detection target site.
  30. 　前記画像処理は、前記検出対象部位ではない部位における解像度又は画素値のコントラストを下げる処理である請求項２８又は２９に記載の検査装置。 The inspection device according to claim 28 or 29, wherein the image processing is a process of lowering the contrast of a resolution or a pixel value in a portion other than the detection target portion.
  31. 　前記画像処理は、前記撮像画像の全体にぼかしフィルタをかける処理である請求項３０に記載の検査装置。 The inspection device according to claim 30, wherein the image processing is a process of applying a blur filter to the entire captured image.
  32. 　前記検査対象は衛生陶器であり、前記検出対象部位は前記衛生陶器の表面に生じた欠陥である請求項２８から３１のいずれかに記載の検査装置。 The inspection device according to any one of claims 28 to 31, wherein the inspection target is a sanitary ware, and the detection target portion is a defect generated on the surface of the sanitary ware.
  33. 　前記欠陥はクラックである請求項３２に記載の検査装置。 The inspection device according to claim 32, wherein the defect is a crack.
  34. 　コンピュータを、
    　検査対象の撮像画像を取得する検査用画像取得部と、
    　前記検査用画像取得部により取得された検査対象の撮像画像から検出対象部位を検出する前に、前記撮像画像に画像処理を施す画像処理部と、
    　検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、前記検査用画像取得部により取得された撮像画像から検出対象部位を検出する検出部と、
    として機能させるための検査プログラム。     Computer,
    An inspection image acquisition unit that acquires an image to be inspected,
    An image processing unit that performs image processing on the captured image before detecting the detection target portion from the captured image of the inspection target acquired by the inspection image acquisition unit.
    The detection target part is detected from the captured image acquired by the inspection image acquisition unit using a detector learned using the learning data generated from the image of the detection target part included in the image to be inspected. With the detector
    Inspection program to function as.
  35. 　コンピュータに、
    　検査対象の撮像画像を取得するステップと、
    　取得された検査対象の撮像画像から検出対象部位を検出する前に、前記撮像画像に画像処理を施すステップと、
    　検査対象の撮像画像に含まれる検出対象部位の画像から生成された学習データを使用して学習された検出器を用いて、取得された撮像画像から検出対象部位を検出するステップと、
    を実行させる検査方法。     On the computer
    Steps to acquire the captured image of the inspection target and
    A step of performing image processing on the captured image before detecting the detection target portion from the acquired captured image of the inspection target, and
    A step of detecting the detection target part from the acquired image using a detector learned using the learning data generated from the image of the detection target part included in the image to be inspected.
    Inspection method to execute.
PCT/JP2021/003830 2020-03-23 2021-02-03 Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method WO2021192627A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2020-051135 2020-03-23
JP2020-051136 2020-03-23
JP2020051136A JP7449739B2 (en) 2020-03-23 2020-03-23 Inspection system, learning device, learning program, learning method, inspection device, inspection program, inspection method
JP2020051135A JP2021148719A (en) 2020-03-23 2020-03-23 Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method

Publications (1)

Publication Number Publication Date
WO2021192627A1 true WO2021192627A1 (en) 2021-09-30

Family

ID=77891356

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/003830 WO2021192627A1 (en) 2020-03-23 2021-02-03 Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method

Country Status (1)

Country Link
WO (1) WO2021192627A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002328096A (en) * 2001-04-27 2002-11-15 Tokyo Electric Power Co Inc:The Program, method, and system for detecting crack defect generated on structure
JP2017053819A (en) * 2015-09-11 2017-03-16 国立大学法人富山大学 Crack detection method and detection program of concrete
JP6294529B1 (en) * 2017-03-16 2018-03-14 阪神高速技術株式会社 Crack detection processing apparatus and crack detection processing program
KR101926561B1 (en) * 2018-03-13 2018-12-07 연세대학교 산학협력단 Road crack detection apparatus of patch unit and method thereof, and computer program for executing the same

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002328096A (en) * 2001-04-27 2002-11-15 Tokyo Electric Power Co Inc:The Program, method, and system for detecting crack defect generated on structure
JP2017053819A (en) * 2015-09-11 2017-03-16 国立大学法人富山大学 Crack detection method and detection program of concrete
JP6294529B1 (en) * 2017-03-16 2018-03-14 阪神高速技術株式会社 Crack detection processing apparatus and crack detection processing program
KR101926561B1 (en) * 2018-03-13 2018-12-07 연세대학교 산학협력단 Road crack detection apparatus of patch unit and method thereof, and computer program for executing the same

Similar Documents

Publication Publication Date Title
CN110349145B (en) Defect detection method, defect detection device, electronic equipment and storage medium
US20140348415A1 (en) System and method for identifying defects in welds by processing x-ray images
JP4416825B2 (en) Image inspection processing apparatus, image inspection processing method, program, and recording medium
CN101207834A (en) Method for checking digital camera articulation
JP2010091361A (en) Method and device for inspecting image
JP7449739B2 (en) Inspection system, learning device, learning program, learning method, inspection device, inspection program, inspection method
CN114283124A (en) Smudginess detection method, device, equipment and storage medium
JP4610656B2 (en) Inspection device, inspection method, program, and recording medium
WO2021192627A1 (en) Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method
WO2020090290A1 (en) Image classifying device, image inspecting device, and image classifying method
JP2021148719A (en) Inspection system, learning device, learning program, learning method, inspection device, inspection program, and inspection method
JP2021135630A (en) Learning device, image inspection device, learned data set, and learning method
JP2011232302A (en) Image inspection method and image inspection device
CN112465780A (en) Method and device for monitoring abnormal film thickness of insulating layer
Tang et al. Thermal fault diagnosis of complex electrical equipment based on infrared image recognition
JP2004045052A (en) Apparatus and method for inspecting image data
CN113870255B (en) Mini LED product defect detection method and related equipment
EP4010873B1 (en) Use of an hdr image in a visual inspection process
WO2024095721A1 (en) Image processing device and image processing method
JP2002259971A (en) Method for detecting uneven image density and inspection device therefor
JP7300155B2 (en) Teaching device in solid preparation appearance inspection, and teaching method in solid preparation appearance inspection
WO2023157238A1 (en) Inspection device, inspection method, and recording medium
JP5479309B2 (en) Mahalanobis reference space generation method, inspection method and inspection apparatus
JP2004347363A (en) Method for inspecting display quality of electronic display through degree of disorder of luminance distribution
JP4121628B2 (en) Screen inspection method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21777139

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21777139

Country of ref document: EP

Kind code of ref document: A1