WO2018235266A1 - Inspection device, inspection method, and inspection program - Google Patents
Inspection device, inspection method, and inspection program Download PDFInfo
- Publication number
- WO2018235266A1 WO2018235266A1 PCT/JP2017/023213 JP2017023213W WO2018235266A1 WO 2018235266 A1 WO2018235266 A1 WO 2018235266A1 JP 2017023213 W JP2017023213 W JP 2017023213W WO 2018235266 A1 WO2018235266 A1 WO 2018235266A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- abnormality
- partial
- determined
- partial image
- contain
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
Definitions
- the present invention relates to an inspection apparatus, an inspection method, and an inspection program.
- an image of an inspection object flowing in a production line is photographed to detect the presence or absence of an abnormality regarding the inspection object.
- the abnormality of the inspection object is, for example, a scratch or dirt attached to the inspection object, or a foreign substance mixed in the inspection object.
- a first processing unit that classifies a test target signal into normal and non-normal by a first learned neural network, and a second neural network after learning
- a second processing unit which extracts a partial signal including an area other than normal from an inspection target signal classified other than normal by a processing unit, and classifies the type of abnormality with respect to the partial signal.
- the first processing unit can detect the presence or absence of an abnormality regarding the inspection object, and the second processing unit can classify the type of abnormality.
- the detection accuracy of the presence or absence of an abnormality depends on the classification accuracy with which the first processing unit classifies the test target signal as normal and non-normal, and the first processing unit when possible abnormalities are various.
- the classification accuracy of is not sufficiently high, and the detection accuracy of the presence or absence of abnormality is not necessarily high.
- an object of the present invention is to provide an inspection apparatus, an inspection method, and an inspection program capable of accurately detecting the presence or absence of an abnormality related to an inspection object even if there are various possible abnormalities.
- An inspection apparatus includes a first division unit that divides an image of an inspection object into a plurality of first partial images, and a plurality of second partial images different from the plurality of first partial images.
- a second division unit for dividing into a plurality of first partial images, and a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality;
- a classification unit; and a second classification unit for classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality.
- a determination unit that determines the presence or absence of an abnormality related to the inspection object based on the overlap between the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality And.
- the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided.
- the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
- the determination unit determines that the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality overlap with each other at the overlapping portion. It may be determined that exists.
- the determination unit does not include the case where the first partial image determined to include an abnormality and the second partial image determined to not include an abnormality or the abnormality is not included. If the first partial image determined to be superimposed on the second partial image determined to contain an abnormality, the first partial image determined to contain the abnormality or the abnormality is It may be determined that no abnormality exists in the portion of the second partial image determined to be included.
- the first partial image determined to include the abnormality
- the erroneous determination of the existence of the abnormality is prevented.
- the first partial image determined to contain an abnormality and the first partial image determined to not contain an abnormality are distinguishably displayed, and it is determined that an abnormality is included.
- the display device may further include a display unit configured to distinguishably display the second partial image and the second partial image that is determined not to include an abnormality, and to display the image.
- the display unit may distinguishably display the first partial image determined to include an abnormality and the second partial image determined to include an abnormality.
- the difference between the classification results by the first classification unit and the second classification unit can be obtained. It can be easily grasped.
- each of the plurality of second partial images may be smaller than each of the plurality of first partial images.
- an abnormality is included in the plurality of first partial images and the plurality of second partial images smaller than each of the plurality of first partial images. Even in the case where there are various items, it is possible to accurately determine the presence or absence and presence of an abnormality.
- At least two of the plurality of second partial images may overlap with any one of the plurality of first partial images.
- the size and the shape of the abnormality are various. Even in this case, the presence or absence of an abnormality can be determined with high accuracy.
- the first division unit divides the image such that the plurality of first partial images do not overlap each other
- the second division divides the image such that the plurality of second partial images do not overlap each other It is also good.
- the first classification unit includes a first learned image in which the first partial image including the abnormality and the first partial image not including the abnormality are learned as the learning data
- the classification unit may include a second learned image in which the second partial image containing the abnormality and the second partial image not containing the abnormality are learned as learning data.
- the first classification unit and the second classification unit classify presence or absence of abnormality from different viewpoints. can do.
- the computer divides the image of the inspection object into a plurality of first partial images, and a plurality of second portions different from the plurality of first partial images. Dividing into a plurality of images, and classifying the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. Classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality; Determining the presence or absence of an abnormality related to the inspection object based on the overlap between the first partial image to be determined and the second partial image determined to contain an abnormality.
- the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided.
- the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
- An inspection program includes a first division unit that divides a computer into an image of an inspection object into a plurality of first partial images, and a plurality of first images different from the plurality of first partial images.
- a second division unit that divides the image into two partial images, and a plurality of first partial images are classified into a first partial image that is determined to contain an abnormality and a first partial image that is determined to contain no abnormality.
- a first classification unit for classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality;
- the inspection target object based on the overlap between the second partial portion different from the second portion and the first partial image determined to include an abnormality and the second partial image determined to include an abnormality.
- the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided.
- the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
- an inspection apparatus an inspection method, and an inspection program that can accurately detect the presence or absence of an abnormality related to an inspection object even if there are various possible abnormalities.
- FIG. 1 is a diagram showing a network configuration of an inspection apparatus 10 according to an embodiment of the present invention.
- the inspection apparatus 10 is a dedicated or general-purpose computer, and may be, for example, a general-purpose computer on which an inspection program according to the present embodiment is installed.
- the inspection apparatus 10 is connected to the learning image database DB and the camera 30 via the communication network N.
- the communication network N is a wired or wireless communication network, and may be, for example, a LAN (Local Area Network) or the Internet.
- the inspection apparatus 10 acquires an image of the inspection object 100 captured by the camera 30 via the communication network N, and determines whether the inspection object 100 has an abnormality based on a learned model.
- the learned model is, for example, a learned neural network.
- the inspection apparatus 10 learns a model using an image stored in the learning image database DB.
- the camera 30 captures an image of the inspection object 100. All or part of the image captured by the camera 30 is stored in the learning image database DB.
- the camera 30 may be a general digital camera that captures visible light, but may be a camera that captures infrared light, X-rays, or the like, or the inspection object 100 may be other than light as with an electron microscope or the like. It may be a camera for shooting.
- FIG. 2 is a diagram showing a physical configuration of the inspection apparatus 10 according to the embodiment of the present invention.
- the inspection device 10 includes a central processing unit (CPU) 10a corresponding to a hardware processor, a random access memory (RAM) 10b corresponding to a memory, a read only memory (ROM) 10c corresponding to a memory, and a communication interface 10d. , An input unit 10e, and a display unit 10f. These components are mutually connected so as to be able to transmit and receive data via a bus.
- CPU central processing unit
- RAM random access memory
- ROM read only memory
- FIG. 2 is a diagram showing a physical configuration of the inspection apparatus 10 according to the embodiment of the present invention.
- the inspection device 10 includes a central processing unit (CPU) 10a corresponding to a hardware processor, a random access memory (RAM) 10b corresponding to a memory, a read only memory (ROM) 10c corresponding to a memory, and a communication interface 10d. , An input unit 10e
- the CPU 10a is a control unit that performs control related to the execution of a program stored in the RAM 10b or the ROM 10c, and performs calculation and processing of data.
- the CPU 10a is an arithmetic device that executes a program (inspection program) related to inspection of an inspection object.
- the CPU 10a receives various input data from the input unit 10e and the communication interface 10d, and displays the calculation result of the input data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.
- the RAM 10 b is a storage unit capable of rewriting data, and is formed of, for example, a semiconductor storage element.
- the RAM 10 b stores programs and data such as applications executed by the CPU 10 a.
- the ROM 10 c is a storage unit that can only read data, and is configured of, for example, a semiconductor storage element.
- the ROM 10 c stores, for example, programs and data such as firmware.
- the communication interface 10 d is an interface that connects the inspection device 10 to the communication network N.
- the input unit 10 e receives an input of data from the user, and includes, for example, a keyboard, a mouse, and a touch panel.
- the display unit 10 f visually displays the calculation result by the CPU 10 a, and is configured of, for example, an LCD (Liquid Crystal Display).
- the inspection program may be stored in a computer-readable storage medium such as the RAM 10 b or the ROM 10 c and provided, or may be provided via the communication network N connected by the communication interface 10 d.
- the CPU 10a executes an inspection program to realize various functions described with reference to the following drawings. Note that these physical configurations are exemplifications and may not necessarily be independent configurations.
- the inspection apparatus 10 may include an LSI (Large-Scale Integration) in which the CPU 10a, the RAM 10b, and the ROM 10c are integrated.
- the inspection apparatus 10 may also include an arithmetic circuit such as a graphics processing unit (GPU), a field-programmable gate array (FPGA), or an application specific integrated circuit (ASIC).
- GPU graphics processing unit
- FPGA field-programmable gate array
- ASIC application specific integrated circuit
- FIG. 3 is a functional block diagram of the inspection apparatus 10 according to the embodiment of the present invention.
- the inspection apparatus 10 includes an acquisition unit 11, a division unit 12, a classification unit 13, a determination unit 14, and a learning unit 16.
- the acquisition unit 11 acquires an image of the inspection object 100 from the camera 30 or the learning image database DB via the communication network N.
- the dividing unit 12 includes a first dividing unit 12a, a second dividing unit 12b, and a third dividing unit 12c.
- the first division unit 12a divides the image of the inspection object 100 into a plurality of first partial images.
- the second division unit 12 b divides the image of the inspection object 100 into a plurality of second partial images different from the plurality of first partial images.
- the third division unit 12c divides the image of the inspection object 100 into a plurality of third partial images different from any of the plurality of first partial images and the plurality of second partial images.
- Each of the plurality of second partial images may be smaller than each of the plurality of first partial images.
- each of the plurality of third partial images may be smaller than each of the plurality of second partial images.
- at least two of the plurality of second partial images may be superimposed on any one of the plurality of first partial images.
- at least two of the plurality of third partial images may overlap with any one of the plurality of second partial images.
- the first division unit 12a may divide the image of the inspection object 100 so that the plurality of first partial images do not overlap each other. However, the first division unit 12a may divide the image of the inspection object 100 such that all or part of the plurality of first partial images overlap each other. The same applies to the second division 12b and the third division 12c.
- the dividing unit 12 divides the image of the inspection object 100 into rectangular partial images.
- the long side and the short side of the first partial image are respectively longer than the long side and the short side of the second partial image
- the long side and the short side of the second partial image are respectively the long side and the third side of the third partial image. May be longer than the short side.
- the dividing unit 12 may divide the image of the inspection object 100 into a partial image having a shape other than a rectangle, for example, a shape such as a polygon or an ellipse.
- the dividing unit 12 according to the present embodiment divides the image of the inspection object 100 into three types of partial images, but the dividing unit 12 divides the image of the inspection object 100 into at least two types of partial images. It may be divided into four or more types of partial images.
- the classification unit 13 includes a first classification unit 13a, a second classification unit 13b, and a third classification unit 13c.
- the first classification unit 13a classifies the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality.
- the second classification unit 13 b classifies the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality.
- the third classification unit 13c classifies the plurality of third partial images into a third partial image determined to contain an abnormality and a third partial image determined to contain no abnormality.
- the first classification unit 13a may include a first learned image in which a first partial image including an abnormality and a first partial image not including an abnormality are learned as learning data.
- the second classification unit 13b may include a second learned model in which the second partial image including the abnormality and the second partial image not including the abnormality are learned as the learning data.
- the third classification unit 13c may include a third learned model in which a third partial image including an abnormality and a third partial image not including an abnormality are learned as learning data.
- the first learned model, the second learned model, and the third learned model may be learned models in which a common model is learned using different learning data, or different models may be different learning data It may be a learned model learned in.
- the first learned model, the second learned model, and the third learned model may be, for example, a learned neural network, but may be any model such as a support vector machine.
- the classification unit 13 may not include a learned model, and may classify whether or not an abnormality is included in a partial image by a predetermined method. For example, the classification unit 13 classifies a partial image including an abnormality and a partial image not including an abnormality by calculating an evaluation value based on the partial image and comparing the evaluation value with a threshold. It may be
- the learning unit 16 acquires a first partial image, a second partial image, and a third partial image for learning from the learning image database DB, and the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. Let each learn. Here, learning is to improve the classification accuracy of the classification unit 13 by inputting learning data to the classification unit 13 and correcting the configuration of the classification unit 13 based on the correctness of the output result.
- the learning unit 16 sets a first partial image for learning and a second partial image as learning data.
- the third partial image are input to the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c, respectively, so that the partial image containing an abnormality and the partial image not containing an abnormality can be classified correctly.
- the weighting of the units included in the neural network is optimized so as to improve the evaluation value.
- the determination unit 14 determines whether there is an abnormality related to the inspection object 100 based on an overlap between the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality. Determine The determination unit 14 may determine the presence or absence of an abnormality regarding the inspection object 100 based on the overlap of at least two types of partial images, but the first partial image determined to contain an abnormality and the abnormality The presence or absence of abnormality regarding the inspection object 100 may be determined based on the overlap between the second partial image determined to be included and the third partial image determined to include an abnormality.
- the determination unit 14 determines that there is an abnormality in the overlapping portion. You may judge. The determination unit 14 may determine that an abnormality exists in the overlapping portion when a partial image including an abnormality is superimposed on at least two types of partial images, but it is determined that an abnormality is included. When the first partial image of the subject, the second partial image determined to contain the abnormality, and the third partial image determined to contain the abnormality overlap, an abnormality is generated in the overlapping portion. It may be determined that it exists.
- the determination unit 14 determines that the first partial image determined to include an abnormality and the second partial image determined to not include an abnormality overlap or do not include an abnormality.
- the first partial image determined to include the abnormality or the abnormality is included It may be determined that no abnormality is present in the portion of the second partial image determined to be corrected.
- the determination unit 14 includes an abnormality when at least two types of partial images to be superimposed include an abnormality in one of the partial images and an abnormality is not included in the other partial image.
- an abnormality is included, but other types of partial images If no abnormality is included, it may be determined that there is no abnormality in a part determined to include the abnormality.
- the display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the first partial image determined to not contain an abnormality, and it is determined that an abnormality is included. And an image of the inspection object 100 is displayed in a distinguishable manner. In addition, the display unit 10 f distinguishably displays the third partial image determined to contain an abnormality and the third partial image determined to contain no abnormality, thereby displaying the image of the inspection object 100. You may display Furthermore, the display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality, and the abnormality is included. The first partial image determined to be present and the second partial image determined to contain an abnormality may be displayed distinguishably from the third partial image determined to contain an abnormality.
- FIG. 4 is a view showing an image P of the inspection object 100 acquired by the inspection apparatus 10 according to the embodiment of the present invention.
- the image P is an image obtained by photographing the inspection object 100 from a predetermined direction.
- the image P shows that the inspection object 100 has a scratch S, and shows the case where the inspection object 100 has an abnormality.
- the case where it is determined whether or not there is an abnormality in the inspection object 100 based on a single image P will be described. It may be determined whether or not the object 100 has an abnormality.
- FIG. 5 is a view showing a first partial image P1 divided by the inspection apparatus 10 according to the embodiment of the present invention.
- the first division unit 12a divides the image P into sixteen first partial images P1.
- the first partial images P1 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of first partial images P1 do not overlap with each other, it is determined whether or not there is an abnormality in the inspection object 100 without excessively increasing the number of the plurality of first partial images P1. Thus, the processing load of the first classification unit 13a can be reduced.
- the first classification unit 13a classifies the sixteen first partial images P1 into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. In the case of this example, the first classification unit 13a determines that one first partial image of the 16 first partial images P1 includes an abnormality, and the other 15 first partial images are determined. Determines that no abnormality is included.
- the display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the first partial image determined to contain no abnormality.
- the display unit 10 f emphasizes the first partial image P 1 a determined to include an abnormality by the first type of hatching, and determines that the first partial image P 1 a does not include an abnormality.
- the partial image P1 is indicated by a broken line. Thereby, it is possible to visually recognize in what part of the inspection object 100 the abnormality is determined to be included, and the classification result by the first classification unit 13a can be easily grasped. .
- the mode in which the first partial image that is determined to be abnormal by the display unit 10 f and the first partial image that is determined to be abnormal is distinguishably displayed is optional, and the abnormality is included.
- the hatching display of the first partial image P1a that is determined to be performed may be subjected to transmission processing to be displayed so that the flaw S can be visually recognized.
- FIG. 6 is a view showing a second partial image P2 divided by the inspection device 10 according to the embodiment of the present invention.
- the second division unit 12b divides the image P into 25 second partial images P2.
- the second partial images P2 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of second partial images P2 do not overlap with each other, it is possible to determine the presence / absence of abnormality of the inspection object 100 without excessively increasing the number of the plurality of second partial images P2. Thus, the processing load of the second classification unit 13b can be reduced.
- the second partial image P2 is smaller than the first partial image P1, and at least two of the second partial images P2 overlap with any one of the plurality of first partial images P1. It is a case where the size and the shape of the abnormality are various by determining whether the abnormality is included in the first partial image P1 and the at least two second partial images P2 to be superimposed on the first partial image P1. Even in this case, the presence or absence of an abnormality can be determined with high accuracy.
- the second classification unit 13b classifies the 25 second partial images P2 into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality. In the case of this example, the second classification unit 13 b determines that three second partial images of the 25 second partial images P 2 contain an abnormality, and the other 22 second partial images are determined. Determines that no abnormality is included.
- the display unit 10 f distinguishably displays the second partial image determined to contain an abnormality and the second partial image determined to contain no abnormality.
- the display unit 10 f emphasizes the second partial images P 2 a, P 2 b, and P 2 c determined to contain an abnormality by the second type of hatching, and determines that no abnormality is included.
- the second partial image P2 to be displayed is indicated by a broken line. Thereby, it is possible to visually recognize which part of the inspection object 100 is determined to contain an abnormality, and the classification result by the second classification unit 13 b can be easily grasped. .
- the display unit 10 f distinguishably displays the first partial image P1 a determined to include an abnormality and the second partial images P2 a, P2 b, and P2 c determined to include an abnormality. Specifically, although the display unit 10f shows the first partial image P1a determined to contain an abnormality by the first type of hatching, the second partial image determined to contain an abnormality. P2a, P2b and P2c are indicated by hatching of the second type.
- the second partial images P2b and P2c determined to contain an abnormality by the second classification unit 13b do not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality in the portion of the second partial images P2b and P2c determined to contain an abnormality. The presence or absence of abnormality can be determined correctly.
- FIG. 7 is a view showing a third partial image P3 divided by the inspection device 10 according to the embodiment of the present invention.
- the third division unit 12c divides the image P into 64 third partial images P3.
- the third partial images P3 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of third partial images P3 do not overlap with each other, it is possible to determine the presence / absence of abnormality of the inspection object 100 without excessively increasing the number of the plurality of third partial images P3. Thus, the processing load of the third classification unit 13c can be reduced.
- the third partial image P3 is smaller than the first partial image P1 and the second partial image P2, and at least two of the third partial images P3 overlap with any one of the plurality of first partial images P1. And overlap with any one of the plurality of second partial images P2.
- the third classification unit 13c classifies the 64 third partial images P3 into a third partial image determined to contain an abnormality and a third partial image determined to contain no abnormality. In the case of this example, the third classification unit 13c determines that four third partial images of the 64 third partial images P3 include an abnormality, and the other 60 third partial images are determined. Determines that no abnormality is included.
- the display unit 10 f distinguishably displays the third partial image determined to contain an abnormality and the third partial image determined to contain no abnormality.
- the display unit 10 f emphasizes the third partial images P 3 a, P 3 b, P 3 c, and P 3 d determined to contain an abnormality by the third type of hatching, and does not include an abnormality.
- the third partial image P3 determined to be represented by a broken line is shown.
- the display unit 10f includes the first partial image P1a determined to contain an abnormality, the second partial images P2a, P2b, and P2c determined to contain an abnormality, and the abnormality.
- the third partial images P3a, P3b, P3c, and P3d determined to be present are displayed in a distinguishable manner.
- the display unit 10f indicates the first partial image P1a determined to contain an abnormality by the first type of hatching, and the second partial image P2a determined to contain an abnormality.
- P2b and P2c are indicated by the second type of hatching
- third partial images P3a, P3b, P3c and P3d determined to contain an abnormality are indicated by the third type of hatching.
- the third partial image P3d determined to contain an abnormality by the third classification unit 13c does not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality in the portion of the third partial image P3 d that is determined to contain an abnormality; The presence or absence of can be determined correctly.
- FIG. 8 is a view showing an image P of the inspection object 100 displayed on the display unit 10 f of the inspection apparatus 10 according to the embodiment of the present invention.
- the display unit 10 f includes the first partial image P 1 a determined to contain an abnormality, the second partial images P 2 a, P 2 b, and P 2 c determined to contain an abnormality, and the abnormality is included.
- the third partial images P3a, P3b, P3c, and P3d to be determined are displayed in a distinguishable manner.
- the classification result by the first classification unit 13a, the classification result by the second classification unit 13b, and the classification result by the third classification unit 13c are visually distinguishably indicated, so that the first classification unit 13a, the second classification, It is possible to easily grasp the difference in the classification result by the unit 13 b and the third classification unit 13 c.
- the determination unit 14 of the inspection apparatus 10 includes the first partial image P1a determined to include an abnormality, the second partial images P2a, P2b, and P2c determined to include an abnormality, and the abnormality. Based on the overlap with the third partial images P3a, P3b, P3c, and P3d that are determined to be present, it is determined whether or not there is an abnormality regarding the inspection object 100.
- the image P is divided into a plurality of first partial images P1, a plurality of second partial images P2, and a plurality of third partial images P3, and the first partial image,
- the number of erroneous determinations is small by determining the presence or absence of an abnormality regarding the inspection object based on the overlap of the partial images Therefore, even if there are various possible abnormalities, it is possible to accurately detect the presence or absence of an abnormality regarding the inspection object.
- the determination unit 14 determines that an abnormality is present in the overlapping portion R. As described above, by determining that an abnormality exists in the superimposed portion of a plurality of types of partial images determined to include an abnormality, not only the presence or absence of an abnormality related to the inspection object, but also the portion where the abnormality exists It can be identified well.
- the determination unit 14 determines that the second partial images P2b and P2c determined to contain an abnormality and the third partial image P3d determined to contain an abnormality do not include an abnormality.
- the second partial images P2b and P2c that are determined to include an abnormality, or the third partial image P3d that is determined to include an abnormality in order to overlap with another type of partial image to be It determines that there is no abnormality.
- it is determined that any one of the first partial image, the second partial image, and the third partial image to be superimposed includes an abnormality, and the other type of partial image is determined not to include an abnormality. If it is determined that there is no abnormality in the portion of the first partial image, the second partial image or the third partial image that is determined to contain the abnormality, erroneous determination of the existence of the abnormality Is prevented.
- each of the plurality of second partial images P2 is smaller than each of the plurality of first partial images P1
- each of the plurality of third partial images P3 is smaller than each of the plurality of second partial images P2 Even in the case of various sizes and shapes, the presence or absence of an abnormality can be determined with high accuracy.
- FIG. 9 is a flowchart of a first process performed by the inspection apparatus 10 according to the embodiment of the present invention.
- the first process divides the image P of the inspection object 100 into a first partial image P1, a second partial image P2, and a third partial image P3, determines the presence or absence of an abnormality, and includes an abnormality. It is the process which determines the presence or absence of abnormality regarding the test object 100 based on the overlap of the partial image determined to be.
- the inspection apparatus 10 first acquires an image P of the inspection object 100 (S10). Then, the image P is divided into a plurality of first partial images P1 by the first division unit 12a (S11), and the image P is divided into a plurality of second partial images P2 by the second division unit 12b (S12). Are divided into a plurality of third partial images P3 by the third division unit 12c (S13).
- the first classification unit 13a classifies the plurality of first partial images P1 into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality.
- the second classification unit 13b determines that the plurality of second partial images P2 are determined to contain no abnormality and the second partial image determined to contain no abnormality.
- the third classification unit 13c determines that the plurality of third partial images P3 are determined not to contain an abnormality and the third partial image determined to contain an abnormality.
- the image is classified into partial images (S16).
- the display unit 10 f determines that the first partial image determined to include an abnormality, the second partial image determined to include an abnormality, and the third part determined to include an abnormality.
- the images are displayed in a distinguishable manner (S17).
- the determination unit 14 determines that the first partial image determined to contain an abnormality, the second partial image determined to contain an abnormality, and the third part determined to contain an abnormality. It is determined whether the image is to be superimposed (S18). When it is determined that these three types of partial images are to be superimposed (S18: Yes), the determination unit 14 determines that there is an abnormality in the superimposed portion (S19). On the other hand, when it is determined that these three types of partial images are not superimposed (S18: No), that is, when it is determined that no abnormality is included in at least one type of partial image, the determination unit 14 determines that It is determined that there is no abnormality in the portion of the partial image determined to be included (S20). Thus, the first process ends.
- FIG. 10 is a flowchart of a second process performed by the inspection apparatus 10 according to the embodiment of the present invention.
- the second process is a process that is executed prior to the first process, and partial images and abnormalities in which abnormality is included in the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. This is a process of learning the classification of partial images not included.
- the learning unit 16 of the inspection apparatus 10 acquires a first partial image for learning from the learning image database DB (S30). Similarly, the learning unit 16 acquires a second partial image for learning (S31) and acquires a third partial image for learning from the learning image database DB (S32).
- the first partial image for learning includes a first partial image including an abnormality and a first partial image not including an abnormality, and each first partial image is tagged with the presence or absence of an abnormality. It is done.
- the second partial image for learning includes a second partial image containing an abnormality and a second partial image not containing an abnormality, and each second partial image is tagged with the presence or absence of an abnormality.
- the third partial image for learning includes a third partial image containing an abnormality and a third partial image not containing an abnormality, and each third partial image is tagged with the presence or absence of an abnormality. It is done.
- the learning unit 16 executes the learning of the first classification unit 13a using the first partial image for learning (S33), and executes the learning of the second classification unit 13b using the second partial image for learning (S33).
- the third classification unit 13c executes learning using the third partial image for learning (S35). Specifically, when the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c include a neural network, the learning unit 16 calculates a first partial image, a second partial image, and a third portion for learning. An image is input to each of the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c, and it is evaluated whether the partial image including the abnormality and the partial image not including the abnormality can be correctly classified. Based on the values, the weightings of the units included in the neural network are optimized to improve the evaluation value.
- the second process ends, and the first classification unit 13a includes the first learned model, the second classification unit 13b includes the second learned model, and the third classification unit 13c It will include the third learned model.
- the first learned model, the second learned model, and the third learned model are different learning data (the first partial image for learning, the second partial image for learning, and the third portion for learning) It may be a model having a different configuration learned using images).
- the first classification unit 13a includes a first learned neural network
- the second classification unit 13b includes a second learned neural network having a configuration different from that of the first learned neural network.
- the unit 13c may include a third learned neural network that is different in configuration from both the first learned neural network and the second learned neural network.
Landscapes
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Immunology (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Pathology (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
The purpose of the present invention is to provide an inspection device, inspection method, and inspection program that make it possible to accurately detect whether there is an abnormality in an object of inspection even if it is possible for various abnormalities to occur. This inspection device 10 comprises a first division unit for dividing an image of an object of inspection 100 into a plurality of first partial images, a second division unit for dividing the image into a plurality of second partial images different from the first partial images, a first classification unit for classifying the plurality of first partial images into first partial images determined to include an abnormality and first partial images determined not to include an abnormality, a second classification unit for classifying the plurality of second partial images into second partial images determined to include an abnormality and second partial images determined not to include an abnormality, and a determination unit for determining whether there is an abnormality in the object of inspection 100 on the basis of overlapping between the first partial images determined to include an abnormality and the second partial images determined to include an abnormality.
Description
本発明は、検査装置、検査方法及び検査プログラムに関する。
The present invention relates to an inspection apparatus, an inspection method, and an inspection program.
従来、生産ラインを流れる検査対象物の画像を撮影して、検査対象物に関する異常の有無を検出することがあった。検査対象物の異常は、例えば検査対象物に付いている傷や汚れであったり、検査対象物に混入した異物であったりする。
Conventionally, an image of an inspection object flowing in a production line is photographed to detect the presence or absence of an abnormality regarding the inspection object. The abnormality of the inspection object is, for example, a scratch or dirt attached to the inspection object, or a foreign substance mixed in the inspection object.
異常検出に関して、例えば下記特許文献1には、学習済みの第1のニューラルネットワークによって検査対象信号を正常と正常以外とに分類する第1処理部と、学習済みの第2のニューラルネットワークによって、第1処理部により正常以外に分類された検査対象信号から正常以外の領域を含む部分信号を抽出し、部分信号について異常の種類を分類する第2処理部と、を備える検査装置が記載されている。
Regarding abnormality detection, for example, in Patent Document 1 below, a first processing unit that classifies a test target signal into normal and non-normal by a first learned neural network, and a second neural network after learning A second processing unit is disclosed which extracts a partial signal including an area other than normal from an inspection target signal classified other than normal by a processing unit, and classifies the type of abnormality with respect to the partial signal. .
特許文献1に記載の検査装置によれば、第1処理部によって検査対象物に関する異常の有無を検出することができ、第2処理部によって異常の種類を分類することができる。しかしながら、異常の有無の検出精度は、第1処理部によって検査対象信号を正常と正常以外とに分類する分類精度に依存しており、起こり得る異常が多様である場合には、第1処理部の分類精度が十分に高くならず、必ずしも異常の有無の検出精度が高くなかった。
According to the inspection apparatus described in Patent Document 1, the first processing unit can detect the presence or absence of an abnormality regarding the inspection object, and the second processing unit can classify the type of abnormality. However, the detection accuracy of the presence or absence of an abnormality depends on the classification accuracy with which the first processing unit classifies the test target signal as normal and non-normal, and the first processing unit when possible abnormalities are various. The classification accuracy of is not sufficiently high, and the detection accuracy of the presence or absence of abnormality is not necessarily high.
そこで、本発明は、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる検査装置、検査方法及び検査プログラムを提供することを目的とする。
Therefore, an object of the present invention is to provide an inspection apparatus, an inspection method, and an inspection program capable of accurately detecting the presence or absence of an abnormality related to an inspection object even if there are various possible abnormalities.
本発明の一態様に係る検査装置は、検査対象物の画像を複数の第1部分画像に分割する第1分割部と、画像を、複数の第1部分画像とは異なる複数の第2部分画像に分割する第2分割部と、複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する第1分類部と、複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する第2分類部と、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像との重なりに基づいて、検査対象物に関する異常の有無を判定する判定部と、を備える。
An inspection apparatus according to an aspect of the present invention includes a first division unit that divides an image of an inspection object into a plurality of first partial images, and a plurality of second partial images different from the plurality of first partial images. A second division unit for dividing into a plurality of first partial images, and a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality; A classification unit; and a second classification unit for classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality. A determination unit that determines the presence or absence of an abnormality related to the inspection object based on the overlap between the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality And.
この態様によれば、画像を複数の第1部分画像と、複数の第1部分画像とは異なる複数の第2部分画像とに分割し、複数の第1部分画像と複数の第2部分画像それぞれに異常が含まれていると判定される場合に、それらの部分画像の重なりに基づいて検査対象物に関する異常の有無を判定することで、誤判定が少なくなり、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる。
According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided. In the case where it is determined that an abnormality is included in the image, by determining the presence or absence of an abnormality regarding the inspection object based on the overlap of the partial images, the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
上記態様において、判定部は、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像とが重畳する場合に、当該重畳箇所に異常が存在すると判定してもよい。
In the above aspect, when the determination unit determines that the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality overlap with each other at the overlapping portion. It may be determined that exists.
この態様によれば、検査対象物に関する異常の有無のみならず、異常が存在する箇所を精度良く特定することができる。
According to this aspect, it is possible to accurately specify not only the presence or absence of an abnormality regarding the inspection object but also the location where the abnormality exists.
上記態様において、判定部は、異常が含まれていると判定される第1部分画像と、異常が含まれていないと判定される第2部分画像とが重畳する場合又は異常が含まれていないと判定される第1部分画像と、異常が含まれていると判定される第2部分画像とが重畳する場合に、当該異常が含まれていると判定される第1部分画像又は当該異常が含まれていると判定される第2部分画像の箇所には異常が存在しないと判定してもよい。
In the above aspect, the determination unit does not include the case where the first partial image determined to include an abnormality and the second partial image determined to not include an abnormality or the abnormality is not included. If the first partial image determined to be superimposed on the second partial image determined to contain an abnormality, the first partial image determined to contain the abnormality or the abnormality is It may be determined that no abnormality exists in the portion of the second partial image determined to be included.
この態様によれば、重畳する第1部分画像及び第2部分画像のいずれか一方に異常が含まれていると判定される場合に、当該異常が含まれていると判定される第1部分画像又は第2部分画像の箇所には異常が存在しないと判定することで、異常の存在の誤判定が防止される。
According to this aspect, when it is determined that the abnormality is included in any one of the first partial image and the second partial image to be superimposed, the first partial image determined to include the abnormality Alternatively, by determining that an abnormality does not exist in the portion of the second partial image, the erroneous determination of the existence of the abnormality is prevented.
上記態様において、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像を区別可能に表示し、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像を区別可能に表示して、画像を表示する表示部をさらに備えてもよい。
In the above aspect, the first partial image determined to contain an abnormality and the first partial image determined to not contain an abnormality are distinguishably displayed, and it is determined that an abnormality is included. The display device may further include a display unit configured to distinguishably display the second partial image and the second partial image that is determined not to include an abnormality, and to display the image.
この態様によれば、検査対象物のどのような箇所に異常が含まれていると判定されたのか、視覚的に認識することができ、第1分類部及び第2分類部による分類結果を容易に把握することができる。
According to this aspect, it is possible to visually recognize in what part of the inspection object the abnormality is determined to be included, and the classification result by the first classification unit and the second classification unit is easy Can be grasped.
上記態様において、表示部は、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像とを区別可能に表示してもよい。
In the above aspect, the display unit may distinguishably display the first partial image determined to include an abnormality and the second partial image determined to include an abnormality.
この態様によれば、第1分類部による分類結果と、第2分類部による分類結果とが視覚的に区別可能に示されることで、第1分類部と第2分類部による分類結果の差異を容易に把握することができる。
According to this aspect, since the classification result by the first classification unit and the classification result by the second classification unit are visually distinguishably shown, the difference between the classification results by the first classification unit and the second classification unit can be obtained. It can be easily grasped.
上記態様において、複数の第2部分画像それぞれは、複数の第1部分画像それぞれよりも小さくてもよい。
In the above aspect, each of the plurality of second partial images may be smaller than each of the plurality of first partial images.
この態様によれば、複数の第1部分画像と、複数の第1部分画像それぞれよりも小さい複数の第2部分画像とに異常が含まれているか否か判定することで、異常のサイズや形状が多様である場合であっても、異常の有無及び存在箇所を精度良く判定することができる。
According to this aspect, it is determined whether or not an abnormality is included in the plurality of first partial images and the plurality of second partial images smaller than each of the plurality of first partial images. Even in the case where there are various items, it is possible to accurately determine the presence or absence and presence of an abnormality.
上記態様において、複数の第2部分画像のうち少なくとも2つは、複数の第1部分画像のうちいずれか1つと重畳してもよい。
In the above aspect, at least two of the plurality of second partial images may overlap with any one of the plurality of first partial images.
この態様によれば、第1部分画像と、第1部分画像に重畳する少なくとも2つの第2部分画像とに異常が含まれているか否か判定することで、異常のサイズや形状が多様である場合であっても、異常の有無及び存在箇所を精度良く判定することができる。
According to this aspect, by determining whether or not the first partial image and the at least two second partial images superimposed on the first partial image include an abnormality, the size and the shape of the abnormality are various. Even in this case, the presence or absence of an abnormality can be determined with high accuracy.
上記態様において、第1分割部は、複数の第1部分画像が互いに重畳しないように画像を分割し、第2分割部は、複数の第2部分画像が互いに重畳しないように画像を分割してもよい。
In the above aspect, the first division unit divides the image such that the plurality of first partial images do not overlap each other, and the second division divides the image such that the plurality of second partial images do not overlap each other It is also good.
この態様によれば、複数の部分画像の枚数を過度に増大させることなく画像を分割して、検査対象物の異常の有無を判定することができ、第1分類部、第2分類部及び判定部の処理負荷を抑えることができる。
According to this aspect, it is possible to divide the image without excessively increasing the number of the plurality of partial images, and to determine the presence or absence of abnormality of the inspection object, and the first classification unit, the second classification unit, and the determination It is possible to reduce the processing load of the department.
上記態様において、第1分類部は、異常が含まれている第1部分画像及び異常が含まれていない第1部分画像を学習用データとして学習された、第1学習済みモデルを含み、第2分類部は、異常が含まれている第2部分画像及び異常が含まれていない第2部分画像を学習用データとして学習された、第2学習済みモデルを含んでもよい。
In the above aspect, the first classification unit includes a first learned image in which the first partial image including the abnormality and the first partial image not including the abnormality are learned as the learning data, The classification unit may include a second learned image in which the second partial image containing the abnormality and the second partial image not containing the abnormality are learned as learning data.
この態様によれば、それぞれ異なる学習用データで学習された第1学習済みモデル及び第2学習済みモデルを用いることで、第1分類部と第2分類部とで異なる観点から異常の有無を分類することができる。
According to this aspect, by using the first learned model and the second learned model that are learned using different learning data, the first classification unit and the second classification unit classify presence or absence of abnormality from different viewpoints. can do.
本発明の他の態様に係る検査方法は、コンピュータが、検査対象物の画像を複数の第1部分画像に分割するステップと、画像を、複数の第1部分画像とは異なる複数の第2部分画像に分割するステップと、複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類するステップと、複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類するステップと、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像との重なりに基づいて、検査対象物に関する異常の有無を判定するステップと、を含む。
In the inspection method according to another aspect of the present invention, the computer divides the image of the inspection object into a plurality of first partial images, and a plurality of second portions different from the plurality of first partial images. Dividing into a plurality of images, and classifying the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. Classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality; Determining the presence or absence of an abnormality related to the inspection object based on the overlap between the first partial image to be determined and the second partial image determined to contain an abnormality.
この態様によれば、画像を複数の第1部分画像と、複数の第1部分画像とは異なる複数の第2部分画像とに分割し、複数の第1部分画像と複数の第2部分画像それぞれに異常が含まれていると判定される場合に、それらの部分画像の重なりに基づいて検査対象物に関する異常の有無を判定することで、誤判定が少なくなり、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる。
According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided. In the case where it is determined that an abnormality is included in the image, by determining the presence or absence of an abnormality regarding the inspection object based on the overlap of the partial images, the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
本発明の他の態様に係る検査プログラムは、コンピュータを、検査対象物の画像を複数の第1部分画像に分割する第1分割部、画像を、複数の第1部分画像とは異なる複数の第2部分画像に分割する第2分割部、複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する第1分類部、複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する、第1分類部とは異なる第2分類部、及び異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像との重なりに基づいて、検査対象物に関する異常の有無を判定する判定部、として機能させる。
An inspection program according to another aspect of the present invention includes a first division unit that divides a computer into an image of an inspection object into a plurality of first partial images, and a plurality of first images different from the plurality of first partial images. A second division unit that divides the image into two partial images, and a plurality of first partial images are classified into a first partial image that is determined to contain an abnormality and a first partial image that is determined to contain no abnormality. A first classification unit for classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality; The inspection target object based on the overlap between the second partial portion different from the second portion and the first partial image determined to include an abnormality and the second partial image determined to include an abnormality. Function as a determination unit that determines the presence or absence of an abnormality related to
この態様によれば、画像を複数の第1部分画像と、複数の第1部分画像とは異なる複数の第2部分画像とに分割し、複数の第1部分画像と複数の第2部分画像それぞれに異常が含まれていると判定される場合に、それらの部分画像の重なりに基づいて検査対象物に関する異常の有無を判定することで、誤判定が少なくなり、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる。
According to this aspect, the image is divided into a plurality of first partial images and a plurality of second partial images different from the plurality of first partial images, and the plurality of first partial images and the plurality of second partial images are respectively provided. In the case where it is determined that an abnormality is included in the image, by determining the presence or absence of an abnormality regarding the inspection object based on the overlap of the partial images, the erroneous determination is reduced, and the possible abnormality may be various. Even if there is an abnormality, the presence or absence of the inspection object can be detected with high accuracy.
本発明によれば、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる検査装置、検査方法及び検査プログラムを提供することができる。
According to the present invention, it is possible to provide an inspection apparatus, an inspection method, and an inspection program that can accurately detect the presence or absence of an abnormality related to an inspection object even if there are various possible abnormalities.
添付図面を参照して、本発明の実施形態について説明する。なお、各図において、同一の符号を付したものは、同一又は同様の構成を有する。
Embodiments of the present invention will be described with reference to the accompanying drawings. In addition, what attached the same code | symbol in each figure has the same or same structure.
図1は、本発明の実施形態に係る検査装置10のネットワーク構成を示す図である。検査装置10は、専用又は汎用のコンピュータであり、例えば、本実施形態に係る検査プログラムがインストールされた汎用のコンピュータであってよい。検査装置10は、通信ネットワークNを介して、学習用画像データベースDB及びカメラ30と接続される。通信ネットワークNは、有線又は無線の通信網であり、例えばLAN(Local Area Network)やインターネットであってよい。
FIG. 1 is a diagram showing a network configuration of an inspection apparatus 10 according to an embodiment of the present invention. The inspection apparatus 10 is a dedicated or general-purpose computer, and may be, for example, a general-purpose computer on which an inspection program according to the present embodiment is installed. The inspection apparatus 10 is connected to the learning image database DB and the camera 30 via the communication network N. The communication network N is a wired or wireless communication network, and may be, for example, a LAN (Local Area Network) or the Internet.
検査装置10は、カメラ30によって撮影された検査対象物100の画像を、通信ネットワークNを介して取得し、学習済みのモデルによって検査対象物100に異常があるか否かを判定する。ここで、学習済みのモデルとは、例えば学習済みのニューラルネットワークである。検査装置10は、学習用画像データベースDBに格納された画像を用いて、モデルの学習を行う。
The inspection apparatus 10 acquires an image of the inspection object 100 captured by the camera 30 via the communication network N, and determines whether the inspection object 100 has an abnormality based on a learned model. Here, the learned model is, for example, a learned neural network. The inspection apparatus 10 learns a model using an image stored in the learning image database DB.
カメラ30は、検査対象物100の画像を撮影する。カメラ30によって撮影された画像の全部又は一部は、学習用画像データベースDBに格納される。カメラ30は、可視光を撮影する一般的なデジタルカメラであってよいが、赤外線やX線等を撮影するカメラであってもよいし、電子顕微鏡等のように光以外によって検査対象物100を撮影するカメラであってもよい。
The camera 30 captures an image of the inspection object 100. All or part of the image captured by the camera 30 is stored in the learning image database DB. The camera 30 may be a general digital camera that captures visible light, but may be a camera that captures infrared light, X-rays, or the like, or the inspection object 100 may be other than light as with an electron microscope or the like. It may be a camera for shooting.
図2は、本発明の実施形態に係る検査装置10の物理的な構成を示す図である。検査装置10は、ハードウェアプロセッサに相当するCPU(Central Processing Unit)10aと、メモリに相当するRAM(Random Access Memory)10bと、メモリに相当するROM(Read Only Memory)10cと、通信インタフェース10dと、入力部10eと、表示部10fとを有する。これら各構成は、バスを介して相互にデータ送受信可能に接続される。
FIG. 2 is a diagram showing a physical configuration of the inspection apparatus 10 according to the embodiment of the present invention. The inspection device 10 includes a central processing unit (CPU) 10a corresponding to a hardware processor, a random access memory (RAM) 10b corresponding to a memory, a read only memory (ROM) 10c corresponding to a memory, and a communication interface 10d. , An input unit 10e, and a display unit 10f. These components are mutually connected so as to be able to transmit and receive data via a bus.
CPU10aは、RAM10b又はROM10cに記憶されたプログラムの実行に関する制御やデータの演算、加工を行う制御部である。CPU10aは、検査対象物の検査に関するプログラム(検査プログラム)を実行する演算装置である。CPU10aは、入力部10eや通信インタフェース10dから種々の入力データを受け取り、入力データの演算結果を表示部10fに表示したり、RAM10bやROM10cに格納したりする。
The CPU 10a is a control unit that performs control related to the execution of a program stored in the RAM 10b or the ROM 10c, and performs calculation and processing of data. The CPU 10a is an arithmetic device that executes a program (inspection program) related to inspection of an inspection object. The CPU 10a receives various input data from the input unit 10e and the communication interface 10d, and displays the calculation result of the input data on the display unit 10f or stores it in the RAM 10b or the ROM 10c.
RAM10bは、データの書き換えが可能な記憶部であり、例えば半導体記憶素子で構成される。RAM10bは、CPU10aが実行するアプリケーション等のプログラムやデータを記憶する。
The RAM 10 b is a storage unit capable of rewriting data, and is formed of, for example, a semiconductor storage element. The RAM 10 b stores programs and data such as applications executed by the CPU 10 a.
ROM10cは、データの読み出しのみが可能な記憶部であり、例えば半導体記憶素子で構成される。ROM10cは、例えばファームウェア等のプログラムやデータを記憶する。
The ROM 10 c is a storage unit that can only read data, and is configured of, for example, a semiconductor storage element. The ROM 10 c stores, for example, programs and data such as firmware.
通信インタフェース10dは、検査装置10を通信ネットワークNに接続するインタフェースである。
The communication interface 10 d is an interface that connects the inspection device 10 to the communication network N.
入力部10eは、ユーザからデータの入力を受け付けるものであり、例えば、キーボードやマウス、タッチパネルで構成される。
The input unit 10 e receives an input of data from the user, and includes, for example, a keyboard, a mouse, and a touch panel.
表示部10fは、CPU10aによる演算結果を視覚的に表示するものであり、例えばLCD(Liquid Crystal Display)により構成される。
The display unit 10 f visually displays the calculation result by the CPU 10 a, and is configured of, for example, an LCD (Liquid Crystal Display).
検査プログラムは、RAM10bやROM10c等のコンピュータによって読み取り可能な記憶媒体に記憶されて提供されてもよいし、通信インタフェース10dにより接続される通信ネットワークNを介して提供されてもよい。検査装置10では、CPU10aが検査プログラムを実行することにより、次図を用いて説明する様々な機能が実現される。なお、これらの物理的な構成は例示であって、必ずしも独立した構成でなくてもよい。例えば、検査装置10は、CPU10aとRAM10bやROM10cが一体化したLSI(Large-Scale Integration)を備えていてもよい。また、検査装置10は、GPU(Graphics Processing Unit)やFPGA(Field-Programmable Gate Array)、ASIC(Application Specific Integrated Circuit)等の演算回路を備えてもよい。
The inspection program may be stored in a computer-readable storage medium such as the RAM 10 b or the ROM 10 c and provided, or may be provided via the communication network N connected by the communication interface 10 d. In the inspection apparatus 10, the CPU 10a executes an inspection program to realize various functions described with reference to the following drawings. Note that these physical configurations are exemplifications and may not necessarily be independent configurations. For example, the inspection apparatus 10 may include an LSI (Large-Scale Integration) in which the CPU 10a, the RAM 10b, and the ROM 10c are integrated. The inspection apparatus 10 may also include an arithmetic circuit such as a graphics processing unit (GPU), a field-programmable gate array (FPGA), or an application specific integrated circuit (ASIC).
図3は、本発明の実施形態に係る検査装置10の機能ブロック図である。検査装置10は、取得部11、分割部12、分類部13、判定部14及び学習部16を備える。
FIG. 3 is a functional block diagram of the inspection apparatus 10 according to the embodiment of the present invention. The inspection apparatus 10 includes an acquisition unit 11, a division unit 12, a classification unit 13, a determination unit 14, and a learning unit 16.
取得部11は、通信ネットワークNを介して、カメラ30又は学習用画像データベースDBより、検査対象物100の画像を取得する。
The acquisition unit 11 acquires an image of the inspection object 100 from the camera 30 or the learning image database DB via the communication network N.
分割部12は、第1分割部12a、第2分割部12b及び第3分割部12cを含む。第1分割部12aは、検査対象物100の画像を複数の第1部分画像に分割する。第2分割部12bは、検査対象物100の画像を、複数の第1部分画像とは異なる複数の第2部分画像に分割する。また、第3分割部12cは、検査対象物100の画像を、複数の第1部分画像及び複数の第2部分画像のいずれとも異なる複数の第3部分画像に分割する。
The dividing unit 12 includes a first dividing unit 12a, a second dividing unit 12b, and a third dividing unit 12c. The first division unit 12a divides the image of the inspection object 100 into a plurality of first partial images. The second division unit 12 b divides the image of the inspection object 100 into a plurality of second partial images different from the plurality of first partial images. The third division unit 12c divides the image of the inspection object 100 into a plurality of third partial images different from any of the plurality of first partial images and the plurality of second partial images.
複数の第2部分画像それぞれは、複数の第1部分画像それぞれよりも小さくてよい。同様に、複数の第3部分画像それぞれは、複数の第2部分画像それぞれよりも小さくてよい。また、複数の第2部分画像のうち少なくとも2つは、複数の第1部分画像のうちいずれか1つと重畳してよい。同様に、複数の第3部分画像のうち少なくとも2つは、複数の第2部分画像のうちいずれか1つと重畳してよい。
Each of the plurality of second partial images may be smaller than each of the plurality of first partial images. Similarly, each of the plurality of third partial images may be smaller than each of the plurality of second partial images. In addition, at least two of the plurality of second partial images may be superimposed on any one of the plurality of first partial images. Similarly, at least two of the plurality of third partial images may overlap with any one of the plurality of second partial images.
第1分割部12aは、複数の第1部分画像が互いに重畳しないように検査対象物100の画像を分割してよい。もっとも、第1分割部12aは、複数の第1部分画像のうち全部又は一部が互いに重畳するように検査対象物100の画像を分割してもよい。第2分割部12b及び第3分割部12cについても同様である。
The first division unit 12a may divide the image of the inspection object 100 so that the plurality of first partial images do not overlap each other. However, the first division unit 12a may divide the image of the inspection object 100 such that all or part of the plurality of first partial images overlap each other. The same applies to the second division 12b and the third division 12c.
より具体的には、本実施形態に係る分割部12は、検査対象物100の画像を矩形の部分画像に分割する。ここで、第1部分画像の長辺及び短辺は、それぞれ第2部分画像の長辺及び短辺より長く、第2部分画像の長辺及び短辺は、それぞれ第3部分画像の長辺及び短辺より長くてよい。もっとも、分割部12は、検査対象物100の画像を、矩形以外の形状、例えば多角形や楕円等の形状を有する部分画像に分割してもよい。なお、本実施形態に係る分割部12は、検査対象物100の画像を3種類の部分画像に分割するが、分割部12は、検査対象物100の画像を少なくとも2種類の部分画像に分割すればよく、4種類以上の部分画像に分割してもよい。
More specifically, the dividing unit 12 according to the present embodiment divides the image of the inspection object 100 into rectangular partial images. Here, the long side and the short side of the first partial image are respectively longer than the long side and the short side of the second partial image, and the long side and the short side of the second partial image are respectively the long side and the third side of the third partial image. May be longer than the short side. However, the dividing unit 12 may divide the image of the inspection object 100 into a partial image having a shape other than a rectangle, for example, a shape such as a polygon or an ellipse. The dividing unit 12 according to the present embodiment divides the image of the inspection object 100 into three types of partial images, but the dividing unit 12 divides the image of the inspection object 100 into at least two types of partial images. It may be divided into four or more types of partial images.
分類部13は、第1分類部13a、第2分類部13b及び第3分類部13cを含む。第1分類部13aは、複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する。第2分類部13bは、複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する。また、第3分類部13cは、複数の第3部分画像を、異常が含まれていると判定される第3部分画像と異常が含まれていないと判定される第3部分画像に分類する。
The classification unit 13 includes a first classification unit 13a, a second classification unit 13b, and a third classification unit 13c. The first classification unit 13a classifies the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. The second classification unit 13 b classifies the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality. The third classification unit 13c classifies the plurality of third partial images into a third partial image determined to contain an abnormality and a third partial image determined to contain no abnormality.
第1分類部13aは、異常が含まれている第1部分画像及び異常が含まれていない第1部分画像を学習用データとして学習された、第1学習済みモデルを含んでよい。また、第2分類部13bは、異常が含まれている第2部分画像及び異常が含まれていない第2部分画像を学習用データとして学習された、第2学習済みモデルを含んでよい。同様に、第3分類部13cは、異常が含まれている第3部分画像及び異常が含まれていない第3部分画像を学習用データとして学習された、第3学習済みモデルを含んでよい。ここで、第1学習済みモデル、第2学習済みモデル及び第3学習済みモデルは、共通のモデルを異なる学習用データで学習した学習済みモデルであってもよいし、異なるモデルを異なる学習用データで学習した学習済みモデルであってもよい。第1学習済みモデル、第2学習済みモデル及び第3学習済みモデルは、例えば、学習済みのニューラルネットワークであってよいが、サポートベクターマシン等の任意のモデルであってよい。
The first classification unit 13a may include a first learned image in which a first partial image including an abnormality and a first partial image not including an abnormality are learned as learning data. In addition, the second classification unit 13b may include a second learned model in which the second partial image including the abnormality and the second partial image not including the abnormality are learned as the learning data. Similarly, the third classification unit 13c may include a third learned model in which a third partial image including an abnormality and a third partial image not including an abnormality are learned as learning data. Here, the first learned model, the second learned model, and the third learned model may be learned models in which a common model is learned using different learning data, or different models may be different learning data It may be a learned model learned in. The first learned model, the second learned model, and the third learned model may be, for example, a learned neural network, but may be any model such as a support vector machine.
なお、分類部13は、学習済みモデルを含まないものであってもよく、所定の方法で部分画像に異常が含まれているか否かを分類するものであってもよい。例えば、分類部13は、部分画像に基づいて評価値を算出して、評価値と閾値を比較することで、異常が含まれている部分画像と異常が含まれていない部分画像を分類するものであってもよい。
The classification unit 13 may not include a learned model, and may classify whether or not an abnormality is included in a partial image by a predetermined method. For example, the classification unit 13 classifies a partial image including an abnormality and a partial image not including an abnormality by calculating an evaluation value based on the partial image and comparing the evaluation value with a threshold. It may be
学習部16は、学習用画像データベースDBから学習用の第1部分画像、第2部分画像及び第3部分画像を取得して、第1分類部13a、第2分類部13b及び第3分類部13cをそれぞれ学習させる。ここで、学習とは、学習用データを分類部13に入力し、出力された結果の正誤に基づいて分類部13の構成を修正することで、分類部13による分類精度を向上させていくことをいう。具体的には、第1分類部13a、第2分類部13b及び第3分類部13cがニューラルネットワークを含む場合、学習部16は、学習用データとして学習用の第1部分画像、第2部分画像及び第3部分画像を第1分類部13a、第2分類部13b及び第3分類部13cにそれぞれ入力し、異常が含まれている部分画像と異常が含まれていない部分画像とを正しく分類できたかを評価する値に基づいて、当該評価値を向上させるように、ニューラルネットワークに含まれるユニットの重み付けを最適化する。
The learning unit 16 acquires a first partial image, a second partial image, and a third partial image for learning from the learning image database DB, and the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. Let each learn. Here, learning is to improve the classification accuracy of the classification unit 13 by inputting learning data to the classification unit 13 and correcting the configuration of the classification unit 13 based on the correctness of the output result. Say Specifically, when the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c include a neural network, the learning unit 16 sets a first partial image for learning and a second partial image as learning data. And the third partial image are input to the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c, respectively, so that the partial image containing an abnormality and the partial image not containing an abnormality can be classified correctly. Based on the value for evaluating the weight, the weighting of the units included in the neural network is optimized so as to improve the evaluation value.
判定部14は、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像との重なりに基づいて、検査対象物100に関する異常の有無を判定する。判定部14は、少なくとも2種類の部分画像の重なりに基づいて、検査対象物100に関する異常の有無を判定してよいが、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像と、異常が含まれていると判定される第3部分画像との重なりに基づいて、検査対象物100に関する異常の有無を判定してもよい。
The determination unit 14 determines whether there is an abnormality related to the inspection object 100 based on an overlap between the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality. Determine The determination unit 14 may determine the presence or absence of an abnormality regarding the inspection object 100 based on the overlap of at least two types of partial images, but the first partial image determined to contain an abnormality and the abnormality The presence or absence of abnormality regarding the inspection object 100 may be determined based on the overlap between the second partial image determined to be included and the third partial image determined to include an abnormality.
判定部14は、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像とが重畳する場合に、当該重畳箇所に異常が存在すると判定してよい。判定部14は、少なくとも2種類の部分画像について、異常が含まれている部分画像が重畳する場合に、当該重畳箇所に異常が存在すると判定してよいが、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像と、異常が含まれていると判定される第3部分画像とが重畳する場合に、当該重畳箇所に異常が存在すると判定してもよい。
When the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality overlap, the determination unit 14 determines that there is an abnormality in the overlapping portion. You may judge. The determination unit 14 may determine that an abnormality exists in the overlapping portion when a partial image including an abnormality is superimposed on at least two types of partial images, but it is determined that an abnormality is included. When the first partial image of the subject, the second partial image determined to contain the abnormality, and the third partial image determined to contain the abnormality overlap, an abnormality is generated in the overlapping portion. It may be determined that it exists.
また、判定部14は、異常が含まれていると判定される第1部分画像と、異常が含まれていないと判定される第2部分画像とが重畳する場合又は異常が含まれていないと判定される第1部分画像と、異常が含まれていると判定される第2部分画像とが重畳する場合に、当該異常が含まれていると判定される第1部分画像又は当該異常が含まれていると判定される第2部分画像の箇所には異常が存在しないと判定してよい。判定部14は、重畳する少なくとも2種類の部分画像について、いずれか一方の部分画像に異常が含まれており、他方の部分画像に異常が含まれていない場合に、当該異常が含まれていると判定される箇所には異常が存在しないと判定してよいが、重畳する3種類以上の部分画像について、いずれか1種類の部分画像に異常が含まれているが、他の種類の部分画像に異常が含まれていない場合に、当該異常が含まれていると判定される箇所には異常が存在しないと判定してもよい。
In addition, the determination unit 14 determines that the first partial image determined to include an abnormality and the second partial image determined to not include an abnormality overlap or do not include an abnormality. When the first partial image to be determined and the second partial image to be determined to include an abnormality overlap, the first partial image determined to include the abnormality or the abnormality is included It may be determined that no abnormality is present in the portion of the second partial image determined to be corrected. The determination unit 14 includes an abnormality when at least two types of partial images to be superimposed include an abnormality in one of the partial images and an abnormality is not included in the other partial image. Although it may be determined that there is no abnormality in the part determined to be, in one or more partial images of three or more types of partial images to be superimposed, an abnormality is included, but other types of partial images If no abnormality is included, it may be determined that there is no abnormality in a part determined to include the abnormality.
表示部10fは、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像を区別可能に表示し、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像を区別可能に表示して、検査対象物100の画像を表示する。また、表示部10fは、異常が含まれていると判定される第3部分画像と異常が含まれていないと判定される第3部分画像を区別可能に表示して、検査対象物100の画像を表示してよい。さらに、表示部10fは、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像とを区別可能に表示し、異常が含まれていると判定される第1部分画像及び異常が含まれていると判定される第2部分画像と、異常が含まれていると判定される第3部分画像とを区別可能に表示してよい。
The display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the first partial image determined to not contain an abnormality, and it is determined that an abnormality is included. And an image of the inspection object 100 is displayed in a distinguishable manner. In addition, the display unit 10 f distinguishably displays the third partial image determined to contain an abnormality and the third partial image determined to contain no abnormality, thereby displaying the image of the inspection object 100. You may display Furthermore, the display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the second partial image determined to contain an abnormality, and the abnormality is included. The first partial image determined to be present and the second partial image determined to contain an abnormality may be displayed distinguishably from the third partial image determined to contain an abnormality.
図4は、本発明の実施形態に係る検査装置10により取得される検査対象物100の画像Pを示す図である。画像Pは、検査対象物100を所定の方向から撮影した画像である。本例では、画像Pは、検査対象物100に傷Sが付いていることを示しており、検査対象物100に異常が有る場合を示している。なお、本例では、一枚の画像Pにより検査対象物100に異常が有るか否かを判定する場合について説明するが、検査装置10は、検査対象物100に関する複数の画像に基づいて、検査対象物100に異常が有るか否かを判定してもよい。
FIG. 4 is a view showing an image P of the inspection object 100 acquired by the inspection apparatus 10 according to the embodiment of the present invention. The image P is an image obtained by photographing the inspection object 100 from a predetermined direction. In the present example, the image P shows that the inspection object 100 has a scratch S, and shows the case where the inspection object 100 has an abnormality. In the present example, the case where it is determined whether or not there is an abnormality in the inspection object 100 based on a single image P will be described. It may be determined whether or not the object 100 has an abnormality.
図5は、本発明の実施形態に係る検査装置10により分割された第1部分画像P1を示す図である。本例において、第1分割部12aは、画像Pを16枚の第1部分画像P1に分割する。第1部分画像P1は、矩形の画像であり、互いに重畳しない。複数の第1部分画像P1が互いに重畳しないように画像Pを分割することで、複数の第1部分画像P1の枚数を過度に増大させることなく、検査対象物100の異常の有無を判定することができ、第1分類部13aの処理負荷を抑えることができる。
FIG. 5 is a view showing a first partial image P1 divided by the inspection apparatus 10 according to the embodiment of the present invention. In the present example, the first division unit 12a divides the image P into sixteen first partial images P1. The first partial images P1 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of first partial images P1 do not overlap with each other, it is determined whether or not there is an abnormality in the inspection object 100 without excessively increasing the number of the plurality of first partial images P1. Thus, the processing load of the first classification unit 13a can be reduced.
第1分類部13aは、16枚の第1部分画像P1を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する。本例の場合、第1分類部13aは、16枚の第1部分画像P1のうち1枚の第1部分画像に異常が含まれていると判定し、他の15枚の第1部分画像には異常が含まれていないと判定する。
The first classification unit 13a classifies the sixteen first partial images P1 into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. In the case of this example, the first classification unit 13a determines that one first partial image of the 16 first partial images P1 includes an abnormality, and the other 15 first partial images are determined. Determines that no abnormality is included.
表示部10fは、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像を区別可能に表示する。本例の場合、表示部10fは、異常が含まれていると判定される第1部分画像P1aを、第1種のハッチングによって強調して示し、異常が含まれていないと判定される第1部分画像P1を、破線による枠線で示している。これにより、検査対象物100のどのような箇所に異常が含まれていると判定されたのか、視覚的に認識することができ、第1分類部13aによる分類結果を容易に把握することができる。なお、表示部10fによって異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像を区別可能に表示する態様は任意であり、異常が含まれていると判定される第1部分画像P1aのハッチング表示を透過処理して傷Sが視認可能なように表示してもよい。
The display unit 10 f distinguishably displays the first partial image determined to contain an abnormality and the first partial image determined to contain no abnormality. In the case of this example, the display unit 10 f emphasizes the first partial image P 1 a determined to include an abnormality by the first type of hatching, and determines that the first partial image P 1 a does not include an abnormality. The partial image P1 is indicated by a broken line. Thereby, it is possible to visually recognize in what part of the inspection object 100 the abnormality is determined to be included, and the classification result by the first classification unit 13a can be easily grasped. . Note that the mode in which the first partial image that is determined to be abnormal by the display unit 10 f and the first partial image that is determined to be abnormal is distinguishably displayed is optional, and the abnormality is included. The hatching display of the first partial image P1a that is determined to be performed may be subjected to transmission processing to be displayed so that the flaw S can be visually recognized.
図6は、本発明の実施形態に係る検査装置10により分割された第2部分画像P2を示す図である。本例において、第2分割部12bは、画像Pを25枚の第2部分画像P2に分割する。第2部分画像P2は、矩形の画像であり、互いに重畳しない。複数の第2部分画像P2が互いに重畳しないように画像Pを分割することで、複数の第2部分画像P2の枚数を過度に増大させることなく、検査対象物100の異常の有無を判定することができ、第2分類部13bの処理負荷を抑えることができる。
FIG. 6 is a view showing a second partial image P2 divided by the inspection device 10 according to the embodiment of the present invention. In the present example, the second division unit 12b divides the image P into 25 second partial images P2. The second partial images P2 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of second partial images P2 do not overlap with each other, it is possible to determine the presence / absence of abnormality of the inspection object 100 without excessively increasing the number of the plurality of second partial images P2. Thus, the processing load of the second classification unit 13b can be reduced.
また、第2部分画像P2は、第1部分画像P1よりも小さく、第2部分画像P2のうち少なくとも2つは、複数の第1部分画像P1のうちいずれか1つと重畳する。第1部分画像P1と、第1部分画像P1に重畳する少なくとも2つの第2部分画像P2とに異常が含まれているか否か判定することで、異常のサイズや形状が多様である場合であっても、異常の有無及び存在箇所を精度良く判定することができる。
The second partial image P2 is smaller than the first partial image P1, and at least two of the second partial images P2 overlap with any one of the plurality of first partial images P1. It is a case where the size and the shape of the abnormality are various by determining whether the abnormality is included in the first partial image P1 and the at least two second partial images P2 to be superimposed on the first partial image P1. Even in this case, the presence or absence of an abnormality can be determined with high accuracy.
第2分類部13bは、25枚の第2部分画像P2を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する。本例の場合、第2分類部13bは、25枚の第2部分画像P2のうち3枚の第2部分画像に異常が含まれていると判定し、他の22枚の第2部分画像には異常が含まれていないと判定する。
The second classification unit 13b classifies the 25 second partial images P2 into a second partial image determined to contain an abnormality and a second partial image determined to contain no abnormality. In the case of this example, the second classification unit 13 b determines that three second partial images of the 25 second partial images P 2 contain an abnormality, and the other 22 second partial images are determined. Determines that no abnormality is included.
表示部10fは、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像を区別可能に表示する。本例の場合、表示部10fは、異常が含まれていると判定される第2部分画像P2a,P2b,P2cを、第2種のハッチングによって強調して示し、異常が含まれていないと判定される第2部分画像P2を、破線による枠線で示している。これにより、検査対象物100のどのような箇所に異常が含まれていると判定されたのか、視覚的に認識することができ、第2分類部13bによる分類結果を容易に把握することができる。
The display unit 10 f distinguishably displays the second partial image determined to contain an abnormality and the second partial image determined to contain no abnormality. In the case of this example, the display unit 10 f emphasizes the second partial images P 2 a, P 2 b, and P 2 c determined to contain an abnormality by the second type of hatching, and determines that no abnormality is included. The second partial image P2 to be displayed is indicated by a broken line. Thereby, it is possible to visually recognize which part of the inspection object 100 is determined to contain an abnormality, and the classification result by the second classification unit 13 b can be easily grasped. .
また、表示部10fは、異常が含まれていると判定される第1部分画像P1aと、異常が含まれていると判定される第2部分画像P2a,P2b,P2cとを区別可能に表示する。具体的には、表示部10fは、異常が含まれていると判定される第1部分画像P1aを第1種のハッチングで示したが、異常が含まれていると判定される第2部分画像P2a,P2b,P2cを第2種のハッチングで示す。
Further, the display unit 10 f distinguishably displays the first partial image P1 a determined to include an abnormality and the second partial images P2 a, P2 b, and P2 c determined to include an abnormality. . Specifically, although the display unit 10f shows the first partial image P1a determined to contain an abnormality by the first type of hatching, the second partial image determined to contain an abnormality. P2a, P2b and P2c are indicated by hatching of the second type.
第2分類部13bによって異常が含まれていると判定される第2部分画像P2b,P2cには、実際には異常が含まれていない。このような場合であっても、後に詳細に説明するように、検査装置10は、異常が含まれていると判定される第2部分画像P2b,P2cの箇所に異常が存在するとは判定せず、異常の有無を正しく判定することができる。
The second partial images P2b and P2c determined to contain an abnormality by the second classification unit 13b do not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality in the portion of the second partial images P2b and P2c determined to contain an abnormality. The presence or absence of abnormality can be determined correctly.
図7は、本発明の実施形態に係る検査装置10により分割された第3部分画像P3を示す図である。本例において、第3分割部12cは、画像Pを64枚の第3部分画像P3に分割する。第3部分画像P3は、矩形の画像であり、互いに重畳しない。複数の第3部分画像P3が互いに重畳しないように画像Pを分割することで、複数の第3部分画像P3の枚数を過度に増大させることなく、検査対象物100の異常の有無を判定することができ、第3分類部13cの処理負荷を抑えることができる。
FIG. 7 is a view showing a third partial image P3 divided by the inspection device 10 according to the embodiment of the present invention. In the present example, the third division unit 12c divides the image P into 64 third partial images P3. The third partial images P3 are rectangular images and do not overlap each other. By dividing the image P so that the plurality of third partial images P3 do not overlap with each other, it is possible to determine the presence / absence of abnormality of the inspection object 100 without excessively increasing the number of the plurality of third partial images P3. Thus, the processing load of the third classification unit 13c can be reduced.
また、第3部分画像P3は、第1部分画像P1及び第2部分画像P2よりも小さく、第3部分画像P3のうち少なくとも2つは、複数の第1部分画像P1のうちいずれか1つと重畳し、複数の第2部分画像P2のうちいずれか1つと重畳する。第1部分画像P1及び第2部分画像P2と、第1部分画像P1及び第2部分画像P2に重畳する少なくとも2つの第3部分画像P3とに異常が含まれているか否か判定することで、異常のサイズや形状が多様である場合であっても、異常の有無及び存在箇所を精度良く判定することができる。
The third partial image P3 is smaller than the first partial image P1 and the second partial image P2, and at least two of the third partial images P3 overlap with any one of the plurality of first partial images P1. And overlap with any one of the plurality of second partial images P2. By determining whether or not an abnormality is included in the first partial image P1 and the second partial image P2 and at least two third partial images P3 superimposed on the first partial image P1 and the second partial image P2, Even if the size and shape of the abnormality are various, the presence or absence of the abnormality can be determined with high accuracy.
第3分類部13cは、64枚の第3部分画像P3を、異常が含まれていると判定される第3部分画像と異常が含まれていないと判定される第3部分画像に分類する。本例の場合、第3分類部13cは、64枚の第3部分画像P3のうち4枚の第3部分画像に異常が含まれていると判定し、他の60枚の第3部分画像には異常が含まれていないと判定する。
The third classification unit 13c classifies the 64 third partial images P3 into a third partial image determined to contain an abnormality and a third partial image determined to contain no abnormality. In the case of this example, the third classification unit 13c determines that four third partial images of the 64 third partial images P3 include an abnormality, and the other 60 third partial images are determined. Determines that no abnormality is included.
表示部10fは、異常が含まれていると判定される第3部分画像と異常が含まれていないと判定される第3部分画像を区別可能に表示する。本例の場合、表示部10fは、異常が含まれていると判定される第3部分画像P3a,P3b,P3c,P3dを、第3種のハッチングによって強調して示し、異常が含まれていないと判定される第3部分画像P3を、破線による枠線で示している。ここで、表示部10fは、異常が含まれていると判定される第1部分画像P1a及び異常が含まれていると判定される第2部分画像P2a,P2b,P2cと、異常が含まれていると判定される第3部分画像P3a,P3b,P3c,P3dとを区別可能に表示する。具体的には、表示部10fは、異常が含まれていると判定される第1部分画像P1aを第1種のハッチングで示し、異常が含まれていると判定される第2部分画像P2a,P2b,P2cを第2種のハッチングで示したが、異常が含まれていると判定される第3部分画像P3a,P3b,P3c,P3dを第3種のハッチングで示す。
The display unit 10 f distinguishably displays the third partial image determined to contain an abnormality and the third partial image determined to contain no abnormality. In the case of this example, the display unit 10 f emphasizes the third partial images P 3 a, P 3 b, P 3 c, and P 3 d determined to contain an abnormality by the third type of hatching, and does not include an abnormality. The third partial image P3 determined to be represented by a broken line is shown. Here, the display unit 10f includes the first partial image P1a determined to contain an abnormality, the second partial images P2a, P2b, and P2c determined to contain an abnormality, and the abnormality. The third partial images P3a, P3b, P3c, and P3d determined to be present are displayed in a distinguishable manner. Specifically, the display unit 10f indicates the first partial image P1a determined to contain an abnormality by the first type of hatching, and the second partial image P2a determined to contain an abnormality. Although P2b and P2c are indicated by the second type of hatching, third partial images P3a, P3b, P3c and P3d determined to contain an abnormality are indicated by the third type of hatching.
第3分類部13cによって異常が含まれていると判定される第3部分画像P3dには、実際には異常が含まれていない。このような場合であっても、後に詳細に説明するように、検査装置10は、異常が含まれていると判定される第3部分画像P3dの箇所に異常が存在するとは判定せず、異常の有無を正しく判定することができる。
The third partial image P3d determined to contain an abnormality by the third classification unit 13c does not actually include an abnormality. Even in such a case, as will be described in detail later, the inspection apparatus 10 does not determine that there is an abnormality in the portion of the third partial image P3 d that is determined to contain an abnormality; The presence or absence of can be determined correctly.
図8は、本発明の実施形態に係る検査装置10の表示部10fに表示される検査対象物100の画像Pを示す図である。表示部10fは、異常が含まれていると判定される第1部分画像P1aと、異常が含まれていると判定される第2部分画像P2a,P2b,P2cと、異常が含まれていると判定される第3部分画像P3a,P3b,P3c,P3dとを区別可能に表示する。第1分類部13aによる分類結果と、第2分類部13bによる分類結果と、第3分類部13cによる分類結果とが視覚的に区別可能に示されることで、第1分類部13a、第2分類部13b及び第3分類部13cによる分類結果の差異を容易に把握することができる。
FIG. 8 is a view showing an image P of the inspection object 100 displayed on the display unit 10 f of the inspection apparatus 10 according to the embodiment of the present invention. The display unit 10 f includes the first partial image P 1 a determined to contain an abnormality, the second partial images P 2 a, P 2 b, and P 2 c determined to contain an abnormality, and the abnormality is included. The third partial images P3a, P3b, P3c, and P3d to be determined are displayed in a distinguishable manner. The classification result by the first classification unit 13a, the classification result by the second classification unit 13b, and the classification result by the third classification unit 13c are visually distinguishably indicated, so that the first classification unit 13a, the second classification, It is possible to easily grasp the difference in the classification result by the unit 13 b and the third classification unit 13 c.
検査装置10の判定部14は、異常が含まれていると判定される第1部分画像P1aと、異常が含まれていると判定される第2部分画像P2a,P2b,P2cと、異常が含まれていると判定される第3部分画像P3a,P3b,P3c,P3dとの重なりに基づいて、検査対象物100に関する異常の有無を判定する。
The determination unit 14 of the inspection apparatus 10 includes the first partial image P1a determined to include an abnormality, the second partial images P2a, P2b, and P2c determined to include an abnormality, and the abnormality. Based on the overlap with the third partial images P3a, P3b, P3c, and P3d that are determined to be present, it is determined whether or not there is an abnormality regarding the inspection object 100.
本実施形態に係る検査装置10によれば、画像Pを複数の第1部分画像P1と、複数の第2部分画像P2と、複数の第3部分画像P3とに分割し、第1部分画像、第2部分画像及び第3部分画像それぞれに異常が含まれていると判定される場合に、それらの部分画像の重なりに基づいて検査対象物に関する異常の有無を判定することで、誤判定が少なくなり、起こり得る異常が多様な場合であっても、検査対象物に関する異常の有無を精度良く検出することができる。
According to the inspection apparatus 10 according to the present embodiment, the image P is divided into a plurality of first partial images P1, a plurality of second partial images P2, and a plurality of third partial images P3, and the first partial image, When it is determined that an abnormality is included in each of the second partial image and the third partial image, the number of erroneous determinations is small by determining the presence or absence of an abnormality regarding the inspection object based on the overlap of the partial images Therefore, even if there are various possible abnormalities, it is possible to accurately detect the presence or absence of an abnormality regarding the inspection object.
本例の場合、異常が含まれていると判定される第1部分画像P1aと、異常が含まれていると判定される第2部分画像P2aと、異常が含まれていると判定される第3部分画像P3a,P3b,P3cとが重畳するため、判定部14は、重畳箇所Rに異常が存在すると判定する。このように、異常が含まれていると判定される複数種類の部分画像の重畳箇所に異常が存在すると判定することで、検査対象物に関する異常の有無のみならず、異常が存在する箇所を精度良く特定することができる。
In the case of this example, it is determined that the first partial image P1a determined to contain an abnormality, the second partial image P2a determined to contain an abnormality, and the abnormality are included. Since the three partial images P3a, P3b, and P3c overlap, the determination unit 14 determines that an abnormality is present in the overlapping portion R. As described above, by determining that an abnormality exists in the superimposed portion of a plurality of types of partial images determined to include an abnormality, not only the presence or absence of an abnormality related to the inspection object, but also the portion where the abnormality exists It can be identified well.
また、判定部14は、異常が含まれていると判定される第2部分画像P2b,P2c及び異常が含まれていると判定される第3部分画像P3dについて、異常が含まれていないと判定される他の種類の部分画像と重畳するため、異常が含まれていると判定される第2部分画像P2b,P2c又は異常が含まれていると判定される第3部分画像P3dの箇所には異常が存在しないと判定する。このように、重畳する第1部分画像、第2部分画像及び第3部分画像のいずれかに異常が含まれていると判定され、他の種類の部分画像には異常が含まれていないと判定される場合に、当該異常が含まれていると判定される第1部分画像、第2部分画像又は第3部分画像の箇所には異常が存在しないと判定することで、異常の存在の誤判定が防止される。
Further, the determination unit 14 determines that the second partial images P2b and P2c determined to contain an abnormality and the third partial image P3d determined to contain an abnormality do not include an abnormality. In the second partial images P2b and P2c that are determined to include an abnormality, or the third partial image P3d that is determined to include an abnormality, in order to overlap with another type of partial image to be It determines that there is no abnormality. Thus, it is determined that any one of the first partial image, the second partial image, and the third partial image to be superimposed includes an abnormality, and the other type of partial image is determined not to include an abnormality. If it is determined that there is no abnormality in the portion of the first partial image, the second partial image or the third partial image that is determined to contain the abnormality, erroneous determination of the existence of the abnormality Is prevented.
さらに、複数の第2部分画像P2それぞれが、複数の第1部分画像P1それぞれよりも小さく、複数の第3部分画像P3それぞれが、複数の第2部分画像P2それぞれよりも小さいことで、異常のサイズや形状が多様である場合であっても、異常の有無及び存在箇所を精度良く判定することができる。
Further, each of the plurality of second partial images P2 is smaller than each of the plurality of first partial images P1, and each of the plurality of third partial images P3 is smaller than each of the plurality of second partial images P2 Even in the case of various sizes and shapes, the presence or absence of an abnormality can be determined with high accuracy.
図9は、本発明の実施形態に係る検査装置10により実行される第1処理のフローチャートである。第1処理は、検査対象物100の画像Pを第1部分画像P1、第2部分画像P2及び第3部分画像P3にそれぞれ分割して、異常の有無を判定して、異常が含まれていると判定される部分画像の重なりに基づいて、検査対象物100に関する異常の有無を判定する処理である。
FIG. 9 is a flowchart of a first process performed by the inspection apparatus 10 according to the embodiment of the present invention. The first process divides the image P of the inspection object 100 into a first partial image P1, a second partial image P2, and a third partial image P3, determines the presence or absence of an abnormality, and includes an abnormality. It is the process which determines the presence or absence of abnormality regarding the test object 100 based on the overlap of the partial image determined to be.
検査装置10は、はじめに検査対象物100の画像Pを取得する(S10)。そして、画像Pを第1分割部12aによって複数の第1部分画像P1に分割し(S11)、画像Pを第2分割部12bによって複数の第2部分画像P2に分割し(S12)、画像Pを第3分割部12cによって複数の第3部分画像P3に分割する(S13)。
The inspection apparatus 10 first acquires an image P of the inspection object 100 (S10). Then, the image P is divided into a plurality of first partial images P1 by the first division unit 12a (S11), and the image P is divided into a plurality of second partial images P2 by the second division unit 12b (S12). Are divided into a plurality of third partial images P3 by the third division unit 12c (S13).
次に、第1分類部13aは、複数の第1部分画像P1を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類し(S14)、第2分類部13bは、複数の第2部分画像P2を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類し(S15)、第3分類部13cは、複数の第3部分画像P3を、異常が含まれていると判定される第3部分画像と異常が含まれていないと判定される第3部分画像に分類する(S16)。
Next, the first classification unit 13a classifies the plurality of first partial images P1 into a first partial image determined to contain an abnormality and a first partial image determined to contain no abnormality. (S14), the second classification unit 13b determines that the plurality of second partial images P2 are determined to contain no abnormality and the second partial image determined to contain no abnormality. (S15), and the third classification unit 13c determines that the plurality of third partial images P3 are determined not to contain an abnormality and the third partial image determined to contain an abnormality. The image is classified into partial images (S16).
その後、表示部10fは、異常が含まれていると判定される第1部分画像、異常が含まれていると判定される第2部分画像及び異常が含まれていると判定される第3部分画像を、それぞれ区別可能な態様で表示する(S17)。
Thereafter, the display unit 10 f determines that the first partial image determined to include an abnormality, the second partial image determined to include an abnormality, and the third part determined to include an abnormality. The images are displayed in a distinguishable manner (S17).
判定部14は、異常が含まれていると判定される第1部分画像と、異常が含まれていると判定される第2部分画像と、異常が含まれていると判定される第3部分画像とが重畳するか否かを判定する(S18)。これら3種類の部分画像が重畳すると判定された場合(S18:Yes)、判定部14は、当該重畳箇所に異常が存在すると判定する(S19)。一方、これら3種類の部分画像が重畳しないと判定された場合(S18:No)、すなわち少なくとも1種類の部分画像には異常が含まれていないと判定される場合、判定部14は、異常が含まれていると判定された部分画像の箇所には異常が存在しないと判定する(S20)。以上により、第1処理が終了する。
The determination unit 14 determines that the first partial image determined to contain an abnormality, the second partial image determined to contain an abnormality, and the third part determined to contain an abnormality. It is determined whether the image is to be superimposed (S18). When it is determined that these three types of partial images are to be superimposed (S18: Yes), the determination unit 14 determines that there is an abnormality in the superimposed portion (S19). On the other hand, when it is determined that these three types of partial images are not superimposed (S18: No), that is, when it is determined that no abnormality is included in at least one type of partial image, the determination unit 14 determines that It is determined that there is no abnormality in the portion of the partial image determined to be included (S20). Thus, the first process ends.
図10は、本発明の実施形態に係る検査装置10により実行される第2処理のフローチャートである。第2処理は、第1処理に先立って実行される処理であり、第1分類部13a、第2分類部13b及び第3分類部13cに対して、異常が含まれている部分画像と異常が含まれていない部分画像の分類を学習させる処理である。
FIG. 10 is a flowchart of a second process performed by the inspection apparatus 10 according to the embodiment of the present invention. The second process is a process that is executed prior to the first process, and partial images and abnormalities in which abnormality is included in the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c. This is a process of learning the classification of partial images not included.
検査装置10の学習部16は、学習用画像データベースDBから、学習用の第1部分画像を取得する(S30)。同様に学習部16は、学習用画像データベースDBから、学習用の第2部分画像の取得と(S31)、学習用の第3部分画像の取得を行う(S32)。ここで、学習用の第1部分画像は、異常が含まれている第1部分画像及び異常が含まれていない第1部分画像を含み、それぞれの第1部分画像には異常の有無がタグ付けされている。また、学習用の第2部分画像は、異常が含まれている第2部分画像及び異常が含まれていない第2部分画像を含み、それぞれの第2部分画像には異常の有無がタグ付けされている。同様に、学習用の第3部分画像は、異常が含まれている第3部分画像及び異常が含まれていない第3部分画像を含み、それぞれの第3部分画像には異常の有無がタグ付けされている。
The learning unit 16 of the inspection apparatus 10 acquires a first partial image for learning from the learning image database DB (S30). Similarly, the learning unit 16 acquires a second partial image for learning (S31) and acquires a third partial image for learning from the learning image database DB (S32). Here, the first partial image for learning includes a first partial image including an abnormality and a first partial image not including an abnormality, and each first partial image is tagged with the presence or absence of an abnormality. It is done. Also, the second partial image for learning includes a second partial image containing an abnormality and a second partial image not containing an abnormality, and each second partial image is tagged with the presence or absence of an abnormality. ing. Similarly, the third partial image for learning includes a third partial image containing an abnormality and a third partial image not containing an abnormality, and each third partial image is tagged with the presence or absence of an abnormality. It is done.
学習部16は、学習用の第1部分画像を用いて第1分類部13aの学習を実行し(S33)、学習用の第2部分画像を用いて第2分類部13bの学習を実行し(S34)、学習用の第3部分画像を用いて第3分類部13cの学習を実行する(S35)。具体的には、第1分類部13a、第2分類部13b及び第3分類部13cがニューラルネットワークを含む場合、学習部16は、学習用の第1部分画像、第2部分画像及び第3部分画像を第1分類部13a、第2分類部13b及び第3分類部13cにそれぞれ入力し、異常が含まれている部分画像と異常が含まれていない部分画像とを正しく分類できたかを評価する値に基づいて、当該評価値を向上させるように、ニューラルネットワークに含まれるユニットの重み付けを最適化する。
The learning unit 16 executes the learning of the first classification unit 13a using the first partial image for learning (S33), and executes the learning of the second classification unit 13b using the second partial image for learning (S33). S34) The third classification unit 13c executes learning using the third partial image for learning (S35). Specifically, when the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c include a neural network, the learning unit 16 calculates a first partial image, a second partial image, and a third portion for learning. An image is input to each of the first classification unit 13a, the second classification unit 13b, and the third classification unit 13c, and it is evaluated whether the partial image including the abnormality and the partial image not including the abnormality can be correctly classified. Based on the values, the weightings of the units included in the neural network are optimized to improve the evaluation value.
以上により第2処理が終了し、第1分類部13aは、第1学習済みモデルを含むこととなり、第2分類部13bは、第2学習済みモデルを含むこととなり、第3分類部13cは、第3学習済みモデルを含むこととなる。ここで、第1学習済みモデル、第2学習済みモデル及び第3学習済みモデルは、それぞれ異なる学習用データ(学習用の第1部分画像、学習用の第2部分画像及び学習用の第3部分画像)を用いて学習された、異なる構成を有するモデルであってよい。具体的には、第1分類部13aは、第1学習済みニューラルネットワークを含み、第2分類部13bは、第1学習済みニューラルネットワークと構成が異なる第2学習済みニューラルネットワークを含み、第3分類部13cは、第1学習済みニューラルネットワーク及び第2学習済みニューラルネットワークいずれとも構成が異なる第3学習済みニューラルネットワークを含んでよい。
Thus, the second process ends, and the first classification unit 13a includes the first learned model, the second classification unit 13b includes the second learned model, and the third classification unit 13c It will include the third learned model. Here, the first learned model, the second learned model, and the third learned model are different learning data (the first partial image for learning, the second partial image for learning, and the third portion for learning) It may be a model having a different configuration learned using images). Specifically, the first classification unit 13a includes a first learned neural network, and the second classification unit 13b includes a second learned neural network having a configuration different from that of the first learned neural network. The unit 13c may include a third learned neural network that is different in configuration from both the first learned neural network and the second learned neural network.
本実施形態に係る検査装置10によれば、それぞれ異なる学習用データで学習された第1学習済みモデル、第2学習済みモデル及び第3学習済みモデルを用いることで、第1分類部13a、第2分類部13b及び第3分類部13cとで異なる観点から異常の有無を分類することができる。
According to the inspection apparatus 10 according to the present embodiment, the first classification unit 13a, the first learned model, the second learned model, and the third learned model that are learned using different learning data, respectively, The presence or absence of an abnormality can be classified from different viewpoints by the second classification unit 13b and the third classification unit 13c.
以上説明した実施形態は、本発明の理解を容易にするためのものであり、本発明を限定して解釈するためのものではない。実施形態が備える各要素並びにその配置、材料、条件、形状及びサイズ等は、例示したものに限定されるわけではなく適宜変更することができる。また、異なる実施形態で示した構成同士を部分的に置換し又は組み合わせることが可能である。
The embodiments described above are for the purpose of facilitating the understanding of the present invention, and are not for the purpose of limiting the present invention. The elements included in the embodiment and the arrangement, the material, the conditions, the shape, the size, and the like of the elements are not limited to those illustrated, and can be changed as appropriate. In addition, configurations shown in different embodiments can be partially substituted or combined with each other.
Claims (11)
- 検査対象物の画像を複数の第1部分画像に分割する第1分割部と、
前記画像を、前記複数の第1部分画像とは異なる複数の第2部分画像に分割する第2分割部と、
前記複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する第1分類部と、
前記複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する第2分類部と、
前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像との重なりに基づいて、前記検査対象物に関する異常の有無を判定する判定部と、
を備える検査装置。 A first division unit that divides an image of an inspection object into a plurality of first partial images;
A second division unit that divides the image into a plurality of second partial images different from the plurality of first partial images;
A first classification unit that classifies the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to not contain an abnormality;
A second classification unit that classifies the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality;
Based on the overlap between the first partial image determined to contain the abnormality and the second partial image determined to contain the abnormality, the presence or absence of an abnormality regarding the inspection object is determined A judgment unit,
Inspection device provided with - 前記判定部は、前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像とが重畳する場合に、当該重畳箇所に異常が存在すると判定する、
請求項1に記載の検査装置。 When the first partial image determined to include the abnormality and the second partial image determined to include the abnormality overlap, the determination unit determines that the overlapping portion has an abnormality. Determine if it exists,
The inspection apparatus according to claim 1. - 前記判定部は、前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていないと判定される第2部分画像とが重畳する場合又は前記異常が含まれていないと判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像とが重畳する場合に、当該異常が含まれていると判定される第1部分画像又は当該異常が含まれていると判定される第2部分画像の箇所には異常が存在しないと判定する、
請求項1又は2に記載の検査装置。 The determination unit determines that the first partial image determined to include the abnormality and the second partial image determined to not include the abnormality overlap or the abnormality is not included. If the first partial image determined to be the second partial image determined to contain the abnormality and the second partial image to be determined to include the abnormality, the first partial image determined to contain the abnormality or the abnormality It is determined that there is no abnormality in the portion of the second partial image that is determined to contain
The inspection apparatus according to claim 1. - 前記異常が含まれていると判定される第1部分画像と前記異常が含まれていないと判定される第1部分画像を区別可能に表示し、前記異常が含まれていると判定される第2部分画像と前記異常が含まれていないと判定される第2部分画像を区別可能に表示して、前記画像を表示する表示部をさらに備える、
請求項1から3のいずれか一項に記載の検査装置。 A first partial image determined to include the abnormality and a first partial image determined to not include the abnormality are distinguishably displayed, and it is determined that the abnormality is included. The display device is further provided with a display unit that displays the two partial images and the second partial images that are determined not to include the abnormality in a distinguishable manner, and displays the images.
The inspection apparatus according to any one of claims 1 to 3. - 前記表示部は、前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像とを区別可能に表示する、
請求項4に記載の検査装置。 The display unit distinguishably displays a first partial image determined to include the abnormality and a second partial image determined to include the abnormality.
The inspection apparatus according to claim 4. - 前記複数の第2部分画像それぞれは、前記複数の第1部分画像それぞれよりも小さい、
請求項1から5のいずれか一項に記載の検査装置。 Each of the plurality of second partial images is smaller than each of the plurality of first partial images,
The inspection apparatus according to any one of claims 1 to 5. - 前記複数の第2部分画像のうち少なくとも2つは、前記複数の第1部分画像のうちいずれか1つと重畳する、
請求項1から6のいずれか一項に記載の検査装置。 At least two of the plurality of second partial images overlap with any one of the plurality of first partial images,
The inspection apparatus according to any one of claims 1 to 6. - 前記第1分割部は、前記複数の第1部分画像が互いに重畳しないように前記画像を分割し、
前記第2分割部は、前記複数の第2部分画像が互いに重畳しないように前記画像を分割する、
請求項1から7のいずれか一項に記載の検査装置。 The first division unit divides the plurality of first partial images so that the plurality of first partial images do not overlap each other,
The second division unit divides the plurality of second partial images so that the plurality of second partial images do not overlap each other.
The inspection apparatus according to any one of claims 1 to 7. - 前記第1分類部は、異常が含まれている第1部分画像及び異常が含まれていない第1部分画像を学習用データとして学習された、第1学習済みモデルを含み、
前記第2分類部は、異常が含まれている第2部分画像及び異常が含まれていない第2部分画像を学習用データとして学習された、第2学習済みモデルを含む、
請求項1から8のいずれか一項に記載の検査装置。 The first classification unit includes a first partial image including an abnormality and a first learned model learned as a learning data, a first partial image including no abnormality.
The second classification unit includes a second partial image in which an abnormality is included and a second learned model in which a second partial image in which an abnormality is not included is learned as learning data.
The inspection apparatus according to any one of claims 1 to 8. - コンピュータが、
検査対象物の画像を複数の第1部分画像に分割するステップと、
前記画像を、前記複数の第1部分画像とは異なる複数の第2部分画像に分割するステップと、
前記複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類するステップと、
前記複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類するステップと、
前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像との重なりに基づいて、前記検査対象物に関する異常の有無を判定するステップと、
を含む検査方法。 The computer is
Dividing the image of the inspection object into a plurality of first partial images;
Dividing the image into a plurality of second partial images different from the plurality of first partial images;
Classifying the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to not contain an abnormality;
Classifying the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality;
Based on the overlap between the first partial image determined to contain the abnormality and the second partial image determined to contain the abnormality, the presence or absence of an abnormality regarding the inspection object is determined Step and
Inspection method including. - コンピュータを、
検査対象物の画像を複数の第1部分画像に分割する第1分割部、
前記画像を、前記複数の第1部分画像とは異なる複数の第2部分画像に分割する第2分割部、
前記複数の第1部分画像を、異常が含まれていると判定される第1部分画像と異常が含まれていないと判定される第1部分画像に分類する第1分類部、
前記複数の第2部分画像を、異常が含まれていると判定される第2部分画像と異常が含まれていないと判定される第2部分画像に分類する、前記第1分類部とは異なる第2分類部、及び
前記異常が含まれていると判定される第1部分画像と、前記異常が含まれていると判定される第2部分画像との重なりに基づいて、前記検査対象物に関する異常の有無を判定する判定部、
として機能させる検査プログラム。 Computer,
A first division unit configured to divide an image of the inspection object into a plurality of first partial images;
A second division unit that divides the image into a plurality of second partial images different from the plurality of first partial images;
A first classification unit that classifies the plurality of first partial images into a first partial image determined to contain an abnormality and a first partial image determined to not contain an abnormality;
Different from the first classification unit, which classifies the plurality of second partial images into a second partial image determined to contain an abnormality and a second partial image determined to not contain an abnormality The inspection object is related to the second classification unit, and the first partial image determined to include the abnormality and the second partial image determined to include the abnormality. A determination unit that determines the presence or absence of an abnormality;
Inspection program to function as.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018515903A JP6387477B1 (en) | 2017-06-23 | 2017-06-23 | Inspection device, inspection method, and inspection program |
PCT/JP2017/023213 WO2018235266A1 (en) | 2017-06-23 | 2017-06-23 | Inspection device, inspection method, and inspection program |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2017/023213 WO2018235266A1 (en) | 2017-06-23 | 2017-06-23 | Inspection device, inspection method, and inspection program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018235266A1 true WO2018235266A1 (en) | 2018-12-27 |
Family
ID=63444288
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2017/023213 WO2018235266A1 (en) | 2017-06-23 | 2017-06-23 | Inspection device, inspection method, and inspection program |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP6387477B1 (en) |
WO (1) | WO2018235266A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020042001A (en) * | 2018-09-07 | 2020-03-19 | 株式会社フジクラ | Evaluation device, evaluation method, evaluation program, and inspection device |
JP2020125618A (en) * | 2019-02-04 | 2020-08-20 | 東京電力ホールディングス株式会社 | Information processing method, program, water intake control system, and learned model creation method |
WO2021010430A1 (en) * | 2019-07-16 | 2021-01-21 | 株式会社テクムズ | Method for detecting defect of item to be inspected, device therefor, and computer program therefor |
WO2022085253A1 (en) * | 2020-10-23 | 2022-04-28 | 株式会社荏原製作所 | Machined-surface determination device, machined-surface determination program, machined-surface determination method, and machining system |
WO2022215382A1 (en) * | 2021-04-05 | 2022-10-13 | パナソニックIpマネジメント株式会社 | Inspection device, inspection method, and program |
US12013351B2 (en) | 2019-09-24 | 2024-06-18 | Ishida Co., Ltd. | Inspection device |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4173441A (en) * | 1977-03-28 | 1979-11-06 | E. I. Du Pont De Nemours And Company | Web inspection system and method therefor |
JPH11111796A (en) * | 1997-10-02 | 1999-04-23 | Mitsubishi Electric Corp | Method and device for analyzing defect |
JP2010266430A (en) * | 2009-04-15 | 2010-11-25 | Jfe Steel Corp | Method and apparatus for inspecting defect of steel plate surface |
JP2011058926A (en) * | 2009-09-09 | 2011-03-24 | Ricoh Elemex Corp | Method and device for inspecting image |
JP2011122985A (en) * | 2009-12-11 | 2011-06-23 | Ricoh Co Ltd | Printed material inspection apparatus, printed material inspection method, program and storage medium |
JP2012026982A (en) * | 2010-07-27 | 2012-02-09 | Panasonic Electric Works Sunx Co Ltd | Inspection device |
JP2015026287A (en) * | 2013-07-26 | 2015-02-05 | 新電元工業株式会社 | Solder inspection device, solder inspection method and electronic component |
JP2016126769A (en) * | 2014-12-26 | 2016-07-11 | 古河機械金属株式会社 | Inspection result output device, inspection result output method, and program |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6126450B2 (en) * | 2013-04-25 | 2017-05-10 | 株式会社ブリヂストン | Inspection device |
-
2017
- 2017-06-23 JP JP2018515903A patent/JP6387477B1/en active Active
- 2017-06-23 WO PCT/JP2017/023213 patent/WO2018235266A1/en active Application Filing
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4173441A (en) * | 1977-03-28 | 1979-11-06 | E. I. Du Pont De Nemours And Company | Web inspection system and method therefor |
JPH11111796A (en) * | 1997-10-02 | 1999-04-23 | Mitsubishi Electric Corp | Method and device for analyzing defect |
JP2010266430A (en) * | 2009-04-15 | 2010-11-25 | Jfe Steel Corp | Method and apparatus for inspecting defect of steel plate surface |
JP2011058926A (en) * | 2009-09-09 | 2011-03-24 | Ricoh Elemex Corp | Method and device for inspecting image |
JP2011122985A (en) * | 2009-12-11 | 2011-06-23 | Ricoh Co Ltd | Printed material inspection apparatus, printed material inspection method, program and storage medium |
JP2012026982A (en) * | 2010-07-27 | 2012-02-09 | Panasonic Electric Works Sunx Co Ltd | Inspection device |
JP2015026287A (en) * | 2013-07-26 | 2015-02-05 | 新電元工業株式会社 | Solder inspection device, solder inspection method and electronic component |
JP2016126769A (en) * | 2014-12-26 | 2016-07-11 | 古河機械金属株式会社 | Inspection result output device, inspection result output method, and program |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020042001A (en) * | 2018-09-07 | 2020-03-19 | 株式会社フジクラ | Evaluation device, evaluation method, evaluation program, and inspection device |
JP7236292B2 (en) | 2018-09-07 | 2023-03-09 | 株式会社フジクラ | Evaluation device, evaluation method, evaluation program, and inspection device |
JP2020125618A (en) * | 2019-02-04 | 2020-08-20 | 東京電力ホールディングス株式会社 | Information processing method, program, water intake control system, and learned model creation method |
JP7293687B2 (en) | 2019-02-04 | 2023-06-20 | 東京電力ホールディングス株式会社 | Information processing method, program, water intake control system, and learned model generation method |
WO2021010430A1 (en) * | 2019-07-16 | 2021-01-21 | 株式会社テクムズ | Method for detecting defect of item to be inspected, device therefor, and computer program therefor |
US12013351B2 (en) | 2019-09-24 | 2024-06-18 | Ishida Co., Ltd. | Inspection device |
WO2022085253A1 (en) * | 2020-10-23 | 2022-04-28 | 株式会社荏原製作所 | Machined-surface determination device, machined-surface determination program, machined-surface determination method, and machining system |
JP7450517B2 (en) | 2020-10-23 | 2024-03-15 | 株式会社荏原製作所 | Machining surface determination device, machining surface determination program, machining surface determination method, and machining system |
WO2022215382A1 (en) * | 2021-04-05 | 2022-10-13 | パナソニックIpマネジメント株式会社 | Inspection device, inspection method, and program |
Also Published As
Publication number | Publication date |
---|---|
JP6387477B1 (en) | 2018-09-05 |
JPWO2018235266A1 (en) | 2019-06-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018235266A1 (en) | Inspection device, inspection method, and inspection program | |
EP3502966B1 (en) | Data generation apparatus, data generation method, and data generation program | |
JP6837848B2 (en) | Diagnostic device | |
JP6707920B2 (en) | Image processing apparatus, image processing method, and program | |
CN109946303A (en) | Check device and method | |
JP2019095217A (en) | Visual inspection device | |
JP2013224833A (en) | Visual inspection device, visual inspection method and computer program | |
JP2015041164A (en) | Image processor, image processing method and program | |
US11769248B2 (en) | Image processing device, image processing method, and image processing non-transitory computer readable medium for verifying detectable range of defect image | |
CN114066857A (en) | Infrared image quality evaluation method and device, electronic equipment and readable storage medium | |
JP6882772B2 (en) | Inspection equipment, inspection method and inspection program | |
US11521392B2 (en) | Image processing apparatus and image processing method for image analysis process | |
US11481673B2 (en) | Signal analysis device, signal analysis method, and signal analysis program | |
CN113016023B (en) | Information processing method and computer-readable non-transitory recording medium | |
US20060093203A1 (en) | Attribute threshold evaluation scheme | |
JP6838651B2 (en) | Image processing equipment, image processing methods and programs | |
JPWO2021130978A1 (en) | Motion analysis system and motion analysis program | |
JP6897100B2 (en) | Judgment device, judgment method, and judgment program | |
US20230080978A1 (en) | Machine learning method and information processing apparatus for machine learning | |
US20210049396A1 (en) | Optical quality control | |
JP7469100B2 (en) | Detection device and program | |
JP7067321B2 (en) | Inspection result presentation device, inspection result presentation method and inspection result presentation program | |
JP2020071716A (en) | Abnormality determination method, feature quantity calculation method, and appearance inspection device | |
JP5140699B2 (en) | Surface state classification apparatus, surface state classification method, and surface state classification program | |
CN113340900A (en) | Image inspection apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2018515903 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17914697 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17914697 Country of ref document: EP Kind code of ref document: A1 |