CN114113101A

CN114113101A - Abnormality determination model generation method, abnormality determination model generation device, and inspection device

Info

Publication number: CN114113101A
Application number: CN202111002232.7A
Authority: CN
Inventors: 中里研一
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2020-08-31
Filing date: 2021-08-30
Publication date: 2022-03-01
Also published as: DE102021119598A1; JP2022040531A

Abstract

Provided are an abnormality determination model generation method, an abnormality determination model generation device, and an inspection device, which can newly add various types of learning data even when the number of abnormal supervised image data is small, and which can provide reliability of inspection of an abnormality appearing in appearance. An abnormality determination model generation method for generating an abnormality determination model for determining an abnormality of an inspection target using inspection image data for measuring the inspection target as input data, the method comprising: a step of generating a learning data set (Img _ tch) for generating an abnormality determination model based on pseudo data (Img _ fak) which is not real image data for measuring an inspection object; and a step of generating an abnormality determination model using the learning dataset (Img _ tch).

Description

Abnormality determination model generation method, abnormality determination model generation device, and inspection device

Technical Field

The present invention relates to an abnormality determination model generation method, an abnormality determination model generation device, and an inspection device.

Background

In recent years, AI (Artificial Intelligence) techniques such as deep learning have been developed, and it has been discussed that AI is utilized for various examinations. As an automated method of inspection to which a general AI technique is applied, a method such as abnormality sensing by supervised machine learning or abnormality sensing by unsupervised machine learning is considered.

In the case where the inspection of the abnormality occurring in the appearance is performed using the supervised machine learning, it is necessary to prepare many pieces of supervised image data in advance, to be marked at each level of normality or abnormality, before the machine learning is carried out. At this time, when the supervised image data is insufficient, unknown abnormality cannot be sensed, and effective learning cannot be performed. Therefore, the following operations are performed: new supervision image data is generated by performing image processing on supervision image data in which an actual abnormality is captured, thereby increasing the number of supervision image data.

For example, patent document 1 proposes a technique in which: in the case where the number of pieces of supervised image data used in machine learning is insufficient, high-quality supervised image data is generated. Specifically, patent document 1 discloses a machine learning system in which: the method includes acquiring supervised image data classified into a plurality of patterns, specifying an insufficient pattern with a small number of the supervised image data based on a predetermined criterion, and generating new supervised image data belonging to the insufficient pattern by spatially inverting or changing the color tone of any one of the supervised image data.

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open No. 2018-169672.

Disclosure of Invention

Problems to be solved by the invention

Here, the machine learning system disclosed in patent document 1 is a system that increases the number of pieces of supervised image data by transforming the color tone of actual supervised image data. Not limited to the conversion of the color tone, the number of the supervisory image data can be increased by inverting or linearly converting the actual supervisory image data. However, in the case where the number of the supervisory image data is increased by the conversion of the color tone or the linear conversion, since the newly added supervisory image data is created based on the characteristics of the original supervisory image data, there is a limit in the contents of the supervisory image data. Therefore, there is room for improvement in the accuracy of the abnormality detection by machine learning.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an abnormality determination model generation method, an abnormality determination model generation device, and an inspection device, which can newly add various types of data for learning even when the number of abnormal supervised image data is small, and can improve the reliability of inspection of an abnormality appearing in the appearance.

Means for solving the problems

In order to solve the above problem, according to an aspect of the present invention, there is provided an abnormality determination model generation method for generating an abnormality determination model for determining an abnormality of an inspection target using inspection image data for measuring the inspection target as input data, the abnormality determination model generation method including: a step of generating a learning data set for generating an abnormality determination model based on pseudo data that is not real image data for measuring an inspection object; and a step of generating an abnormality determination model using the learning data set.

In order to solve the above problem, according to another aspect of the present invention, there is provided an abnormality determination model generation device that generates an abnormality determination model for determining an abnormality of an inspection target using inspection image data obtained by measuring the inspection target as input data, the abnormality determination model generation device including a learning unit that generates the abnormality determination model using a learning data set generated based on pseudo data that is not real image data obtained by measuring the inspection target.

In order to solve the above problem, according to still another aspect of the present invention, there is provided an inspection apparatus for performing an inspection of an inspection target based on inspection image data for measuring the inspection target, the inspection apparatus including: a storage unit that stores an abnormality determination model generated using a learning dataset generated based on pseudo data that is not real image data for measuring an inspection target; and a determination unit that inputs inspection image data for measuring the inspection target to the abnormality determination model and determines whether or not there is an abnormality in the inspection target.

Effects of the invention

As described above, according to the present invention, even when there is little abnormal supervised image data, various kinds of supervised image data can be newly added, and the reliability of the inspection of the abnormality appearing in the appearance can be improved.

Drawings

Fig. 1 is a schematic diagram showing a configuration example of an inspection system including an inspection apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram showing a functional configuration of the inspection apparatus according to the embodiment.

Fig. 3 is a flowchart showing an example of the abnormality determination model generation method according to the embodiment.

Fig. 4 is an explanatory diagram illustrating a method of creating a data set for learning.

Fig. 5 is a flowchart showing an example of an abnormality determination method using the inspection apparatus according to the embodiment.

Fig. 6 is a schematic diagram showing an application example of an inspection system including the inspection device according to the embodiment.

Fig. 7 is a schematic diagram showing another application example of the inspection system including the inspection device according to the embodiment.

Detailed Description

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, the same reference numerals are given to components having substantially the same functional configuration, and thus, redundant description is omitted.

< 1. overview of inspection apparatus

First, an outline of an inspection apparatus according to an embodiment of the present invention will be described.

Fig. 1 is a schematic diagram showing a configuration example of an inspection system 100 including an inspection apparatus 20 according to the present embodiment. The inspection system 100 includes an inspection device 20, a detector 11, an input unit 15, and a display unit 13. The inspection system 100 is constructed as a system that automatically determines an abnormality of an inspection target using an abnormality determination model based on image data of the inspection target.

The detector 11 measures the object to be inspected to generate image data, and outputs the image data to the inspection apparatus 20. As the detector 11, a camera or LiDAR (Light Detection And Ranging) or Light Imaging Detection And Ranging) is exemplified. The imaging camera includes an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and generates image data of an imaging target. The LiDAR irradiates the measurement target with laser light while scanning the laser light, and generates image data reflecting the state of the appearance of the measurement target by observing the scattered or reflected light of the laser light.

The detector 11 is not limited to an imaging camera or LiDAR as long as it can generate image data reflecting the state of the appearance of the measurement target. The number of detectors 11 provided in the inspection system 100 is not limited to 1, and may be 2 or more. The detector 11 may comprise multiple kinds of devices such as a camera and a LiDAR.

The input unit 15 receives an operation input from a user and transmits an operation signal to the inspection apparatus 20. For example, the input unit 15 is configured to include at least one of a keyboard, a touch panel, a tablet computer, a smartphone, an operation button, a switch, and the like.

The display unit 13 is driven based on a drive signal output from the inspection device 20, and performs predetermined display of an inspection result and the like. For example, the display unit 13 is configured to include an optical panel such as a liquid crystal panel. The input unit 15 and the display unit 13 may be integrated to form a touch panel.

The inspection device 20 includes a control unit 30, a memory 21, and a storage device 23. A part or the whole of the control Unit 30 is constituted by a processor such as a CPU (Central Processing Unit). Specifically, the control Unit 30 may include a CPU and a GPU (graphics Processing Unit). The inspection device 20 also includes interfaces for transmitting and receiving signals to and from the detector 11, the input unit 15, and the display unit 13, respectively.

The Memory 21 is formed of at least one Memory element such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage device 23 is configured by at least one storage medium such as an HDD (Hard Disk Drive), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SSD (Solid State Drive), a USB (Universal Serial Bus) flash memory, and a storage device. The memory 21 or the storage device 23 functions as a storage unit, and stores a program of arithmetic processing executed by the processor, various arithmetic parameters used in the arithmetic processing, acquired data, arithmetic results, and the like. Further, the memory 21 or the storage device 23 stores an abnormality determination model used in the examination.

< 2. schematic Structure of inspection apparatus

Thus far, the overall structure of the inspection system 100 is explained. Next, a schematic configuration example of the inspection apparatus 20 will be specifically described.

Fig. 2 is a block diagram showing a functional configuration of the inspection apparatus 20. The control unit 30 of the inspection apparatus 20 includes an image processing unit 31, an arithmetic unit 33, and a display control unit 39. The calculation unit 33 includes a learning unit 35 and a determination unit 37. Specifically, each function of the control unit 30 may be a software function realized by the execution of a program by a processor, or a part thereof may be realized by a hardware function.

(2-1. image processing section)

The image processing unit 31 processes the image data output from the detector 11 to generate inspection image data for use in an inspection. For example, the image processing unit 31 may perform a process of adjusting the contrast using at least one of an expansion filter, a contraction filter, an averaging filter, or a median filter, or may perform a process of adjusting the contrast or emphasizing an edge using an edge extraction filter or an edge emphasis filter, with respect to the image data output from the detector 11. The image processing unit 31 may perform trimming (trimming) processing for extracting image data of a predetermined size, range, or the like from the image data output from the detector 11. The detector 11 may be provided with the function of the image processing unit 31.

(2-2. arithmetic part)

The arithmetic unit 33 includes a learning unit 35 and a determination unit 37, the learning unit 35 generates an abnormality determination model used for determining an abnormality of the inspection target, and the determination unit 37 determines an abnormality of the inspection target using the generated abnormality determination model.

The learning unit 35 generates an abnormality determination model for use in abnormality determination. Specifically, the learning unit 35 performs machine learning of the abnormality determination model using, as input data, image data (learning data) indicating a characteristic pattern of an abnormality of the inspection target. The generated abnormality determination model is stored in the memory 21 or the storage device 23 and used for the abnormality determination process by the determination unit 37. In the present embodiment, the inspection device 20 including the learning unit 35 also functions as an abnormality determination model generation device.

In the present embodiment, pseudo data is used as data for learning, instead of real image data obtained by actually measuring an inspection target in which an abnormality has occurred. The pseudo data may be created by referring to a characteristic pattern of a defect that actually occurs, or by manually drawing a characteristic pattern of an abnormality that may occur based on past knowledge or experience of a user, for example. The data for learning may include pseudo data generated by randomly performing image conversion processing on the basis of pseudo data created by artificially plotting a characteristic pattern of an abnormality. Thus, when a new inspection system 100 is constructed, or when an inspection target is newly installed, or the like, an abnormality determination model can be generated before an abnormality actually occurs. As such an image conversion process, for example, a random image generation algorithm typified by GAN (generic adaptive Network) can be used. However, the image generation algorithm is not limited to GAN.

In the case where it is desired to determine not only the presence or absence of an abnormality but also the type of the abnormality when determining the abnormality, when creating pseudo data by manually drawing a feature pattern of the abnormality, the pseudo data may be classified for each type of the abnormality. For example, when a defect occurs in the surface of the inspection object, when a damage occurs, when a crack occurs, or the like, and when the characteristic patterns appearing depending on the type of abnormality are different, the pseudo data may be classified for each type of abnormality.

The learning unit 35 uses the pseudo data as data for learning, and generates an abnormality determination model using a machine learning model. The machine learning model may be any one of existing machine learning models that can be used for machine learning using image data as supervision data. For example, the machine learning model may be a calculation model using a filter such as linear regression or kalman filter, a support vector machine, a neural network such as random forest, neighbor method, or deep learning, or a bayesian network.

Further, the learning unit 35 may update the abnormality determination model based on the inspection image data generated by the image processing unit 31 and information of the abnormality determination result obtained by the determination unit 37 described later. Thus, the abnormality determination model generated using pseudo data created by artificially drawing the characteristic pattern of an abnormality can be automatically corrected based on the result of determination of normality or abnormality of the inspection image data actually acquired. Therefore, the accuracy of the inspection result using the abnormality determination model can be improved.

The determination unit 37 determines an abnormality of the inspection target based on the inspection image data generated by the image processing unit 31 by using the abnormality determination model generated by the learning unit 35 and stored in the memory 21 or the storage device 23. Specifically, the determination unit 37 inputs the inspection image data to the abnormality determination model, and determines the presence or absence of an abnormality of the inspection target based on the output from the abnormality determination model. For example, the probability that a feature pattern corresponding to a feature pattern of an abnormality included in suspected data input as data for learning is included in inspection image data is calculated using an abnormality determination model. The determination unit 37 determines that there is an abnormality when the probability output from the abnormality determination model is equal to or greater than a predetermined threshold value.

Note that the method of determining an abnormality using the abnormality determination model is not limited to the above example. When the abnormality determination model is used to determine not only the probability of occurrence of an abnormality but also the classification of the type of the abnormality, the determination unit 37 determines not only the presence or absence of the abnormality of the inspection target but also the type of the abnormality.

(2-3. display control part)

The display control unit 39 generates an image signal to be displayed on the display unit 13, and controls display on the display unit 13. The display control unit 39 displays at least the processing results of the learning unit 35 and the determination unit 37 on the display unit 13. For example, when the learning unit 35 executes the machine learning, the display control unit 39 displays a setting screen of the conditions of the machine learning or a progress state on the display unit 13. Further, the display control unit 39 displays the determination result on the display unit 13 when the determination unit 37 performs the abnormality determination process. However, the display content on the display unit 13 is not particularly limited.

< 3. method for generating abnormality determination model >

The outline of the structure of the inspection apparatus 20 according to the present embodiment is described above. Next, a method of generating an abnormality determination model including the operation of the learning unit 35 will be described.

Fig. 3 is a flowchart showing an example of the abnormality determination model generation method.

First, the user manually draws a characteristic pattern of an abnormality that may occur in the inspection target to create pseudo data (step S11). For example, a user traces a characteristic pattern of anomalies to generate suspect data, which anomalies can be considered by the user as anomalies that may be produced in the examination object. The characteristic patterns of the anomalies that can be considered can be plotted on the basis of a rule of thumb, for example. Alternatively, the pattern may be a randomly drawn feature pattern depending on the kind of abnormality. The object of abnormality determination by the inspection device 20 according to the present embodiment is an abnormality occurring in the appearance of the inspection object. Specifically, the abnormality to be a target of the abnormality determination includes at least one of a defect or scratch, a breakage, a deformation, adhesion of a contaminant, and the like, which are generated in the surface of the inspection target, for example. The user draws a pattern of an abnormal shape that can be considered in consideration of the usage environment and usage condition of the inspection object, the pattern of the abnormal shape that occurred in the past, and the like, and generates pseudo data.

For example, the user may draw a characteristic pattern of an abnormality using graphics drawing software and perform data conversion processing to generate pseudo data, or may read a characteristic pattern of an abnormality created by handwriting using data creation software to generate pseudo data. Further, a method of manually drawing a characteristic pattern of an abnormality to create pseudo data is not particularly limited. Furthermore, when the characteristic patterns of the generated suspected data are depicted with excessive specificity, an overfitting may occur at the time of machine learning. Therefore, it is preferable that the drawing pattern of the generated pseudo data is not a pattern for drawing an abnormal shape in detail but a simplified pattern.

Next, the user adds new pseudo data by performing image conversion processing at random based on the pseudo data created in step S11, and generates a data set for machine learning (step S13). For example, the user performs tone conversion, linear conversion, and the like of the pseudo data created in step S11 using a device capable of executing a random image generation algorithm such as GAN, and further creates pseudo data. At this time, when the number of the suspected data is too small, overfitting may occur and it may not correspond to the unknown abnormal characteristic pattern. Therefore, it is preferable to accumulate an appropriate amount of data for learning while verifying the validity of the abnormality determination model using image data obtained by actually measuring the inspection target in an abnormal state as verification image data.

Fig. 4 is an explanatory diagram illustrating a method of creating a data set for machine learning.

For example, when an abnormality determination model for inspecting a defect occurring on the surface of an inspection target is generated, image data Img _ act having a characteristic pattern as shown in the upper stage of fig. 4 is obtained when the defect occurring on the surface of the inspection target is actually measured.

In the example shown in fig. 4, the user creates pseudo data Img _ fak that represents a characteristic pattern of a defect that may occur, with reference to the characteristic pattern of the defect that actually occurs, or based on past findings or rules of thumb of the user. Further, the user randomly performs image conversion processing based on the created pseudo data Img _ fak to add pseudo data, and generates a data set Img _ tch for learning including a predetermined number of pseudo data.

If the number of dummy data created in step S11 is sufficient as a dataset for opportunistic learning, step S13 of adding dummy data by image conversion processing may be omitted. The generated learning data set Img _ tch may include image data obtained by actually measuring an inspection target in which an abnormality occurs.

Next, the user inputs the generated data set for machine learning into the inspection device 20, and executes machine learning by the learning unit 35 to generate an abnormality determination model (step S15). As described above, the learning unit 35 performs machine learning of the abnormality determination model using the existing machine learning model that can be used for machine learning using image data as supervision data.

Next, the user verifies the validity of the generated abnormality determination model using the verification image data that actually measures the abnormality occurring in the inspection target (step S17). As the verification image data used for verifying the validity, real image data that is not used for machine learning of the abnormality determination model is used. The verification image data includes, for example, image data Img _ act in which an abnormality occurs and image data in a normal state in which no abnormality occurs as shown in fig. 4. The number of verification image data may be larger than the number of pseudo data used for machine learning of the abnormality determination model. The user acquires data on the accuracy of the determination result output by inputting the verification image data to the abnormality determination model.

Next, the user determines whether the acquired accuracy satisfies a desired requirement (step S19). If the accuracy does not satisfy the requirement (S19/no), the user returns to step S11 to newly create pseudo data Img _ fak and again create the abnormality determination model. Instead of returning to step S11, the process may return to step S13 to add pseudo data by image conversion processing. Thereby, it is adapted so that the accuracy of the result of the abnormality determination using the abnormality determination model is improved.

On the other hand, when the accuracy satisfies the required condition (S19/yes), the user ends the job of generating the abnormality determination model. The generated abnormality determination model is stored in the memory 21 or the storage device 23.

The abnormality determination model may be a model that not only determines the presence or absence of an abnormality but also classifies and outputs the type of the abnormality. For example, the anomaly determination model is generated by machine learning by associating the pseudo data Img _ fak of the learning data set Img _ tch used for machine learning of the anomaly determination model with the type of anomaly. This makes it possible to estimate an abnormality occurring in the inspection target from the determination result obtained by inputting the inspection image data to the abnormality determination model, and to select a treatment or urgency level for the abnormality.

< 4. action of inspection apparatus

Next, the operation of the abnormality determination process performed by the determination unit 37 of the inspection apparatus 20 will be described.

Fig. 5 is a flowchart showing the operation of the abnormality determination processing for the inspection target, which is mainly executed by the determination unit 37.

First, the image processing unit 31 acquires image data for measuring the inspection target output from the detector 11, and performs image processing on the image data to generate inspection image data (step S21).

Next, the determination unit 37 inputs the generated inspection image data to the abnormality determination model, and determines whether or not there is an abnormality of the inspection target (step S23). For example, the determination unit 37 acquires data of the probability of occurrence of an abnormality, and determines that an abnormality has occurred as a determination result output from the abnormality determination model when the probability of occurrence of an abnormality is equal to or greater than a predetermined threshold.

Next, the display control unit 39 generates a drive signal for the display unit 13 and causes the display unit 13 to display information of the determination result (step S25). The display control unit 39 may store a log of the determination result in the memory 21 or the storage device 23 together with the display on the display unit 13. In addition, when an abnormality occurs in the inspection target, the display control unit 39 may generate a warning sound or the like and notify the user of the abnormality.

In the case where the abnormality determination model is a model that can not only determine the presence or absence of an abnormality but also classify the type of the abnormality that has occurred, the display control unit 39 may cause the display unit 13 to display information on the type of the abnormality based on the data of the determination result output from the abnormality determination model. At this time, the display control unit 39 may display information such as a treatment method and an emergency degree corresponding to the type of the abnormality.

The inspection apparatus 20 can monitor an abnormality of the inspection target by continuously acquiring the inspection image data measured on the inspection target by the detector 11 fixed at the fixed point and repeating the processing of steps S21 to S25. Alternatively, the inspection device 20 may determine the presence or absence of an abnormality of the inspection target by executing the processing of steps S21 to S25 based on the inspection image data obtained by measuring the inspection target by the detector 11 at an arbitrary timing by the user.

The inspection target is configured to be movable relative to the detector 11 and arranged to pass through the measurement range of the detector 11 in sequence, and the inspection apparatus 20 can monitor abnormality of the inspection target by continuously acquiring the inspection image data measured by the detector 11 with the process of steps S21 to S25 being repeated. Thus, even if the test object is long, the presence or absence of an abnormality can be determined using the abnormality determination model without simultaneously measuring the entire object.

< 5. application example >

Next, several application examples of the inspection system 100 using the inspection device 20 according to the present embodiment will be described.

(5-1. first example)

Fig. 6 is an explanatory diagram showing an example of an inspection system 100A applied in a manufacturing line of products.

In the example shown in fig. 6, 2

detectors

11a, 11b are provided as a scalable manufacturing line that uses a plurality of robotic arms 51 a-51 c, 53 to assemble parts to produce products for equipment flowing on a conveyor belt 55. In such a manufacturing line, for example, the robot arms 51a to 51c or the leg 55a of the conveyor belt 55 may be defective due to contact with another object or slippage.

In this case, the user manually draws a characteristic pattern of a defect that can be considered in advance, and further performs image conversion processing to generate a data set Img _ tch for learning that includes a plurality of pseudo data Img _ fak. The user performs machine learning using the learning data set Img _ tch as input data in advance to generate an abnormality determination model. Then, the inspection device 20 repeatedly executes the following processing: the inspection image data generated from the image data output from the

detectors

11a and 11b is input to the abnormality determination model, and the presence or absence of an abnormality is determined based on the output from the abnormality determination model.

Thus, even when the inspection target such as the robot arms 51a to 51c, 55 includes a movable portion, it is possible to monitor whether or not an abnormality such as a defect occurs in the manufacturing line. Further, since the presence or absence of an abnormality is determined using an abnormality determination model that learns pseudo data created by artificially drawing a characteristic pattern of an abnormality such as a defect as learning data, it is possible to sense an abnormality assumed in advance even when there is no image data of an abnormality that actually occurs. The application example shown in fig. 6 can be used, for example, as an inspection system for the appearance of various inspection objects such as boxes on containers moving in a warehouse, floors or walls in a factory or in a warehouse.

(5-2. second example)

Fig. 7 shows an example of an inspection system 100B applied to inspection of inspection objects 63a to 63e placed on and conveyed by a conveyor belt 61.

In the example shown in fig. 7, 2

detectors

11c and 11d are provided toward specific measurement ranges Ra and Rb of the conveyor belt 61, respectively. It is assumed that defects, scratches, stains, and damages may occur in the inspection objects 63a to 63 e.

In this case, the user manually draws in advance a characteristic pattern of an abnormality in appearance that can be considered and a characteristic pattern of a shape that does not exist in the appearance of the normal inspection target products 63a to 63e, and further performs image conversion processing to generate a data set Img _ tch for learning that includes a plurality of pseudo data Img _ fak. The user performs machine learning using the learning data set Img _ tch as input data in advance to generate an abnormality determination model. Then, in a state where the inspection objects 63a to 63e sequentially pass through each of the measurement ranges Ra and Rb, the inspection apparatus 20 repeatedly executes: the inspection image data generated from the image data output from the

detectors

11c and 11d is input to the abnormality determination model, and the presence or absence of an abnormality in the inspection target products 63a to 63e is determined based on the output from the abnormality determination model.

This enables the inspection of the inspection objects 63a to 63e to be continuously performed automatically, and the inspection objects 63a to 63e sequentially pass through the measurement ranges Ra and Rb of the

detectors

11c and 11 d. Further, since the presence or absence of an abnormality is determined using an abnormality determination model that learns pseudo data created by artificially drawing a characteristic pattern of an abnormality as learning data, it is possible to sense an abnormality assumed in advance or an unknown abnormality even when there is no image data of an abnormality that actually occurs.

In the application example shown in fig. 7, the measurement ranges Ra and Rb of the

detectors

11c and 11d are fixed, and the abnormality determination is performed using the image data measured on the inspection objects 63a to 63b passing through the measurement ranges Ra and Rb, but the inspection system may be configured such that the measurement range of the detector is relatively moved with respect to the inspection object when the inspection object is not moved. Further, instead of the inspection target being placed on a conveyor belt and transported, the inspection target itself may be moved relative to the measurement range of the detector. The application example shown in fig. 7 can also be used as an inspection system for the appearance of a vehicle such as an electric train or an automobile.

< 6. Effect >

As described above, according to the abnormality determination model generation method and the abnormality determination model generation device according to the present embodiment, the image data obtained by actually measuring the inspection target in which an abnormality occurs is not used, and the pseudo data such as the data manually created by the user is used as the learning data for generating the abnormality determination model used for abnormality determination. Therefore, since the data for learning including the characteristic pattern of the abnormality can be freely generated, even before the abnormality actually occurs or when the number of samples in which the specific abnormality occurs is small, the abnormality determination model that can sense the abnormality assumed in advance based on the knowledge or the rule of thumb or the unknown abnormality that is not assumed can be generated. Further, since the learning data recognized as abnormal can be freely generated in advance, the burden of classifying the actually measured image data as normal or abnormal or classifying the image data for each type of abnormality can be reduced, and the cost can be reduced.

Further, according to the present embodiment, validity is verified using verification image data obtained by actually measuring an inspection target for an abnormality determination model generated by machine learning using pseudo data as learning data. Therefore, it is possible to generate an abnormality determination model with high determination accuracy while creating data for learning so that the accuracy of the generated abnormality determination model becomes a desired accuracy. Therefore, even when pseudo data created by manually drawing a characteristic pattern of an abnormality by a user is used, an abnormality determination model with high determination accuracy can be generated.

Further, according to the present embodiment, pseudo data created by manually drawing a characteristic pattern of an abnormality by a user can be used as data for learning. Therefore, it is possible to create data for learning having various shapes of patterns as characteristic patterns, and it is possible to create an abnormality determination model capable of sensing various abnormalities. Further, since pseudo data created by manually drawing an abnormal characteristic pattern by a user can be used as the data for learning, the data for learning including a simplified pattern of a shape can be used, and the possibility of occurrence of overfitting during machine learning can be reduced.

Further, according to the inspection apparatus of the present embodiment, the appearance of the inspection target can be inspected using the abnormality determination model generated by the abnormality determination model generation method and the abnormality determination model generation apparatus. Therefore, even before an abnormality actually occurs or when the number of samples in which a specific abnormality occurs is small, it is possible to easily detect an abnormality occurring in the appearance of the inspection target, thereby improving the determination accuracy. Further, according to the inspection apparatus of the present embodiment, since the abnormality determination is performed using the abnormality determination model in which the overfitting is suppressed, the abnormality determination model being generated using the data for learning including the characteristic pattern of the abnormality which is drawn in a simplified manner, it is possible to improve the accuracy of determining the abnormality which occurs in the appearance of the inspection target.

While preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It should be understood that a person having ordinary knowledge in the art to which the present invention pertains can obviously conceive of various modifications and alterations within the scope of the technical idea described in the claims, and naturally falls within the technical scope of the present invention.

For example, the present invention can be applied if the inspection object to which the abnormality determination model generation method, the abnormality determination model generation device, and the inspection device according to the present invention are applied is an inspection object that can extract a characteristic pattern from image data of the inspection object.

Note that the learning data set Img _ tch may be configured not only from pseudo data, but may include at least one of normal data and abnormal data obtained by actually measuring the test object.

In the above embodiment, the example in which the inspection device 20 includes the learning unit 35 and the determination unit 37 has been described, but the present invention is not limited to such an example. For example, the learning unit 35 may be provided in another device and may be configured as an independent abnormality determination model generation device. In this case, the abnormality determination model is generated by performing machine learning in advance using the abnormality determination model generation device including the learning unit 35, and the abnormality determination model is stored in the memory 21 or the storage device 23 of the inspection device 20 including the determination unit 37.

Further, when the abnormality determination is not performed in real time but performed offline using the inspection apparatus 20, the inspection apparatus 20 may be configured not to include the image processing unit 31. For example, the inspection image data extracted from the image data output from the detector 11 may be stored in a storage medium, and the determination unit 37 of the inspection device 20 may read the inspection image data stored in the storage medium. In the case of such a configuration, the inspection device 20 can also determine an abnormality of the inspection target based on the abnormality determination model.

Description of reference numerals

The system comprises a 1 … inspection device, a 3 … detector, a 7 … display part, a 10 … control part, an 11 … image processing part, a 13 … arithmetic part, a 15 … learning part, a 17 … judging part, a 19 … display control part, a 21 … memory and a 23 … storage device.

Claims

1. An abnormality determination model generation method for generating an abnormality determination model for determining an abnormality of an inspection target using inspection image data for measuring the inspection target as input data, the abnormality determination model generation method comprising:

a step of generating a learning data set (Img _ tch) for generating the abnormality determination model based on pseudo data (Img _ fak) that is not real image data for measuring an inspection target; and

and a step of generating the abnormality determination model using the learning data set (Img _ tch).

2. The abnormality determination model generation method according to claim 1, wherein the characteristic pattern of the abnormality of the suspected data (Img _ fak) is a characteristic pattern of an abnormality created by a user.

3. The abnormality determination model generation method according to claim 1 or 2, characterized in that the characteristic pattern of the abnormality of the suspected data (Img _ fak) is a pattern of a shape of an abnormality formed on the surface of the inspection object.

4. An abnormality determination model generation device that generates an abnormality determination model for determining an abnormality of an inspection object using inspection image data for measuring the inspection object as input data,

the abnormality determination model generation device (20) includes a learning unit (35) that generates the abnormality determination model using a learning data set (Img _ tch) generated based on pseudo data (Img _ fak) that is not real image data for measuring an inspection target.

5. An inspection apparatus that performs an inspection of an inspection object based on inspection image data for measuring the inspection object, the inspection apparatus (20) comprising:

storage units (21, 23) that store an abnormality determination model that is generated using a data set for learning (Img _ tch) that is generated on the basis of pseudo data (Img _ fak) that is not real image data for measuring an inspection target; and

and a determination unit (37) that inputs the inspection image data measured on the inspection object to the abnormality determination model and determines whether or not there is an abnormality in the inspection object.

6. The inspection apparatus according to claim 5, wherein the inspection object is an inspection object configured to be relatively movable with respect to a measurement range (Ra, Rb) of a detector (11) of the inspection image data and configured to sequentially pass through the measurement range (Ra, Rb) of the detector (11).