WO2022124152A1 - 検査経路生成装置、検査経路生成方法 - Google Patents
検査経路生成装置、検査経路生成方法 Download PDFInfo
- Publication number
- WO2022124152A1 WO2022124152A1 PCT/JP2021/044041 JP2021044041W WO2022124152A1 WO 2022124152 A1 WO2022124152 A1 WO 2022124152A1 JP 2021044041 W JP2021044041 W JP 2021044041W WO 2022124152 A1 WO2022124152 A1 WO 2022124152A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- inspection
- route
- unit
- information
- posture
- Prior art date
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 685
- 238000000034 method Methods 0.000 title claims description 114
- 230000008859 change Effects 0.000 claims abstract description 105
- 238000003384 imaging method Methods 0.000 claims abstract description 70
- 230000008569 process Effects 0.000 claims description 77
- 238000011179 visual inspection Methods 0.000 claims description 68
- 238000012545 processing Methods 0.000 claims description 25
- 230000036544 posture Effects 0.000 description 92
- 238000004364 calculation method Methods 0.000 description 39
- 238000010586 diagram Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 5
- 238000004904 shortening Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000007429 general method Methods 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 1
- 238000005352 clarification Methods 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/9515—Objects of complex shape, e.g. examined with use of a surface follower device
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/9515—Objects of complex shape, e.g. examined with use of a surface follower device
- G01N2021/9518—Objects of complex shape, e.g. examined with use of a surface follower device using a surface follower, e.g. robot
Definitions
- the present invention relates to an apparatus and a method for generating an inspection path when a visual inspection apparatus for visual inspection of an inspection object moves a plurality of inspection positions.
- Patent Document 1 includes a moving mechanism for moving an inspected object so that an image of a plurality of imaging regions set on the inspection surface is performed, and the moving locus of the imaging region on the inspection surface is a movement locus of the inspected object.
- An inspection system that can image an inspected object in a plurality of imaging regions (inspection regions) more smoothly by including a section that moves around the center of gravity is disclosed.
- Patent Document 1 In the inspection system disclosed in Patent Document 1, a spiral movement locus that revolves around the center of gravity of the inspected body is set, and the inspected body is imaged and inspected in a plurality of inspection regions on the movement locus. , The amount of movement between inspection areas is minimized to shorten the inspection time.
- the posture of the camera at the time of imaging is not taken into consideration. Therefore, when a defect existing in the inspected object is imaged in a plurality of inspection areas, the position of the defect on the captured image also changes in a spiral shape according to the movement of the camera.
- the inspection route generation device generates an inspection path of an appearance inspection device that inspects the appearance of an inspection object based on an image captured by the image pickup unit, and the image pickup unit captures an image of the inspection object.
- the image pickup is based on a storage unit that stores inspection position information representing a plurality of inspection positions and inspection device configuration information representing the configuration of the visual inspection device, and the inspection position information and the inspection device configuration information.
- the path length and the amount of change in the posture of the imaging unit when the unit moves between the plurality of inspection positions are calculated, and the inspection route is determined based on the calculated path length and the amount of the change in attitude. It is provided with a routing unit to be used.
- the inspection route generation method generates an inspection path of an appearance inspection device that inspects the appearance of an inspection object based on an image captured by the image pickup unit, and the image pickup unit performs the inspection by an arithmetic device.
- the process of determining the inspection route based on the above and the process of determining the inspection route are executed.
- FIG. 1 It is a figure which shows the structural example of the visual inspection system including the inspection path generation apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the data of a robot configuration information. It is a figure which shows an example of the data of inspection equipment composition information. It is a figure which shows an example of the data of the inspection target area information. It is a figure which shows an example of the data of inspection condition information. It is a figure which shows an example of the connection relation between nodes represented by the connection information between nodes. It is a figure which shows an example of the data of the movement cost information. It is a figure which shows an example of the data of output information.
- a process performed by executing a program may be described, but the program is executed by a processor (for example, CPU, GPU) to appropriately store a predetermined process as a storage resource (a storage resource (for example, CPU, GPU). Since it is performed using, for example, a memory) and / or an interface device (for example, a communication port), the main body of processing may be a processor. Similarly, the main body of the process of executing the program may be a controller, an apparatus, a system, a computer, or a node having a processor. The main body of the process of executing the program may be an arithmetic unit, and may include a dedicated circuit (for example, FPGA or ASIC) for performing a specific process.
- a dedicated circuit for example, FPGA or ASIC
- the program may be installed in a device such as a calculator from the program source.
- the program source may be, for example, a program distribution server or a computer-readable storage medium.
- the program distribution server includes a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers.
- two or more programs may be realized as one program, or one program may be realized as two or more programs.
- FIG. 1 is a diagram showing a configuration example of a visual inspection system including an inspection route generation device according to a first embodiment of the present invention.
- the visual inspection system shown in FIG. 1 is configured by connecting the inspection route generation device 100 and the visual inspection device 200 to each other via a communication line 300.
- the communication line 300 is configured by using, for example, Ethernet (registered trademark), wireless LAN, or the like, and transmits various information transmitted and received between the inspection route generation device 100 and the visual inspection device 200.
- the visual inspection device 200 includes an imaging unit 210, a robot unit 220, and an arithmetic processing unit 230.
- the image pickup unit 210 takes an image of an inspection object (not shown) and outputs the obtained captured image to the arithmetic processing unit 230.
- the robot unit 220 is configured by combining a plurality of joints rotatably configured along a predetermined rotation direction and a link connecting each joint, and is configured according to the control of the arithmetic processing unit 230. By adjusting the angle of each joint, the image pickup unit 210 attached to the tip portion is moved to an arbitrary inspection position, and the posture is set according to an arbitrary image pickup direction.
- the arithmetic processing unit 230 controls the robot unit 220 and determines whether or not the appearance of the inspection object is normal based on the captured image input from the imaging unit 210. As a result, the appearance of the inspection object is inspected.
- the inspection route generation device 100 visually inspects the order of each inspection position that the image pickup unit 210 passes through when the appearance inspection device 200 performs the appearance inspection of the inspection object, and the posture of the image pickup unit 210 at each inspection position. Generated as an inspection path of the device 200. Then, the generated inspection route information is transmitted to the visual inspection device 200 to support the visual inspection of the inspection object by the visual inspection device 200.
- the inspection route generation device 100 includes an input device 110, an output device 120, a storage unit 130, and an arithmetic unit 140.
- the input device 110 is a device for inputting each information to be described later stored in the storage unit 130 by the operation of the user.
- the input device 110 is configured by using, for example, a keyboard, a mouse, a touch panel, or the like.
- the output device 120 is a device for outputting information such as the generated inspection route and providing it to the user.
- the output device 120 is configured by using, for example, a display, a printer, a smartphone, a tablet PC, or the like.
- the storage unit 130 stores information input from the user using the input device 110 and information such as the calculation result of the calculation device 140.
- the storage unit 130 is configured by using a well-known information storage device capable of reading and writing data, such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive).
- the storage unit 130 stores the inspection device configuration information 131, the inspection object information 132, the inspection condition information 133, the route graph information 134, and the output information 135.
- Information other than this, for example, information such as a program executed by the arithmetic unit 140 may be stored in the storage unit 130.
- the inspection device configuration information 131 is information regarding the configuration of the visual inspection device 200, and includes the robot configuration information 1310 and the inspection equipment configuration information 1311.
- the robot configuration information 1310 is information representing the shape of the robot unit 220 in the visual inspection device 200, the range of each joint angle, the limit speed, and the like.
- the inspection equipment configuration information 1311 is information representing the arrangement of the image pickup unit 210 and the robot unit 220 constituting the visual inspection device 200.
- the inspection object information 132 is information about an inspection object to be visually inspected by the visual inspection apparatus 200, and includes shape information 1320 and inspection target area information 1321.
- the shape information 1320 is information representing the shape of the object to be inspected.
- the inspection target area information 1321 is information representing an area to be inspected for appearance in the inspection target object.
- the inspection condition information 133 is information regarding constraint conditions when the visual inspection device 200 performs a visual inspection of an inspection object, and includes inspection position information 1330, inspectable area information 1331, and camera movable range information 1332.
- the inspection position information 1330 is information indicating an inspection position in which the image pickup unit 210 takes an image of an inspection object and a posture of the image pickup unit 210 at each inspection position.
- the inspectable area information 1331 is information representing an inspectable area for each inspection position.
- the inspectable area represented by the inspectable area information 1331 is an visual inspection from an image captured for each inspection position acquired by the imaging unit 210 among a plurality of surface areas to be visually inspected in the inspection object. A surface area on which the device 200 can perform a visual inspection.
- the camera movable range information 1332 is information representing the movable range of the imaging unit 210 at each inspection position.
- the movable range represented by the camera movable range information 1332 corresponds to the constraint condition of the posture of the imaging unit 210 at each inspection position.
- the route graph information 134 is information related to the route graph used in the calculation performed by the arithmetic unit 140 when generating the inspection route of the visual inspection apparatus 200, and includes the inter-node connection information 1340 and the movement cost information 1341.
- the node-to-node connection information 1340 is information representing the connection relationship between the nodes corresponding to each inspection position.
- the movement cost information 1341 is information representing the movement cost according to the length of the movement time when the image pickup unit 210 moves between each node represented by the connection information between nodes 1340.
- the output information 135 is information representing the calculation result by the calculation device 140, and includes inspection path information 1350, camera angle information 1351, and robot posture information 1352.
- the inspection route information 1350 is information representing the optimum inspection route of the visual inspection apparatus 200 generated by the arithmetic of the arithmetic unit 140.
- the camera angle information 1351 is information representing the rotation angle of the image pickup unit 210 at each inspection position in the inspection path represented by the inspection path information 1350.
- the robot posture information 1352 is information representing the posture of the robot unit 220 at each inspection position in the inspection path represented by the inspection route information 1350.
- the arithmetic unit 140 is a part in which the inspection path generation device 100 performs various arithmetic processes for generating an inspection path, and is composed of well-known elements such as a CPU, GPU, RAM, and ROM.
- the arithmetic unit 140 has, as its functions, a route determination unit 141, an inspectable area calculation unit 142, a camera movable range calculation unit 143, a route graph generation unit 144, a movement cost calculation unit 145, a minimum cost route selection unit 146, and an output information generation.
- Each functional block of unit 147 is provided.
- the arithmetic unit 140 can realize these functional blocks by, for example, executing a predetermined program stored in advance.
- the route determination unit 141 determines the inspection route by using each functional block of the inspectable area calculation unit 142 to the output information generation unit 147. The details of the processing performed by the routing unit 141 will be described later with reference to FIGS. 9 and 10.
- the inspectable area calculation unit 142 calculates an inspectable area for each inspection position, and stores the calculation result in the storage unit 130 as inspectable area information 1331.
- the camera movable range calculation unit 143 calculates the movable range of the imaging unit 210 for each inspection position, and stores the calculation result in the storage unit 130 as the camera movable range information 1332.
- the route graph generation unit 144 sets the nodes corresponding to each inspection position and generates a route graph showing the connection relationship between the nodes. Then, the generated route graph information is stored in the storage unit 130 as the inter-node connection information 1340.
- the movement cost calculation unit 145 calculates the movement cost when the imaging unit 210 moves between each node of the route graph based on the inspection device configuration information 131 and the inspection condition information 133, and the calculation result is the movement cost information 1341. Is stored in the storage unit 130.
- the minimum cost route selection unit 146 selects the minimum cost route via all the nodes based on the route graph generated by the route graph generation unit 144 and the movement cost calculated by the movement cost calculation unit 145. As a result, the visual inspection apparatus 200 generates an optimum inspection route when performing a visual inspection of the inspection object.
- the output information generation unit 147 determines the rotation angle (imaging direction) of the imaging unit 210 at each node and the angle of each joint of the robot unit 220 at each node with respect to the minimum cost route selected by the minimum cost route selection unit 146. To determine. Then, the output information 135 is generated and stored in the storage unit 130 by using these determined information and the information of the inspection route generated by the minimum cost route selection unit 146.
- FIG. 2 is a diagram showing an example of the data of the robot configuration information 1310 among the inspection device configuration information 131 stored in the storage unit 130.
- the robot configuration information 1310 has the data shown in the data columns 101 to 103 for each component constituting the robot unit 220.
- names such as "robot", "link”, and "joint” are stored as classification names of each component constituting the robot unit 220.
- the item name of the data is stored in the data column 102.
- data values, file names, and the like are stored as information corresponding to the item names of the data column 102.
- FIG. 3 is a diagram showing an example of the data of the inspection equipment configuration information 1311 among the inspection equipment configuration information 131 stored in the storage unit 130.
- the inspection equipment configuration information 1311 has the data shown in the data columns 111 to 114 for each equipment constituting the visual inspection apparatus 200.
- the data column 111 stores the names of the facilities constituting the visual inspection device 200.
- the data column 112 stores information corresponding to each equipment to be installed, that is, information indicating to which other equipment each equipment is installed.
- the data column 113 stores information regarding the relative position of each equipment with respect to the equipment to be attached. Information regarding the relative posture of each equipment with respect to the equipment to be attached is stored in the data column 114.
- FIG. 4 is a diagram showing an example of the data of the inspection target area information 1321 among the inspection target information 132 stored in the storage unit 130.
- the inspection target area information 1321 has the data shown in the data columns 211 to 218 for each inspection target to be visually inspected by the visual inspection apparatus 200.
- the part name of each inspection target is stored in the data column 211.
- the data representing the areas F1 to Fn of the inspection object is stored in the storage unit 130 as the shape information 1320 of the inspection object information 132, and can be represented in a well-known polygon data format such as STL.
- FIG. 5 is a diagram showing an example of the data of the inspection condition information 133 stored in the storage unit 130.
- the inspection condition information 133 contains data columns 301 to 307, 311 to 316, 321 for each inspection position in which the image pickup unit 210 takes an image of the inspection object when the appearance inspection device 200 performs the appearance inspection. It has the data shown in ⁇ 322.
- Information representing each inspection position is stored in the data column 301.
- each inspection position is set to A to L, and information representing these inspection positions A to L is stored.
- Information indicating the relative position of each inspection position with respect to the inspection object is stored in the data columns 302 to 304.
- Information indicating the relative posture of the image pickup unit 210 at each inspection position with respect to the inspection object is stored in the data columns 305 to 307.
- the inspection position information 1330 is configured by each of the data of these data columns 302 to 307.
- the visual inspection apparatus 200 can perform a visual inspection of the regions F1 to Fn from the captured image of the inspection object acquired by the imaging unit 210 at each inspection position.
- “1" indicates that the region can be inspected from the captured image obtained at each inspection position
- "0" indicates that the region can be inspected from the captured image obtained at each inspection position. Indicates that the area cannot be inspected.
- the inspectable area information 1331 is configured by each of the data of these data columns 311 to 316.
- Information indicating the range in which the robot unit 220 can move the image pickup unit 210 at each inspection position is stored in the data columns 321 to 322, respectively.
- the minimum rotation angle stored in the data string 321 and the maximum rotation angle stored in the data string 322 represent a rotation angle range when the image pickup unit 210 is rotated in the yaw direction.
- "30" is stored in the data column 321
- "200" is stored in the data column 322. This indicates that at the inspection position A, the imaging unit 210 can take the posture required for imaging the inspection object within the range of the yaw rotation angle of 30 ° to 200 °.
- the camera movable range information 1332 is configured by each of the data of these data columns 321 and 322.
- the camera movable range information 1332 may be represented by a roll rotation angle, a pitch rotation angle, or the like, or a combination thereof.
- the camera movable range information 1332 may be used.
- FIG. 6 is a diagram showing an example of the connection relationship between the nodes represented by the node-to-node connection information 1340 among the route graph information 134 stored in the storage unit 130.
- the node-to-node connection information 1340 for example, as shown in FIG. 6, nodes corresponding to the inspection positions A to G are set, and the connection relationship between these nodes is represented by an arrow.
- the information of the arrow between each node shown in FIG. 6 is stored in the storage unit 130 as the inter-node connection information 1340, so that when the visual inspection device 200 performs a visual inspection of the inspection object, the imaging unit 210 moves to the inspection position A.
- the combination of the order of moving ⁇ G is expressed.
- FIG. 7 is a diagram showing an example of the data of the movement cost information 1341 among the route graph information 134 stored in the storage unit 130.
- the movement cost is large for each combination of the column 411 representing the start point node and the row 422 representing the end point node for the connection relationship between each node represented by the connection information between nodes 1340.
- a numerical value indicating the value is stored.
- No numerical value is stored for the combination of the start point B and the end point C in FIG. 7 between the nodes whose movement costs are known to be high, for example.
- the numerical value of the movement cost stored between each node is an asymmetrical numerical value so that the numerical value changes when the start point and the end point are exchanged in order to impose restrictions such as patrolling the inspection position clockwise. It has become.
- These moving cost values are obtained by obtaining the moving time between each inspection position, the posture change of the imaging unit 210, the traveling direction, etc. based on the inspection device configuration information 131 and the inspection condition information 133 in the moving cost calculation unit 145. Calculated.
- FIG. 8 is a diagram showing an example of the data of the output information 135 stored in the storage unit 130.
- the output information 135 is provided in the data columns 501 to 507, 511, 521 to 525 for each inspection position in which the image pickup unit 210 takes an image of the inspection object when the appearance inspection device 200 performs the appearance inspection.
- Information representing each inspection position is stored in the data sequence 501 in the order in which the imaging unit 210 passes through the inspection route generated by the minimum cost route selection unit 146.
- Information indicating the relative position of each inspection position with respect to the inspection object is stored in the data columns 502 to 504.
- Information indicating the relative posture of the image pickup unit 210 at each inspection position with respect to the inspection object is stored in the data columns 505 to 507.
- the inspection route information 1350 is configured by each of the data of these data columns 502 to 507.
- the data of each row in the inspection route information 1350 corresponds to the data of each row in the inspection position information 1330, and the same data is stored in the same inspection position.
- Information indicating the rotation angle of the imaging unit 210 at each inspection position is stored in the data column 511.
- the camera angle information 1351 is configured by the data of the data string 511.
- the camera angle information 1351 may be represented by, for example, a roll rotation angle or a pitch rotation angle, and these may be combined to represent the camera angle.
- Information 1351 may be used.
- the data columns 521 to 525 store information representing the angles of these joints J1 to Jn at each inspection position when the joints of the robot unit 220 are J1 to Jn (n is a natural number).
- the robot posture information 1352 is configured by each of the data of these data columns 521 to 525.
- FIG. 9 is an example of a flowchart of the inspection route planning process according to the first embodiment of the present invention. The process shown in the flowchart of FIG. 9 is executed by the routing unit 141 in the arithmetic unit 140.
- step S101 the inspection device configuration information 131, the inspection object information 132, and the inspection position information 1330 are read from the storage unit 130 and input to the arithmetic unit 140 as the information used for the processing in step S102. It is assumed that these pieces of information are stored in advance in the storage unit 130 based on the information or the like previously input by the user using the input device 110.
- step S102 an inspection route determination process for determining the optimum inspection route is executed based on each information input in step S101.
- the details of the inspection route determination process executed in step S102 will be described later with reference to the flowchart of FIG.
- step S103 the information of the inspection route determined in step S102 is stored in the storage unit 130 as output information 135, and is output to the output device 120.
- FIG. 10 is an example of a flowchart of the inspection route determination process performed in step S102.
- the route determination unit 141 in the arithmetic unit 140, the route determination unit 141 has the inspectable area calculation unit 142, the camera movable range calculation unit 143, the route graph generation unit 144, the movement cost calculation unit 145, and the minimum cost route selection unit. It is executed by using these functional blocks by appropriately calling 146 and the output information generation unit 147.
- step S200 the inspectable area of the visual inspection device 200 at each inspection position is calculated based on the inspection object information 132 and the inspection position information 1330 read in step S101 of FIG.
- the inspectable area calculation unit 142 based on the relative position and posture of the imaging unit 210 at each inspection position represented by the inspection position information 1330 by the inspectable area calculation unit 142, and the shape information 1320 included in the inspection object information 132.
- the position and orientation of the image pickup unit 210 and the positions and orientations of the regions F1 to Fn of the inspection object are compared for each inspection position. As a result, if each of these differences is within a predetermined range, the area is determined as an inspectable area, and if it is outside the predetermined range, the area is determined not to be an inspectable area.
- the inspectable area information calculated in step S200 is stored in the storage unit 130 as the inspectable area information 1331 included in the inspection condition information 133. Further, based on the inspection target area information 1321 included in the inspection target area information 132, the inspectable area calculated is extracted, and the area of each extracted inspectable area is based on the shape information 1320. To calculate. The area of each inspectable area calculated here is used when calculating the total inspectable areas Ao and Ac in steps S204 and S207 described later.
- step S201 the movable range of the imaging unit 210 according to the posture in which the robot unit 220 can be taken at each inspection position is calculated based on the inspection device configuration information 131 and the inspection position information 1330 read in step S101 of FIG.
- the shape of the robot unit 220, the angle range and limit speed of each joint, and the appearance inspection are performed by the camera movable range calculation unit 143.
- Information such as the shape and arrangement of each facility in the device 200 is acquired, and based on this information, the range of postures that the robot unit 220 can take at each inspection position is calculated.
- the movable range of the image pickup unit 210 at each inspection position that is, the constraint condition of the posture of the image pickup unit 210 is calculated.
- the information on the movable range of the imaging unit 210 calculated in step S201 is stored in the storage unit 130 as the camera movable range information 1332 included in the inspection condition information 133.
- step S202 the movement cost between each inspection position is calculated based on the inspection position information 1330 read in step S101 of FIG. 9 and the camera movable range information 1332 calculated in step S201, and a route graph is generated.
- the route graph generation unit 144 sets the nodes corresponding to each inspection position based on the inspection position information 1330, and sets the movement route expressing the connection relationship of the image pickup unit 210 by the line connecting each node. .. Then, the movement cost calculation unit 145 calculates the path length (distance between nodes) of each set movement path, the posture change amount of the image pickup unit 210, the traveling direction, etc. based on the inspection position information 1330 and the camera movable range information 1332.
- the travel route information and travel cost information calculated in step S202 are stored in the storage unit 130 as inter-node connection information 1340 and travel cost information 1341 included in the route graph information 134, respectively.
- FIG. 11 is an explanatory diagram of a method of calculating the movement cost. Below, four nodes A to D are set as shown in FIG. 11, and an example of calculating the movement cost between these nodes will be described.
- the movement cost calculation unit 145 is the length (distance) of each movement path connecting the two points for all combinations when two are selected from the four nodes A, B, C, and D shown in FIG. ,
- the amount of change in posture of the imaging unit 210 and the traveling direction are calculated.
- FIG. 11 shows an example of the distance of each movement path calculated in this way, the amount of change in the posture of the image pickup unit 210, and the traveling direction. For example, in the movement path starting from the node A and ending at the node B, the distance is 0.2, the posture change amount of the image pickup unit 210 is 30, and the image pickup unit 210 moves in the counterclockwise direction. ing.
- the movement cost calculation unit 145 calculates the movement cost between each node by evaluating the length of time required for the imaging unit 210 to move each movement route based on the above information calculated for each movement route. It can be calculated and the movement cost information 1341 can be set. At this time, for a movement path in which the movement cost is clearly excessive as compared with other movement paths, such as a movement path in which the amount of change in posture of the imaging unit 210 is equal to or more than a predetermined threshold value, the example of FIG. It is preferable not to set the movement cost in the movement cost information 1341 as in the combination of the start point B and the end point C in. By doing so, in the processing after step S203 described later, it is possible to efficiently determine the inspection route by excluding the candidate of the inspection route which increases the moving cost in advance.
- the minimum cost route selection unit 146 generates an initial inspection route candidate Ro using the route graph generated in step S202.
- the initial inspection route candidate Ro is generated by randomly selecting the route route order of the nodes in the route graph, and this is set as the initial value of the inspection route candidate Rc. At this time, it is not always necessary to select a route that goes through all the nodes. Any route that passes through at least two or more nodes can be selected as the initial inspection route candidate Ro. Further, for the movement route for which the cost is not set in the movement cost information 1341, for example, the movement route corresponding to the combination of the start point B and the end point C in FIG. 7, the initial inspection route candidate Ro is generated so as not to include the movement route. It is preferable to do so.
- the minimum cost route selection unit 146 calculates the total movement cost Co of the initial inspection route candidate Ro generated in step S203 and the total inspectable area Ao.
- the total travel cost Co is calculated by summing the travel costs calculated in step S202 for each travel route included in the initial inspection route candidate Ro according to the route order of the nodes selected in the initial inspection route candidate Ro. be able to.
- the total inspectable area Ao can be calculated by totaling the areas of the inspectable areas calculated in step S200 for each inspection position corresponding to each node of the initial inspection route candidate Ro while avoiding duplication. can.
- the total movement cost Co of the initial inspection route candidate Ro and the total inspectable area Ao calculated in step S204 correspond to the total movement cost Cc of the inspection route candidate Rc and the initial values of the total inspectable area Ac, respectively.
- steps S205 to S210 loop processing is executed.
- step S206 From step S206 until the total movement cost Cc of the inspection route candidate Rc set during the loop processing and the value of the total inspectable area Ac each exceed a predetermined threshold value or the number of loops exceeds a predetermined threshold value.
- the processing up to S210 is repeatedly executed.
- the minimum cost route selection unit 146 generates a new inspection route candidate Rc from the route graph generated in step S202.
- an unselected inspection route can be generated as an inspection route candidate Rc by randomly selecting the route passage order of the nodes in the route graph.
- the inspection route candidate Rc may be generated by exchanging, adding, deleting, or the like, some nodes of the current inspection route R in the loop processing.
- step S207 the minimum cost route selection unit 146 calculates the total movement cost Cc and the total inspectable area Ac of the inspection route candidate Rc generated in step S206.
- the total movement cost Cc and the total inspectable area Ac can be calculated by the same method as in step S204.
- step S210 the minimum cost route selection unit 146 updates the inspection route R, the total movement cost C of the inspection route R, and the total inspectable area A.
- the inspection route candidate Rc generated in the latest step S206 is set in the inspection route R
- the total movement cost Cc of the inspection route candidate Rc and the total inspectable area Ac are set to the total movement cost C of the inspection route R and These values are updated by setting each in the total inspectable area A.
- the inspection route R having the smallest total movement cost Cc and the largest total inspectable region Ac among the plurality of inspection route candidate Rc is finally determined. ..
- step S211 the output information generation unit 147 calculates the rotation angle of the image pickup unit 210 with respect to each inspection position of the inspection path R finally determined by the loop processing of steps S205 to S210.
- a combination of rotation angles such that the change in the yaw rotation angle of the imaging unit 210 is minimized within the camera movable range calculated in step S201 is selected for the inspection path R.
- Select for each inspection position The same calculation method can be applied to the case where the roll rotation angle, the pitch rotation angle, and the like are set for the image pickup unit 210 as described above. As a result, the posture P of the imaging unit 210 in the inspection path R is calculated.
- step S212 output information 135 is generated based on the inspection path R finally determined by the loop processing of steps S205 to S210 and the posture P calculated in step S212.
- the output information generation unit 147 determines the angle of each joint of the robot unit 220 in each node of the inspection path R based on the inspection device configuration information 131 and the inspection position information 1330 read in step S101 of FIG. do. Then, the inspection position information 1330 read in step S101 is rearranged according to the route order of each inspection position in the inspection path R, and the posture P corresponding to each inspection position and the angle of each joint are set to set the inspection path information 1350. , Camera angle information 1351 and robot attitude information 1352 are determined. As a result, the output information 135 can be generated.
- the output information 135 generated in step S212 is stored in the storage unit 130 and is output to the output device 120 in step S103 of FIG. At this time, information such as the total movement cost C of the inspection route R and the total inspectable area A may be included in the output information 135.
- the movement cost between each node may not be calculated when the route graph is generated in step S202.
- the node for the entire inspection route candidate when calculating the total movement cost Co of the initial inspection route candidate Ro in step S204, or when calculating the total movement cost Cc of the inspection route candidate Rc in step S207, the node for the entire inspection route candidate. It is preferable to calculate the total movement costs Co and Cc in consideration of the total distance, the amount of change in the posture of the imaging unit 210, the change in the traveling direction, and the like.
- the calculation method of the total travel cost Cc when the route passing through the four nodes A to D shown in FIG. 11 in the order of A ⁇ B ⁇ C ⁇ D is selected as the inspection route candidate Rc will be described below.
- ⁇ , ⁇ , and ⁇ are weighting coefficients, and arbitrary values are set for each.
- the inspection route candidate Rc that minimizes the total movement cost Cc represented by the equation (1) is finally selected as the inspection route R.
- the inspection route candidate Rc that minimizes the total movement cost Cc represented by the equation (1) is finally selected as the inspection route R.
- steps S208 and S209 may be exchanged, or only one of steps S208 or S209 may be carried out.
- step S208 the inspection route candidate Rc having the lowest movement cost is finally selected as the inspection route R in the loop processing of steps S205 to S210. Therefore, an inspection route that can realize the visual inspection of the inspection object by the visual inspection apparatus 200 in a short time can be obtained.
- step S209 it is determined whether or not the total inspectable area Ac of the inspection route candidate Rc is equal to or greater than a predetermined threshold value, and the process of step S210 is performed only when the threshold value is equal to or greater than the threshold value. You may do it. By doing so, it is possible to prevent the inspection route from being generated in which the inspectable area of the inspection object is narrow and therefore the desired inspection accuracy cannot be obtained.
- step S208 when only step S208 is performed, the inspection route candidate Rc having the widest inspectable area is finally selected as the inspection route R in the loop processing of steps S205 to S210. Therefore, in the visual inspection of the inspection object by the visual inspection apparatus 200, an inspection route capable of maximizing the inspectable area can be obtained.
- step S208 it is determined whether or not the total movement cost Cc of the inspection route candidate Rc is equal to or less than a predetermined threshold value, and the process of step S210 is performed only when it is equal to or less than the threshold value. You may. By doing so, it is possible to prevent the generation of an inspection route in which the visual inspection cannot be performed within a desired time due to the high movement cost.
- the total cost W1 which is the weighted sum of the total movement cost Cc of the inspection route candidate Rc and the total inspectable area Ac is calculated by the following equation (2), and this total cost is calculated.
- W1 may be used to evaluate the test route candidate Rc.
- W1 ⁇ Cc + ⁇ (A all -Ac) ... Equation (2)
- ⁇ and ⁇ are weighting coefficients, and arbitrary values are set for each. Further, A all is the total area of the inspection target area in the inspection target.
- the total cost W1 for all the movement routes included in the route graph generated in step S202 is calculated by the above equation (2), and then the route for minimizing the total cost W1 is obtained. It may be obtained as the inspection route R.
- the inspection path R that minimizes the total cost W1 may be calculated by a general method such as a brute force method or a greedy method.
- the total cost W2 different from the total cost W1 is calculated for all the routes represented by the route graph generated in step S202, and then the total cost W2 is calculated.
- the route to be minimized may be obtained as the inspection route R.
- the total cost W2 is calculated using, for example, the following equation (3) as a weighted sum of the total movement cost Cc of each inspection route candidate Rc obtained from the route graph, the total inspectable region Ac, and the total posture change amount Y. can do.
- W2 ⁇ Cc + ⁇ (A all -Ac) + ⁇ (MAX (Y) -MIN (Y)) ... Equation (3)
- MAX (Y) is the maximum value of the total posture change amount Y
- MIN (Y) is the minimum value of the total posture change amount Y.
- the inspection path R can be optimized at one time including the amount of change in the posture of the imaging unit 210.
- the optimum solution of the inspection path R in this case can be solved as a traveling salesman problem, or can be calculated by a general method such as a brute force method or a greedy method.
- FIG. 12 is a diagram showing an example of an output screen in the inspection route generation device according to the first embodiment of the present invention.
- the output screen 1100 shown in FIG. 12 is displayed in the output device 120 based on the output information 135 output in step S103 of FIG.
- the output screen 1100 includes selection frames 1101 to 1103, operation buttons 1104 to 1107, an inspection route list 1108, and a display frame 1109.
- the user can select any inspection equipment as the appearance inspection device 200 by using the input device 110.
- Information on the selected inspection equipment is displayed in the selection frame 1101.
- the inspection equipment configuration information 131 corresponding to the inspection equipment is stored in the storage unit 130.
- the user can select an arbitrary inspection object by using the input device 110.
- the information of the selected inspection object is displayed in the selection frame 1102.
- the inspection object information 132 and the inspection position information 1330 corresponding to the inspection object are stored. It is stored in the unit 130.
- the user After the inspection device configuration information 131, the inspection object information 132, and the inspection position information 1330 are stored in the storage unit 130 as described above, the user performs an operation of selecting the operation button 1106 on the output screen 1100.
- the arithmetic unit 140 can be made to execute the route planning process.
- the route determination unit 141 executes the process according to the flowcharts of FIGS. 9 and 10.
- the inspection route list 1108 displays a list of inspection routes generated by the route planning process. Here, based on the output information 135 determined in the route planning process plus the above-mentioned total movement cost C and total inspectable area A, for example, the execution time of the appearance inspection in each inspection route and the posture change of the imaging unit 210. Information such as the amount and coverage of the area to be inspected is displayed. By selecting any of the inspection routes in the inspection route list 1108, the user can select an execution recipe when the visual inspection apparatus 200 performs the visual inspection. The information of the selected execution recipe is displayed in the selection frame 1103.
- the appearance inspection by the execution recipe is displayed in the display frame 1109. From the display contents of the display frame 1109, the user visually confirms the movement of the visual inspection device 200 in the visual inspection, the posture change of the image pickup unit 210, the inspectable area of the inspection object, and the like, and determines the inspection route to be adopted. can do.
- FIG. 13 is a diagram showing another example of an output screen in the inspection route generation device according to the first embodiment of the present invention.
- the output screen 1200 shown in FIG. 13 is displayed in the output device 120 based on the output information 135 output in step S103 of FIG. 9, similarly to the output screen 1100 of FIG.
- the output screen 1200 includes a display frame 1201 instead of the display frame 1109 in the output screen 1100 of FIG.
- the display frame 1201 displays a change in the posture of the image pickup unit 210, for example, a change in the yaw rotation angle in the visual inspection according to the selected execution recipe. From the display contents of the display frame 1201, the user can confirm how much the posture change of the imaging unit 210 in the visual inspection is, and determine the inspection route to be adopted.
- the inspection route generation device 100 generates an inspection route of the appearance inspection device 200 that inspects the appearance of the inspection object based on the image captured by the image pickup unit 210.
- the inspection route generation device 100 is a storage unit that stores inspection position information 1330 representing a plurality of inspection positions in which the image pickup unit 210 captures an image of an inspection object, and inspection device configuration information 131 representing the configuration of the visual inspection device 200. It includes 130 and a routing unit 141. Based on the inspection position information 1330 and the inspection device configuration information 131, the route determination unit 141 determines the path length and the amount of change in the posture of the image pickup unit 210 between the inspection positions when the image pickup unit 210 moves through a plurality of inspection positions.
- step S202 It is calculated (step S202), and the inspection route is determined based on the calculated route length and the amount of change in posture (steps S203 to S210). As described above, it is possible to provide an inspection route capable of shortening the inspection time while suppressing a decrease in inspection accuracy for the visual inspection apparatus 200 that inspects the appearance of the inspection object.
- the route determination unit 141 sets the movement costs Co and Cc based on the path length between each inspection position when the image pickup unit 210 moves between a plurality of inspection positions, the amount of change in the posture of the image pickup unit 210, and the traveling direction. It is calculated (steps S202, S204, S206), and the inspection route is determined based on the calculated movement costs Co and Cc (steps S208 to S210). Since this is done, it is possible to reliably determine the inspection route having the shortest inspection time within a predetermined inspection accuracy range.
- the route determination unit 141 calculates a constraint condition for the posture of the image pickup unit 210 at each inspection position based on the inspection device configuration information 131 (step S201), and changes the posture of the image pickup unit 210 based on the calculated constraint condition. Calculate the amount (step S202). As a result, the amount of change in the posture of the image pickup unit 210 can be accurately calculated in consideration of the range of postures that the image pickup unit 210 can take in the visual inspection apparatus 200.
- the route determination unit 141 does not set the movement cost when the amount of change in the posture of the imaging unit 210 between arbitrary inspection positions is equal to or more than a predetermined threshold value. May be good. By doing so, when a plurality of inspection route candidate Rc are generated in the loop processing of steps S205 to S210 and the inspection route R is determined from the generated plurality of inspection route candidate Rc, the inspection route corresponding to the condition is satisfied.
- the inspection route R is determined by excluding the candidate Rc. As a result, it is possible to efficiently determine the inspection route by excluding the inspection route candidates having an excessive inspection time in advance.
- the inspection route generation device 100 comprises an inspection position information 1330 indicating a plurality of inspection positions in which the image pickup unit 210 of the visual inspection device 200 captures an image of an inspection object by the arithmetic unit 140, and a configuration of the visual inspection device 200.
- a process of calculating the path length and the amount of change in the posture of the imaging unit 210 between them (step S202) and a process of determining the inspection route based on the calculated path length and the amount of change in the posture (steps S203 to S210) are executed. do.
- FIG. 14 is a diagram showing a configuration example of a visual inspection system including an inspection route generation device according to a second embodiment of the present invention.
- the visual inspection system shown in FIG. 14 is configured by connecting the inspection route generation device 100A and the visual inspection device 200 to each other via a communication line 300.
- the difference between the inspection route generation device 100 of FIG. 1 described in the first embodiment and the inspection route generation device 100A shown in FIG. 14 is the route determination unit 141 and the inspectable area calculation unit already described in the arithmetic unit 140A.
- the route change unit 151 changes the inspection route determined by the route determination unit 141 by using each functional block of the change target inspection position selection unit 152, the candidate position setting unit 153, and the candidate position selection unit 154. The details of the processing performed by the route changing unit 151 will be described later with reference to FIGS. 15 and 16.
- the change target inspection position selection unit 152 selects any of a plurality of inspection positions in the inspection route determined by the route determination unit 141 as the change target inspection position.
- the change target inspection position selection unit 152 selects the change target inspection position based on, for example, the movement cost between inspection positions or the designation from the user.
- the candidate position setting unit 153 has a plurality of candidate positions that are candidates for the inspection position to be changed and the posture of the imaging unit 210 at each candidate position around the change target inspection position selected by the change target inspection position selection unit 152. Set the combination with.
- the candidate position selection unit 154 does not reduce the inspectable area from the combination of the plurality of candidate positions set by the candidate position setting unit 153 and the posture of the image pickup unit 210, and the total movement cost in the entire inspection route is not reduced.
- the combination that minimizes is selected as the changed inspection position and the posture of the imaging unit 210.
- FIG. 15 is an example of a flowchart of the inspection route planning process according to the second embodiment of the present invention. The process shown in the flowchart of FIG. 15 is executed by the route determination unit 141 and the route change unit 151 in the arithmetic unit 140A.
- step S301 the inspection device configuration information 131, the inspection object information 132, and the inspection position information 1330 are read from the storage unit 130 and input to the arithmetic unit 140A as the information used for the processing in step S302. It is assumed that these pieces of information are stored in advance in the storage unit 130 based on the information or the like previously input by the user using the input device 110.
- step S302 an inspection route determination process for determining the optimum inspection route is executed based on each information input in step S301.
- the inspection route determination process is executed according to the flowchart of FIG. 10 described in the first embodiment.
- step S303 the inspection route change process for changing the inspection route determined in step S302 is executed.
- the details of the inspection route change process executed in step S303 will be described later with reference to the flowchart of FIG.
- step S304 it is determined whether or not the visual inspection by the inspection route changed in step S303 is feasible in the visual inspection apparatus 200. For example, from the inspection device configuration information 131 input in step S301, information such as the shape of the robot unit 220 in the visual inspection device 200, the angle range of each joint, the limit speed, and the shape of each facility constituting the visual inspection device 200. By acquiring information such as arrangement and performing buffering determination or the like based on this information, it is determined whether or not the operation according to the changed inspection path can be executed in the robot unit 220. As a result, if it is determined that it is feasible, the process proceeds to the process of step S305, and if it is determined that it is not feasible, the process proceeds to the process of step S302. When the process transitions to the process of step S302, the inspection route determination process is executed again in step S302, so that an inspection route different from the previous one is determined.
- step S305 the information of the inspection route changed in step S303 is stored in the storage unit 130 as output information 135, and is output to the output device 120.
- FIG. 16 is an example of a flowchart of the inspection route change process performed in step S303.
- the route changing unit 151 in the arithmetic unit 140A, has the inspectable area calculation unit 142, the minimum cost route selection unit 146, the change target inspection position selection unit 152, the candidate position setting unit 153, and the candidate position selection. Execution is performed using these functional blocks by calling unit 154 as appropriate.
- step S401 the inspection route R determined in step S302 of FIG. 15, the posture P of the inspection route R, the total movement cost C, and the total inspectable area A are input to the route change unit 151.
- steps S402 to S407 loop processing is executed.
- the processes from steps S403 to S407 are repeatedly executed until the number of loops exceeds a predetermined threshold value.
- step S403 the change target inspection position selection unit 152 compares the change in the posture P of the imaging unit 210 with the front and rear inspection positions among the plurality of inspection positions included in the inspection path R input in step S401.
- the larger inspection position is selected as the inspection position X to be changed.
- FIG. 17 is an explanatory diagram of a method of selecting an inspection position to be changed.
- the state of change in the posture P at each inspection position and the yaw rotation angle of the imaging unit 210 at each inspection position with respect to the inspection path R passing through the inspection positions A to G exemplified in FIG. 6 in order. Shows the range.
- the range of the yaw rotation angle that the image pickup unit 210 can take is determined in consideration of the movable range of the robot unit 220.
- the arrows in the graph of FIG. 17 indicate changes in the posture P set for each inspection position, that is, changes in the yaw rotation angle of the imaging unit 210.
- the change in posture P is larger between the time of moving from the inspection position D to the inspection position E and the time of moving from the inspection position E to the inspection position F than when moving between other inspection positions. You can see that. Therefore, for this inspection path R, the inspection position E is selected as the inspection position X to be changed.
- the inspection position X to be changed may be selected by using another method.
- the user may arbitrarily select any inspection position to select the inspection position X to be changed.
- an inspection position that matches the condition preset by the user may be selected as the inspection position X to be changed.
- step S404 in the vicinity of the change target inspection position X set in step S403 by the candidate position setting unit 153, the candidate position X'instead of the change target inspection position X and the candidate position X thereof.
- N sets (N is an arbitrary natural number) are randomly generated in combination with the candidate posture P in'.
- FIG. 18 is an explanatory diagram of a method of setting a candidate position and a candidate posture.
- FIG. 18 shows examples of inspection positions D, E, and F, candidate positions X', and candidate postures P'among the inspection positions A to G exemplified in FIG.
- the inspection position E is selected as the inspection position X to be changed in step S403 as described above, as shown in FIG. 18, the combination of the candidate position X'and the candidate posture P'is random around the inspection position E. N sets are generated in.
- step S404 only one of the candidate position X'and the candidate posture P'may be generated for the changed target inspection position X, and the original value may be used as it is for the other. That is, the inspection path R can be changed by changing at least one of the position and the posture of the imaging unit 210 at the inspection position X to be changed.
- the inspectable area calculation unit 142 calculates the inspectable area of the appearance inspection device 200 at each candidate position X'generated in step S404.
- the inspectable area of the appearance inspection device 200 at each candidate position X' can be calculated by the same method as in step S200 of FIG. .. That is, it is included in the relative position of the imaging unit 210 at each candidate position X', the candidate posture P'representing the relative posture of the imaging unit 210 at each candidate position X', and the inspection object information 132.
- the position and orientation of the imaging unit 210 and the positions and orientations of the regions F1 to Fn of the inspection object are compared for each candidate position X'. As a result, if each of these differences is within a predetermined range, the area is determined as an inspectable area, and if it is outside the predetermined range, the area is determined not to be an inspectable area. Further, based on the inspection target area information 1321 included in the inspection target area information 132, the inspectable area calculated is extracted, and the area of each extracted inspectable area is based on the shape information 1320. To calculate. The area of each inspectable area calculated here is used when calculating the total inspectable area A'in step S406 described later.
- step S406 the candidate position selection unit 154 calculates the total movement cost C'and the total inspectable area A'of the inspection path R when the change target inspection position X in the inspection path R is changed to each candidate position X'. do.
- the same method as in steps S204 and S207 of FIG. 10 is used by using the area of each inspectable area calculated in step S405. Then, the total movement cost C'and the total inspectable area A'are calculated respectively.
- the total movement cost C'based on the amount of change in posture and the traveling direction of 210 and the total inspectable area A'representing the area of the inspection object that can be inspected by the visual inspection device 200 can be calculated.
- step S407 the candidate position selection unit 154 updates the inspection path R and the posture P based on the total movement cost C'calculated in step S406 and the total inspectable area A'.
- the total inspectable area A' is equal to or greater than the total inspectable area A of the original inspection path R.
- the candidate position X' that minimizes the total movement cost C'is set as the changed inspection position.
- the inspection path R is updated by replacing the changed target inspection position X with this inspection position
- the posture P is updated by setting the posture at this inspection position as the candidate posture P'.
- the position and posture of the image pickup unit 210 after the change with respect to the change target inspection position X can be determined, and the inspection path R and the posture P can be updated so that the inspection accuracy is maintained and the inspection time is the shortest. ..
- step S408 the inspection path R and the posture P updated in step S407 are output as the inspection path R and the posture P after the change by the inspection route change process.
- the changed inspection path R and posture P are reflected in the inspection route generation device 100A.
- the storage unit 130 may retain both the information on the inspection path R and the posture P before the change and the information on the inspection route R and the posture P after the change.
- the inspection route generation device 100A includes a route changing unit 151 that changes the inspection route determined by the route determining unit 141.
- the storage unit 130 stores the inspection object information 132 representing the shape of the inspection object.
- the route changing unit 151 calculates an inspectable area representing an inspectable object area that can be inspected by the visual inspection device 200 when the inspection route is changed based on the inspection object information 132 (steps S405 and S406).
- the changed inspection route is determined based on the calculated inspectable area (step S407).
- the route changing unit 151 selects one of the plurality of inspection positions as the change target inspection position X (step S403), and at least one of the position and the posture of the image pickup unit 210 at the selected change target inspection position X. By changing the inspection route, the inspection route is changed (step S404). Since this is done, the inspection route can be appropriately changed by making the best use of the original inspection route to some extent.
- the route changing unit 151 moves each of the imaging units 210 when moving a plurality of inspection positions of the inspection route R.
- the movement cost C' based on the path length between the inspection positions, the amount of change in the posture of the imaging unit 210, and the traveling direction, and the inspectable area A'are calculated (step S406), and the calculated movement cost C'and the inspectable area are calculated.
- the position and posture of the image pickup unit 210 after the change with respect to the inspection position X to be changed are determined, and the inspection path R after the change is determined based on the position and attitude of the image pickup unit 210 after the change. (Step S407).
- a plurality of candidate positions X' are set in the vicinity of the inspection position X to be changed (step S404), and the inspectable area A'is not reduced in the set plurality of candidate positions X', and the inspectable area A'is not reduced.
- the route change unit 151 may select the change target inspection position X based on the information input from the user. By doing so, the user can arbitrarily select the inspection position X to be changed and change the inspection route.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biochemistry (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Pathology (AREA)
- Immunology (AREA)
- Image Analysis (AREA)
- Manipulator (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
Description
本発明による検査経路生成方法は、撮像部で撮像した画像に基づいて検査対象物の外観検査を行う外観検査装置の検査経路を生成するものであって、演算装置により、前記撮像部が前記検査対象物の撮像を行う複数の検査位置を表す検査位置情報と、前記外観検査装置の構成を表す検査装置構成情報と、を取得する処理と、取得した前記検査位置情報および前記検査装置構成情報に基づいて、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長および前記撮像部の姿勢変化量を算出する処理と、算出した前記経路長および前記姿勢変化量に基づいて前記検査経路を決定する処理と、を実行する。
上記した以外の課題、構成および効果は、以下の実施形態の説明により明らかにされる。
本発明の第1の実施形態について以下に説明する。
図1は、本発明の第1の実施形態に係る検査経路生成装置を含む外観検査システムの構成例を示す図である。図1に示す外観検査システムは、検査経路生成装置100と外観検査装置200が通信回線300を介して互いに接続されることにより構成されている。通信回線300は、例えばイーサネット(登録商標)や無線LAN等を用いて構成されており、検査経路生成装置100と外観検査装置200の間で送受信される様々な情報を伝送する。
次に、本発明の第1の実施形態に係る検査経路生成装置において用いられるデータについて、図2~図8を参照して以下に説明する。
次に、本発明の第1の実施形態に係る検査経路生成装置において実行される検査経路の計画処理について、図9、図10のフローチャートおよび図11の説明図を参照して以下に説明する。
Cc=αL+βD+γY …式(1)
式(1)において、α、β、γは重み係数であり、それぞれ任意の値が設定される。
W1=δCc+ε(Aall-Ac) …式(2)
式(2)において、δ、εは重み係数であり、それぞれ任意の値が設定される。また、Aallは検査対象物における検査対象領域の総面積である。
W2=δCc+ε(Aall-Ac)+γ(MAX(Y)-MIN(Y)) …式(3)
式(3)において、MAX(Y)は総姿勢変化量Yの最大値、MIN(Y)は総姿勢変化量Yの最小値である。
次に、本発明の第1の実施形態に係る検査経路生成装置における処理結果の出力例について、図12および図13を参照して以下に説明する。
本発明の第2の実施形態について以下に説明する。
図14は、本発明の第2の実施形態に係る検査経路生成装置を含む外観検査システムの構成例を示す図である。図14に示す外観検査システムは、検査経路生成装置100Aと外観検査装置200が通信回線300を介して互いに接続されることにより構成されている。第1の実施形態で説明した図1の検査経路生成装置100と、図14に示す検査経路生成装置100Aとの違いは、演算装置140Aにおいて、既に説明した経路決定部141、検査可能領域算出部142、カメラ可動範囲算出部143、経路グラフ生成部144、移動コスト算出部145、最小コスト経路選択部146、出力情報生成部147の各機能ブロックに加えて、経路変更部151、変更対象検査位置選択部152、候補位置設定部153および候補位置選択部154の各機能ブロックをさらに有する点である。以下では、これらの相違点を中心に、本実施形態について説明する。
次に、本発明の第2の実施形態に係る検査経路生成装置において実行される検査経路の計画処理について、図15、図16のフローチャートおよび図17、図18の説明図を参照して以下に説明する。
110・・・入力装置
120・・・出力装置
130・・・記憶部
131・・・検査装置構成情報
132・・・検査対象物情報
133・・・検査条件情報
134・・・経路グラフ情報
135・・・出力情報
140,140A・・・演算装置
141・・・経路決定部
142・・・検査可能領域算出部
143・・・カメラ可動範囲算出部
144・・・経路グラフ生成部
145・・・移動コスト算出部
146・・・最小コスト経路選択部
147・・・出力情報生成部
151・・・経路変更部
152・・・変更対象検査位置選択部
153・・・候補位置設定部
154・・・候補位置選択部
200・・・外観検査装置
210・・・撮像部
220・・・ロボット部
230・・・演算処理部
300・・・通信回線
1310・・・ロボット構成情報
1311・・・検査設備構成情報
1320・・・形状情報
1321・・・検査対象領域情報
1330・・・検査位置情報
1331・・・検査可能領域情報
1332・・・カメラ可動範囲情報
1340・・・ノード間接続情報
1341・・・移動コスト情報
1350・・・検査経路情報
1351・・・カメラ角度情報
1352・・・ロボット姿勢情報
Claims (18)
- 撮像部で撮像した画像に基づいて検査対象物の外観検査を行う外観検査装置の検査経路を生成する検査経路生成装置であって、
前記撮像部が前記検査対象物の撮像を行う複数の検査位置を表す検査位置情報と、前記外観検査装置の構成を表す検査装置構成情報と、を記憶する記憶部と、
前記検査位置情報および前記検査装置構成情報に基づいて、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長および前記撮像部の姿勢変化量を算出し、算出した前記経路長および前記姿勢変化量に基づいて前記検査経路を決定する経路決定部と、を備える検査経路生成装置。 - 請求項1に記載の検査経路生成装置において、
前記経路決定部により決定された前記検査経路を変更する経路変更部を備え、
前記記憶部は、前記検査対象物の形状を表す検査対象物情報を記憶し、
前記経路変更部は、前記検査対象物情報に基づいて、前記検査経路を変更した場合に前記外観検査装置が検査可能な前記検査対象物の領域を表す検査可能領域を算出し、算出した前記検査可能領域に基づいて変更後の前記検査経路を決定する検査経路生成装置。 - 請求項2に記載の検査経路生成装置において、
前記経路変更部は、
前記複数の検査位置のうちいずれかを変更対象検査位置として選択し、
選択した前記変更対象検査位置において前記撮像部の位置および姿勢の少なくとも一方を変化させることで、前記検査経路を変更する検査経路生成装置。 - 請求項3に記載の検査経路生成装置において、
前記経路変更部は、
前記変更対象検査位置において前記撮像部の位置および姿勢の少なくとも一方を変化させた場合に、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長、前記撮像部の姿勢変化量および進行方向に基づく移動コストと、前記検査可能領域とを算出し、
算出した前記移動コストおよび前記検査可能領域に基づいて、前記変更対象検査位置に対する変更後の前記撮像部の位置および姿勢を決定し、
決定した変更後の前記撮像部の位置および姿勢に基づいて変更後の前記検査経路を決定する検査経路生成装置。 - 請求項4に記載の検査経路生成装置において、
前記経路変更部は、前記変更対象検査位置の近傍に複数の候補位置を設定し、設定した前記複数の候補位置の中で、前記検査可能領域が縮小せず、かつ前記移動コストが最小となる候補位置を、前記変更対象検査位置に対する変更後の前記撮像部の位置として決定する検査経路生成装置。 - 請求項3に記載の検査経路生成装置において、
前記経路変更部は、ユーザから入力された情報に基づいて前記変更対象検査位置を選択する検査経路生成装置。 - 請求項1に記載の検査経路生成装置において、
前記経路決定部は、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長、前記撮像部の姿勢変化量および進行方向に基づいて移動コストを算出し、算出した前記移動コストに基づいて前記検査経路を決定する検査経路生成装置。 - 請求項1に記載の検査経路生成装置において、
前記経路決定部は、前記検査装置構成情報に基づいて各検査位置における前記撮像部の姿勢の制約条件を算出し、算出した前記制約条件に基づいて前記姿勢変化量を算出する検査経路生成装置。 - 請求項1に記載の検査経路生成装置において、
前記経路決定部は、前記検査経路の候補を複数生成し、生成した複数の前記検査経路の候補の中から、任意の検査位置間での前記撮像部の姿勢変化量が所定の閾値以上となる前記検査経路の候補を除外して、前記検査経路を決定する検査経路生成装置。 - 撮像部で撮像した画像に基づいて検査対象物の外観検査を行う外観検査装置の検査経路を生成する検査経路生成方法であって、
演算装置により、
前記撮像部が前記検査対象物の撮像を行う複数の検査位置を表す検査位置情報と、前記外観検査装置の構成を表す検査装置構成情報と、を取得する処理と、
取得した前記検査位置情報および前記検査装置構成情報に基づいて、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長および前記撮像部の姿勢変化量を算出する処理と、
算出した前記経路長および前記姿勢変化量に基づいて前記検査経路を決定する処理と、を実行する検査経路生成方法。 - 請求項10に記載の検査経路生成方法において、
前記演算装置により、
前記検査対象物の形状を表す検査対象物情報を取得する処理と、
取得した前記検査対象物情報に基づいて、前記検査経路を変更した場合に前記外観検査装置が検査可能な前記検査対象物の領域を表す検査可能領域を算出する処理と、
算出した前記検査可能領域に基づいて変更後の前記検査経路を決定することにより、前記検査経路を変更する処理と、を実行する検査経路生成方法。 - 請求項11に記載の検査経路生成方法において、
前記検査経路を変更する処理では、前記複数の検査位置のうちいずれかを変更対象検査位置として選択し、選択した前記変更対象検査位置において前記撮像部の位置および姿勢の少なくとも一方を変化させることで、前記検査経路を変更する検査経路生成方法。 - 請求項12に記載の検査経路生成方法において、
前記演算装置により、前記変更対象検査位置において前記撮像部の位置および姿勢の少なくとも一方を変化させた場合に、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長、前記撮像部の姿勢変化量および進行方向に基づく移動コストを算出する処理と、前記検査可能領域を算出する処理とを実行し、
前記検査経路を変更する処理では、算出した前記移動コストおよび前記検査可能領域に基づいて、前記変更対象検査位置に対する変更後の前記撮像部の位置および姿勢を決定し、決定した変更後の前記撮像部の位置および姿勢に基づいて変更後の前記検査経路を決定する検査経路生成方法。 - 請求項13に記載の検査経路生成方法において、
前記移動コストを算出する処理および前記検査可能領域を算出する処理では、前記変更対象検査位置の近傍に複数の候補位置を設定して、前記移動コストおよび前記検査可能領域をそれぞれ算出し、
前記検査経路を変更する処理では、設定した前記複数の候補位置の中で、前記検査可能領域が縮小せず、かつ前記移動コストが最小となる候補位置を、前記変更対象検査位置に対する変更後の前記撮像部の位置として決定する検査経路生成方法。 - 請求項12に記載の検査経路生成方法において、
前記検査可能領域を算出する処理では、ユーザから入力された情報を取得し、取得した前記情報に基づいて前記変更対象検査位置を選択する検査経路生成方法。 - 請求項10に記載の検査経路生成方法において、
前記検査経路を決定する処理では、前記撮像部が前記複数の検査位置を移動する際の各検査位置間での経路長、前記撮像部の姿勢変化量および進行方向に基づいて移動コストを算出し、算出した前記移動コストに基づいて前記検査経路を決定する検査経路生成方法。 - 請求項10に記載の検査経路生成方法において、
前記撮像部の姿勢変化量を算出する処理では、取得した前記検査装置構成情報に基づいて各検査位置における前記撮像部の姿勢の制約条件を算出し、算出した前記制約条件に基づいて前記姿勢変化量を算出する検査経路生成方法。 - 請求項10に記載の検査経路生成方法において、
前記検査経路を決定する処理では、前記検査経路の候補を複数生成し、生成した複数の前記検査経路の候補の中から、任意の検査位置間での前記撮像部の姿勢変化量が所定の閾値以上となる前記検査経路の候補を除外して、前記検査経路を決定する検査経路生成方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/265,276 US20240042607A1 (en) | 2020-12-11 | 2021-12-01 | Inspection route generation device and inspection route generation method |
CN202180080879.3A CN116569026A (zh) | 2020-12-11 | 2021-12-01 | 检查路径生成装置、检查路径生成方法 |
EP21903256.2A EP4261530A1 (en) | 2020-12-11 | 2021-12-01 | Inspection path generation device and inspection path generation method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020-206099 | 2020-12-11 | ||
JP2020206099A JP2022093035A (ja) | 2020-12-11 | 2020-12-11 | 検査経路生成装置、検査経路生成方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022124152A1 true WO2022124152A1 (ja) | 2022-06-16 |
Family
ID=81973911
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/044041 WO2022124152A1 (ja) | 2020-12-11 | 2021-12-01 | 検査経路生成装置、検査経路生成方法 |
Country Status (5)
Country | Link |
---|---|
US (1) | US20240042607A1 (ja) |
EP (1) | EP4261530A1 (ja) |
JP (1) | JP2022093035A (ja) |
CN (1) | CN116569026A (ja) |
WO (1) | WO2022124152A1 (ja) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5633717A (en) * | 1996-06-26 | 1997-05-27 | Forensic Technology Wai Inc. | Method for monitoring and adjusting the position of an object under optical observation for imaging |
JP2016224707A (ja) * | 2015-05-29 | 2016-12-28 | リコーエレメックス株式会社 | 検査システム |
JP2017203744A (ja) * | 2016-05-13 | 2017-11-16 | 三菱重工業株式会社 | 航空機パネル外観検査方法 |
JP2019007891A (ja) * | 2017-06-27 | 2019-01-17 | ファナック株式会社 | 機械学習装置、ロボット制御システム及び機械学習方法 |
JP2019078651A (ja) * | 2017-10-25 | 2019-05-23 | ファナック株式会社 | 検査システム |
JP2019158499A (ja) * | 2018-03-12 | 2019-09-19 | オムロン株式会社 | 外観検査システム、設定装置、画像処理装置、設定方法およびプログラム |
JP2020099215A (ja) * | 2018-12-20 | 2020-07-02 | 三菱重工機械システム株式会社 | 植物管理システム |
-
2020
- 2020-12-11 JP JP2020206099A patent/JP2022093035A/ja active Pending
-
2021
- 2021-12-01 WO PCT/JP2021/044041 patent/WO2022124152A1/ja active Application Filing
- 2021-12-01 CN CN202180080879.3A patent/CN116569026A/zh active Pending
- 2021-12-01 EP EP21903256.2A patent/EP4261530A1/en active Pending
- 2021-12-01 US US18/265,276 patent/US20240042607A1/en active Pending
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5633717A (en) * | 1996-06-26 | 1997-05-27 | Forensic Technology Wai Inc. | Method for monitoring and adjusting the position of an object under optical observation for imaging |
JP2016224707A (ja) * | 2015-05-29 | 2016-12-28 | リコーエレメックス株式会社 | 検査システム |
JP2017203744A (ja) * | 2016-05-13 | 2017-11-16 | 三菱重工業株式会社 | 航空機パネル外観検査方法 |
JP2019007891A (ja) * | 2017-06-27 | 2019-01-17 | ファナック株式会社 | 機械学習装置、ロボット制御システム及び機械学習方法 |
JP2019078651A (ja) * | 2017-10-25 | 2019-05-23 | ファナック株式会社 | 検査システム |
JP2019158499A (ja) * | 2018-03-12 | 2019-09-19 | オムロン株式会社 | 外観検査システム、設定装置、画像処理装置、設定方法およびプログラム |
JP2020099215A (ja) * | 2018-12-20 | 2020-07-02 | 三菱重工機械システム株式会社 | 植物管理システム |
Also Published As
Publication number | Publication date |
---|---|
EP4261530A1 (en) | 2023-10-18 |
CN116569026A (zh) | 2023-08-08 |
US20240042607A1 (en) | 2024-02-08 |
JP2022093035A (ja) | 2022-06-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11403439B2 (en) | Method and apparatus for robust reduction of shape error in laser powder deposition based additive manufacturing process due to uncertainty | |
EP3166081A2 (en) | Method and system for positioning a virtual object in a virtual simulation environment | |
US20200258297A1 (en) | Method in constructing a model of a scenery and device therefor | |
US20170091999A1 (en) | Method and system for determining a configuration of a virtual robot in a virtual environment | |
EP2993001A1 (en) | Method and apparatus for industrial robotic energy saving optimization using fly-by | |
US11468211B2 (en) | Method of operating a CAD system model for modelling an article to be manufactured | |
WO2018143019A1 (ja) | 情報処理装置、情報処理方法およびプログラム記録媒体 | |
JP2021196705A (ja) | 画像処理装置、画像処理方法およびプログラム | |
US20210056245A1 (en) | Method and system for performing a simulation of a retraction cable motion | |
WO2022124152A1 (ja) | 検査経路生成装置、検査経路生成方法 | |
CN111951348A (zh) | 确定框选区域的方法、装置及电子设备 | |
JP4809645B2 (ja) | メッシュ分割方法、有限要素解析装置、及びコンピュータプログラム | |
US20220230702A1 (en) | Optical Flow Based Forecasting of Binding States in Molecular Dynamic Simulations | |
WO2022212313A1 (en) | Ml-based discrepancy surrogate for digital twin simulation | |
JP7164008B2 (ja) | データ生成方法、データ生成装置及びプログラム | |
JP5509952B2 (ja) | シミュレーション方法、シミュレーション装置、プログラム、及び記憶媒体 | |
US8704828B1 (en) | Inverse kinematic melting for posing models | |
WO2020189269A1 (ja) | 画像処理方法、画像処理装置、プログラム | |
JP6123398B2 (ja) | 欠陥箇所予測装置、識別モデル生成装置、欠陥箇所予測プログラムおよび欠陥箇所予測方法 | |
WO2022244444A1 (ja) | 情報処理システム、情報処理方法、プログラム | |
WO2023248968A1 (ja) | 画像加工方法、画像加工装置、及び画像加工プログラム | |
US20140303946A1 (en) | Analysis device and simulation method | |
JP3872469B2 (ja) | データ処理方法、データ処理装置 | |
JP2014087875A (ja) | 経路探索方法、経路評価方法、経路探索装置、制御装置、ロボット、及び、プログラム | |
JP2024033451A (ja) | 情報処理装置、情報処理装置の制御方法、及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21903256 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180080879.3 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18265276 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2021903256 Country of ref document: EP Effective date: 20230711 |