WO2023037443A1 - ロボット制御装置、学習装置および推論装置 - Google Patents
ロボット制御装置、学習装置および推論装置 Download PDFInfo
- Publication number
- WO2023037443A1 WO2023037443A1 PCT/JP2021/032995 JP2021032995W WO2023037443A1 WO 2023037443 A1 WO2023037443 A1 WO 2023037443A1 JP 2021032995 W JP2021032995 W JP 2021032995W WO 2023037443 A1 WO2023037443 A1 WO 2023037443A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- robot
- person
- information
- processing unit
- state
- Prior art date
Links
- 238000012545 processing Methods 0.000 claims abstract description 138
- 238000012544 monitoring process Methods 0.000 claims abstract description 82
- 230000002093 peripheral effect Effects 0.000 claims abstract description 48
- 238000005259 measurement Methods 0.000 claims abstract description 14
- 230000033001 locomotion Effects 0.000 claims description 120
- 230000009471 action Effects 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 18
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000000034 method Methods 0.000 description 31
- 238000010586 diagram Methods 0.000 description 18
- 241000282412 Homo Species 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000036544 posture Effects 0.000 description 7
- 230000002787 reinforcement Effects 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006378 damage Effects 0.000 description 5
- 230000007423 decrease Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000004904 shortening Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000011960 computer-aided design Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000000245 forearm Anatomy 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 210000000689 upper leg Anatomy 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1674—Programme controls characterised by safety, monitoring, diagnostic
- B25J9/1676—Avoiding collision or forbidden zones
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/06—Safety devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40198—Contact with human allowed if under pain tolerance limit
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40201—Detect contact, collision with human
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40202—Human robot coexistence
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40203—Detect position of operator, create non material barrier to protect operator
Definitions
- the present disclosure relates to a robot control device, a learning device, and an inference device that control a robot that operates while sharing a workspace with people.
- Patent Document 1 the distance between a robot and a worker present in the vicinity of a robot recognized from image information captured by a camera is measured, and when the measured distance is a distance that interferes with the worker, the robot performs a task.
- a method for controlling a robot is disclosed in which it is estimated which parts of a person will be interfered with and in what manner, and based on the result, the motion of the robot is restricted.
- the conventional technology has the problem that the possibility of the worker being caught between the robot and the surrounding object is not taken into consideration. For example, in a state where a part of the worker's body exists between the robot and surrounding objects, as a result of estimating the mode of interference between the worker and the robot from the distance between the worker and the robot, the robot's movement is slowed down. , there is a possibility that part of the worker's body will be caught between the robot and the surrounding object because the existence of the surrounding object is not considered.
- the present disclosure has been made in view of the above, and an object thereof is to obtain a robot control device capable of suppressing the occurrence of a person being caught between a robot and a peripheral object.
- the present disclosure provides a robot control device that controls a robot that operates while sharing a work area with a person, comprising: an image recognition processing unit; a robot control processing unit; , and a monitoring processing unit.
- the image recognition processing unit recognizes first information, which is information about a person present in the monitored area, based on measurement data in the monitored area obtained from the vision sensor.
- the robot control processing unit controls the motion of the robot according to a motion program for operating the robot.
- the monitoring processing unit determines the possibility of a human being caught between the robot and the surrounding object based on the surrounding object data and the first information obtained from the image recognition processing unit.
- the peripheral object data is data indicating the three-dimensional arrangement state of peripheral objects, which are robots and objects other than robots, in the monitoring area.
- the robot control device has the effect of being able to suppress the occurrence of a person being caught between the robot and surrounding objects.
- FIG. 1 is a block diagram showing an example of a configuration of a robot system including a robot control device according to Embodiment 1;
- FIG. A diagram showing an example of human influence level information Flowchart showing an example of the procedure of the robot control method according to Embodiment 1 Flowchart showing an example of the procedure of the robot control method according to Embodiment 1 Diagram showing the relationship between humans, robots, and surrounding objects
- FIG. 2 is a block diagram showing an example of the configuration of a robot system including a robot control device according to Embodiment 2; Diagram for explaining how to generate an access frequency map A diagram showing an example of an access frequency map FIG.
- a robot control device, a learning device, and an inference device according to embodiments of the present disclosure will be described below in detail based on the drawings.
- FIG. 1 is a block diagram showing an example of a configuration of a robot system including a robot control device according to Embodiment 1.
- the robot system 1 includes a robot 10 , a vision sensor 20 and a robot controller 30 .
- the robot 10 has a plurality of arms and a driving unit provided at each joint that is a connection point of each arm and controlling the joint angle.
- the robot 10 can take various postures in accordance with motion commands from the robot control device 30 . By taking various postures, the robot 10 can operate within a predetermined range of positions centered on the fixed position of the robot 10 .
- An example of the drive unit is an electric motor typified by a servomotor or a stepping motor, or a cylinder using pneumatic pressure or hydraulic pressure.
- the vision sensor 20 captures an image of a predetermined area including the robot 10, that is, the monitoring area of the robot system 1.
- the vision sensor 20 is a sensor capable of acquiring measurement data including a range image including the depth of objects in the monitoring area and a color image for distinguishing between humans and peripheral objects other than humans.
- An example of vision sensor 20 is a two-dimensional or three-dimensional camera. Color information can be obtained with a two-dimensional camera, and position information can be obtained with a three-dimensional camera.
- a stereo system, a time of flight (ToF) system, or a projector system can be used as an imaging system of the three-dimensional camera.
- peripheral objects are tables, walls, shelves, doors, processing machines.
- the robot control device 30 controls the motion of the robot 10 according to a predetermined motion program, which is a program for operating the robot 10 . During the control process of the robot 10, the robot control device 30 controls the robot 10 so that the robot 10 does not come into contact with the person when there is a person around the robot 10 based on the imaging result of the vision sensor 20. control behavior.
- the robot control device 30 includes an operation program storage unit 31, a robot control processing unit 32, an image recognition processing unit 33, a human influence level information storage unit 34, a peripheral object data storage unit 35, a monitoring processing unit 36, Prepare.
- the motion program storage unit 31 stores motion programs that describe motions of the robot 10 .
- the robot control processing unit 32 loads the operation program from the operation program storage unit 31, executes it, and controls the robot 10 according to the execution result. Further, when the monitoring processing unit 36 issues a command to limit the movement of the robot 10 while controlling the robot 10 according to the operation program, the robot control processing unit 32 controls the robot 10 according to the command.
- the image recognition processing unit 33 recognizes the first information, which is information about a person present in the monitored area, based on the measurement data in the monitored area obtained from the vision sensor 20 .
- An example of measurement data is a range image or a color image.
- the image recognition processing unit 33 uses a color image in the measurement data to determine whether a person exists in the monitoring area, and recognizes a human body part when a person exists in the monitoring area.
- Human body parts include the head, body, upper arm, forearm, hand, thigh, lower leg, foot, and the like.
- the image recognition processing unit 33 can recognize the human body part by comparing the data of the human body part stored in advance with the measurement data.
- the image recognition processing unit 33 uses the distance image in the measurement data to determine the position and orientation of the recognized person, more specifically, the position and orientation of the human body part.
- the first position/orientation information which is the information included
- the first state information which is information including the state of the person
- the first position/orientation information indicates the position and orientation of a person and human body parts in a camera coordinate system, which is a coordinate system used for measurement data measured by the vision sensor 20 .
- the first state information is information indicating how the human body part is operating, and for example, information including the moving direction and speed of the human body part. When stopped, the moving direction and speed are "0".
- the first position and orientation information and the first state information are included in the first information.
- the image recognition processing unit 33 may further recognize the second information, which is information about the robot 10, for the robot 10 as well.
- the second information includes second position/orientation information, which is information including the position and orientation, and second state information, which is information including the state, for the parts of the robot 10 .
- the human body influence information storage unit 34 stores human body influence information, which is information indicating the degree of influence on the human body when the robot 10 comes into contact with a human.
- the human body influence degree information defines the degree of influence exerted on the human body when the robot 10 comes into contact with each part of the human body.
- FIG. 2 is a diagram showing an example of human influence level information.
- the human body influence level information is information in which the state of contact between a person and the robot 10 is input data and the influence level on the human body is output data.
- the input data has items of human contact site, human state, robot 10 contact site, and robot 10 state. Note that the items of input data in FIG. 2 are only examples, and the items are not limited to these.
- the output data has an item of degree of influence on the human body.
- Each item constituting the input data is information obtained as a result of recognition processing by the image recognition processing unit 33 and as a result of specific processing by the monitoring processing unit 36 .
- the degree of impact on the human body are the degree of impact on the human body and the degree of injury or damage to the human body. For example, when the human body is damaged, the degree of influence on the human body is "high”, and when the human body is not damaged, the degree of influence on the human body is "low”.
- the human contact part is "head"
- the human state is "moving”
- the contact part of the robot 10 is "all parts”
- the robot 10 state is "moving speed”.
- the degree of influence on the human body is “high”.
- the information on the degree of influence on the human body is, for example, ISO/TS (International Organization for Standardization/Technical Specifications) 15066:2016 table A. 2 are the "Biomechanical limits”.
- the peripheral object data storage unit 35 stores peripheral object data, which is information including the three-dimensional arrangement state of the robot 10 existing in the monitoring area and peripheral objects other than the robot 10 .
- the peripheral object data includes three-dimensional CAD (Computer-Aided Design) data indicating the shape and size of the object, and position and orientation data indicating the position and orientation of the object.
- the position and orientation data is, for example, data indicating the position and orientation of the robot 10 with respect to the installation position.
- Peripheral object data is data capable of three-dimensionally reproducing the arrangement state of objects including the robot 10 in the monitoring area.
- the peripheral object data is expressed using the robot coordinate system, which is the coordinate system used in the peripheral object data.
- the monitoring processing unit 36 uses the recognition result of the image recognition processing unit 33 to determine the degree of impact at the time of contact. is likely to be given, a command to restrict the movement of the robot 10 is output to the robot control processing unit 32. Each specific process will be described below.
- the monitoring processing unit 36 measures the measurement distance, which is the distance between the person and the robot 10, based on the recognition result of the image recognition processing unit 33.
- the monitoring processing unit 36 determines whether the measured distance is a non-contact distance, which is a distance at which the robot 10 does not come into contact with a person due to the motion of the robot 10 .
- the non-contact distance is obtained from operation stop data, which is data indicating in which direction and how much distance is required to stop the robot 10 from the time the image is captured by the vision sensor 20 .
- the monitoring processing unit 36 measures the measured distance between the human and the robot 10 including the coasting distance until the robot 10 stops.
- the monitoring processing unit 36 does not limit the robot 10 and allows the current operation to continue.
- the monitoring processing unit 36 transmits to the robot control processing unit 32 a command to slow down the operation speed of the robot 10 .
- the monitoring processing unit 36 determines whether the robot 10 is making contact with the human. . In one example, the monitoring processing unit 36 determines whether the robot 10 will come into contact with a person based on the first information of the person, which is the recognition result of the image recognition processing unit 33, and the second information of the robot 10. As described above, the first information includes first position/orientation information and first state information of human body parts, and the second information includes second position/orientation information and second state information of parts of robot 10 .
- the monitoring processing unit 36 determines the contact portion and contact state between the robot 10 and the human body part based on the predicted motion of the human body part and the predicted motion of the robot 10 . to predict.
- the monitoring processing unit 36 identifies contact position/orientation information including contacting human body parts and robot 10 parts and postures, and contacting state information indicating the contacting human body parts and robot 10 states.
- the contact position/orientation information includes, for example, the human body part and contact angle that come into contact with the robot 10, and the part and contact angle of the robot 10 that come into contact with the human.
- the state information at the time of contact includes the moving direction and speed of the parts of the human body and the robot 10 at the time of contact. In the state information at the time of contact, when both the moving direction and the speed are "0", it means that the robot is stopped.
- the second position/orientation information and the second state information of the parts of the robot 10 are used by the robot control processing unit 32 instead of the recognition result by the image recognition processing unit 33 .
- a simulation result of operating the robot 10 according to a program may be used.
- the monitoring processing unit 36 allows the robot 10 to continue its movement while being restricted.
- the monitoring processing unit 36 refers to the human body impact level information in the human body impact level information storage unit 34 based on the identified parts and states of the robot 10 and the person coming into contact with the human body. Extract the degree of impact. That is, the monitoring processing unit 36 extracts the human body influence degree information corresponding to the combination of the position/orientation information at the time of contact and the state information at the time of contact. Then, the monitoring processing unit 36 outputs an operation command of the robot 10 to the robot control processing unit 32 according to the obtained degree of influence on the human body. In one example, the monitoring processing unit 36 transmits to the robot control processing unit 32 a command to further restrict the movement of the robot 10 when the degree of influence on the human body is high. Examples of restrictions on the movement of the robot 10 include stopping the robot 10 and moving the robot 10 away from the person.
- the monitoring processing unit 36 when the contact between the robot 10 and a person does not have a high degree of influence on the human body, the monitoring processing unit 36 generates surrounding object data indicating the three-dimensional arrangement state of the robot 10 and surrounding objects in the monitoring area. and the first information obtained from the image recognition processing unit 33, the possibility of a person being caught between the robot 10 and a peripheral object is determined, and the movement of the robot 10 is further restricted or the warning sound is suppressed. Determine whether output is required.
- the monitoring processing unit 36 adds a first This is done by adding the position information of the person based on the information and predicting the motion of the person and the robot 10 based on the first information.
- the monitoring processing unit 36 may determine the possibility of a person being caught between the robot 10 and a peripheral object, using not only the first information but also the second information. In this case, the monitoring processing unit 36 predicts the motion of the robot 10 from the second information including the second position/orientation information and the second state information.
- the monitoring processing unit 36 adds the position information of the person in the camera coordinate system recognized by the image recognition processing unit 33, that is, the position information of each part of the human body to the peripheral object data in the robot coordinate system, By simulating the motion of the human and the motion of the robot 10, it is determined whether or not the human is caught between the robot 10 and a surrounding object.
- the camera coordinate system and the robot coordinate system are calibrated in advance. Thereby, a coordinate conversion matrix between the camera coordinate system and the robot coordinate system is calculated.
- this coordinate transformation matrix the position of the person in the camera coordinate system recognized by the vision sensor 20 can be transformed into the robot coordinate system.
- the position and orientation of surrounding objects are also represented using coordinates with the robot 10 as a reference.
- the monitoring processing unit 36 can grasp the positional relationship between the robot 10, the surrounding objects, and the person in the robot coordinate system, and determines from this positional relationship whether or not the person is in a position where the person is caught.
- a simulation of human motion can be performed based on the first information.
- the motion of the robot 10 can be simulated using the motion program or based on the second information.
- the monitoring processing unit 36 continues the movement of the robot 10 while the movement is restricted when there is no possibility of a person getting caught. In other words, the monitoring processing unit 36 does not further limit the operation. Further, when there is a possibility that a person is caught in the machine, the monitoring processing unit 36 issues a command to the robot control processing unit to restrict the movement of the robot 10, such as stopping the robot 10 or moving the robot 10 away from the person. 32 or output a warning sound.
- 3 and 4 are flowcharts showing an example of the procedure of the robot control method according to Embodiment 1.
- FIG. It should be noted that processing will be described here on the assumption that a person exists within the monitored area.
- the vision sensor 20 captures an image of the monitored area and transmits the captured data to the robot control device 30 as measurement data.
- the image recognition processing unit 33 of the robot control device 30 recognizes the first position/orientation information and the first state information of the human body part present in the monitoring area from the received measurement data (step S11).
- the first position/orientation information is information including the position and orientation of the human body part
- the first state information is information including the state of the human body part.
- the monitoring processing unit 36 uses the first position/orientation information and the first state information of the human body part recognized by the image recognition processing unit 33 to measure the measured distance between the recognized human and the robot 10 (step) S12).
- the monitoring processing unit 36 determines whether the measured distance is a non-contact distance in which the human and the robot 10 do not contact each other (step S13).
- the non-contact distance is the distance that the robot 10 moves from the state captured by the vision sensor 20 until the robot 10 receives a stop signal and stops. If the measured distance is greater than the non-contact distance, the robot 10 will not come into contact with the human, and if the measured distance is smaller than the non-contact distance, the robot 10 may come into contact with the human.
- step S13 If the measured distance is greater than the non-contact distance (Yes in step S13), there is no possibility that the robot 10 will come into contact with a person. , the operation program continues, and the process ends.
- the monitoring processing unit 36 When the measured distance is smaller than the non-contact distance (No in step S13), the monitoring processing unit 36 outputs a command to change the operating speed of the robot 10 to the robot control processing unit 32 (step S14). ). Upon receiving the command, the robot control processing unit 32 changes the operating speed of the robot 10 according to the command (step S15). Specifically, the monitoring processing unit 36 outputs a deceleration command to the robot control processing unit 32 in order to decelerate the operating speed of the robot 10 .
- the operating speed of the robot 10 included in the deceleration command is such an operating speed that, even if the robot 10 comes into contact with a human body, it has almost no effect on the human body at the contact site, and is, for example, 0.25 m/s. .
- the monitoring processing unit 36 uses the recognition result of the image recognition processing unit 33 to determine whether there is a possibility that the motion of the robot 10 will come into contact with a person (step S16).
- the image recognition processing unit 33 recognizes the second position/orientation information and the second state information of the parts of the robot 10, and recognizes the first position/orientation information and the first state information of the human body parts, The second position/orientation information and the second state information are used to predict the motions of the human and the robot 10, and determine the possibility of contact between the robot 10 and the human.
- the possibility of contact between the robot 10 and a person is calculated by combining a simulation of operating the robot 10 according to an operation program and a prediction using the first position/orientation information and first state information of the human body part. judge.
- the monitoring processing unit 36 When it is determined that there is no possibility that the motion of the robot 10 will come into contact with a person (No in step S16), the monitoring processing unit 36 maintains the motion speed of the robot 10 changed in step S15. Then, the operation of the robot 10 is continued, and the process ends. Further, when it is determined that the motion of the robot 10 may come into contact with a person (Yes in step S16), the monitoring processing unit 36 uses the recognition result of the image recognition processing unit 33 to 10 and contact position/orientation information of the human body part is specified (step S17). Also, the monitoring processing unit 36 acquires contact state information of the robot 10 and the human body using the recognition result of the image recognition processing unit 33 (step S18). Examples of contact state information of the human body include head direction and movement speed, shoulder direction and movement speed, and the like. Similarly, an example of the contact state information of the robot 10 is the orientation and movement speed of the arm tip.
- the monitoring processing unit 36 obtains the degree of influence on the human body by referring to the degree of influence information on the human body based on the contact position/orientation information and the contact state information of the robot 10 and the human body (step S19). Specifically, the monitoring processing unit 36 receives, as input data, a combination of contact position/orientation information and contact state information of the robot 10 and the human body, and determines the degree of influence on the human body corresponding to the input data from the human body influence degree information. Get some output data. The monitoring processing unit 36 determines whether the degree of influence on the human body is higher than a predetermined reference value (step S20).
- the degree of impact on the human body is associated with information such as whether the human body part that the robot 10 contacts is a vital point, the movement speed of the robot 10, the human body part that may come into contact, the position, the state, and the like.
- the monitoring processing unit 36 adds the position information of the person recognized by the image recognition processing unit 33 to the surrounding object data (Ste S21), the motions of the robot 10 and humans are predicted in the environment including surrounding objects (step S22).
- the monitoring processing unit 36 uses peripheral object data to simulate the motion of the robot 10 and a person. At this time, contact position/orientation information and contact state information of the robot 10 and the human body, an operation program, and the like can be used.
- FIG. 5 is a diagram showing the relationship between humans, robots, and surrounding objects. This figure shows a case where an area between the robot 10 and a workbench 510, which is a peripheral object, is an entrapment possibility area R1, and a hand 501 of a person 500 exists in the entrapment possibility area R1. In such a case, it is determined that there is a possibility that the hand 501 of the person 500 will be caught between the robot 10 and the workbench 510 .
- the monitoring processing unit 36 outputs to the robot control processing unit 32 a command to restrict the movement of the robot 10 so that the person and the robot 10 do not come into contact with each other (step S24).
- the robot control processing unit 32 limits the motion of the robot 10 based on the command (step S25). Restrictions on the motion of the robot 10 include stopping the motion of the robot 10, further deceleration, motion of the robot 10 in a direction away from the person, and the like.
- the monitoring processing unit 36 outputs a command to limit the movement of the robot 10 to the robot control processing unit 32, and the robot control processing unit 32 limits the movement of the robot 10.
- the monitoring processing unit 36 may output a warning sound. With this, the processing ends.
- step S23 If there is no possibility of a person being caught between the robot 10 and a surrounding object (No in step S23), it is recognized that the effect on the human body is not so serious, and the operation of the robot 10 is changed to the current state. Leave as is and the process ends.
- the image recognition processing unit 33 uses the measurement data from the vision sensor 20 to obtain the first position/orientation information and the first state information of the human body part existing in the monitoring area. to get The monitoring processing unit 36 adds the position of the person to the surrounding object data including the three-dimensional arrangement state, shape and size of the robot 10 and surrounding objects, predicts the movements of the person and the robot 10, Determine whether there is a possibility that a person is caught between surrounding objects.
- the robot controls a command to restrict the movement of the robot 10 so that the person is not caught between the robot 10 and the surrounding object.
- Output to the processing unit 32 is a command to restrict the movement of the robot 10 so that the person is not caught between the robot 10 and the surrounding object.
- Embodiment 2 Conventionally, when there is a person in the area where the robot 10 and the surrounding object are arranged, the possibility of the robot 10 coming into contact with the person and the possibility of the person being caught between the robot 10 and the surrounding object are considered. No technique has been proposed for shortening the movement path of the robot 10 to the target position as much as possible while reducing it. In the second embodiment, the movement path of robot 10 to the target position is shortened as much as possible while reducing the possibility that robot 10 will come into contact with a person and the possibility that a person will be caught between robot 10 and surrounding objects.
- a robot control device capable of
- FIG. 6 is a block diagram showing an example of the configuration of a robot system including a robot control device according to the second embodiment.
- symbol is attached
- the robot control device 30 further includes a human analysis processing unit 37, an access frequency map storage unit 38, a learning device 39, a learned model storage unit 40, and an inference device 41 in addition to the configuration of the first embodiment. .
- the human analysis processing unit 37 generates an access frequency map, which is information indicating the access status of people within the monitoring area during a predetermined period, from the recognition results of the image recognition processing unit 33 .
- FIG. 7 is a diagram for explaining a method of generating an access frequency map.
- FIG. 7 shows image data 600 of the monitored area captured by the vision sensor 20 .
- image data 600 obtained by imaging the monitoring area from above is shown.
- Image data 600 of a monitoring area is divided into a plurality of small rectangular areas 601 .
- the substantially lower half is the area where the robot 10 and the surrounding object 520 are arranged, and the substantially upper half is the area 610 in which the person 500 can move.
- a circular area centered on the position of the robot 10 is a robot motion area 620 in which parts of the robot 10 can move.
- the access frequency of each small area 601 is 0.
- the person analysis processing unit 37 determines which small area 601 the position of the person 500 recognized by the image recognition processing unit 33 belongs to, and adds "1" to the access frequency of the small area 601 to which the person 500 belongs.
- the person analysis processing unit 37 generates an access frequency map by performing this process for a predetermined period.
- FIG. 8 is a diagram showing an example of an access frequency map.
- the access frequency map is generated from the image data 600 of the monitored area in FIG.
- the frequency with which the person 500 belongs to each small area 601 is represented by performing the above processing.
- the access frequency map it is possible to know in which position the person 500 is likely to exist in the monitored area.
- the access frequency map storage unit 38 stores the access frequency map in the monitoring area generated by the human analysis processing unit 37.
- the access frequency map is data prepared for machine learning.
- learning device 39 Based on the motion path of robot 10 and the states of human 500, robot 10, and peripheral object 520, learning device 39 suppresses the motion of robot 10 from being decelerated or stopped, and prevents human 500 from robot 10.
- a trained model is generated for learning the movement path of the robot 10 that suppresses contact with the robot 10 and the pinching of the person 500 between the robot 10 and the surrounding object 520 .
- the learned model storage unit 40 stores the learned model learned by the learning device 39 .
- the inference device 41 inputs the target position of the robot 10 and the states of the robot 10, the person 500, and the peripheral object 520 to the learned model stored in the learned model storage unit 40, thereby obtaining the robot 10, A motion path of the robot 10 that is suitable for the state of the person 500 and the surrounding object 520 is inferred.
- the learning by the learning device 39 and the inference by the inference device 41 will be explained in detail below.
- ⁇ Learning phase> 9 is a block diagram showing an example of a configuration of a learning device in a robot control device according to Embodiment 2.
- FIG. The learning device 39 includes a data acquisition section 391 and a model generation section 392 .
- the data acquisition unit 391 acquires the motion path of the robot 10 and state data indicating the states of the person 500, the robot 10, and surrounding objects as data for learning.
- the state data includes first information of the person 500, target position and second information of the robot 10, and peripheral object data.
- the first information is first position and orientation information
- the second information is second position and orientation information.
- the motion path and target position of the robot 10 can be obtained by simulating the motion program.
- the motion path of the robot 10 is the motion in the states of the person 500, the robot 10, and the surrounding object 520, that is, in the combination of the first position and orientation information of the person 500, the target position and second position and orientation information of the robot 10, and the surrounding object data. is the route.
- the model generating unit 392 determines whether the motion of the robot 10 is decelerated or stopped based on the states of the robot 10, the person 500, and the surrounding object 520 based on learning data including the motion path of the robot 10 and state data.
- the motion path of the robot 10 is learned by suppressing the contact of the robot 10 with the person 500 and the pinching of the person 500 between the robot 10 and the surrounding object 520 . That is, a trained model is generated for inferring the movement path of the robot 10 that reduces the possibility of contacting the person 500 and pinching the person 500 from the states of the person 500 , the robot 10 and the surrounding object 520 .
- supervised learning can be used as the learning algorithm used by the model generation unit 392 .
- reinforcement learning an agent, who is the subject of action in an environment, observes the parameters of the environment, which is the current state, and decides what action to take. The environment dynamically changes according to the actions of the agent, and the agent is rewarded according to the change in the environment. The agent repeats this and learns the course of action that yields the most rewards through a series of actions.
- Q learning Q-learning
- TD-learning TD learning
- a general update formula for the action-value function Q(s, a) is represented by the following formula (1).
- s t represents the state of the environment at time t
- a t represents the action at time t
- Action a t changes the state to s t+1
- r t+1 represents the reward obtained by changing the state
- ⁇ represents the discount rate
- ⁇ represents the learning coefficient.
- ⁇ is in the range of 0 ⁇ 1
- ⁇ is in the range of 0 ⁇ 1.
- the action path of the robot 10 is the action at , the state of the person 500, the robot 10 and the surrounding object 520 is the state st , and the best action at in the state st at time t is learned.
- the update formula represented by formula (1) increases the action value Q if the action value Q of action a with the highest Q value at time t+1 is greater than the action value Q of action a executed at time t.
- the action value Q is decreased.
- the action value function Q(s, a) is updated so that the action value Q of action a at time t approaches the best action value Q at time t+1.
- the best action value Q in a certain environment is sequentially propagated to the action value Q in the previous environment.
- the model generation unit 392 when generating a trained model by reinforcement learning, includes a reward calculation unit 393 and a function update unit 394.
- the reward calculation unit 393 calculates a reward based on the motion path of the robot 10 and the states of the person 500, the robot 10, and the surrounding object 520.
- the reward calculation unit 393 calculates at least one reward among the operation time of the robot 10, the degree of impact on the human body when the robot 10 contacts the person 500, and the access frequency of the person 500 within the robot operation area 620. Based on the criteria, calculate the reward r.
- the operation time of the robot 10 is, for example, the time it takes to move from a position A to another position B, which is a target position. The shorter the operating time of the robot 10, the higher the productivity, so the shorter the operating time, the better.
- Methods of shortening the operation time include a method of increasing the operation speed of the robot 10 and a method of shortening the operation path of the robot 10 .
- the reward calculation unit 393 increases the reward r.
- the reward calculation unit 393 increases the reward r by giving “1”, which is the value of the reward. Note that the reward value is not limited to "1”.
- the reward calculation unit 393 reduces the reward r. In one example, the reward calculation unit 393 reduces the reward r by giving “ ⁇ 1”, which is the value of the reward. Note that the reward value is not limited to "-1".
- the function updating unit 394 updates the function for determining the motion path of the robot 10 according to the reward calculated by the reward calculating unit 393 and outputs the function to the learned model storage unit 40 .
- the action value function Q(s t , a t ) represented by Equation (1) is used as a function for calculating the motion path of the robot 10 .
- the above learning is repeatedly executed.
- the learned model storage unit 40 stores the action value function Q(s t , a t ) updated by the function updating unit 394, that is, the learned model.
- FIG. 10 is a flow chart showing an example of the procedure of learning processing of the learning device included in the robot control device according to the second embodiment.
- the data acquisition unit 391 acquires the motion path of the robot 10 and state data indicating the states of the person 500, the robot 10, and the surrounding objects 520 as learning data (step S51).
- the state data for example, includes first position/posture information of the person 500, target position and second position/posture information of the robot 10, and peripheral object data.
- the model generation unit 392 calculates a reward based on the motion path of the robot 10 and state data indicating the states of the person 500, the robot 10, and the surrounding object 520, and determines whether to increase the reward (step S52).
- the reward calculation unit 393 acquires the motion path of the robot 10 and the states of the person 500, the robot 10, and the surrounding object 520, and determines the motion time of the robot 10, and the frequency of access of the person 500 within the robot operation area 620 to determine whether to increase or decrease the reward based on predetermined reward criteria.
- step S52 If it is determined in step S52 to increase the reward, the reward calculation unit 393 increases the reward (step S53). On the other hand, when it is determined to decrease the reward in step S52, the reward calculator 393 decreases the reward (step S54).
- step S55 the function updating unit 394 updates the action value function Q(s t , a t ) are updated (step S55).
- the learning device 39 repeats the above steps S51 to S55 and stores the generated action-value function Q(s t , a t ) as a learned model in the learned model storage unit 40 .
- the learning device 39 according to Embodiment 2 stores the learned model in the learned model storage unit 40 provided outside the learning device 39
- the learned model storage unit 40 is stored inside the learning device 39. be prepared for
- ⁇ Utilization phase> 11 is a block diagram showing an example of a configuration of an inference device in a robot control device according to Embodiment 2.
- the inference device 41 includes a data acquisition unit 411 and an inference unit 412 .
- the data acquisition unit 411 acquires state data indicating the states of the person 500, the robot 10, and the surrounding objects 520.
- the state data for example, includes first position/posture information of the person 500, target position and second position/posture information of the robot 10, and peripheral object data.
- the inference unit 412 infers the motion path of the robot 10 using the learned model. That is, by inputting the state data acquired by the data acquisition unit 411, that is, the states of the person 500, the robot 10, and the surrounding object 520, into this trained model, the motion path of the robot 10, more specifically, the person 500, A motion path of the robot 10 suitable for the state of the robot 10 and surrounding objects 520 can be inferred.
- the learned model learned by the model generation unit 392 of the learning device 39 of the robot control device 30 is used to output the motion path of the robot 10 , but the learned model is output from another robot system 1 . and output the motion path of the robot 10 based on this learned model.
- FIG. 12 is a flowchart illustrating an example of an inference processing procedure of an inference device included in the robot control device according to the second embodiment.
- the data acquisition unit 411 acquires state data indicating the states of the person 500, the robot 10, and the peripheral object 520 as inference data (step S71).
- the inference unit 412 inputs the state data, which is inference data, that is, the states of the person 500, the robot 10, and the peripheral object 520 to the learned model stored in the learned model storage unit 40 (step S72), A motion path of the robot 10 is obtained. After that, the inference unit 412 outputs the obtained motion path of the robot 10 as data to the robot control processing unit 32 (step S73).
- the robot control processing unit 32 controls the robot 10 using the output motion path of the robot 10 (step S74).
- the slowdown or stop of the motion of the robot 10 is suppressed, the degree of influence on the human body is reduced, the degree of influence of the human body 500 on the human body and pinching of the human 500 are suppressed, and the operation rate of the robot 10 is improved. It is possible to control the robot 10 while simultaneously improving and improving, and the robot system 1 with high versatility can be realized.
- reinforcement learning is applied to the learning algorithm used by the inference unit 412
- the present invention is not limited to this.
- the learning algorithm supervised learning, unsupervised learning, or semi-supervised learning can be applied in addition to reinforcement learning.
- Deep learning Deep Learning
- neural networks genetic programming
- function Machine learning may be performed according to logic programming, support vector machines, and the like.
- FIG. 6 shows the case where the learning device 39 and the reasoning device 41 are built in the robot control device 30, but the learning device 39 and the reasoning device 41 are connected to the robot control device 30 via a network, for example. It may be a device that is connected and separate from the robot controller 30 . Also, the learning device 39 and the reasoning device 41 may exist on a cloud server.
- the model generation unit 392 may learn the motion path of the robot 10 using learning data acquired from a plurality of robot control devices 30 .
- the model generation unit 392 may acquire learning data from a plurality of robot control devices 30 used in the same area, or may acquire learning data from a plurality of robot control devices 30 operating independently in different areas.
- the motion path of the robot 10 may be learned by using learning data obtained from the robot 10 . It is also possible to add or remove the robot control device 30 that collects learning data from the target in the middle.
- the learning device 39 that has learned the motion path of the robot 10 with respect to a certain robot control device 30 is applied to another robot control device 30, and the motion path of the robot 10 is re-learned with respect to the other robot control device 30. may be updated by
- the learning device 39 slows down or decelerates the motion of the robot 10 from the states of the person 500, the robot 10, and the peripheral object 520 based on learning data including the motion path of the robot 10 and state data.
- the motion path of the robot 10 is learned by suppressing being stopped and suppressing contact with the person 500 and pinching of the person 500. - ⁇ As a result, the movement of the robot 10 is prevented from being decelerated or stopped due to the states of the person 500, the robot 10, and the peripheral object 520, and the movement path of the robot 10 can be learned with a reduced degree of influence on the human body. It has the effect of being able to
- the inference device 41 uses the learned model to suppress the deceleration or stoppage of the motion of the robot 10 based on the states of the person 500, the robot 10, and the surrounding object 520, and A motion path of the robot 10 with reduced influence is inferred and output to the robot control processing unit 32 .
- the robot control processing unit 32 uses the learned model to suppress the deceleration or stoppage of the motion of the robot 10 based on the states of the person 500, the robot 10, and the surrounding object 520, and A motion path of the robot 10 with reduced influence is inferred and output to the robot control processing unit 32 .
- FIG. 13 is a block diagram showing an example hardware configuration of the robot control device 30 according to the first and second embodiments.
- the robot control device 30 can be realized by a hardware configuration including an arithmetic device 301 and a storage device 302.
- An example of the arithmetic unit 301 is a CPU (Central Processing Unit, central processing unit, processor, microprocessor, microcomputer, processor, DSP (Digital Signal Processor)) or system LSI (Large Scale Integration).
- Examples of the storage device 302 are RAM (Random Access Memory) or ROM (Read Only Memory).
- the robot control device 30 is implemented by the arithmetic device 301 reading and executing a program for executing the operation of the robot control device 30 stored in the storage device 302 . It can also be said that this program causes a computer to execute the procedure or method of the robot control device 30, for example, the robot control method shown in FIGS.
- the storage device 302 stores an operation program, human influence degree information, peripheral object data, an access frequency map, and a learned model.
- the storage device 302 is also used as a temporary memory when the arithmetic device 301 executes various processes.
- the program executed by the computing device 301 may be stored in a computer-readable storage medium in an installable or executable format and provided as a computer program product. Also, the program executed by the arithmetic device 301 may be provided to the robot control device 30 via a network such as the Internet.
- the robot control device 30 may be realized by dedicated hardware. Also, the functions of the robot control device 30 may be partly realized by dedicated hardware and partly by software or firmware.
- Robot system 10 Robot, 20 Vision sensor, 30 Robot controller, 31 Operation program storage unit, 32 Robot control processing unit, 33 Image recognition processing unit, 34 Human influence level information storage unit, 35 Peripheral object data storage unit, 36 Monitoring processing unit 37 Human analysis processing unit 38 Access frequency map storage unit 39 Learning device 40 Trained model storage unit 41 Inference device 391, 411 Data acquisition unit 392 Model generation unit 393 Reward calculation unit 394 Function update unit, 412 reasoning unit, 500 people, 501 hands, 510 workbench, 520 surrounding objects, 600 image data, 601 small area, 620 robot motion area.
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Manipulator (AREA)
Abstract
Description
図1は、実施の形態1に係るロボット制御装置を含むロボットシステムの構成の一例を示すブロック図である。ロボットシステム1は、ロボット10と、ビジョンセンサ20と、ロボット制御装置30と、を備える。
従来では、ロボット10と周辺物体とが配置される領域に人が存在する場合に、ロボット10が人と接触する可能性、およびロボット10と周辺物体との間への人の挟み込みの可能性を低減しながら、目的位置までのロボット10の動作経路をなるべく短くする技術については提案されていなかった。実施の形態2では、ロボット10が人と接触する可能性、およびロボット10と周辺物体との間への人の挟み込みの可能性を低減しながら、目的位置までのロボット10の動作経路をなるべく短くすることができるロボット制御装置について説明する。
図9は、実施の形態2に係るロボット制御装置における学習装置の構成の一例を示すブロック図である。学習装置39は、データ取得部391と、モデル生成部392と、を備える。
図11は、実施の形態2に係るロボット制御装置における推論装置の構成の一例を示すブロック図である。推論装置41は、データ取得部411と、推論部412と、を備える。
Claims (16)
- 人と作業領域を共有して動作するロボットを制御するロボット制御装置であって、
ビジョンセンサから得られる監視領域における計測データを基に、前記監視領域に存在する人に関する情報である第1情報を認識する画像認識処理部と、
前記ロボットを動作させる動作プログラムに従って前記ロボットの動作を制御するロボット制御処理部と、
前記監視領域における前記ロボットおよび前記ロボット以外の物体である周辺物体の3次元的な配置状態を示す周辺物体データと、前記画像認識処理部から得られる前記第1情報と、を基に、前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を判定する監視処理部と、
を備えることを特徴とするロボット制御装置。 - 前記監視処理部は、前記監視領域における前記ロボットおよび前記周辺物体の位置、形状および大きさを含む前記周辺物体データ上に、前記第1情報に基づいて前記人の位置情報を追加し、前記第1情報に基づいて前記人と前記ロボットとの動作を予測して、前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を判定することを特徴とする請求項1に記載のロボット制御装置。
- 前記画像認識処理部は、前記ロボットに関する情報である第2情報をさらに認識し、
前記監視処理部は、前記第1情報および前記第2情報に基づいて、前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を判定することを特徴とする請求項2に記載のロボット制御装置。 - 前記第2情報は、前記ロボットの位置および姿勢を含む第2位置姿勢情報と、前記ロボットの移動方向および速度を含む第2状態情報と、を含み、
前記監視処理部は、前記第2位置姿勢情報および前記第2状態情報から前記ロボットの動作を予測することを特徴とする請求項3に記載のロボット制御装置。 - 前記監視処理部は、前記第1情報から前記人の動作を予測し、前記動作プログラムから前記ロボットの動作を予測し、前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を判定することを特徴とする請求項2に記載のロボット制御装置。
- 前記第1情報は、前記人の位置および姿勢を含む第1位置姿勢情報と、前記人の移動方向および速度を含む第1状態情報と、を含み、
前記監視処理部は、前記第1位置姿勢情報および前記第1状態情報から前記人の動作を予測することを特徴とする請求項3から5のいずれか1つに記載のロボット制御装置。 - 前記第1位置姿勢情報は、人体部位の位置および姿勢を含み、
前記第1状態情報は、前記人体部位の移動方向および速度を含み、
前記ロボットの動作によって前記人に接触した場合の前記人体部位毎における人体へ与える影響度を示す人体影響度情報を記憶する人体影響度情報記憶部をさらに備え、
前記監視処理部は、前記人体部位の予測した動作と、前記ロボットの予測した動作と、から、前記ロボットと前記人体部位との接触部位および接触の状態を予測し、前記予測した前記接触部位および前記接触の状態に対応する前記人体へ与える影響度を前記人体影響度情報から取得し、取得した前記人体へ与える影響度に応じて前記ロボットの動作の指令を前記ロボット制御処理部に出力することを特徴とする請求項6に記載のロボット制御装置。 - 前記ロボットの動作を制限する指令は、前記ロボットの停止、減速、または前記人から離れる方向への前記ロボットの動作であることを特徴とする請求項1から7のいずれか1つに記載のロボット制御装置。
- 前記監視処理部は、前記人の挟み込みの可能性がある場合に、前記ロボットの動作を制限する指令を前記ロボット制御処理部に出力する、あるいは警告音を出力することを特徴とする請求項1から8のいずれか1つに記載のロボット制御装置。
- 前記人、前記ロボットおよび前記周辺物体の状態を示す状態データと、前記人、前記ロボットおよび前記周辺物体の状態における前記ロボットの動作経路と、を含む学習用データを取得するデータ取得部と、
前記学習用データを用いて、前記状態データから前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを生成するモデル生成部と、
を有する学習装置をさらに備え、
前記状態データは、前記第1情報、前記ロボットの目的位置、前記ロボットに関する情報である第2情報、および前記周辺物体データを含むことを特徴とする請求項1,2,5のいずれか1つに記載のロボット制御装置。 - 前記人、前記ロボットおよび前記周辺物体の状態を示す状態データと、前記人、前記ロボットおよび前記周辺物体の状態における前記ロボットの動作経路と、を含む学習用データを取得するデータ取得部と、
前記学習用データを用いて、前記状態データから前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを生成するモデル生成部と、
を有する学習装置をさらに備え、
前記状態データは、前記第1情報、前記ロボットの目的位置、前記第2情報、および前記周辺物体データを含むことを特徴とする請求項3または4に記載のロボット制御装置。 - 前記モデル生成部は、
前記ロボットの動作時間、前記ロボットが前記人に接触した場合の人体へ与える影響度、および前記ロボットの動作領域内における前記人のアクセス頻度のうちの少なくとも1つの報酬基準に基づいて、報酬を計算する報酬計算部と、
前記報酬計算部によって計算される報酬に従って、前記ロボットの動作経路を決定するための関数を更新する関数更新部と、
を有することを特徴とする請求項10または11に記載のロボット制御装置。 - 前記人、前記ロボットおよび前記周辺物体の状態を示す状態データを取得するデータ取得部と、
前記人、前記ロボットおよび前記周辺物体の状態から前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを用いて、前記データ取得部で取得した前記状態データから前記ロボットの動作経路を出力する推論部と、
を有する推論装置をさらに備え、
前記状態データは、前記第1情報、前記ロボットの目的位置、前記ロボットに関する情報である第2情報、および前記周辺物体データを含むことを特徴とする請求項1,2,5,10のいずれか1つに記載のロボット制御装置。 - 前記人、前記ロボットおよび前記周辺物体の状態を示す状態データを取得するデータ取得部と、
前記人、前記ロボットおよび前記周辺物体の状態から前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを用いて、前記データ取得部で取得した前記状態データから前記ロボットの動作経路を出力する推論部と、
を有する推論装置をさらに備え、
前記状態データは、前記第1情報、前記ロボットの目的位置、前記第2情報、および前記周辺物体データを含むことを特徴とする請求項3,4,11のいずれか1つに記載のロボット制御装置。 - ロボット、前記ロボットを含む監視領域に存在する人および前記ロボット以外の物体である周辺物体の状態を示す状態データと、前記人、前記ロボットおよび前記周辺物体の状態における前記ロボットの動作経路と、を含む学習用データを取得するデータ取得部と、
前記学習用データを用いて、前記状態データから前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを生成するモデル生成部と、
を備え、
前記状態データは、前記人に関する情報である第1情報、前記ロボットの目的位置、前記ロボットに関する情報である第2情報、および前記監視領域における前記ロボットおよび前記ロボット以外の物体である周辺物体の3次元的な配置状態を示す周辺物体データを含むことを特徴とする学習装置。 - ロボット、前記ロボットを含む監視領域に存在する人および前記ロボット以外の物体である周辺物体の状態を示す状態データを取得するデータ取得部と、
前記人、前記ロボットおよび前記周辺物体の状態から前記ロボットの動作が減速または停止されることを抑制し、かつ前記人への接触および前記ロボットと前記周辺物体との間への前記人の挟み込みの可能性を低減した前記ロボットの動作経路を推論するための学習済モデルを用いて、前記データ取得部で取得した前記状態データから前記ロボットの動作経路を出力する推論部と、
を備え、
前記状態データは、前記人に関する情報である第1情報、前記ロボットの目的位置、前記ロボットに関する情報である第2情報、および前記監視領域における前記ロボットおよび前記ロボット以外の物体である周辺物体の3次元的な配置状態を示す周辺物体データを含むことを特徴とする推論装置。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE112021007561.6T DE112021007561T5 (de) | 2021-09-08 | 2021-09-08 | Robotersteuerung, Lernvorrichtung und Inferenzvorrichtung |
CN202180098461.5A CN117377561A (zh) | 2021-09-08 | 2021-09-08 | 机器人控制装置、学习装置及推断装置 |
JP2022506319A JP7098080B1 (ja) | 2021-09-08 | 2021-09-08 | ロボット制御装置 |
PCT/JP2021/032995 WO2023037443A1 (ja) | 2021-09-08 | 2021-09-08 | ロボット制御装置、学習装置および推論装置 |
US18/571,741 US12017372B2 (en) | 2021-09-08 | 2021-09-08 | Robot control device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2021/032995 WO2023037443A1 (ja) | 2021-09-08 | 2021-09-08 | ロボット制御装置、学習装置および推論装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023037443A1 true WO2023037443A1 (ja) | 2023-03-16 |
Family
ID=82356947
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/032995 WO2023037443A1 (ja) | 2021-09-08 | 2021-09-08 | ロボット制御装置、学習装置および推論装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US12017372B2 (ja) |
JP (1) | JP7098080B1 (ja) |
CN (1) | CN117377561A (ja) |
DE (1) | DE112021007561T5 (ja) |
WO (1) | WO2023037443A1 (ja) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008137127A (ja) * | 2006-12-04 | 2008-06-19 | Matsushita Electric Ind Co Ltd | ロボットの制御方法 |
JP2009090400A (ja) * | 2007-10-05 | 2009-04-30 | National Institute Of Advanced Industrial & Technology | ロボット、ロボット制御装置、ロボット制御プログラム、ロボット制御プログラムを作成するためのシミュレータ |
JP2019030941A (ja) * | 2017-08-08 | 2019-02-28 | ファナック株式会社 | 制御装置及び学習装置 |
JP2019063974A (ja) * | 2017-10-05 | 2019-04-25 | ファナック株式会社 | ロボットの構成部材の速度を制限する制御装置 |
JP2019098410A (ja) * | 2017-11-28 | 2019-06-24 | ファナック株式会社 | ロボットの速度を制限する制御装置 |
JP2020066092A (ja) * | 2018-10-24 | 2020-04-30 | ファナック株式会社 | ロボットの制御方法 |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS60131192A (ja) * | 1983-12-16 | 1985-07-12 | 松下電器産業株式会社 | ロボツトの安全柵 |
JP4155804B2 (ja) * | 2002-11-26 | 2008-09-24 | ソニー株式会社 | 脚式移動ロボットの制御装置 |
WO2014036549A2 (en) * | 2012-08-31 | 2014-03-06 | Rethink Robotics, Inc. | Systems and methods for safe robot operation |
US9452532B2 (en) | 2014-01-27 | 2016-09-27 | Panasonic Intellectual Property Management Co., Ltd. | Robot, device and method for controlling robot, and computer-readable non-transitory recording medium |
JP6677441B2 (ja) | 2014-10-20 | 2020-04-08 | 株式会社デンソーウェーブ | ロボット、ロボットの形状設計方法 |
JP2017077608A (ja) | 2015-10-21 | 2017-04-27 | ファナック株式会社 | ロボットの安全監視装置 |
JP2017144490A (ja) * | 2016-02-15 | 2017-08-24 | オムロン株式会社 | 制御装置、制御システム、制御方法およびプログラム |
JP6431017B2 (ja) * | 2016-10-19 | 2018-11-28 | ファナック株式会社 | 機械学習により外力の検出精度を向上させた人協調ロボットシステム |
TWI774666B (zh) * | 2017-03-17 | 2022-08-21 | 達明機器人股份有限公司 | 協作型機器手臂的防夾方法 |
JP7011910B2 (ja) | 2017-09-01 | 2022-01-27 | 川崎重工業株式会社 | ロボットシステム |
JP6690626B2 (ja) | 2017-10-31 | 2020-04-28 | 株式会社安川電機 | ロボットシステムおよびロボットの退避方法 |
JP2020082307A (ja) * | 2018-11-29 | 2020-06-04 | トヨタ自動車株式会社 | ロボットのリスクアセスメント方法 |
JP6997068B2 (ja) | 2018-12-19 | 2022-01-17 | ファナック株式会社 | ロボット制御装置、ロボット制御システム、及びロボット制御方法 |
JP2020142344A (ja) | 2019-03-08 | 2020-09-10 | トヨタ自動車株式会社 | 環境監視システム |
JP2021020287A (ja) * | 2019-07-30 | 2021-02-18 | セイコーエプソン株式会社 | 検出方法 |
US20220105634A1 (en) * | 2021-12-17 | 2022-04-07 | Intel Corporation | Situation-aware safety assessment of robot-human activities |
-
2021
- 2021-09-08 WO PCT/JP2021/032995 patent/WO2023037443A1/ja active Application Filing
- 2021-09-08 CN CN202180098461.5A patent/CN117377561A/zh active Pending
- 2021-09-08 JP JP2022506319A patent/JP7098080B1/ja active Active
- 2021-09-08 US US18/571,741 patent/US12017372B2/en active Active
- 2021-09-08 DE DE112021007561.6T patent/DE112021007561T5/de active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008137127A (ja) * | 2006-12-04 | 2008-06-19 | Matsushita Electric Ind Co Ltd | ロボットの制御方法 |
JP2009090400A (ja) * | 2007-10-05 | 2009-04-30 | National Institute Of Advanced Industrial & Technology | ロボット、ロボット制御装置、ロボット制御プログラム、ロボット制御プログラムを作成するためのシミュレータ |
JP2019030941A (ja) * | 2017-08-08 | 2019-02-28 | ファナック株式会社 | 制御装置及び学習装置 |
JP2019063974A (ja) * | 2017-10-05 | 2019-04-25 | ファナック株式会社 | ロボットの構成部材の速度を制限する制御装置 |
JP2019098410A (ja) * | 2017-11-28 | 2019-06-24 | ファナック株式会社 | ロボットの速度を制限する制御装置 |
JP2020066092A (ja) * | 2018-10-24 | 2020-04-30 | ファナック株式会社 | ロボットの制御方法 |
Also Published As
Publication number | Publication date |
---|---|
US20240123627A1 (en) | 2024-04-18 |
CN117377561A (zh) | 2024-01-09 |
DE112021007561T5 (de) | 2024-05-02 |
JP7098080B1 (ja) | 2022-07-08 |
JPWO2023037443A1 (ja) | 2023-03-16 |
US12017372B2 (en) | 2024-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Halme et al. | Review of vision-based safety systems for human-robot collaboration | |
Sharkawy et al. | Human–robot collisions detection for safe human–robot interaction using one multi-input–output neural network | |
Flacco et al. | A depth space approach for evaluating distance to objects: with application to human-robot collision avoidance | |
US10953538B2 (en) | Control device and learning device | |
KR101942167B1 (ko) | 전방향 휠을 가지고 선형 예측 위치 및 속도 제어기를 기초한 휴머노이드 로봇 | |
De Luca et al. | Integrated control for pHRI: Collision avoidance, detection, reaction and collaboration | |
KR102165967B1 (ko) | 3차원 공간 감시 장치, 3차원 공간 감시 방법, 및 3차원 공간 감시 프로그램 | |
Najmaei et al. | Applications of artificial intelligence in safe human–robot interactions | |
CN111906778B (zh) | 基于多重感知的机器人安全控制方法及装置 | |
JP2017030137A (ja) | 人の行動パターンを学習する機械学習装置、ロボット制御装置、ロボットシステム、および機械学習方法 | |
JP2010120139A (ja) | 産業用ロボットの安全制御装置 | |
JP2019188580A (ja) | 情報処理装置、制御方法、ロボットシステム、コンピュータプログラム、及び記憶媒体 | |
US20210394362A1 (en) | Information processing device, control method, and program | |
JP7243979B2 (ja) | ロボット干渉判定装置、ロボット干渉判定方法、ロボット制御装置、ロボット制御システム、人動作予測装置、および人動作予測方法 | |
WO2017141567A1 (ja) | 作業領域推定装置、制御装置、制御システム、作業領域推定方法およびプログラム | |
WO2023037443A1 (ja) | ロボット制御装置、学習装置および推論装置 | |
JP5776486B2 (ja) | ロボット制御装置、その制御方法及びプログラム | |
JP5306934B2 (ja) | ロボットおよび制御システム | |
CN115903773A (zh) | 移动体控制装置、移动体、学习装置及方法、及存储介质 | |
CN116806184A (zh) | 机器人的机械阻抗控制方法、控制***及机器人 | |
Zhu et al. | Dynamic motion planning algorithm in human-robot collision avoidance | |
JPWO2023037443A5 (ja) | ロボット制御装置 | |
WO2022074827A1 (ja) | 命題設定装置、命題設定方法及び記憶媒体 | |
US20230364791A1 (en) | Temporal logic formula generation device, temporal logic formula generation method, and storage medium | |
JP7364032B2 (ja) | 制御装置、制御方法及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 2022506319 Country of ref document: JP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21956740 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180098461.5 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 112021007561 Country of ref document: DE Ref document number: 18571741 Country of ref document: US |