WO2023195436A1 - Work machine - Google Patents

Work machine Download PDF

Info

Publication number
WO2023195436A1
WO2023195436A1 PCT/JP2023/013757 JP2023013757W WO2023195436A1 WO 2023195436 A1 WO2023195436 A1 WO 2023195436A1 JP 2023013757 W JP2023013757 W JP 2023013757W WO 2023195436 A1 WO2023195436 A1 WO 2023195436A1
Authority
WO
WIPO (PCT)
Prior art keywords
area
recognition
dump truck
machine
information
Prior art date
Application number
PCT/JP2023/013757
Other languages
French (fr)
Japanese (ja)
Inventor
匡士 小谷
哲平 齋藤
理優 成川
英明 伊東
英史 石本
慧 佐藤
Original Assignee
日立建機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日立建機株式会社 filed Critical 日立建機株式会社
Publication of WO2023195436A1 publication Critical patent/WO2023195436A1/en

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • the present invention relates to working machines.
  • Multi-jointed working machines eg, hydraulic excavators
  • front working devices eg, attachments such as booms, arms, and buckets
  • This type of working machine performs a loading operation in which excavated objects such as earth and sand are loaded onto a transporting machine (for example, a dump truck) that is a loaded machine.
  • a transporting machine for example, a dump truck
  • Patent Document 1 discloses a data acquisition unit that acquires a captured image showing a target for loading and unloading objects to be transported by a working machine, an area specifying unit that identifies an area including the target for loading and unloading from the captured image, and an area including the target for loading and unloading.
  • An image processing system comprising: an unloading target specifying section that identifies at least one predetermined surface of the unloading target.
  • Patent Document 1 uses an image recognition system using machine learning to detect the position and orientation of the transport machine from an image captured by a camera attached to the work machine, in order to identify the transport machine.
  • image recognition system using machine learning it is difficult to guarantee the validity and recognition accuracy of recognition results for sites or transport machines for which learning data has not been collected.
  • the present invention has been made in view of the above, and provides work that enables appropriate control of loading work on a transport machine by accurately recognizing the position and orientation of the transport machine in various situations.
  • the purpose is to provide machinery.
  • a working machine of the present invention is a working machine that loads objects onto a transport machine, and includes an external world measuring device that measures the surrounding environment of a vehicle body of the working machine, and a measuring device of the external world measuring device.
  • an information processing device that recognizes the transportation machine existing around the vehicle body based on the result; a control device that controls the operation of the vehicle body based on the recognition result of the information processing device; and a control device that controls the location of the transportation machine from the outside.
  • a transport machine information acquisition device that acquires vehicle class information, the information processing device recognizes the transport machine based on the position and vehicle class information of the transport machine acquired by the transport machine information acquisition device.
  • the control device corrects the result and controls the operation of the vehicle body based on the corrected recognition result of the transport machine.
  • FIG. 1 is a side view schematically showing the external configuration of a hydraulic excavator that is an example of a working machine according to a first embodiment
  • FIG. 2 is a block diagram showing the configuration of a hydraulic system and a control system installed in the hydraulic excavator shown in FIG. 1.
  • FIG. 3 is a diagram illustrating an example of the operation of the hydraulic excavator shown in FIG. 2.
  • FIG. 3 is a flowchart showing an example of the operation of the hydraulic excavator shown in FIG. 2.
  • 3 is a block diagram illustrating the functional configuration of the information processing apparatus shown in FIG. 2.
  • FIG. 7 is a diagram of each coordinate system shown in FIG. 6 viewed from above.
  • FIG. 9 is a diagram of the vehicle body coordinate system shown in FIG. 8 viewed from above.
  • 5 is a flowchart illustrating an example of recognition result verification processing.
  • FIG. 4 is a diagram illustrating the degree of polymerization of a recognition region and an estimated region.
  • FIG. 3 is a diagram showing a table that defines criteria for determining validity of recognition results and methods for correcting recognition areas for each task of a hydraulic excavator. The figure explaining the correction method of the recognition area at the time of loading task acquisition. The figure explaining the correction method of the recognition area at the time of reaching task acquisition. The figure which shows the example of a display of a display device when the accuracy of a recognition result is below a threshold value.
  • FIG. 3 is a block diagram illustrating the functional configuration of an information processing device according to a second embodiment.
  • FIG. 17 is a diagram illustrating processing of the truck attitude estimating section shown in FIG. 16;
  • FIG. 3 is a block diagram illustrating the functional configuration of an information processing apparatus according to a third embodiment.
  • the working machine of this embodiment is a working machine that performs a loading operation of loading an object such as excavated earth and sand onto a transport machine that is a loaded machine.
  • a hydraulic excavator equipped with a bucket is used as a working machine
  • a dump truck is used as an example of a transport machine.
  • the working machine of this embodiment may be a hydraulic excavator equipped with an attachment other than a bucket, or may be a working machine other than a hydraulic excavator.
  • the transport machine of this embodiment may be a transport machine other than a dump truck.
  • FIG. 1 is a side view schematically showing the external configuration of a hydraulic excavator 1, which is an example of a working machine according to the first embodiment.
  • the hydraulic excavator 1 includes an articulated front working device 2 that holds an object and rotates in the vertical or longitudinal direction, and a machine body 3 on which the front working device 2 is mounted.
  • the machine main body 3 includes a lower traveling body 5 that travels by a traveling right hydraulic motor 4a and a traveling left hydraulic motor 4b provided on the right and left parts of the lower traveling body 5, and a rotating device attached to the upper part of the lower traveling body 5.
  • the upper rotating body 7 is attached to the upper rotating body 7 and is rotated by a rotating hydraulic motor 6 of a rotating device.
  • the right travel hydraulic motor 4a and the left travel hydraulic motor 4b are also collectively referred to as "travel hydraulic motors 4a, 4b.”
  • the front working device 2 is an articulated working device configured by a plurality of front members attached to the front part of the upper revolving body 7.
  • the upper revolving body 7 mounts the front working device 2 and rotates.
  • the front working device 2 includes a boom 8 connected to the front part of the upper revolving body 7 so as to be rotatable in the vertical direction, an arm 9 connected to the tip of the boom 8 so as to be rotatable in the vertical direction, and an arm 9 .
  • the bucket 10 is rotatably connected to the tip of the bucket 10 in the vertical direction.
  • the boom 8 is connected to the upper revolving body 7 by a boom pin 8a, and rotates as the boom cylinder 11 expands and contracts.
  • the arm 9 is connected to the tip of the boom 8 by an arm pin 9a, and rotates as the arm cylinder 12 expands and contracts.
  • the bucket 10 is connected to the tip of the arm 9 by a bucket pin 10a and a bucket link 16, and rotates as the bucket cylinder 13 expands and contracts.
  • a boom angle sensor 14 that detects the rotation angle of the boom 8 is attached to the boom pin 8a.
  • An arm angle sensor 15 for detecting the rotation angle of the arm 9 is attached to the arm pin 9a.
  • a bucket angle sensor 17 that detects the rotation angle of the bucket 10 is attached to the bucket link 16 .
  • each rotation angle of the boom 8, arm 9, and bucket 10 is obtained by detecting each angle of the boom 8, arm 9, and bucket 10 with respect to a reference plane such as a horizontal plane using an inertial measurement device, and converting it into each rotation angle. may be done. Further, each rotation angle of the boom 8, arm 9, and bucket 10 may be obtained by detecting each stroke of the boom cylinder 11, arm cylinder 12, and bucket cylinder 13 with a stroke sensor and converting it into each rotation angle. .
  • a tilt angle sensor 18 is attached to the upper revolving body 7 to detect the tilt angle of the machine body 3 with respect to a reference plane such as a horizontal plane.
  • a turning angle sensor 19 is attached to the turning device between the lower traveling body 5 and the upper rotating body 7 to detect a turning angle that is a relative angle of the upper rotating body 7 with respect to the lower traveling body 5.
  • An angular velocity sensor (not shown) that detects the angular velocity of the upper rotating body 7 is attached to the upper rotating body 7 .
  • the boom angle sensor 14, the arm angle sensor 15, the bucket angle sensor 17, the tilt angle sensor 18, and the turning angle sensor 19 are also collectively referred to as the "attitude detection device 53."
  • the attitude detection device 53 detects each rotation angle of the front working device 2, the rotation angle of the upper rotating body 7, and the like.
  • a display device 55 configured with a touch screen or the like capable of receiving input from an operator is installed in the operator's cab provided in the upper revolving structure 7. Moreover, in the operator's cab provided in the upper revolving structure 7, an operating device for operating the plurality of hydraulic actuators 4a, 4b, 6, 11, 12, and 13 is installed. Specifically, the operating device operates the right travel lever 23a for operating the right travel hydraulic motor 4a, the left travel lever 23b for operating the left travel hydraulic motor 4b, the boom cylinder 11, and the bucket cylinder 13. A right operating lever 22a for operating the arm cylinder 12 and a left operating lever 22b for operating the swing hydraulic motor 6 is provided. In this embodiment, the right travel lever 23a, the left travel lever 23b, the right operation lever 22a, and the left operation lever 22b are also collectively referred to as "operation levers 22, 23.”
  • an external world measuring device 70 is attached to the upper revolving body 7 to measure the surrounding environment of the vehicle body of the hydraulic excavator 1.
  • the external world measuring device 70 measures the depth of an object (distance to the object) existing around the hydraulic excavator 1 .
  • the external world measuring device 70 acquires depth information of the object as a measurement result.
  • the external world measuring device 70 may be, for example, LiDAR (Light Detection And Ranging) or a stereo camera.
  • a plurality of external world measuring devices 70 may be attached to the hydraulic excavator 1.
  • a positioning device 60 is attached to the upper revolving body 7 to measure the position and orientation of the hydraulic excavator 1 at the site.
  • the positioning device 60 acquires positioning information including the position and orientation of the hydraulic excavator 1 at the site as a measurement result.
  • the positioning device 60 may be, for example, a GNSS receiver or a surveying device such as a TS (Total Station).
  • the positioning device 60 is not limited to these, and may be, for example, a camera fixed at the site.
  • the positioning device 60 may measure the position and orientation of the hydraulic excavator 1 at the site based on information calculated from the detected image of the hydraulic excavator 1. In this case, in order to accurately measure the orientation of the hydraulic excavator 1, it is desirable that at least two positioning devices 60 are provided.
  • a truck information acquisition device 56 is attached to the upper revolving body 7 to acquire truck information including the position, attitude, and vehicle size information of the dump truck 200 from the outside.
  • the vehicle class information includes information on the vehicle type and dimensions of the dump truck 200.
  • the truck information acquisition device 56 may be, for example, a communication terminal that performs wireless communication with a position/dispatch management system for the dump truck 200 such as a Fleet Management System (FMS).
  • FMS Fleet Management System
  • the truck information acquisition device 56 can acquire truck information by receiving the truck information of the dump truck 200 transmitted from the dump truck 200 position/dispatch management system.
  • the track information acquisition device 56 may be configured as software that implements the functions of the track information acquisition device 56, and may be incorporated into the information processing device 54.
  • the truck information acquisition device 56 can acquire information regarding the position adjustment amount of the dump truck 200 based on the adjustment function from the position/vehicle allocation management system at the same time as the truck information of the dump truck 200.
  • a task acquisition device 58 is attached to the upper revolving body 7 to acquire a task indicating the content of the next operation (or content of work) to be performed by the hydraulic excavator 1 .
  • the task acquisition device 58 may be, for example, a communication terminal that performs wireless communication with a management device that manages a work site.
  • the task acquisition device 58 can acquire the task by receiving the task of the hydraulic excavator 1 transmitted from the management device.
  • the task acquisition device 58 may be configured by a communication terminal shared with the track information acquisition device 56.
  • the task acquisition device 58 may be configured as software that implements the functions of the task acquisition device 58, and may be incorporated into the information processing device 54.
  • the task acquisition device 58 may acquire the task by automatically determining the task based on the operation of the operating levers 22 and 23 by the operator.
  • the task acquisition device 58 is an input terminal that can be operated by an operator, and may acquire tasks by accepting input from the operator. In this embodiment, there is no particular limitation on the task acquisition method.
  • FIG. 2 is a block diagram showing the configuration of a hydraulic system and a control system installed in the hydraulic excavator 1 shown in FIG. 1.
  • An engine 103 which is a prime mover mounted on the upper revolving structure 7, drives a hydraulic pump 102 and a pilot pump 104.
  • the control device 40 controls the rotating operation of the front working device 2, the traveling operation of the lower traveling body 5, and the rotating upper rotating body 7 according to operation information (operating amount and operating direction) of the operating levers 22 and 23 by the operator.
  • the turning operation (that is, the operation of the vehicle body of the hydraulic excavator 1 constituted by these) is controlled.
  • the control device 40 detects operation information (operation amount and operation direction) of the operation levers 22 and 23 by the operator using sensors 52a to 52f such as rotary encoders or potentiometers, and detects operation information according to the detected operation information.
  • a control command is output to the electromagnetic proportional valves 47a to 47l.
  • the electromagnetic proportional valves 47a to 47l are provided in the pilot line 100, and are activated when a control command from the control device 40 is input, output pilot pressure to the flow control valve 101, and operate the flow control valve 101.
  • the flow control valve 101 controls pressure oil supplied from the hydraulic pump 102 to each of the swing hydraulic motor 6, arm cylinder 12, boom cylinder 11, bucket cylinder 13, right travel hydraulic motor 4a, and left travel hydraulic motor 4b. Control is performed according to pilot pressure from electromagnetic proportional valves 47a to 47l. Note that the electromagnetic proportional valves 47a and 47b output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the swing hydraulic motor 6. The electromagnetic proportional valves 47c and 47d output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the arm cylinder 12. The electromagnetic proportional valves 47e and 47f output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the boom cylinder 11.
  • the electromagnetic proportional valves 47g and 47h output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the bucket cylinder 13.
  • the electromagnetic proportional valves 47i and 47j output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the travel right hydraulic motor 4a.
  • the electromagnetic proportional valves 47k and 47l output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the travel left hydraulic motor 4b.
  • the boom cylinder 11, arm cylinder 12, and bucket cylinder 13 are expanded and contracted by the supplied pressure oil, respectively, and rotate the boom 8, arm 9, and bucket 10. As a result, the position and attitude of the bucket 10 change.
  • the swing hydraulic motor 6 is rotated by the supplied pressure oil, and swings the upper revolving structure 7.
  • the right travel hydraulic motor 4a and the left travel hydraulic motor 4b are rotated by the supplied pressure oil and cause the lower traveling body 5 to travel.
  • control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 (the rotational operation of the front working device 2, the traveling operation of the lower traveling body 5, and the movement of the upper rotating body 7) based on the recognition result of the information processing device 54. It is possible to control the rotation movement automatically (semi-automatically or fully automatically).
  • the information processing device 54 recognizes the dump truck 200 existing around the hydraulic excavator 1 based on the measurement results of the external world measurement device 70.
  • the information processing device 54 includes a CPU (Central Processing Unit) 73, a RAM (Random Access Memory) 72, a ROM (Read Only Memory) 71, an external I/F (Interface) 74, etc., which are connected to each other by a bus 75.
  • CPU Central Processing Unit
  • RAM Random Access Memory
  • ROM Read Only Memory
  • I/F Interface
  • the external I/F 74 includes a control device 40, a display device 55, an external world measurement device 70, a positioning device 60, an attitude detection device 53, a task acquisition device 58, a track information acquisition device 56, and a storage device 57 (for example, a hard disk drive or large (capacitive flash memory, etc.) is connected.
  • FIG. 3 is a diagram illustrating an example of the operation of the hydraulic excavator 1 shown in FIG. 2.
  • the dump truck 200 stops at a predetermined position where the hydraulic excavator 1 can load the object.
  • the task acquisition device 58 of the hydraulic excavator 1 acquires a loading task for loading excavated objects such as earth and sand onto the dump truck 200 as the next task.
  • the external world measurement device 70 of the hydraulic excavator 1 first measures the vessel of the dump truck 200.
  • the information processing device 54 of the hydraulic excavator 1 calculates a recognition area and an estimation area, which will be described later, for the presence area 210 of the dump truck 200.
  • the information processing device 54 of the hydraulic excavator 1 calculates the position of the bucket 10 (hereinafter also referred to as "loading position") when loading objects from the bucket 10 into the vessel of the dump truck 200. Then, the control device 40 of the hydraulic excavator 1 loads the bucket 10 so that the front working device 2 does not collide with the dump truck 200 based on the calculated recognition area and estimated area of the dump truck 200 and the loading position. Outputs a control command to move to a certain position. Thereby, the hydraulic excavator 1 can appropriately automatically control the loading work without colliding with the dump truck 200.
  • the dump truck 200 is assumed to stop at a predetermined position where the hydraulic excavator 1 can load the object.
  • the position of the dump truck 200 in the present embodiment at the work site is managed by a dump truck 200 position/dispatch management system such as FMS.
  • the method of controlling the dump truck 200 until the dump truck 200 stops at a predetermined position is not particularly limited.
  • the dump truck 200 may be stopped at a predetermined position by the operator of the dump truck 200.
  • the operator of the hydraulic excavator 1 may visually confirm that the dump truck 200 has stopped at a predetermined position, or the information processing device 54 may recognize the fact that the dump truck 200 has stopped at a predetermined position. You may.
  • the external world measuring device 70 in FIG. 3 is attached so as to face the left side of the hydraulic excavator 1, it may be attached so as to face the front of the hydraulic excavator 1. Furthermore, the hydraulic excavator 1 may further include an external world measurement device 70 attached to face the right side or rear of the hydraulic excavator 1.
  • FIG. 4 is a flowchart showing an example of the operation of the hydraulic excavator 1 shown in FIG. 2.
  • step S111 the task acquisition device 58 of the hydraulic excavator 1 acquires an excavation task for excavating the ground to be excavated.
  • step S112 the control device 40 of the hydraulic excavator 1 controls the rotational operation of the front working device 2 to cause the hydraulic excavator 1 to perform an excavation operation, according to the acquired excavation task.
  • step S113 the task acquisition device 58 of the hydraulic excavator 1 acquires a loading task for loading an object such as excavated earth and sand onto the dump truck 200.
  • step S114 the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2 and the rotation operation of the upper rotating structure 7 according to the acquired loading task, so that the hydraulic excavator 1 performs a loading operation. Have them do it.
  • step S115 the control device 40 of the hydraulic excavator 1 determines whether to finish loading the dump truck 200.
  • the operator of the hydraulic excavator 1 performs visual confirmation and then operates specific control levers 22 and 23 or inputs information to the display device 55 to finish loading the dump truck 200.
  • the control device 40 may be notified of this.
  • the control device 40 may determine to finish loading the dump truck 200 upon being notified of these operations from the operators.
  • the control device 40 finishes loading the object into the dump truck 200 based on the determination result of the load determination device that measures the load of the object loaded onto the dump truck 200 and determines that the load is appropriate. It may be determined that If it is determined that loading into the dump truck 200 is finished, the control device 40 moves to step S118. If it is determined that loading into the dump truck 200 is not finished, the control device 40 moves to step S116.
  • step S116 the task acquisition device 58 of the hydraulic excavator 1 acquires a reaching task to move the front working device 2 to the next ground to be excavated.
  • step S117 the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2, the traveling operation of the lower traveling body 5, and the turning operation of the upper rotating body 7, according to the acquired reaching task. to cause the hydraulic excavator 1 to perform a reaching operation. After that, the control device 40 moves to step S111.
  • step S118 the control device 40 of the hydraulic excavator 1 determines whether or not to end the work.
  • the control device 40 may determine that the task is to be completed when the task acquisition device 58 has acquired the notification of completion of the task. Alternatively, the control device 40 may determine that the work is finished when the operator or manager stops the engine 103 of the hydraulic excavator 1. If it is determined that the work is to be completed, the control device 40 interrupts the operation of the hydraulic excavator 1 and ends the work. If it is determined that the work is not finished, the control device 40 moves to step S116. The hydraulic excavator 1 continues the reaching operation, the digging operation, and the loading operation.
  • the tasks and operations of the hydraulic excavator 1 are not limited to those shown in FIG. 4, and may include tasks and operations not shown in FIG. 4, such as moving or leveling the ground.
  • the information processing device 54 recognizes the dump truck 200 based on the measurement results of the external world measurement device 70 each time the task acquisition device 58 acquires each task. Then, the information processing device 54 verifies whether the recognition result of the dump truck 200 is invalid to the extent that it cannot be corrected, or whether the recognition result of the dump truck 200 is valid to the extent that it can be corrected (hereinafter referred to as “validation”. (referred to as “verification”). Then, if the recognition result of the dump truck 200 is valid, the information processing device 54 corrects the recognition result.
  • the process of verifying the validity of the recognition result of the dump truck 200 and correcting the recognition result is also referred to as "recognition result verification process.”
  • recognition result verification process the process of verifying the validity of the recognition result of the dump truck 200 and correcting the recognition result.
  • FIG. 5 is a block diagram illustrating the functional configuration of the information processing device 54 shown in FIG. 2.
  • FIG. 6 is a side view of each of the coordinate systems 300 to 500 set in the hydraulic excavator 1.
  • FIG. 7 is a diagram of each of the coordinate systems 300 to 500 shown in FIG. 6 viewed from above.
  • the information processing device 54 includes an attitude calculation section 81 , a coordinate transformation section 82 , a positioning information calculation section 83 , a task acquisition section 80 , a recognition section 84 , an area estimation section 85 , a verification section 86 , and a correction section 87 and.
  • a vehicle body coordinate system 400 shown in FIGS. 6 and 7 is preset in the information processing device 54 as a reference coordinate system for specifying the positions and postures of the components of the hydraulic excavator 1.
  • the vehicle body coordinate system 400 of the hydraulic excavator 1 of the present embodiment is defined as a right-handed coordinate system whose origin is the point on the swing center line 120 of the upper rotating body 7 where the lower traveling body 5 and the ground G contact.
  • the forward direction of the lower traveling body 5 is defined as the positive direction of the X-axis.
  • the direction in which the turning center line 120 extends upward is defined as the positive direction of the Z-axis.
  • the vehicle body coordinate system 400 of the hydraulic excavator 1 is orthogonal to each of the X-axis and the Z-axis, and the left side is defined as the positive direction of the Y-axis. Furthermore, in the vehicle body coordinate system 400 of the hydraulic excavator 1, the turning angle ⁇ sw of the upper rotating body 7 is defined as 0 degrees when the front working device 2 is parallel to the X axis.
  • a sensor coordinate system 300 as shown in FIGS. 6 and 7 is defined as a reference coordinate system of the external world measuring device 70.
  • a site coordinate system 500 as shown in FIGS. 6 and 7 is defined as a site reference coordinate system.
  • the posture calculation unit 81 calculates the postures of the components of the hydraulic excavator 1 in the vehicle body coordinate system 400 of the hydraulic excavator 1 from the detection signal of the posture detection device 53. Specifically, the attitude calculation unit 81 calculates the rotation angle ⁇ bm of the boom 8 with respect to the X axis from the detection signal of the rotation angle of the boom 8 output from the boom angle sensor 14. Posture calculation unit 81 calculates the rotation angle ⁇ am of arm 9 with respect to boom 8 from the detection signal of the rotation angle of arm 9 output from arm angle sensor 15 . The posture calculation unit 81 calculates the rotation angle ⁇ bk of the bucket 10 with respect to the arm 9 from the detection signal of the rotation angle of the bucket 10 output from the bucket angle sensor 17.
  • the attitude calculation unit 81 calculates the turning angle ⁇ sw of the upper rotating structure 7 with respect to the X axis (the lower traveling structure 5) from the detection signal of the turning angle of the upper rotating structure 7 output from the turning angle sensor 19. Furthermore, the attitude calculation unit 81 calculates the turning angular velocity ⁇ sw of the upper rotating structure 7 from the turning angle ⁇ sw of the upper rotating structure 7.
  • the attitude calculation unit 81 calculates the inclination angle of the machine body 3 (lower traveling body 5) with respect to the reference plane DP from the detection signal of the inclination angle of the machine body 3 output from the inclination angle sensor 18.
  • the reference plane DP is, for example, a horizontal plane orthogonal to the direction of gravity.
  • the tilt angle includes a rotation angle ⁇ p around the Y-axis and a rotation angle ⁇ r around the X-axis.
  • the coordinate conversion unit 82 uses the attitude information of the hydraulic excavator 1 output from the attitude calculation unit 81 to convert the coordinate system expressing the depth information acquired by the external world measurement device 70 from the sensor coordinate system 300 of the hydraulic excavator 1. It is converted to the vehicle body coordinate system 400.
  • the depth information acquired by the external world measuring device 70 is given as a set of three-dimensional point data (ie, point group data) expressed in the sensor coordinate system 300.
  • Rsv is a rotation matrix from the sensor coordinate system 300 to the vehicle body coordinate system 400.
  • ⁇ s, ⁇ s, and ⁇ s are angles formed by each axis of the external world measuring device 70 (each axis of the sensor coordinate system 300) in the vehicle body coordinate system 400.
  • these angles can be determined by, for example, measuring the attitude of the external world measuring device 70 in the vehicle body coordinate system 400 in advance and storing it in the storage device 57 in advance.
  • a posture measuring sensor is attached to the external world measuring device 70, and a coordinate transformation matrix is used using the angle detected by the posture measuring sensor. may be calculated.
  • ⁇ sw is a turning angle of the upper rotating body 7, and is output from the attitude calculation unit 81.
  • Tsv is a translation vector from the origin of the vehicle body coordinate system 400 to the origin of the sensor coordinate system 300.
  • Lsx, Lsy, and Lsz are equal to the origin coordinates of the sensor coordinate system 300 viewed from the vehicle body coordinate system 400.
  • the mounting position of the external world measuring device 70 is often fixed with respect to the hydraulic excavator 1. Therefore, in that case, the attachment position of the external world measurement device 70 to the hydraulic excavator 1 may be measured in advance and stored in the storage device 57 in advance.
  • the positioning information calculation unit 83 calculates the position of the origin of the vehicle body coordinate system 400 in the site coordinate system 500 of the hydraulic excavator 1 and the orientation ⁇ dir of the front work device 2 in the site coordinate system 500 from the positioning information acquired by the positioning device 60. calculate.
  • the task acquisition unit 80 acquires the next task to be performed by the hydraulic excavator 1 from the task acquisition device 58.
  • the tasks performed by the hydraulic excavator 1 include an excavation task, a loading task, and a reaching task.
  • the excavation task is a task that specifies an excavation operation in which the hydraulic excavator 1 excavates the ground to be excavated and holds an object such as earth and sand in the bucket 10.
  • the loading task is a task that specifies a loading operation from the state where the hydraulic excavator 1 has finished excavating to moving the bucket 10 above the vessel of the dump truck 200 and discharging the object from the bucket 10 to the vessel. .
  • the reaching task is a task for specifying a reaching operation in which the hydraulic excavator 1 moves the bucket 10 from the state where the loading operation has been completed to the position where the next ground to be excavated is located.
  • the tasks performed by the hydraulic excavator 1 are not limited to these, and for example, a ground leveling task in which the hydraulic excavator 1 specifies a leveling operation to level the surrounding ground until the dump truck 200 stops at a predetermined position on the site.
  • the excavator 1 may include a ground leveling task that specifies a movement operation when changing the ground to be excavated.
  • the recognition unit 84 recognizes the dump truck 200 from the measurement results of the external world measuring device 70. Specifically, the recognition unit 84 determines the position, orientation, and shape of the dump truck 200 using point cloud data acquired as the measurement results of the external world measuring device 70 and converted into the vehicle body coordinate system 400 by the coordinate conversion unit 82. Recognize. Then, the recognition unit 84 calculates a recognition area that is an area where the recognized dump truck 200 exists. That is, the recognition area is the recognition result of the recognition unit 84 regarding the presence area 210 of the dump truck 200.
  • a three-dimensional mesh model obtained by measuring the dump truck 200 is stored in the storage device 57 in advance. Then, the position, orientation, and shape are compared between the point group data converted into the vehicle body coordinate system 400 obtained from the coordinate conversion unit 82 and the three-dimensional mesh model. Thereby, the recognition unit 84 can calculate the position, orientation, and shape of the target dump truck 200, and therefore can calculate the recognition area of the dump truck 200.
  • the method of calculating the recognition area of the dump truck 200 is not limited to this, and for example, the recognition of the dump truck 200 is performed by extracting a specific plane of the dump truck 200 from point cloud data obtained from the external world measuring device 70.
  • a method of calculating the area may also be used.
  • a neural network that calculates the recognition area of the dump truck 200 from point cloud data including the dump truck 200 may be constructed in advance, and the features of the dump truck 200 to be recognized may be machine learned.
  • a method of calculating using a device may also be used.
  • the position, attitude, and presence area 210 of the dump truck 200 are defined as follows.
  • FIG. 8 is a side view of the vehicle body coordinate system 600 of the dump truck 200.
  • FIG. 9 is a diagram of the vehicle body coordinate system 600 shown in FIG. 8 viewed from above.
  • the vehicle body coordinate system 600 of the dump truck 200 has the center of the rear wheel axle of the dump truck 200 as its origin, the X axis is the direction from the rear wheel to the front wheel, the Y axis is the direction in which the rear wheel axle extends, and the dump truck 200 Let the height direction be the Z axis. It is assumed that the position of the dump truck 200 points to the origin Pd (Xd, Yd, Zd) of the vehicle body coordinate system 600.
  • the attitude of the dump truck 200 refers to an angle ⁇ d ( ⁇ roll, ⁇ pitch, ⁇ yaw) formed by each axis of the site coordinate system 500 and each axis of the vehicle body coordinate system 600 of the dump truck 200.
  • the existence area 210 of the dump truck 200 refers to a rectangular area Sd (Pd1 to Pd4) whose vertices are points Pd1 to Pd4 at the four corners of the vessel of the dump truck 200.
  • the area 210 of the dump truck 200 is not limited to this, and may be, for example, a hexahedron (for example, a rectangular parallelepiped or a cube) that includes the entire dump truck 200.
  • the recognition area of the dump truck 200 which is the recognition result of the recognition unit 84 regarding the existence area 210 of the dump truck 200, is Sdr (Pdr1 to Pdr4).
  • the area estimating unit 85 calculates an estimated area, which is an area where the dump truck 200 is estimated to exist, from the position and vehicle class information (truck information) of the dump truck 200 acquired by the truck information acquisition device 56. That is, the estimated area is the estimation result of the area estimating unit 85 regarding the area 210 where the dump truck 200 exists. The estimated area is calculated for comparison with the recognition area of the dump truck 200 calculated by the recognition unit 84 in order to verify the validity of the recognition result of the dump truck 200.
  • truck information vehicle class information
  • the method of calculating the estimated area of the dump truck 200 by the area estimation unit 85 is, for example, a method of calculating the estimated area of the dump truck 200 from truck information, which is stored in advance in the storage device 57 for each vehicle type. Then, using the truck information acquired by the truck information acquisition device 56 and the calculation method corresponding to the truck information, an estimated area of the target dump truck 200 in the site coordinate system 500 is calculated, and the estimated area of the target dump truck 200 is calculated. It is converted to the vehicle body coordinate system 400. Thereby, the area estimation unit 85 can calculate an estimated area of the dump truck 200 that can be compared with the recognition area of the dump truck 200 calculated by the recognition unit 84. In this embodiment, the estimated area of the dump truck 200, which is the estimation result of the area estimating unit 85 regarding the existing area 210 of the dump truck 200, is assumed to be Sde (Pde1 to Pde4).
  • Conversion from the site coordinate system 500 to the vehicle body coordinate system 400 of the hydraulic excavator 1 can be performed using the following equation.
  • point data Pv (Xv, Yv, Zv) in the vehicle body coordinate system 400 of the hydraulic excavator 1 to point data Pg (Xg, Yg, Zg) in the site coordinate system 500 for example, the following equation (4) is used.
  • ⁇ Equation (6) is used.
  • Rvg is a rotation matrix from the vehicle body coordinate system 400 to the site coordinate system 500.
  • ⁇ r, ⁇ p, and ⁇ y are angles formed by each axis of the vehicle body coordinate system 400 in the site coordinate system 500. The angles formed by these can be calculated using the value calculated by the attitude calculation section 81, the turning angle, the inclination angle of the machine body 3, and the direction calculated by the positioning information calculation section 83.
  • Tvg is a translation vector from the origin of the site coordinate system 500 to the origin of the vehicle body coordinate system 400.
  • x0, y0, and z0 are equal to the origin coordinates of the vehicle body coordinate system 400 viewed from the site coordinate system 500.
  • the calculation result of the positioning information calculation unit 83 can be used for Tvg.
  • the area estimation unit 85 can adjust the position of the dump truck 200 included in the truck information using the position adjustment amount of the dump truck 200 acquired simultaneously with the truck information by the truck information acquisition device 56. Then, the area estimation unit 85 can calculate the estimated area of the dump truck 200 from the adjusted position of the dump truck 200.
  • the verification unit 86 verifies the validity of the recognition result of the dump truck 200 based on the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85. Specifically, the verification unit 86 calculates the accuracy of the recognition result of the dump truck 200 by comparing the recognition area with the estimated area, and determines the validity of the recognition result of the dump truck 200 according to the calculated accuracy. Verify. Specifically, the verification unit 86 calculates the degree of overlap, which indicates the degree of overlap between the recognition area and the estimated area, as the accuracy of the recognition result of the dump truck 200. Then, when the calculated degree of polymerization is higher than the threshold value, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (correctable). At this time, the verification unit 86 can change the threshold of the degree of overlap, which is a criterion for determining the validity of the recognition result, according to the task acquired by the task acquisition unit 80.
  • the correction unit 87 corrects the recognition area calculated by the recognition unit 84 based on the estimated area calculated by the area estimation unit 85. Specifically, the correction unit 87 compares the recognition area calculated by the recognition unit 84 with the estimated area calculated by the area estimation unit 85, and corrects the recognition area calculated by the recognition unit 84. Accordingly, the correction unit 87 corrects the recognition result of the dump truck 200. At this time, the correction unit 87 can change the method of correcting the recognition area (recognition result) calculated by the recognition unit 84, depending on the task acquired by the task acquisition unit 80.
  • the information processing device 54 uses the verification unit 86 and the correction unit 87 to perform a recognition result verification process to verify the validity of the recognition result of the dump truck 200 and correct the recognition result.
  • the recognition result verification process will be described in detail below.
  • FIG. 10 is a flowchart illustrating an example of recognition result verification processing.
  • FIG. 11 is a diagram illustrating the degree of polymerization of the recognition area and the estimation area.
  • FIG. 12 is a diagram showing a table in which criteria for determining the validity of recognition results and a method for correcting the recognition area are determined for each task of the hydraulic excavator 1.
  • FIG. 13 is a diagram illustrating a method of correcting a recognition area when acquiring a loading task.
  • FIG. 14 is a diagram illustrating a method of correcting a recognition area when acquiring a reaching task.
  • FIG. 15 is a diagram showing a display example of the display device 55 when the accuracy of the recognition result is less than or equal to the threshold value.
  • step S121 the verification unit 86 of the information processing device 54 acquires the task acquired by the task acquisition unit 80.
  • step S122 the verification unit 86 of the information processing device 54 acquires the recognition area calculated by the recognition unit 84.
  • step S123 the verification unit 86 of the information processing device 54 obtains the estimated area calculated by the area estimation unit 85.
  • step S124 the verification unit 86 of the information processing device 54 calculates the accuracy of the recognition result of the dump truck 200. As shown in FIG. 11, in the present embodiment, the verification unit 86 determines the accuracy of the recognition result of the dump truck 200 between the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85. A degree of polymerization Acover indicating the degree of overlap is calculated. The degree of polymerization Acover is given by the following formula (7).
  • Sde indicates the area of the estimated region calculated by the region estimation unit 85.
  • Scover indicates the area of a region where the recognition region (Sdr) calculated by the recognition section 84 and the estimation region (Sde) calculated by the region estimation section 85 overlap. For example, Scover can be reduced to a problem of finding the area of an intersection area between two convex polygons, and therefore can be solved as a numerical calculation problem.
  • step S125 the verification unit 86 of the information processing device 54 determines whether the calculated degree of polymerization Acover is higher than the threshold value Ath. Thereby, the verification unit 86 verifies the validity of the recognition result of the dump truck 200. If the calculated degree of polymerization Acover is higher than the threshold Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (correctable), and proceeds to step S126. If the calculated degree of polymerization Acover is less than or equal to the threshold value Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is not valid (cannot be corrected), and proceeds to step S127.
  • the verification unit 86 can change the threshold Ath of the degree of polymerization Acover, which is a criterion for determining the validity of the recognition result, according to the task acquired by the task acquisition unit 80. .
  • the table shown in FIG. 12 is stored in the storage device 57 in advance. As shown in column 1201 of the table shown in FIG. 12, when the task acquired by the task acquisition unit 80 is a loading task, the verification unit 86 sets Ath1 as the threshold value Ath. When the task acquired by the task acquisition unit 80 is a reaching task, the verification unit 86 sets Ath2 as the threshold value Ath.
  • Ath1 and Ath2 are based on the calculation accuracy of each area of the area estimation unit 85 and recognition unit 84 at the time of functional verification performed in advance, the presence or absence of an object in the bucket 10 at the time of task acquisition, and the accuracy of the front work device 2 and dump truck 200. It is desirable that the distance be determined in consideration of distance, etc.
  • the hydraulic excavator 1 starts with the object in the bucket 10 and the distance between the front working device 2 and the dump truck 200 is large. As a result, it is considered that the damage caused when the front working device 2 collides with the dump truck 200 is greater during the loading operation than during the reaching operation. Therefore, it is desirable that the threshold value Ath1 that is set when a loading task is acquired is the same value as or greater than the threshold value Ath2 that is set when a leeching task is acquired.
  • step S126 the correction unit 87 of the information processing device 54 corrects the recognition result of the dump truck 200. Specifically, the correction unit 87 corrects the recognition area calculated by the recognition unit 84. Then, the correction unit 87 sends the determination result that the recognition result of the dump truck 200 is appropriate (correctable), the accuracy of the recognition result (degree of superposition), and the corrected recognition area to the control device 40. Output. Further, the correction unit 87 displays the information outputted to the control device 40 on the display device 55 to notify the operator, or transmits the information to a management device that manages the work site to notify the manager. It's okay. After that, the information processing device 54 ends the recognition result verification process shown in FIG.
  • the correction unit 87 can change the method of correcting the recognition area calculated by the recognition unit 84 according to the task acquired by the task acquisition unit 80.
  • the correction unit 87 converts the recognition area calculated by the recognition unit 84 into the recognition area shown in FIG.
  • the correction is made so that the area (Sdc) includes the recognition area (Sdr) and the estimated area (Sde).
  • the area that includes the recognition area (Sdr) and the estimation area (Sde) is an area (Sdc) that includes the OR area occupied by the recognition area (Sdr) or the estimation area (Sde).
  • the correction unit 87 defines the recognition area (Sdr) calculated by the recognition unit 84 as a rectangular area that includes the recognition area (Sdr) and the estimated area (Sde), and whose area is Correction is made to the minimum area (Sdc).
  • the hydraulic excavator 1 moves the bucket 10 from the ground outside the dump truck 200 toward the upper side of the vessel inside the dump truck 200 to the loading position. Therefore, when the recognition area calculated by the recognition unit 84 is corrected to include the recognition area and the estimated area, the front working device 2, which moves from the outside to the inside of the dump truck 200 during the loading operation, The risk of colliding with the dump truck 200 is reduced. Therefore, when the task acquired by the task acquisition unit 80 is a loading task, the correction unit 87 corrects the recognition area calculated by the recognition unit 84 so that it becomes an area that includes the recognition area and the estimated area. do.
  • the correction unit 87 can calculate the position of the corrected recognition area in the height direction (Z-axis direction) as follows. That is, in order to avoid a collision between the front working device 2 and the dump truck 200, the correction unit 87 adjusts the maximum value of the Z-axis component of each vertex coordinate of the recognition area and the estimation area to the height direction of the recognition area after correction. Can be used as a position.
  • the correction unit 87 calculates the corrected orientation vdc of the recognition area using the orientation vector vdr of the recognition area and the orientation vector vde of the estimation area, for example, from the following equations (8) to (9). Can be calculated.
  • ⁇ dr ( ⁇ xdr, ⁇ ydr, ⁇ zdr) used to calculate the gain Gk in the above equation (9) is information regarding the calculation accuracy of the recognition area in the recognition unit 84.
  • ⁇ dr for example, the measurement accuracy of the external world measurement device 70 and the calculation accuracy of the recognition unit 84 can be measured in advance.
  • information on recognition accuracy output by the discriminator may be used as ⁇ dr.
  • ⁇ de ( ⁇ xde, ⁇ yde, ⁇ zde) used to calculate the gain Gk in the above equation (9) is information regarding the calculation accuracy of the estimated region in the region estimation unit 85.
  • the position adjustment amount of the dump truck 200 that can be acquired by the truck information acquisition device 56 at the same time as the truck information can be used. That is, the correction unit 87 can correct the recognition area calculated by the recognition unit 84 using the position adjustment amount of the dump truck 200 acquired by the truck information acquisition device 56.
  • the track information acquisition device 56 cannot acquire the position adjustment amount, the result of a preliminary investigation of the error in the positioning information acquired by the positioning device 60 at the site of the hydraulic excavator 1 can be used as ⁇ de.
  • the correction unit 87 calculates the loading position Pdc (Xdc, Ydc, Zdc) of the bucket 10. Specifically, the correction unit 87 calculates, for example, the center position Pdr0 of the vessel in the recognition area calculated by the recognition unit 84 and the center position Pde0 of the vessel in the estimation area calculated by the area estimation unit 85. Then, the correction unit 87 can calculate, for example, from the following equation (10) using the center position Pdr0 of the vessel in the recognition area and the center position Pde0 of the vessel in the estimation area.
  • the correction unit 87 can calculate the loading position of the bucket 10 based on the position adjustment amount of the dump truck 200 acquired by the truck information acquisition device 56, the recognition area, and the estimated area.
  • the correction unit 87 outputs the calculated loading position of the bucket 10 to the control device 40.
  • the correction unit 87 estimates the recognition area (Sdr) calculated by the recognition unit 84 to be a recognition area (Sdr) as shown in FIG. Correction is made so that the area (Sdc) is included in the area overlapping with the area (Sde).
  • the area (Sdc) included in the area where the recognition area (Sdr) and the estimation area (Sde) overlap is the area (Sdc) included in the AND area occupied by both the recognition area (Sdr) and the estimation area (Sde). It is.
  • the correction unit 87 defines the recognition area (Sdr) calculated by the recognition unit 84 as a rectangular area that includes the recognition area (Sdr) and the estimated area (Sde), and whose area is Correction is made to the maximum area (Sdc).
  • the hydraulic excavator 1 moves the bucket 10 from the loading position above the vessel inside the dump truck 200 to the ground outside the dump truck 200. Therefore, when the recognition area calculated by the recognition unit 84 is corrected to an area included in the area where the recognition area and the estimated area overlap, the front work that moves from the inside to the outside of the dump truck 200 during the reaching operation The risk of the device 2 colliding with the dump truck 200 is reduced. Therefore, when the task acquired by the task acquisition unit 80 is a reaching task, the correction unit 87 sets the recognition area calculated by the recognition unit 84 to be an area included in the area where the recognition area and the estimated area overlap. Correct it as follows.
  • the correction unit 87 uses the Z-axis component of each vertex coordinate of the recognition area and the estimated area as the position in the height direction (Z-axis direction) of the recognition area after correction. The maximum value of can be used. Further, the correction unit 87 can calculate the orientation of the corrected recognition area from the above equations (8) to (9), as in the case where the loading task is acquired.
  • step S127 the verification unit 86 of the information processing device 54 determines that the recognition result of the dump truck 200 is invalid (impossible to correct), the accuracy of the recognition result (degree of superimposition), the recognition area, and the estimated area. Output to the control device 40. Furthermore, the verification unit 86 displays the information output to the control device 40 on the display device 55 to notify the operator, or sends it to the management device that manages the work site to notify the manager. . At this time, the verification unit 86 confirms with the user, such as the operator or administrator, whether or not the operation of the hydraulic excavator 1 may be continued.
  • the verification unit 86 notifies the user of the determination result that the recognition result of the dump truck 200 is not valid (cannot be corrected), and also informs the user whether or not the operation of the hydraulic excavator 1 can be continued.
  • a screen 551 to be confirmed is displayed on the display device 55.
  • the screen 551 includes a message 552 for notifying the user of the determination result that the recognition result of the dump truck 200 is invalid (impossible to correct) and for confirming with the user whether or not the operation of the hydraulic excavator 1 can be continued.
  • the screen 551 includes a button 553 for the user to input a confirmation result that the operation of the hydraulic excavator 1 will be continued.
  • Screen 551 includes a button 554 for the user to input a confirmation result that the operation of hydraulic excavator 1 is not to be continued.
  • Screen 551 includes a button 555 for the user to input switching the hydraulic excavator 1 from automatic control to manual operation by the operator. Further, the screen 551 shows the calculation results of the recognition area (Sdr) and the estimation area (Sde) of the dump truck 200, and notifies the user of the positional relationship between the dump truck 200 and the hydraulic excavator 1.
  • the verification unit 86 Upon receiving the confirmation result from the user as to whether or not continuation is possible, the verification unit 86 outputs the confirmation result to the control device 40. After that, the information processing device 54 ends the recognition result verification process shown in FIG.
  • the control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 (for example, the rotational movement of the front working device 2, the lower traveling body 5 (traveling operation of the upper revolving structure 7 and turning operation of the upper revolving structure 7).
  • the control device 40 performs the following based on the task acquired by the task acquisition device 58 and the corrected recognition area output from the information processing device 54: Controls the operation of the hydraulic excavator 1. Specifically, when the loading task is acquired, the control device 40 causes the front working device 2 to move on a trajectory such that the front working device 2 does not collide with the dump truck 200 before the bucket 10 reaches the loading position from the ground. The loading operation of the hydraulic excavator 1 is controlled so as to do so. When the reaching task is acquired, the control device 40 moves on a trajectory such that the front working device 2 does not collide with the dump truck 200 before the bucket 10 reaches the next excavation target ground from the loading position. Thus, the reaching operation of the hydraulic excavator 1 is controlled.
  • the control device 40 responds to the confirmation result. to control the operation of the hydraulic excavator 1.
  • the hydraulic excavator 1 of the first embodiment is a working machine that loads objects onto the dump truck 200.
  • the hydraulic excavator 1 includes an external world measuring device 70 that measures the surrounding environment of the vehicle body of the hydraulic excavator 1 .
  • the hydraulic excavator 1 includes an information processing device 54 that recognizes dump trucks 200 existing around the vehicle body of the hydraulic excavator 1 based on the measurement results of the external world measuring device 70.
  • the hydraulic excavator 1 includes a control device 40 that controls the operation of the vehicle body of the hydraulic excavator 1 based on the recognition result of the information processing device 54.
  • the hydraulic excavator 1 includes a truck information acquisition device 56 that acquires the position and vehicle class information of the dump truck 200 from the outside.
  • the information processing device 54 corrects the recognition result of the dump truck 200 based on the position and vehicle class information of the dump truck 200 acquired by the truck information acquisition device 56.
  • the control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 based on the corrected recognition result of the dump truck 200.
  • the hydraulic excavator 1 of the first embodiment can control the loading operation, etc. for the dump truck 200 using accurate recognition results of the dump truck 200 even in various situations. Therefore, according to the first embodiment, it is possible to provide a working machine that can appropriately control the loading work on the transport machine by accurately recognizing the position and orientation of the transport machine even in various situations. I can do it.
  • the hydraulic excavator 1 of the first embodiment verifies the validity of the recognition result of the dump truck 200 based on information from a third party acquired by the truck information acquisition device 56, and corrects the recognition result. be able to.
  • the hydraulic excavator 1 of the first embodiment can appropriately control the loading operation and the like based on the corrected recognition result. That is, the hydraulic excavator 1 according to the first embodiment can control the operation of the vehicle body of the hydraulic excavator 1 based on accurate recognition results such as the position and posture of the dump truck 200. Therefore, the hydraulic excavator 1 of the first embodiment can control the loading operation and the reaching operation while reducing the risk of collision with the dump truck 200, thereby improving the safety and productivity of the loading operation. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
  • the information processing device 54 includes a recognition unit 84 that recognizes the dump truck 200 from the measurement results of the external world measurement device 70 and calculates a recognition area where the recognized dump truck 200 exists.
  • the information processing device 54 includes an area estimation unit 85 that calculates an estimated area where the dump truck 200 is estimated to exist based on the position and vehicle class information of the dump truck 200 acquired by the truck information acquisition device 56.
  • the information processing device 54 includes a correction unit 87 that corrects the recognition area based on the estimated area.
  • the hydraulic excavator 1 of the first embodiment can correct the recognition result based on a relatively simple index of the area where the dump truck 200 exists. Therefore, the hydraulic excavator 1 of the first embodiment can accurately and easily recognize the position and orientation of the dump truck 200.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be easily provided.
  • the information processing device 54 calculates the accuracy of the recognition result of the dump truck 200 by comparing the recognition area with the estimated area.
  • the correction unit 87 corrects the recognition area according to the calculated accuracy.
  • the hydraulic excavator 1 of the first embodiment can quantitatively verify the validity of the recognition result of the dump truck 200, and can accurately and finely correct the recognition area.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, by accurately and precisely recognizing the position and posture of the dump truck 200 even in various situations, the hydraulic pressure that can more appropriately control the loading operation on the dump truck 200 is realized.
  • a shovel 1 can be provided.
  • the information processing device 54 calculates the degree of overlap, which indicates the degree of overlap between the recognition area and the estimated area, as the accuracy of the recognition result.
  • the correction unit 87 corrects the recognition area according to the calculated degree of superposition.
  • the hydraulic excavator 1 of the first embodiment can calculate the accuracy of the recognition result based on a clear standard, and can correct the recognition area more accurately and finely.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, by recognizing the position and posture of the dump truck 200 more accurately and precisely in various situations, it is possible to more appropriately control the loading operation on the dump truck 200.
  • a hydraulic excavator 1 can be provided.
  • the truck information acquisition device 56 further acquires information regarding the amount of position adjustment of the dump truck 200.
  • the area estimation unit 85 calculates an estimated area using the position adjustment amount acquired by the track information acquisition device 56.
  • the hydraulic excavator 1 of the first embodiment can calculate the estimated area using a value that is more in line with reality than the error in the position information of the dump truck 200 obtained through preliminary verification or the like. Therefore, the hydraulic excavator 1 of the first embodiment can correct the recognition area more accurately.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator is capable of more accurately controlling the loading operation on the dump truck 200 by more accurately recognizing the position and posture of the dump truck 200 in various situations. 1 can be provided.
  • the information processing device 54 further includes a task acquisition unit 80 that acquires the next task to be performed by the hydraulic excavator 1.
  • the correction unit 87 changes the recognition area correction method according to the acquired task.
  • the hydraulic excavator 1 of Embodiment 1 can correct the recognition area to the optimum recognition area for each acquired task, so the hydraulic excavator 1 can perform the optimum operation for each task.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator is capable of more accurately controlling the loading operation on the dump truck 200 by more accurately recognizing the position and posture of the dump truck 200 in various situations. 1 can be provided.
  • the correction unit 87 corrects the recognition area so that it becomes an area that includes the recognition area and the estimation area.
  • the hydraulic excavator 1 of the first embodiment the risk of the front working device 2 moving from the outside to the inside of the dump truck 200 colliding with the dump truck 200 during the loading operation can be further reduced.
  • the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
  • the hydraulic excavator 1 further includes a bucket 10 that holds an object.
  • the truck information acquisition device 56 further acquires the position adjustment amount of the dump truck 200.
  • the correction unit 87 performs a dump operation from the bucket 10 based on the position adjustment amount acquired by the truck information acquisition device 56, the recognition area, and the estimated area.
  • the position of the bucket 10 when loading objects onto the truck 200 is calculated.
  • the control device 40 controls the loading operation of the hydraulic excavator 1 based on the position of the bucket 10 calculated by the correction section 87.
  • the hydraulic excavator 1 of the first embodiment can set the control target value of the control device 40, which is the loading position of the bucket 10, to a value with higher validity. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
  • the correction unit 87 recognizes the area so that it is included in the area where the recognition area and the estimated area overlap. Correct the area.
  • the hydraulic excavator 1 of the first embodiment can further reduce the risk of the front working device 2 moving from the inside to the outside of the dump truck 200 colliding with the dump truck 200 during the reaching operation. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
  • the information processing device 54 determines that the recognition result of the dump truck 200 cannot be corrected, the information processing device 54 (verification unit 86) notifies the user of the determination result that the recognition result cannot be corrected, and also adjusts the operation of the vehicle body of the hydraulic excavator 1.
  • a screen 551 is displayed on the display device 55 for asking the user whether or not to continue.
  • the information processing device 54 receives a confirmation result from the user as to whether or not continuation is possible
  • the information processing device 54 outputs the confirmation result to the control device 40.
  • the control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 according to the confirmation result.
  • the hydraulic excavator 1 of the first embodiment can continue to operate the hydraulic excavator 1 under appropriate judgment by asking the operator or manager for instructions. Therefore, according to the first embodiment, it is possible to provide the hydraulic excavator 1 that can appropriately control the loading work on the dump truck 200 even when the recognition result of the dump truck 200 cannot be corrected. can.
  • Embodiment 2 The working machine of Embodiment 2 will be described using FIGS. 16 and 17. In the work machine of Embodiment 2, descriptions of the same configurations and operations as in Embodiment 1 will be omitted.
  • FIG. 16 is a block diagram illustrating the functional configuration of the information processing device 54 of the second embodiment.
  • FIG. 17 is a diagram illustrating the processing of the truck attitude estimation unit 88 shown in FIG. 16.
  • the truck information acquisition device 56 of the first embodiment acquires truck information including the position, attitude, and vehicle class information of the dump truck 200.
  • the attitude of the dump truck 200 may not be managed.
  • the hydraulic excavator 1 of the second embodiment estimates the attitude of the dump truck 200 from topographical information around the hydraulic excavator 1.
  • the hydraulic excavator 1 of the second embodiment includes a topographic information acquisition device 59 that acquires topographic information around the hydraulic excavator 1, as shown in FIG.
  • the information processing device 54 of the second embodiment includes a truck attitude estimation unit 88 that estimates the attitude of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59.
  • the area estimating unit 85 of the second embodiment calculates an estimated area using the attitude of the dump truck 200 estimated by the truck attitude estimating unit 88.
  • the terrain information acquisition device 59 may be attached to the upper revolving body 7 of the hydraulic excavator 1.
  • the terrain information acquisition device 59 may share a sensor with the external world measurement device 70 to measure the terrain around the hydraulic excavator 1, and the information processing device 54 may acquire the terrain information from the measurement results.
  • the topographic information acquisition device 59 may have a wireless communication function and may obtain topographic information by receiving topographic information transmitted from an external system that measures the topography of the site. In this embodiment, there are no particular limitations on the method of acquiring terrain information.
  • the topographical information in this embodiment is given as point cloud data.
  • the terrain information is not limited to point cloud data, and may be data in a grid format, for example, in which the area around the hydraulic excavator 1 is divided into grids and height information is held for each grid.
  • the truck attitude estimation unit 88 acquires the position, orientation, and vehicle size information of the dump truck 200 included in the truck information acquired by the truck information acquisition device 56 and the terrain information acquired by the terrain information acquisition device 59.
  • the truck attitude estimating unit 88 extracts topographical information within the presence area 210 of the dump truck 200 based on the acquired information.
  • the truck attitude estimation unit 88 estimates the plane of the terrain within the existence area 210 from the extracted terrain information. For example, least squares approximation can be used for plane estimation. Then, the truck attitude estimating unit 88 estimates the attitude of the dump truck 200 by calculating the inclination angle of the plane obtained by plane estimation.
  • the truck attitude estimation unit 88 indicates the attitude of the dump truck 200 from the angle formed by the normal vector nt of the plane obtained by the plane estimation and the Y axis and the Z axis of the site coordinate system 500. Calculate angles ⁇ roll and ⁇ pitch.
  • the area estimation unit 85 of the second embodiment is based on the position, orientation, and vehicle size information of the dump truck 200 acquired by the truck information acquisition device 56 and the attitude of the dump truck 200 estimated by the truck attitude estimation unit 88. , calculates the estimated area of the dump truck 200.
  • the method of calculating the estimated area is the same as in the first embodiment.
  • the hydraulic excavator 1 of the second embodiment further includes the terrain information acquisition device 59.
  • the information processing device 54 of the second embodiment includes a truck attitude estimation unit 88 that estimates the attitude of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59.
  • the area estimating unit 85 of the second embodiment calculates an estimated area using the attitude of the dump truck 200 estimated by the truck attitude estimating unit 88.
  • the hydraulic excavator 1 of the second embodiment can accurately correct the recognition area even if the dump truck 200 position/dispatch management system does not manage the attitude of the dump truck 200. Therefore, according to the second embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
  • Embodiment 3 The hydraulic excavator 1 of Embodiment 3 will be described using FIG. 18. In the hydraulic excavator 1 of Embodiment 3, descriptions of the same configuration and operation as in Embodiment 1 will be omitted.
  • FIG. 18 is a block diagram illustrating the functional configuration of the information processing device 54 of the third embodiment.
  • the information processing device 54 of the third embodiment recognizes the dump truck 200 using only the point group data of the dump truck 200 and its surroundings among the point group data acquired by the external world measurement device 70.
  • the information processing device 54 of the third embodiment extracts point cloud data within a predetermined region including the estimated region calculated by the region estimation unit 85 from the point cloud data acquired by the external world measuring device 70. It further includes a filter section 89.
  • the recognition unit 84 of the third embodiment recognizes the dump truck 200 from the point cloud data extracted by the filter unit 89 and calculates the recognition area of the dump truck 200. The method of calculating the recognition area is the same as in the first embodiment.
  • the point cloud data acquired by the external measurement device 70 includes data from sources other than the dump truck 200 (other work machines or the ground), it may generally cause a decrease in the accuracy of the recognition process or an increase in the processing time. There is sex.
  • the hydraulic excavator 1 according to the third embodiment can use the estimated area calculated by the area estimating section 85 to remove point cloud data that causes disturbance. Thereby, the hydraulic excavator 1 of the third embodiment can improve the accuracy of recognition processing and shorten processing time. Therefore, according to the third embodiment, by accurately and quickly recognizing the position and orientation of the dump truck 200 even in various situations, it is possible to appropriately and quickly control the loading work on the dump truck 200.
  • a hydraulic excavator 1 can be provided.
  • the present invention is not limited to the above-described embodiments, and includes various modifications.
  • the above embodiments have been described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
  • each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Further, each of the above-mentioned configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tapes, and files that implement each function can be stored in a memory, a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, SD card, or DVD.
  • a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, SD card, or DVD.
  • control lines and information lines are shown that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.
  • SYMBOLS 1 Hydraulic excavator (work machine), 54... Information processing device, 55... Display device, 56... Truck information acquisition device (transportation machine information acquisition device), 59... Terrain information acquisition device, 70... External world measurement device, 80... Task Acquisition unit, 84... Recognition unit, 85... Area estimation unit, 86... Verification unit, 87... Correction unit, 88... Truck posture estimation unit, 89... Filter unit, 200... Dump truck (transportation machine), Sde... Estimation area, Sdr...recognition area

Landscapes

  • Engineering & Computer Science (AREA)
  • Mining & Mineral Resources (AREA)
  • Civil Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Structural Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Operation Control Of Excavators (AREA)
  • Image Analysis (AREA)

Abstract

Provided is a work machine that is capable of verifying validity of recognition results of a transportation device, and appropriately controlling loading work to the transportation device. A hydraulic power shovel 1 according to the present invention comprises: an environment measurement device 70 that measures the surrounding environment of the hydraulic power shovel 1; an information processing device 54 that recognizes a dump truck 200 that is present in the vicinity of the hydraulic power shovel 1 on the basis of measurement results of the environment measurement device 70; a control device 40 that controls operations of a body of the hydraulic power shovel 1 on the basis of recognition results of the information processing device 54; and a truck information acquisition device 56 that acquires position and size class information of the dump truck 200 by external communication. The information processing device 54 corrects recognition results of the dump truck 200 on the basis of information acquired by the truck information acquisition device 56. The control device 40 controls the operations of the body of the hydraulic power shovel 1 on the basis of recognition results of the dump truck 200 that are corrected.

Description

作業機械working machine
　本発明は、作業機械に関する。 The present invention relates to working machines.
　油圧アクチュエータにより駆動されるフロント作業装置（例えばブーム、アーム、及び、バケット等のアタッチメント）等を有する多関節型の作業機械（例えば油圧ショベル）が知られている。この種の作業機械は、掘削した土砂等の対象物を、被積込機械である運搬機械（例えばダンプトラック）に積み込む積込作業を行う。 Multi-jointed working machines (eg, hydraulic excavators) are known that have front working devices (eg, attachments such as booms, arms, and buckets) driven by hydraulic actuators. This type of working machine performs a loading operation in which excavated objects such as earth and sand are loaded onto a transporting machine (for example, a dump truck) that is a loaded machine.
　積込作業を行う際、フロント作業装置の高さ（例えば、バケットの高さ）が運搬機械より低い位置で旋回させると、フロント作業装置が運搬機械と衝突する可能性がある。そこで、積込作業を行う作業機械のオペレータの操作を支援する機能や、作業機械が自動で積込作業を行う技術が求められている。作業機械の積込作業を半自動又は全自動（以下「自動」と総称する）で行う際には、作業機械が運搬機械の位置及び姿勢を正確に認識する必要がある。 When performing loading work, if the front working device is turned at a position where the height (for example, the height of the bucket) is lower than the transporting machine, there is a possibility that the front working device will collide with the transporting machine. Therefore, there is a need for a function that supports the operation of the operator of a work machine that performs loading work, and a technology that allows the work machine to automatically perform the loading work. When loading a work machine semi-automatically or fully automatically (hereinafter collectively referred to as "automatic"), the work machine needs to accurately recognize the position and orientation of the transport machine.
　運搬機械を認識する従来技術としては、例えば、特許文献１に記載の技術がある。特許文献１には、作業機械の運搬物の積み下ろし対象が写る撮像画像を取得するデータ取得部と、前記撮像画像から前記積み下ろし対象を含む領域を特定する領域特定部と、前記積み下ろし対象を含む領域から前記積み下ろし対象の少なくとも一つの所定の面を特定する積み下ろし対象特定部と、を備える画像処理システムが開示されている。 An example of a conventional technology for recognizing a transport machine is the technology described in Patent Document 1. Patent Document 1 discloses a data acquisition unit that acquires a captured image showing a target for loading and unloading objects to be transported by a working machine, an area specifying unit that identifies an area including the target for loading and unloading from the captured image, and an area including the target for loading and unloading. An image processing system is disclosed, comprising: an unloading target specifying section that identifies at least one predetermined surface of the unloading target.
特開２０２０－１２６３６３号公報Japanese Patent Application Publication No. 2020-126363
　特許文献１に開示の技術は、運搬機械を特定するために、機械学習を用いた画像認識システムによって、作業機械に取り付けられたカメラの撮像画像から運搬機械の位置及び姿勢を検出している。しかしながら、機械学習を用いた認識システムでは、学習データを収集していない現場又は運搬機械に対して認識結果の妥当性及び認識精度を保証することが困難である。 The technology disclosed in Patent Document 1 uses an image recognition system using machine learning to detect the position and orientation of the transport machine from an image captured by a camera attached to the work machine, in order to identify the transport machine. However, in a recognition system using machine learning, it is difficult to guarantee the validity and recognition accuracy of recognition results for sites or transport machines for which learning data has not been collected.
　また、運搬機械の認識システムとして他社システムを組み込んだ場合、作業機械の開発会社に対して、認識システムのアルゴリズムの細部まで公開されない可能性がある。作業機械の開発会社は、認識システムの認識結果を検証することが容易ではない。作業機械の開発会社は、認識システムの認識結果を用いて運搬機械に衝突することなく適切に積込作業を制御することが可能であるか、事前に判断することが困難である。 Additionally, if another company's system is incorporated as a recognition system for a transport machine, there is a possibility that the details of the recognition system's algorithm will not be disclosed to the working machine development company. It is not easy for work machine development companies to verify the recognition results of recognition systems. It is difficult for work machine development companies to judge in advance whether it is possible to appropriately control loading operations without colliding with transport machines using the recognition results of recognition systems.
　本発明は、上記に鑑みてなされたものであり、種々の状況においても運搬機械の位置及び姿勢を正確に認識することで、運搬機械への積込作業を適切に制御することが可能な作業機械を提供することを目的とする。 The present invention has been made in view of the above, and provides work that enables appropriate control of loading work on a transport machine by accurately recognizing the position and orientation of the transport machine in various situations. The purpose is to provide machinery.
　上記課題を解決するために、本発明の作業機械は、対象物を運搬機械に積み込む作業機械であって、前記作業機械の車体の周辺環境を計測する外界計測装置と、前記外界計測装置の計測結果に基づいて前記車体の周辺に存在する前記運搬機械を認識する情報処理装置と、前記情報処理装置の認識結果に基づいて前記車体の動作を制御する制御装置と、外部から前記運搬機械の位置及び車格情報を取得する運搬機械情報取得装置と、を備え、前記情報処理装置は、前記運搬機械情報取得装置により取得された前記運搬機械の位置及び車格情報に基づいて前記運搬機械の認識結果を補正し、前記制御装置は、補正された前記運搬機械の認識結果に基づいて前記車体の動作を制御することを特徴とする。 In order to solve the above problems, a working machine of the present invention is a working machine that loads objects onto a transport machine, and includes an external world measuring device that measures the surrounding environment of a vehicle body of the working machine, and a measuring device of the external world measuring device. an information processing device that recognizes the transportation machine existing around the vehicle body based on the result; a control device that controls the operation of the vehicle body based on the recognition result of the information processing device; and a control device that controls the location of the transportation machine from the outside. and a transport machine information acquisition device that acquires vehicle class information, the information processing device recognizes the transport machine based on the position and vehicle class information of the transport machine acquired by the transport machine information acquisition device. The control device corrects the result and controls the operation of the vehicle body based on the corrected recognition result of the transport machine.
　本発明によれば、種々の状況においても運搬機械の位置及び姿勢を正確に認識することで、運搬機械への積込作業を適切に制御することが可能な作業機械を提供することができる。
　上記以外の課題、構成および効果は、以下の実施形態の説明により明らかにされる。 According to the present invention, it is possible to provide a working machine that can appropriately control loading work on a transporting machine by accurately recognizing the position and orientation of the transporting machine in various situations.
Problems, configurations, and effects other than those described above will be made clear by the description of the embodiments below.
実施形態１の作業機械の一例である油圧ショベルの外観構成を模式的に示す側面図。1 is a side view schematically showing the external configuration of a hydraulic excavator that is an example of a working machine according to a first embodiment; FIG. 図１に示す油圧ショベルに搭載された油圧システム及び制御システムの構成を示すブロック図。2 is a block diagram showing the configuration of a hydraulic system and a control system installed in the hydraulic excavator shown in FIG. 1. FIG. 図２に示す油圧ショベルの動作の一例を説明する図。3 is a diagram illustrating an example of the operation of the hydraulic excavator shown in FIG. 2. FIG. 図２に示す油圧ショベルの動作の一例を示すフローチャート。3 is a flowchart showing an example of the operation of the hydraulic excavator shown in FIG. 2. 図２に示す情報処理装置の機能的構成を説明するブロック図。3 is a block diagram illustrating the functional configuration of the information processing apparatus shown in FIG. 2. FIG. 油圧ショベルに設定される各座標系を側方から視た図。A side view of each coordinate system set on the hydraulic excavator. 図６に示す各座標系を上方から視た図。FIG. 7 is a diagram of each coordinate system shown in FIG. 6 viewed from above. ダンプトラックの車体座標系を側方から視た図。A side view of the body coordinate system of a dump truck. 図８に示す車体座標系を上方から視た図。FIG. 9 is a diagram of the vehicle body coordinate system shown in FIG. 8 viewed from above. 認識結果検証処理の一例を示すフローチャート。5 is a flowchart illustrating an example of recognition result verification processing. 認識領域及び推定領域の重合度を説明する図。FIG. 4 is a diagram illustrating the degree of polymerization of a recognition region and an estimated region. 油圧ショベルのタスク毎に、認識結果の妥当性の判定基準と認識領域の補正方法とを定めたテーブルを示す図。FIG. 3 is a diagram showing a table that defines criteria for determining validity of recognition results and methods for correcting recognition areas for each task of a hydraulic excavator. 積込タスク取得時の認識領域の補正方法を説明する図。The figure explaining the correction method of the recognition area at the time of loading task acquisition. リーチングタスク取得時の認識領域の補正方法を説明する図。The figure explaining the correction method of the recognition area at the time of reaching task acquisition. 認識結果の確度が閾値以下である場合の表示装置の表示例を示す図。The figure which shows the example of a display of a display device when the accuracy of a recognition result is below a threshold value. 実施形態２の情報処理装置の機能的構成を説明するブロック図。FIG. 3 is a block diagram illustrating the functional configuration of an information processing device according to a second embodiment. 図１６に示すトラック姿勢推定部の処理を説明する図。FIG. 17 is a diagram illustrating processing of the truck attitude estimating section shown in FIG. 16; 実施形態３の情報処理装置の機能的構成を説明するブロック図。FIG. 3 is a block diagram illustrating the functional configuration of an information processing apparatus according to a third embodiment.
　以下、本発明の実施形態について図面を用いて説明する。なお、各実施形態において同一の符号を付された構成については、特に言及しない限り、各実施形態において同様の機能を有し、その説明を省略する。 Hereinafter, embodiments of the present invention will be described using the drawings. In addition, unless otherwise mentioned, the configurations with the same reference numerals in each embodiment have the same functions in each embodiment, and the description thereof will be omitted.
　本実施形態の作業機械は、掘削した土砂等の対象物を、被積込機械である運搬機械に積み込む積込作業を行う作業機械である。以下では、作業機械としてバケットを備える油圧ショベル、運搬機械としてダンプトラックを例に挙げて説明する。しかし、本実施形態の作業機械は、バケット以外のアタッチメントを備える油圧ショベルであってもよいし、油圧ショベル以外の作業機械であってもよい。本実施形態の運搬機械は、ダンプトラック以外の運搬機械であってもよい。 The working machine of this embodiment is a working machine that performs a loading operation of loading an object such as excavated earth and sand onto a transport machine that is a loaded machine. In the following description, a hydraulic excavator equipped with a bucket is used as a working machine, and a dump truck is used as an example of a transport machine. However, the working machine of this embodiment may be a hydraulic excavator equipped with an attachment other than a bucket, or may be a working machine other than a hydraulic excavator. The transport machine of this embodiment may be a transport machine other than a dump truck.
［実施形態１］
　図１～図１５を用いて、実施形態１の油圧ショベル１について説明する。 [Embodiment 1]
A hydraulic excavator 1 according to a first embodiment will be described using FIGS. 1 to 15.
＜油圧ショベルの外観構成＞
　図１は、実施形態１の作業機械の一例である油圧ショベル１の外観構成を模式的に示す側面図である。 <Exterior configuration of hydraulic excavator>
FIG. 1 is a side view schematically showing the external configuration of a hydraulic excavator 1, which is an example of a working machine according to the first embodiment.
　油圧ショベル１は、対象物を保持して上下方向又は前後方向に回動する多関節型のフロント作業装置２と、フロント作業装置２を搭載する機械本体３とを備える。 The hydraulic excavator 1 includes an articulated front working device 2 that holds an object and rotates in the vertical or longitudinal direction, and a machine body 3 on which the front working device 2 is mounted.
　機械本体３は、下部走行体５の右部及び左部に設けられた走行右油圧モータ４ａ及び走行左油圧モータ４ｂにより走行する下部走行体５と、下部走行体５の上部に旋回装置を介して取り付けられ、旋回装置の旋回油圧モータ６により旋回する上部旋回体７とを備える。なお、本実施形態では、走行右油圧モータ４ａ及び走行左油圧モータ４ｂを総称して、「走行油圧モータ４ａ，４ｂ」ともいう。 The machine main body 3 includes a lower traveling body 5 that travels by a traveling right hydraulic motor 4a and a traveling left hydraulic motor 4b provided on the right and left parts of the lower traveling body 5, and a rotating device attached to the upper part of the lower traveling body 5. The upper rotating body 7 is attached to the upper rotating body 7 and is rotated by a rotating hydraulic motor 6 of a rotating device. In this embodiment, the right travel hydraulic motor 4a and the left travel hydraulic motor 4b are also collectively referred to as "travel hydraulic motors 4a, 4b."
　フロント作業装置２は、上部旋回体７の前部に取り付けられた複数のフロント部材によって構成された多関節型の作業装置である。上部旋回体７は、フロント作業装置２を搭載して旋回する。フロント作業装置２は、上部旋回体７の前部に上下方向に回動可能に連結されたブーム８と、ブーム８の先端部に上下方向に回動可能に連結されたアーム９と、アーム９の先端部に上下方向に回動可能に連結されたバケット１０とを含む。 The front working device 2 is an articulated working device configured by a plurality of front members attached to the front part of the upper revolving body 7. The upper revolving body 7 mounts the front working device 2 and rotates. The front working device 2 includes a boom 8 connected to the front part of the upper revolving body 7 so as to be rotatable in the vertical direction, an arm 9 connected to the tip of the boom 8 so as to be rotatable in the vertical direction, and an arm 9 . The bucket 10 is rotatably connected to the tip of the bucket 10 in the vertical direction.
　ブーム８は、ブームピン８ａによって上部旋回体７に連結され、ブームシリンダ１１の伸縮によって回動する。アーム９は、アームピン９ａによってブーム８の先端部に連結され、アームシリンダ１２の伸縮によって回動する。バケット１０は、バケットピン１０ａ及びバケットリンク１６によってアーム９の先端部に連結され、バケットシリンダ１３の伸縮によって回動する。 The boom 8 is connected to the upper revolving body 7 by a boom pin 8a, and rotates as the boom cylinder 11 expands and contracts. The arm 9 is connected to the tip of the boom 8 by an arm pin 9a, and rotates as the arm cylinder 12 expands and contracts. The bucket 10 is connected to the tip of the arm 9 by a bucket pin 10a and a bucket link 16, and rotates as the bucket cylinder 13 expands and contracts.
　ブームピン８ａには、ブーム８の回動角度を検出するブーム角度センサ１４が取り付けられている。アームピン９ａには、アーム９の回動角度を検出するアーム角度センサ１５が取り付けられている。バケットリンク１６には、バケット１０の回動角度を検出するバケット角度センサ１７が取り付けられている。 A boom angle sensor 14 that detects the rotation angle of the boom 8 is attached to the boom pin 8a. An arm angle sensor 15 for detecting the rotation angle of the arm 9 is attached to the arm pin 9a. A bucket angle sensor 17 that detects the rotation angle of the bucket 10 is attached to the bucket link 16 .
　なお、ブーム８、アーム９及びバケット１０の各回動角度は、水平面等の基準面に対するブーム８、アーム９及びバケット１０の各角度を慣性計測装置により検出し、各回動角度に換算することによって取得されてもよい。また、ブーム８、アーム９及びバケット１０の各回動角度は、ブームシリンダ１１、アームシリンダ１２及びバケットシリンダ１３の各ストロークをストロークセンサにより検出し、各回動角度に換算することによって取得されてもよい。 Note that each rotation angle of the boom 8, arm 9, and bucket 10 is obtained by detecting each angle of the boom 8, arm 9, and bucket 10 with respect to a reference plane such as a horizontal plane using an inertial measurement device, and converting it into each rotation angle. may be done. Further, each rotation angle of the boom 8, arm 9, and bucket 10 may be obtained by detecting each stroke of the boom cylinder 11, arm cylinder 12, and bucket cylinder 13 with a stroke sensor and converting it into each rotation angle. .
　上部旋回体７には、水平面等の基準面に対する機械本体３の傾斜角度を検出する傾斜角度センサ１８が取り付けられている。下部走行体５と上部旋回体７との間の旋回装置には、下部走行体５に対する上部旋回体７の相対的な角度である旋回角度を検出する旋回角度センサ１９が取り付けられている。上部旋回体７には、上部旋回体７の角速度を検出する角速度センサ（不図示）が取り付けられている。なお、本実施形態では、ブーム角度センサ１４、アーム角度センサ１５、バケット角度センサ１７、傾斜角度センサ１８及び旋回角度センサ１９を総称して、「姿勢検出装置５３」ともいう。姿勢検出装置５３は、フロント作業装置２の各回動角度及び上部旋回体７の旋回角度等を検出する。 A tilt angle sensor 18 is attached to the upper revolving body 7 to detect the tilt angle of the machine body 3 with respect to a reference plane such as a horizontal plane. A turning angle sensor 19 is attached to the turning device between the lower traveling body 5 and the upper rotating body 7 to detect a turning angle that is a relative angle of the upper rotating body 7 with respect to the lower traveling body 5. An angular velocity sensor (not shown) that detects the angular velocity of the upper rotating body 7 is attached to the upper rotating body 7 . In addition, in this embodiment, the boom angle sensor 14, the arm angle sensor 15, the bucket angle sensor 17, the tilt angle sensor 18, and the turning angle sensor 19 are also collectively referred to as the "attitude detection device 53." The attitude detection device 53 detects each rotation angle of the front working device 2, the rotation angle of the upper rotating body 7, and the like.
　上部旋回体７に設けられた運転室内には、オペレータの入力を受け付け可能なタッチスクリーン等によって構成された表示装置５５が設置されている。また、上部旋回体７に設けられた運転室内には、複数の油圧アクチュエータ４ａ，４ｂ，６，１１，１２，１３を操作する操作装置が設置されている。具体的には、操作装置は、走行右油圧モータ４ａを操作するための走行右レバー２３ａと、走行左油圧モータ４ｂを操作するための走行左レバー２３ｂと、ブームシリンダ１１及びバケットシリンダ１３を操作するための操作右レバー２２ａと、アームシリンダ１２及び旋回油圧モータ６を操作するための操作左レバー２２ｂとを備える。なお、本実施形態では、走行右レバー２３ａ、走行左レバー２３ｂ、操作右レバー２２ａ及び操作左レバー２２ｂを総称して、「操作レバー２２，２３」ともいう。 A display device 55 configured with a touch screen or the like capable of receiving input from an operator is installed in the operator's cab provided in the upper revolving structure 7. Moreover, in the operator's cab provided in the upper revolving structure 7, an operating device for operating the plurality of hydraulic actuators 4a, 4b, 6, 11, 12, and 13 is installed. Specifically, the operating device operates the right travel lever 23a for operating the right travel hydraulic motor 4a, the left travel lever 23b for operating the left travel hydraulic motor 4b, the boom cylinder 11, and the bucket cylinder 13. A right operating lever 22a for operating the arm cylinder 12 and a left operating lever 22b for operating the swing hydraulic motor 6 is provided. In this embodiment, the right travel lever 23a, the left travel lever 23b, the right operation lever 22a, and the left operation lever 22b are also collectively referred to as "operation levers 22, 23."
　また、上部旋回体７には、油圧ショベル１の車体の周辺環境を計測する外界計測装置７０が取り付けられている。外界計測装置７０は、油圧ショベル１の周辺に存在する物体の深度（物体までの距離）を計測する。外界計測装置７０は、計測結果として、当該物体の深度情報を取得する。外界計測装置７０は、例えば、ＬｉＤＡＲ（Light Detection And Ranging）であってもよいし、ステレオカメラであってもよい。外界計測装置７０は、油圧ショベル１に複数取り付けられていてもよい。 Further, an external world measuring device 70 is attached to the upper revolving body 7 to measure the surrounding environment of the vehicle body of the hydraulic excavator 1. The external world measuring device 70 measures the depth of an object (distance to the object) existing around the hydraulic excavator 1 . The external world measuring device 70 acquires depth information of the object as a measurement result. The external world measuring device 70 may be, for example, LiDAR (Light Detection And Ranging) or a stereo camera. A plurality of external world measuring devices 70 may be attached to the hydraulic excavator 1.
　また、上部旋回体７には、油圧ショベル１の現場における位置及び方位を計測する測位装置６０が取り付けられている。測位装置６０は、計測結果として、油圧ショベル１の現場における位置及び方位を含む測位情報を取得する。測位装置６０は、例えば、ＧＮＳＳ受信機や、ＴＳ（Total Station）等の測量機器であってもよい。測位装置６０は、これらに限らず、例えば、現場に固定されたカメラであってもよい。この場合、測位装置６０は、検出された油圧ショベル１の画像から演算された情報に基づいて、油圧ショベル１の現場における位置及び方位を計測してもよい。この場合、油圧ショベル１の方位を正確に計測するために、測位装置６０は、少なくとも２つ設けられていることが望ましい。 Additionally, a positioning device 60 is attached to the upper revolving body 7 to measure the position and orientation of the hydraulic excavator 1 at the site. The positioning device 60 acquires positioning information including the position and orientation of the hydraulic excavator 1 at the site as a measurement result. The positioning device 60 may be, for example, a GNSS receiver or a surveying device such as a TS (Total Station). The positioning device 60 is not limited to these, and may be, for example, a camera fixed at the site. In this case, the positioning device 60 may measure the position and orientation of the hydraulic excavator 1 at the site based on information calculated from the detected image of the hydraulic excavator 1. In this case, in order to accurately measure the orientation of the hydraulic excavator 1, it is desirable that at least two positioning devices 60 are provided.
　また、上部旋回体７には、ダンプトラック２００の位置、姿勢及び車格情報を含むトラック情報を外部から取得するトラック情報取得装置５６が取り付けられている。車格情報は、ダンプトラック２００の車種及び寸法の情報を含む。トラック情報取得装置５６は、例えば、ＦＭＳ（Fleet Management System）のようなダンプトラック２００の位置・配車管理システムとの無線通信を行う通信端末であってもよい。トラック情報取得装置５６は、ダンプトラック２００の位置・配車管理システムから送信されたダンプトラック２００のトラック情報を受信することによって、トラック情報を取得することができる。トラック情報取得装置５６は、トラック情報取得装置５６の機能を実装したソフトウェアとして構成され、情報処理装置５４に組み込まれていてもよい。 Additionally, a truck information acquisition device 56 is attached to the upper revolving body 7 to acquire truck information including the position, attitude, and vehicle size information of the dump truck 200 from the outside. The vehicle class information includes information on the vehicle type and dimensions of the dump truck 200. The truck information acquisition device 56 may be, for example, a communication terminal that performs wireless communication with a position/dispatch management system for the dump truck 200 such as a Fleet Management System (FMS). The truck information acquisition device 56 can acquire truck information by receiving the truck information of the dump truck 200 transmitted from the dump truck 200 position/dispatch management system. The track information acquisition device 56 may be configured as software that implements the functions of the track information acquisition device 56, and may be incorporated into the information processing device 54.
　ＦＭＳのようなダンプトラック２００の位置・配車管理システムの多くは、ダンプトラック２００のＧＮＳＳ受信機から取得したダンプトラック２００の位置情報を、地図上の経路に合致させる調整機能を備えている。トラック情報取得装置５６は、ダンプトラック２００のトラック情報と同時に、当該調整機能に基づくダンプトラック２００の位置調整量に関する情報を、位置・配車管理システムから取得することができる。 Many of the dump truck 200 position/dispatch management systems such as FMS have an adjustment function that matches the dump truck 200 position information obtained from the dump truck 200's GNSS receiver with the route on the map. The truck information acquisition device 56 can acquire information regarding the position adjustment amount of the dump truck 200 based on the adjustment function from the position/vehicle allocation management system at the same time as the truck information of the dump truck 200.
　また、上部旋回体７には、油圧ショベル１が次に行う動作内容（又は作業内容）を示すタスクを取得するタスク取得装置５８が取り付けられている。タスク取得装置５８は、例えば、作業現場を管理する管理装置との無線通信を行う通信端末であってもよい。タスク取得装置５８は、管理装置から送信された油圧ショベル１のタスクを受信することによって、タスクを取得することができる。タスク取得装置５８は、トラック情報取得装置５６と共有する通信端末によって構成されていてもよい。タスク取得装置５８は、タスク取得装置５８の機能を実装したソフトウェアとして構成され、情報処理装置５４に組み込まれていてもよい。また、タスク取得装置５８は、オペレータによる操作レバー２２，２３の操作から自動的にタスクを判別することによってタスクを取得してもよい。タスク取得装置５８は、オペレータが操作可能な入力端末であり、オペレータの入力を受け付けることによってタスクを取得してもよい。本実施形態では、タスクの取得方法については特に限定されない。 Additionally, a task acquisition device 58 is attached to the upper revolving body 7 to acquire a task indicating the content of the next operation (or content of work) to be performed by the hydraulic excavator 1 . The task acquisition device 58 may be, for example, a communication terminal that performs wireless communication with a management device that manages a work site. The task acquisition device 58 can acquire the task by receiving the task of the hydraulic excavator 1 transmitted from the management device. The task acquisition device 58 may be configured by a communication terminal shared with the track information acquisition device 56. The task acquisition device 58 may be configured as software that implements the functions of the task acquisition device 58, and may be incorporated into the information processing device 54. Further, the task acquisition device 58 may acquire the task by automatically determining the task based on the operation of the operating levers 22 and 23 by the operator. The task acquisition device 58 is an input terminal that can be operated by an operator, and may acquire tasks by accepting input from the operator. In this embodiment, there is no particular limitation on the task acquisition method.
＜油圧ショベルの内部構成＞
　図２は、図１に示す油圧ショベル１に搭載された油圧システム及び制御システムの構成を示すブロック図である。 <Internal configuration of hydraulic excavator>
FIG. 2 is a block diagram showing the configuration of a hydraulic system and a control system installed in the hydraulic excavator 1 shown in FIG. 1.
　上部旋回体７に搭載された原動機であるエンジン１０３は、油圧ポンプ１０２とパイロットポンプ１０４とを駆動する。制御装置４０は、オペレータによる操作レバー２２，２３の操作情報（操作量及び操作方向）に応じて、フロント作業装置２の回動動作、下部走行体５の走行動作、及び、上部旋回体７の旋回動作（すなわちこれらによって構成される油圧ショベル１の車体の動作）を制御する。具体的には、制御装置４０は、オペレータによる操作レバー２２，２３の操作情報（操作量及び操作方向）をロータリエンコーダ又はポテンショメータ等のセンサ５２ａ～５２ｆにより検出し、検出された操作情報に応じた制御指令を電磁比例弁４７ａ～４７ｌに出力する。電磁比例弁４７ａ～４７ｌは、パイロットライン１００に設けられており、制御装置４０からの制御指令が入力されると作動し、流量制御弁１０１にパイロット圧を出力して、流量制御弁１０１を作動させる。 An engine 103, which is a prime mover mounted on the upper revolving structure 7, drives a hydraulic pump 102 and a pilot pump 104. The control device 40 controls the rotating operation of the front working device 2, the traveling operation of the lower traveling body 5, and the rotating upper rotating body 7 according to operation information (operating amount and operating direction) of the operating levers 22 and 23 by the operator. The turning operation (that is, the operation of the vehicle body of the hydraulic excavator 1 constituted by these) is controlled. Specifically, the control device 40 detects operation information (operation amount and operation direction) of the operation levers 22 and 23 by the operator using sensors 52a to 52f such as rotary encoders or potentiometers, and detects operation information according to the detected operation information. A control command is output to the electromagnetic proportional valves 47a to 47l. The electromagnetic proportional valves 47a to 47l are provided in the pilot line 100, and are activated when a control command from the control device 40 is input, output pilot pressure to the flow control valve 101, and operate the flow control valve 101. let
　流量制御弁１０１は、旋回油圧モータ６、アームシリンダ１２、ブームシリンダ１１、バケットシリンダ１３、走行右油圧モータ４ａ及び走行左油圧モータ４ｂのそれぞれに対して油圧ポンプ１０２から供給される圧油を、電磁比例弁４７ａ～４７ｌからのパイロット圧に応じて制御する。なお、電磁比例弁４７ａ，４７ｂは、旋回油圧モータ６に供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。電磁比例弁４７ｃ，４７ｄは、アームシリンダ１２に供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。電磁比例弁４７ｅ，４７ｆは、ブームシリンダ１１に供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。電磁比例弁４７ｇ，４７ｈは、バケットシリンダ１３に供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。電磁比例弁４７ｉ，４７ｊは、走行右油圧モータ４ａに供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。電磁比例弁４７ｋ，４７ｌは、走行左油圧モータ４ｂに供給される圧油を制御するためのパイロット圧を流量制御弁１０１に出力する。 The flow control valve 101 controls pressure oil supplied from the hydraulic pump 102 to each of the swing hydraulic motor 6, arm cylinder 12, boom cylinder 11, bucket cylinder 13, right travel hydraulic motor 4a, and left travel hydraulic motor 4b. Control is performed according to pilot pressure from electromagnetic proportional valves 47a to 47l. Note that the electromagnetic proportional valves 47a and 47b output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the swing hydraulic motor 6. The electromagnetic proportional valves 47c and 47d output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the arm cylinder 12. The electromagnetic proportional valves 47e and 47f output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the boom cylinder 11. The electromagnetic proportional valves 47g and 47h output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the bucket cylinder 13. The electromagnetic proportional valves 47i and 47j output pilot pressure to the flow control valve 101 for controlling the pressure oil supplied to the travel right hydraulic motor 4a. The electromagnetic proportional valves 47k and 47l output pilot pressure to the flow rate control valve 101 for controlling the pressure oil supplied to the travel left hydraulic motor 4b.
　ブームシリンダ１１、アームシリンダ１２及びバケットシリンダ１３は、それぞれ、供給された圧油によって伸縮し、ブーム８、アーム９及びバケット１０を回動させる。これにより、バケット１０の位置及び姿勢が変化する。旋回油圧モータ６は、供給された圧油によって回転し、上部旋回体７を旋回させる。走行右油圧モータ４ａ及び走行左油圧モータ４ｂは、供給された圧油によって回転し、下部走行体５を走行させる。 The boom cylinder 11, arm cylinder 12, and bucket cylinder 13 are expanded and contracted by the supplied pressure oil, respectively, and rotate the boom 8, arm 9, and bucket 10. As a result, the position and attitude of the bucket 10 change. The swing hydraulic motor 6 is rotated by the supplied pressure oil, and swings the upper revolving structure 7. The right travel hydraulic motor 4a and the left travel hydraulic motor 4b are rotated by the supplied pressure oil and cause the lower traveling body 5 to travel.
　また、制御装置４０は、情報処理装置５４の認識結果に基づいて、油圧ショベル１の車体の動作（フロント作業装置２の回動動作、下部走行体５の走行動作、及び、上部旋回体７の旋回動作）を自動（半自動又は全自動）で制御することが可能である。情報処理装置５４は、外界計測装置７０の計測結果に基づいて油圧ショベル１の周辺に存在するダンプトラック２００を認識する。情報処理装置５４は、ＣＰＵ（Central Processing Unit）７３、ＲＡＭ（Random Access Memory）７２、ＲＯＭ（Read Only Memory）７１、及び、外部Ｉ／Ｆ（Interface）７４等が、バス７５により互いに接続されたコンピュータによって構成されている。外部Ｉ／Ｆ７４には、制御装置４０、表示装置５５、外界計測装置７０、測位装置６０、姿勢検出装置５３、タスク取得装置５８、トラック情報取得装置５６及び記憶装置５７（例えば、ハードディスクドライブ又は大容量フラッシュメモリ等）が接続されている。 Further, the control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 (the rotational operation of the front working device 2, the traveling operation of the lower traveling body 5, and the movement of the upper rotating body 7) based on the recognition result of the information processing device 54. It is possible to control the rotation movement automatically (semi-automatically or fully automatically). The information processing device 54 recognizes the dump truck 200 existing around the hydraulic excavator 1 based on the measurement results of the external world measurement device 70. The information processing device 54 includes a CPU (Central Processing Unit) 73, a RAM (Random Access Memory) 72, a ROM (Read Only Memory) 71, an external I/F (Interface) 74, etc., which are connected to each other by a bus 75. It is composed by a computer. The external I/F 74 includes a control device 40, a display device 55, an external world measurement device 70, a positioning device 60, an attitude detection device 53, a task acquisition device 58, a track information acquisition device 56, and a storage device 57 (for example, a hard disk drive or large (capacitive flash memory, etc.) is connected.
＜油圧ショベルの動作概要＞
　図３は、図２に示す油圧ショベル１の動作の一例を説明する図である。 <Overview of hydraulic excavator operation>
FIG. 3 is a diagram illustrating an example of the operation of the hydraulic excavator 1 shown in FIG. 2.
　ダンプトラック２００は、油圧ショベル１が対象物を積み込み可能な所定位置に停車する。油圧ショベル１のタスク取得装置５８は、次のタスクとして、掘削された土砂等の対象物をダンプトラック２００に積み込む積込タスクを取得する。積込タスクが取得された場合、まず、油圧ショベル１の外界計測装置７０は、ダンプトラック２００のベッセルを計測する。続いて、油圧ショベル１の情報処理装置５４は、ダンプトラック２００の存在領域２１０について後述の認識領域及び推定領域を演算する。油圧ショベル１の情報処理装置５４は、バケット１０からダンプトラック２００のベッセルに対象物を積み込む時のバケット１０の位置（以下「積込位置」とも称する）を演算する。そして、油圧ショベル１の制御装置４０は、演算されたダンプトラック２００の認識領域及び推定領域並びに積込位置に基づいて、フロント作業装置２がダンプトラック２００に衝突しないように、バケット１０を積込位置に移動させる制御指令を出力する。これにより、油圧ショベル１は、ダンプトラック２００に衝突することなく適切に積込作業を自動制御することができる。 The dump truck 200 stops at a predetermined position where the hydraulic excavator 1 can load the object. The task acquisition device 58 of the hydraulic excavator 1 acquires a loading task for loading excavated objects such as earth and sand onto the dump truck 200 as the next task. When the loading task is acquired, the external world measurement device 70 of the hydraulic excavator 1 first measures the vessel of the dump truck 200. Subsequently, the information processing device 54 of the hydraulic excavator 1 calculates a recognition area and an estimation area, which will be described later, for the presence area 210 of the dump truck 200. The information processing device 54 of the hydraulic excavator 1 calculates the position of the bucket 10 (hereinafter also referred to as "loading position") when loading objects from the bucket 10 into the vessel of the dump truck 200. Then, the control device 40 of the hydraulic excavator 1 loads the bucket 10 so that the front working device 2 does not collide with the dump truck 200 based on the calculated recognition area and estimated area of the dump truck 200 and the loading position. Outputs a control command to move to a certain position. Thereby, the hydraulic excavator 1 can appropriately automatically control the loading work without colliding with the dump truck 200.
　本実施形態では、図３に示すように、ダンプトラック２００は、油圧ショベル１が対象物を積み込み可能な所定位置に停車することを想定している。具体的には、本実施形態のダンプトラック２００は、ＦＭＳのようなダンプトラック２００の位置・配車管理システムによって、作業現場におけるダンプトラック２００の位置が管理されているものとする。しかしながら、ダンプトラック２００が所定位置に停車するまでのダンプトラック２００の制御方法については、特に限定されない。例えば、ダンプトラック２００は、ダンプトラック２００のオペレータの操縦によって所定位置に停車してもよい。また、本実施形態では、ダンプトラック２００が所定位置に停車したことを、油圧ショベル１のオペレータが目視で確認してもよいし、外界計測装置７０の計測結果に基づいて情報処理装置５４が認識してもよい。 In this embodiment, as shown in FIG. 3, the dump truck 200 is assumed to stop at a predetermined position where the hydraulic excavator 1 can load the object. Specifically, it is assumed that the position of the dump truck 200 in the present embodiment at the work site is managed by a dump truck 200 position/dispatch management system such as FMS. However, the method of controlling the dump truck 200 until the dump truck 200 stops at a predetermined position is not particularly limited. For example, the dump truck 200 may be stopped at a predetermined position by the operator of the dump truck 200. Further, in the present embodiment, the operator of the hydraulic excavator 1 may visually confirm that the dump truck 200 has stopped at a predetermined position, or the information processing device 54 may recognize the fact that the dump truck 200 has stopped at a predetermined position. You may.
　なお、図３の外界計測装置７０は、油圧ショベル１の左側方を向くように取り付けられているが、油圧ショベル１の前方を向くように取り付けられていてもよい。更に、油圧ショベル１は、油圧ショベル１の右側方又は後方を向くように取り付けられた外界計測装置７０を更に備えていてもよい。 Although the external world measuring device 70 in FIG. 3 is attached so as to face the left side of the hydraulic excavator 1, it may be attached so as to face the front of the hydraulic excavator 1. Furthermore, the hydraulic excavator 1 may further include an external world measurement device 70 attached to face the right side or rear of the hydraulic excavator 1.
　図４は、図２に示す油圧ショベル１の動作の一例を示すフローチャートである。 FIG. 4 is a flowchart showing an example of the operation of the hydraulic excavator 1 shown in FIG. 2.
　ステップＳ１１１において、油圧ショベル１のタスク取得装置５８は、掘削対象の地面を掘削する掘削タスクを取得する。 In step S111, the task acquisition device 58 of the hydraulic excavator 1 acquires an excavation task for excavating the ground to be excavated.
　ステップＳ１１２において、油圧ショベル１の制御装置４０は、取得された掘削タスクに応じて、フロント作業装置２の回動動作を制御して油圧ショベル１に掘削動作を行わせる。 In step S112, the control device 40 of the hydraulic excavator 1 controls the rotational operation of the front working device 2 to cause the hydraulic excavator 1 to perform an excavation operation, according to the acquired excavation task.
　ステップＳ１１３において、油圧ショベル１のタスク取得装置５８は、掘削された土砂等の対象物をダンプトラック２００に積み込む積込タスクを取得する。 In step S113, the task acquisition device 58 of the hydraulic excavator 1 acquires a loading task for loading an object such as excavated earth and sand onto the dump truck 200.
　ステップＳ１１４は、油圧ショベル１の制御装置４０は、取得された積込タスクに応じて、フロント作業装置２の回動動作及び上部旋回体７の旋回動作を制御して油圧ショベル１に積込動作を行わせる。 In step S114, the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2 and the rotation operation of the upper rotating structure 7 according to the acquired loading task, so that the hydraulic excavator 1 performs a loading operation. Have them do it.
　ステップＳ１１５において、油圧ショベル１の制御装置４０は、ダンプトラック２００への積込を終了するか否かを判定する。油圧ショベル１のオペレータは、目視による確認を行った上で、特定の操作レバー２２，２３を操作したり、表示装置５５に情報を入力したりして、ダンプトラック２００への積込を終了することを制御装置４０に通知してもよい。制御装置４０は、これらのオペレータからの操作が通知されたことによって、ダンプトラック２００への積込を終了すると判定してもよい。或いは、制御装置４０は、ダンプトラック２００に積み込まれた対象物の荷重を計測して当該荷重が適正であることを判定する荷重判定装置の判定結果から、ダンプトラック２００への積込を終了することを判定してもよい。ダンプトラック２００への積込を終了すると判定した場合、制御装置４０は、ステップＳ１１８に移行する。ダンプトラック２００への積込を終了しないと判定した場合、制御装置４０は、ステップＳ１１６に移行する。 In step S115, the control device 40 of the hydraulic excavator 1 determines whether to finish loading the dump truck 200. The operator of the hydraulic excavator 1 performs visual confirmation and then operates specific control levers 22 and 23 or inputs information to the display device 55 to finish loading the dump truck 200. The control device 40 may be notified of this. The control device 40 may determine to finish loading the dump truck 200 upon being notified of these operations from the operators. Alternatively, the control device 40 finishes loading the object into the dump truck 200 based on the determination result of the load determination device that measures the load of the object loaded onto the dump truck 200 and determines that the load is appropriate. It may be determined that If it is determined that loading into the dump truck 200 is finished, the control device 40 moves to step S118. If it is determined that loading into the dump truck 200 is not finished, the control device 40 moves to step S116.
　ステップＳ１１６において、油圧ショベル１のタスク取得装置５８は、次の掘削対象の地面にフロント作業装置２を移動させるリーチングタスクを取得する。 In step S116, the task acquisition device 58 of the hydraulic excavator 1 acquires a reaching task to move the front working device 2 to the next ground to be excavated.
　ステップＳ１１７において、油圧ショベル１の制御装置４０は、取得されたリーチングタスクに応じて、フロント作業装置２の回動動作、下部走行体５の走行動作、及び、上部旋回体７の旋回動作を制御して油圧ショベル１にリーチング動作を行わせる。その後、制御装置４０は、ステップＳ１１１に移行する。 In step S117, the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2, the traveling operation of the lower traveling body 5, and the turning operation of the upper rotating body 7, according to the acquired reaching task. to cause the hydraulic excavator 1 to perform a reaching operation. After that, the control device 40 moves to step S111.
　ステップＳ１１８において、油圧ショベル１の制御装置４０は、作業を終了するか否かを判定する。制御装置４０は、タスク取得装置５８が作業終了通知を取得したことによって、作業を終了すると判定してもよい。或いは、制御装置４０は、オペレータ又は管理者が油圧ショベル１のエンジン１０３を停止したことによって、作業を終了すると判定してもよい。作業を終了すると判定した場合、制御装置４０は、油圧ショベル１の動作を中断して作業を終了する。作業を終了しないと判定した場合、制御装置４０は、ステップＳ１１６に移行する。油圧ショベル１は、リーチング動作、掘削動作及び積込動作を継続する。 In step S118, the control device 40 of the hydraulic excavator 1 determines whether or not to end the work. The control device 40 may determine that the task is to be completed when the task acquisition device 58 has acquired the notification of completion of the task. Alternatively, the control device 40 may determine that the work is finished when the operator or manager stops the engine 103 of the hydraulic excavator 1. If it is determined that the work is to be completed, the control device 40 interrupts the operation of the hydraulic excavator 1 and ends the work. If it is determined that the work is not finished, the control device 40 moves to step S116. The hydraulic excavator 1 continues the reaching operation, the digging operation, and the loading operation.
　油圧ショベル１のタスク及び動作は、図４に示したタスク及び動作に限定されず、例えば移動又は整地等の、図４に示されていないタスク及び動作を含んでいてもよい。 The tasks and operations of the hydraulic excavator 1 are not limited to those shown in FIG. 4, and may include tasks and operations not shown in FIG. 4, such as moving or leveling the ground.
　油圧ショベル１の動作において、タスク取得装置５８が各タスクを取得した際に、ダンプトラック２００の位置、姿勢及び形状等の情報が必要となる場合がある。この場合、情報処理装置５４は、タスク取得装置５８が各タスクを取得する度に、都度、外界計測装置７０の計測結果に基づいてダンプトラック２００を認識する。そして、情報処理装置５４は、ダンプトラック２００の認識結果が補正不能な程度に妥当でないか、又は、ダンプトラック２００の認識結果が補正可能な程度に妥当であるかを検証（以下「妥当性を検証」と称する）する。そして、情報処理装置５４は、ダンプトラック２００の認識結果が妥当であれば、当該認識結果を補正する。 In the operation of the hydraulic excavator 1, when the task acquisition device 58 acquires each task, information such as the position, posture, shape, etc. of the dump truck 200 may be required. In this case, the information processing device 54 recognizes the dump truck 200 based on the measurement results of the external world measurement device 70 each time the task acquisition device 58 acquires each task. Then, the information processing device 54 verifies whether the recognition result of the dump truck 200 is invalid to the extent that it cannot be corrected, or whether the recognition result of the dump truck 200 is valid to the extent that it can be corrected (hereinafter referred to as "validation"). (referred to as “verification”). Then, if the recognition result of the dump truck 200 is valid, the information processing device 54 corrects the recognition result.
　本実施形態では、ダンプトラック２００の認識結果の妥当性を検証し、当該認識結果を補正する処理を「認識結果検証処理」とも称する。以下では、認識結果検証処理を行う情報処理装置５４の機能について説明する。 In this embodiment, the process of verifying the validity of the recognition result of the dump truck 200 and correcting the recognition result is also referred to as "recognition result verification process." Below, the functions of the information processing device 54 that performs recognition result verification processing will be explained.
＜情報処理装置の機能的構成＞
　図５は、図２に示す情報処理装置５４の機能的構成を説明するブロック図である。図６は、油圧ショベル１に設定される各座標系３００～５００を側方から視た図である。図７は、図６に示す各座標系３００～５００を上方から視た図である。 <Functional configuration of information processing device>
FIG. 5 is a block diagram illustrating the functional configuration of the information processing device 54 shown in FIG. 2. As shown in FIG. FIG. 6 is a side view of each of the coordinate systems 300 to 500 set in the hydraulic excavator 1. FIG. 7 is a diagram of each of the coordinate systems 300 to 500 shown in FIG. 6 viewed from above.
　情報処理装置５４は、姿勢演算部８１と、座標変換部８２と、測位情報演算部８３と、タスク取得部８０と、認識部８４と、領域推定部８５と、検証部８６と、補正部８７と、を備える。 The information processing device 54 includes an attitude calculation section 81 , a coordinate transformation section 82 , a positioning information calculation section 83 , a task acquisition section 80 , a recognition section 84 , an area estimation section 85 , a verification section 86 , and a correction section 87 and.
　情報処理装置５４には、油圧ショベル１の構成要素の位置及び姿勢を特定する基準座標系として、図６及び図７に示す車体座標系４００が予め設定される。本実施形態の油圧ショベル１の車体座標系４００は、上部旋回体７の旋回中心線１２０のうち、下部走行体５と地面Ｇとが接する点を原点とする右手座標系として定義されている。油圧ショベル１の車体座標系４００は、下部走行体５の前進方向をＸ軸の正方向として定義されている。油圧ショベル１の車体座標系４００は、旋回中心線１２０が上方に延びる方向をＺ軸の正方向として定義されている。油圧ショベル１の車体座標系４００は、Ｘ軸及びＺ軸のそれぞれに直交し、左方をＹ軸の正方向として定義されている。また、油圧ショベル１の車体座標系４００において、上部旋回体７の旋回角度θｓｗは、フロント作業装置２がＸ軸と平行となる状態を０度として定義されている。 A vehicle body coordinate system 400 shown in FIGS. 6 and 7 is preset in the information processing device 54 as a reference coordinate system for specifying the positions and postures of the components of the hydraulic excavator 1. The vehicle body coordinate system 400 of the hydraulic excavator 1 of the present embodiment is defined as a right-handed coordinate system whose origin is the point on the swing center line 120 of the upper rotating body 7 where the lower traveling body 5 and the ground G contact. In the vehicle body coordinate system 400 of the hydraulic excavator 1, the forward direction of the lower traveling body 5 is defined as the positive direction of the X-axis. In the vehicle body coordinate system 400 of the hydraulic excavator 1, the direction in which the turning center line 120 extends upward is defined as the positive direction of the Z-axis. The vehicle body coordinate system 400 of the hydraulic excavator 1 is orthogonal to each of the X-axis and the Z-axis, and the left side is defined as the positive direction of the Y-axis. Furthermore, in the vehicle body coordinate system 400 of the hydraulic excavator 1, the turning angle θsw of the upper rotating body 7 is defined as 0 degrees when the front working device 2 is parallel to the X axis.
　また、本実施形態では、外界計測装置７０の基準座標系として、図６及び図７に示すようなセンサ座標系３００を定義する。本実施形態では、現場の基準座標系として、図６及び図７に示すような現場座標系５００を定義する。 Furthermore, in this embodiment, a sensor coordinate system 300 as shown in FIGS. 6 and 7 is defined as a reference coordinate system of the external world measuring device 70. In this embodiment, a site coordinate system 500 as shown in FIGS. 6 and 7 is defined as a site reference coordinate system.
＜姿勢演算部＞
　姿勢演算部８１は、姿勢検出装置５３の検出信号から、油圧ショベル１の車体座標系４００における油圧ショベル１の構成要素の姿勢等を演算する。具体的には、姿勢演算部８１は、ブーム角度センサ１４から出力されたブーム８の回動角度の検出信号から、Ｘ軸に対するブーム８の回動角度θｂｍを演算する。姿勢演算部８１は、アーム角度センサ１５から出力されたアーム９の回動角度の検出信号から、ブーム８に対するアーム９の回動角度θａｍを演算する。姿勢演算部８１は、バケット角度センサ１７から出力されたバケット１０の回動角度の検出信号から、アーム９に対するバケット１０の回動角度θｂｋを演算する。姿勢演算部８１は、旋回角度センサ１９から出力された上部旋回体７の旋回角度の検出信号から、Ｘ軸（下部走行体５）に対する上部旋回体７の旋回角度θｓｗを演算する。更に、姿勢演算部８１は、上部旋回体７の旋回角度θｓｗから、上部旋回体７の旋回角速度ωｓｗを演算する。 <Posture calculation section>
The posture calculation unit 81 calculates the postures of the components of the hydraulic excavator 1 in the vehicle body coordinate system 400 of the hydraulic excavator 1 from the detection signal of the posture detection device 53. Specifically, the attitude calculation unit 81 calculates the rotation angle θbm of the boom 8 with respect to the X axis from the detection signal of the rotation angle of the boom 8 output from the boom angle sensor 14. Posture calculation unit 81 calculates the rotation angle θam of arm 9 with respect to boom 8 from the detection signal of the rotation angle of arm 9 output from arm angle sensor 15 . The posture calculation unit 81 calculates the rotation angle θbk of the bucket 10 with respect to the arm 9 from the detection signal of the rotation angle of the bucket 10 output from the bucket angle sensor 17. The attitude calculation unit 81 calculates the turning angle θsw of the upper rotating structure 7 with respect to the X axis (the lower traveling structure 5) from the detection signal of the turning angle of the upper rotating structure 7 output from the turning angle sensor 19. Furthermore, the attitude calculation unit 81 calculates the turning angular velocity ωsw of the upper rotating structure 7 from the turning angle θsw of the upper rotating structure 7.
　更に、姿勢演算部８１は、傾斜角度センサ１８から出力された機械本体３の傾斜角度の検出信号から、基準面ＤＰに対する機械本体３（下部走行体５）の傾斜角度を演算する。基準面ＤＰは、例えば、重力方向に直交する水平面である。傾斜角度は、Ｙ軸周りの回転角度θｐと、Ｘ軸周りの回転角度θｒとを含む。 Furthermore, the attitude calculation unit 81 calculates the inclination angle of the machine body 3 (lower traveling body 5) with respect to the reference plane DP from the detection signal of the inclination angle of the machine body 3 output from the inclination angle sensor 18. The reference plane DP is, for example, a horizontal plane orthogonal to the direction of gravity. The tilt angle includes a rotation angle θp around the Y-axis and a rotation angle θr around the X-axis.
＜座標変換部＞
　座標変換部８２は、姿勢演算部８１から出力された油圧ショベル１の姿勢情報を用いて、外界計測装置７０により取得された深度情報を表現する座標系を、センサ座標系３００から油圧ショベル１の車体座標系４００に変換する。外界計測装置７０により取得された深度情報は、センサ座標系３００で表現された３次元の点データの集合（すなわち点群データ）として与えられる。 <Coordinate conversion section>
The coordinate conversion unit 82 uses the attitude information of the hydraulic excavator 1 output from the attitude calculation unit 81 to convert the coordinate system expressing the depth information acquired by the external world measurement device 70 from the sensor coordinate system 300 of the hydraulic excavator 1. It is converted to the vehicle body coordinate system 400. The depth information acquired by the external world measuring device 70 is given as a set of three-dimensional point data (ie, point group data) expressed in the sensor coordinate system 300.
　センサ座標系３００における点データＰｓ（Ｘs，Ｙｓ，Ｚｓ）から、油圧ショベル１の車体座標系４００における点データＰｖ（Ｘｖ，Ｙｖ，Ｚｖ）への変換には、例えば、下記の式（１）～式（３）が用いられる。 To convert point data Ps (Xs, Ys, Zs) in the sensor coordinate system 300 to point data Pv (Xv, Yv, Zv) in the vehicle body coordinate system 400 of the hydraulic excavator 1, for example, the following equation (1) is used. ~Equation (3) is used.
Figure JPOXMLDOC01-appb-M000001
Figure JPOXMLDOC01-appb-M000001
Figure JPOXMLDOC01-appb-M000002
Figure JPOXMLDOC01-appb-M000002
Figure JPOXMLDOC01-appb-M000003
Figure JPOXMLDOC01-appb-M000003
　上記の式（１）～式（３）において、Ｒｓｖは、センサ座標系３００から車体座標系４００への回転行列である。αｓ、βｓ、γｓは、車体座標系４００における外界計測装置７０の各軸（センサ座標系３００の各軸）が成す角度である。外界計測装置７０が油圧ショベル１に固定されている場合、これらの成す角度は、例えば、予め車体座標系４００における外界計測装置７０の姿勢を測定しておき、記憶装置５７に予め保存しておけばよい。また、外界計測装置７０が油圧ショベル１に対して姿勢を変化させながら計測を行う場合、外界計測装置７０に姿勢計測センサを取り付ける等して、姿勢計測センサが検出した角度を用いて座標変換行列を算出してもよい。θｓｗは、上部旋回体７の旋回角度であり、姿勢演算部８１から出力される。 In the above equations (1) to (3), Rsv is a rotation matrix from the sensor coordinate system 300 to the vehicle body coordinate system 400. αs, βs, and γs are angles formed by each axis of the external world measuring device 70 (each axis of the sensor coordinate system 300) in the vehicle body coordinate system 400. When the external world measuring device 70 is fixed to the hydraulic excavator 1, these angles can be determined by, for example, measuring the attitude of the external world measuring device 70 in the vehicle body coordinate system 400 in advance and storing it in the storage device 57 in advance. Bye. In addition, when the external world measuring device 70 performs measurement while changing the posture with respect to the hydraulic excavator 1, a posture measuring sensor is attached to the external world measuring device 70, and a coordinate transformation matrix is used using the angle detected by the posture measuring sensor. may be calculated. θsw is a turning angle of the upper rotating body 7, and is output from the attitude calculation unit 81.
　上記の式（１）～式（３）において、Ｔｓｖは車体座標系４００の原点からセンサ座標系３００の原点への並進ベクトルである。Ｌｓｘ，Ｌｓｙ，Ｌｓｚは、車体座標系４００から視たセンサ座標系３００の原点座標に等しい。外界計測装置７０の取り付け位置は、油圧ショベル１に対して固定されている場合が多い。したがってその場合は、予め外界計測装置７０の油圧ショベル１への取り付け位置を計測しておき、記憶装置５７に予め保存しておけばよい。 In the above equations (1) to (3), Tsv is a translation vector from the origin of the vehicle body coordinate system 400 to the origin of the sensor coordinate system 300. Lsx, Lsy, and Lsz are equal to the origin coordinates of the sensor coordinate system 300 viewed from the vehicle body coordinate system 400. The mounting position of the external world measuring device 70 is often fixed with respect to the hydraulic excavator 1. Therefore, in that case, the attachment position of the external world measurement device 70 to the hydraulic excavator 1 may be measured in advance and stored in the storage device 57 in advance.
＜測位情報演算部＞
　測位情報演算部８３は、測位装置６０により取得された測位情報から、油圧ショベル１の現場座標系５００における車体座標系４００の原点の位置と、フロント作業装置２の現場座標系５００における方位θｄｉｒを演算する。 <Positioning information calculation unit>
The positioning information calculation unit 83 calculates the position of the origin of the vehicle body coordinate system 400 in the site coordinate system 500 of the hydraulic excavator 1 and the orientation θdir of the front work device 2 in the site coordinate system 500 from the positioning information acquired by the positioning device 60. calculate.
＜タスク取得部＞
　タスク取得部８０は、油圧ショベル１が次に行うタスクをタスク取得装置５８から取得する。本実施形態において、油圧ショベル１が行うタスクは、掘削タスク、積込タスク、リーチングタスクを含む。掘削タスクは、油圧ショベル１が掘削対象の地面を掘削し、バケット１０に土砂等の対象物を保持するまでの掘削動作を指定するタスクである。積込タスクは、油圧ショベル１が掘削動作を終了した状態からバケット１０をダンプトラック２００のベッセル上方まで移動させ、バケット１０からベッセルに対象物を放出するまでの積込動作を指定するタスクである。リーチングタスクは、油圧ショベル１が積込動作を終了した状態から次の掘削対象の地面がある位置までバケット１０を移動させるリーチング動作を指定するタスクである。なお、油圧ショベル１が行うタスクは、これらに限定されず、例えば、ダンプトラック２００が現場の所定位置に停車するまでの間に油圧ショベル１が周囲の地面を均す整地動作を指定する整地タスクや、油圧ショベル１が掘削対象の地面を変更する際の移動動作を指定する整地タスクを含んでもよい。 <Task acquisition part>
The task acquisition unit 80 acquires the next task to be performed by the hydraulic excavator 1 from the task acquisition device 58. In this embodiment, the tasks performed by the hydraulic excavator 1 include an excavation task, a loading task, and a reaching task. The excavation task is a task that specifies an excavation operation in which the hydraulic excavator 1 excavates the ground to be excavated and holds an object such as earth and sand in the bucket 10. The loading task is a task that specifies a loading operation from the state where the hydraulic excavator 1 has finished excavating to moving the bucket 10 above the vessel of the dump truck 200 and discharging the object from the bucket 10 to the vessel. . The reaching task is a task for specifying a reaching operation in which the hydraulic excavator 1 moves the bucket 10 from the state where the loading operation has been completed to the position where the next ground to be excavated is located. Note that the tasks performed by the hydraulic excavator 1 are not limited to these, and for example, a ground leveling task in which the hydraulic excavator 1 specifies a leveling operation to level the surrounding ground until the dump truck 200 stops at a predetermined position on the site. Alternatively, the excavator 1 may include a ground leveling task that specifies a movement operation when changing the ground to be excavated.
＜認識部＞
　認識部８４は、外界計測装置７０の計測結果からダンプトラック２００を認識する。具体的には、認識部８４は、外界計測装置７０の計測結果として取得されて座標変換部８２により車体座標系４００に変換された点群データを用いて、ダンプトラック２００の位置、姿勢及び形状を認識する。そして、認識部８４は、認識されたダンプトラック２００が存在する領域である認識領域を演算する。すなわち、認識領域は、ダンプトラック２００の存在領域２１０についての認識部８４の認識結果である。 <Recognition part>
The recognition unit 84 recognizes the dump truck 200 from the measurement results of the external world measuring device 70. Specifically, the recognition unit 84 determines the position, orientation, and shape of the dump truck 200 using point cloud data acquired as the measurement results of the external world measuring device 70 and converted into the vehicle body coordinate system 400 by the coordinate conversion unit 82. Recognize. Then, the recognition unit 84 calculates a recognition area that is an area where the recognized dump truck 200 exists. That is, the recognition area is the recognition result of the recognition unit 84 regarding the presence area 210 of the dump truck 200.
　認識部８４によるダンプトラック２００の認識領域の演算方法は、例えば、予めダンプトラック２００を計測した３次元メッシュモデルを記憶装置５７に保持しておく。そして、座標変換部８２から取得した車体座標系４００に変換された点群データと３次元メッシュモデルとの間において位置、姿勢及び形状を照合する。これにより、認識部８４は、対象とするダンプトラック２００の位置、姿勢及び形状を演算することができるので、ダンプトラック２００の認識領域を演算することができる。なお、ダンプトラック２００の認識領域の演算方法は、これに限定されず、例えば、外界計測装置７０から得られた点群データからダンプトラック２００の特定の平面を抽出する処理によってダンプトラック２００の認識領域を演算する方法であってもよい。或いは、ダンプトラック２００を含む点群データからダンプトラック２００の認識領域を演算するニューラルネットワーク（Neural Network）を予め構築しておき、認識対象のダンプトラック２００の特徴量を機械学習させておいた識別器を用いて演算する方法であってもよい。 In order to calculate the recognition area of the dump truck 200 by the recognition unit 84, for example, a three-dimensional mesh model obtained by measuring the dump truck 200 is stored in the storage device 57 in advance. Then, the position, orientation, and shape are compared between the point group data converted into the vehicle body coordinate system 400 obtained from the coordinate conversion unit 82 and the three-dimensional mesh model. Thereby, the recognition unit 84 can calculate the position, orientation, and shape of the target dump truck 200, and therefore can calculate the recognition area of the dump truck 200. Note that the method of calculating the recognition area of the dump truck 200 is not limited to this, and for example, the recognition of the dump truck 200 is performed by extracting a specific plane of the dump truck 200 from point cloud data obtained from the external world measuring device 70. A method of calculating the area may also be used. Alternatively, a neural network that calculates the recognition area of the dump truck 200 from point cloud data including the dump truck 200 may be constructed in advance, and the features of the dump truck 200 to be recognized may be machine learned. A method of calculating using a device may also be used.
　本実施形態では、ダンプトラック２００の位置、姿勢及び存在領域２１０を、次のように定義する。 In this embodiment, the position, attitude, and presence area 210 of the dump truck 200 are defined as follows.
　図８は、ダンプトラック２００の車体座標系６００を側方から視た図である。図９は、図８に示す車体座標系６００を上方から視た図である。 FIG. 8 is a side view of the vehicle body coordinate system 600 of the dump truck 200. FIG. 9 is a diagram of the vehicle body coordinate system 600 shown in FIG. 8 viewed from above.
　本実施形態では、ダンプトラック２００の車体座標系６００は、ダンプトラック２００の後輪軸の中心を原点とし、後輪から前輪に向かう方向をＸ軸、後輪軸が延びる方向をＹ軸、ダンプトラック２００の高さ方向をＺ軸とする。ダンプトラック２００の位置は、車体座標系６００の原点Ｐｄ（Ｘｄ，Ｙｄ，Ｚｄ）を指すものとする。ダンプトラック２００の姿勢は、現場座標系５００の各軸と、ダンプトラック２００の車体座標系６００の各軸とが成す角度θｄ（θｒｏｌｌ，θｐｉｔｃｈ，θｙａｗ）を指すものとする。ダンプトラック２００の存在領域２１０は、ダンプトラック２００のベッセルの四隅の点Ｐｄ１～Ｐｄ４を頂点とする矩形の領域Ｓｄ（Ｐｄ１～Ｐｄ４）を指すものとする。なお、ダンプトラック２００の存在領域２１０は、これに限定されず、例えば、ダンプトラック２００全体を包含する六面体（例えば直方体又は立方体）であってもよい。また、本実施形態では、ダンプトラック２００の存在領域２１０についての認識部８４の認識結果であるダンプトラック２００の認識領域をＳｄｒ（Ｐｄｒ１～Ｐｄｒ４）とする。 In this embodiment, the vehicle body coordinate system 600 of the dump truck 200 has the center of the rear wheel axle of the dump truck 200 as its origin, the X axis is the direction from the rear wheel to the front wheel, the Y axis is the direction in which the rear wheel axle extends, and the dump truck 200 Let the height direction be the Z axis. It is assumed that the position of the dump truck 200 points to the origin Pd (Xd, Yd, Zd) of the vehicle body coordinate system 600. The attitude of the dump truck 200 refers to an angle θd (θroll, θpitch, θyaw) formed by each axis of the site coordinate system 500 and each axis of the vehicle body coordinate system 600 of the dump truck 200. The existence area 210 of the dump truck 200 refers to a rectangular area Sd (Pd1 to Pd4) whose vertices are points Pd1 to Pd4 at the four corners of the vessel of the dump truck 200. Note that the area 210 of the dump truck 200 is not limited to this, and may be, for example, a hexahedron (for example, a rectangular parallelepiped or a cube) that includes the entire dump truck 200. Further, in this embodiment, the recognition area of the dump truck 200, which is the recognition result of the recognition unit 84 regarding the existence area 210 of the dump truck 200, is Sdr (Pdr1 to Pdr4).
＜領域推定部＞
　領域推定部８５は、トラック情報取得装置５６により取得されたダンプトラック２００の位置及び車格情報（トラック情報）から、ダンプトラック２００が存在すると推定される領域である推定領域を演算する。すなわち、推定領域は、ダンプトラック２００の存在領域２１０についての領域推定部８５の推定結果である。推定領域は、ダンプトラック２００の認識結果の妥当性を検証するべく、認識部８４により演算されたダンプトラック２００の認識領域と比較するために演算される。 <Area estimation section>
The area estimating unit 85 calculates an estimated area, which is an area where the dump truck 200 is estimated to exist, from the position and vehicle class information (truck information) of the dump truck 200 acquired by the truck information acquisition device 56. That is, the estimated area is the estimation result of the area estimating unit 85 regarding the area 210 where the dump truck 200 exists. The estimated area is calculated for comparison with the recognition area of the dump truck 200 calculated by the recognition unit 84 in order to verify the validity of the recognition result of the dump truck 200.
　領域推定部８５によるダンプトラック２００の推定領域の演算方法は、例えば、トラック情報からダンプトラック２００の推定領域を演算する方法を車種毎に予め記憶装置５７に保持しておく。そして、トラック情報取得装置５６により取得されたトラック情報と、当該トラック情報に対応する演算方法とを用いて、対象とするダンプトラック２００の現場座標系５００における推定領域を演算し、油圧ショベル１の車体座標系４００に変換する。これにより、領域推定部８５は、認識部８４により演算されたダンプトラック２００の認識領域と比較可能な、ダンプトラック２００の推定領域を演算することができる。なお、本実施形態では、ダンプトラック２００の存在領域２１０についての領域推定部８５の推定結果であるダンプトラック２００の推定領域をＳｄｅ（Ｐｄｅ１～Ｐｄｅ４）とする。 The method of calculating the estimated area of the dump truck 200 by the area estimation unit 85 is, for example, a method of calculating the estimated area of the dump truck 200 from truck information, which is stored in advance in the storage device 57 for each vehicle type. Then, using the truck information acquired by the truck information acquisition device 56 and the calculation method corresponding to the truck information, an estimated area of the target dump truck 200 in the site coordinate system 500 is calculated, and the estimated area of the target dump truck 200 is calculated. It is converted to the vehicle body coordinate system 400. Thereby, the area estimation unit 85 can calculate an estimated area of the dump truck 200 that can be compared with the recognition area of the dump truck 200 calculated by the recognition unit 84. In this embodiment, the estimated area of the dump truck 200, which is the estimation result of the area estimating unit 85 regarding the existing area 210 of the dump truck 200, is assumed to be Sde (Pde1 to Pde4).
　現場座標系５００から油圧ショベル１の車体座標系４００への変換は、下記の式を用いて行うことができる。油圧ショベル１の車体座標系４００における点データＰｖ（Ｘｖ，Ｙｖ，Ｚｖ）から、現場座標系５００における点データＰｇ（Ｘｇ，Ｙｇ，Ｚｇ）への変換には、例えば、下記の式（４）～式（６）が用いられる。 Conversion from the site coordinate system 500 to the vehicle body coordinate system 400 of the hydraulic excavator 1 can be performed using the following equation. To convert point data Pv (Xv, Yv, Zv) in the vehicle body coordinate system 400 of the hydraulic excavator 1 to point data Pg (Xg, Yg, Zg) in the site coordinate system 500, for example, the following equation (4) is used. ~Equation (6) is used.
Figure JPOXMLDOC01-appb-M000004
Figure JPOXMLDOC01-appb-M000004
Figure JPOXMLDOC01-appb-M000005
Figure JPOXMLDOC01-appb-M000005
Figure JPOXMLDOC01-appb-M000006
Figure JPOXMLDOC01-appb-M000006
　上記の式（４）～式（６）において、Ｒｖｇは、車体座標系４００から現場座標系５００への回転行列である。θｒ、θｐ、θｙは、現場座標系５００における車体座標系４００の各軸が成す角度である。これらの成す角度は、姿勢演算部８１によって演算された値、旋回角度、機械本体３の傾斜角度、及び、測位情報演算部８３によって演算された方位を用いて算出することができる。 In the above equations (4) to (6), Rvg is a rotation matrix from the vehicle body coordinate system 400 to the site coordinate system 500. θr, θp, and θy are angles formed by each axis of the vehicle body coordinate system 400 in the site coordinate system 500. The angles formed by these can be calculated using the value calculated by the attitude calculation section 81, the turning angle, the inclination angle of the machine body 3, and the direction calculated by the positioning information calculation section 83.
　上記の式（４）～式（６）において、Ｔｖｇは、現場座標系５００の原点から車体座標系４００の原点への並進ベクトルである。ｘ０，ｙ０，ｚ０は、現場座標系５００から視た車体座標系４００の原点座標に等しい。Ｔｖｇは、測位情報演算部８３の演算結果を用いることができる。 In the above equations (4) to (6), Tvg is a translation vector from the origin of the site coordinate system 500 to the origin of the vehicle body coordinate system 400. x0, y0, and z0 are equal to the origin coordinates of the vehicle body coordinate system 400 viewed from the site coordinate system 500. The calculation result of the positioning information calculation unit 83 can be used for Tvg.
　また、領域推定部８５は、トラック情報取得装置５６によってトラック情報と同時に取得されたダンプトラック２００の位置調整量を用いてトラック情報に含まれるダンプトラック２００の位置を調整することができる。そして、領域推定部８５は、調整されたダンプトラック２００の位置から、ダンプトラック２００の推定領域を演算することができる。 Furthermore, the area estimation unit 85 can adjust the position of the dump truck 200 included in the truck information using the position adjustment amount of the dump truck 200 acquired simultaneously with the truck information by the truck information acquisition device 56. Then, the area estimation unit 85 can calculate the estimated area of the dump truck 200 from the adjusted position of the dump truck 200.
＜検証部＞
　検証部８６は、認識部８４により演算された認識領域と、領域推定部８５により演算された推定領域とに基づいて、ダンプトラック２００の認識結果の妥当性を検証する。具体的には、検証部８６は、認識領域を推定領域と比較することによって、ダンプトラック２００の認識結果の確度を演算し、演算された確度に応じてダンプトラック２００の認識結果の妥当性を検証する。詳細には、検証部８６は、認識領域と推定領域との重なり具合を示す重合度を、ダンプトラック２００の認識結果の確度として演算する。そして、検証部８６は、演算された重合度が閾値より高い場合、ダンプトラック２００の認識結果が妥当である（補正可能である）と判定する。この際、検証部８６は、タスク取得部８０により取得されたタスクに応じて、認識結果の妥当性の判定基準となる重合度の閾値を変更することができる。 <Verification Department>
The verification unit 86 verifies the validity of the recognition result of the dump truck 200 based on the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85. Specifically, the verification unit 86 calculates the accuracy of the recognition result of the dump truck 200 by comparing the recognition area with the estimated area, and determines the validity of the recognition result of the dump truck 200 according to the calculated accuracy. Verify. Specifically, the verification unit 86 calculates the degree of overlap, which indicates the degree of overlap between the recognition area and the estimated area, as the accuracy of the recognition result of the dump truck 200. Then, when the calculated degree of polymerization is higher than the threshold value, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (correctable). At this time, the verification unit 86 can change the threshold of the degree of overlap, which is a criterion for determining the validity of the recognition result, according to the task acquired by the task acquisition unit 80.
＜補正部＞
　補正部８７は、領域推定部８５により演算された推定領域に基づいて、認識部８４により演算された認識領域を補正する。具体的には、補正部８７は、認識部８４により演算された認識領域を領域推定部８５により演算された推定領域と比較し、認識部８４により演算された認識領域を補正する。これによって、補正部８７は、ダンプトラック２００の認識結果を補正する。この際、補正部８７は、タスク取得部８０により取得されたタスクに応じて、認識部８４により演算された認識領域（認識結果）の補正方法を変更することができる。 <Correction section>
The correction unit 87 corrects the recognition area calculated by the recognition unit 84 based on the estimated area calculated by the area estimation unit 85. Specifically, the correction unit 87 compares the recognition area calculated by the recognition unit 84 with the estimated area calculated by the area estimation unit 85, and corrects the recognition area calculated by the recognition unit 84. Accordingly, the correction unit 87 corrects the recognition result of the dump truck 200. At this time, the correction unit 87 can change the method of correcting the recognition area (recognition result) calculated by the recognition unit 84, depending on the task acquired by the task acquisition unit 80.
　情報処理装置５４は、検証部８６及び補正部８７を用いて、ダンプトラック２００の認識結果の妥当性を検証し、当該認識結果を補正する認識結果検証処理を行う。以下では、認識結果検証処理について詳細に説明する。 The information processing device 54 uses the verification unit 86 and the correction unit 87 to perform a recognition result verification process to verify the validity of the recognition result of the dump truck 200 and correct the recognition result. The recognition result verification process will be described in detail below.
＜認識結果検証処理＞
　図１０は、認識結果検証処理の一例を示すフローチャートである。図１１は、認識領域及び推定領域の重合度を説明する図である。図１２は、油圧ショベル１のタスク毎に、認識結果の妥当性の判定基準と認識領域の補正方法とを定めたテーブルを示す図である。図１３は、積込タスク取得時の認識領域の補正方法を説明する図である。図１４は、リーチングタスク取得時の認識領域の補正方法を説明する図である。図１５は、認識結果の確度が閾値以下である場合の表示装置５５の表示例を示す図である。 <Recognition result verification process>
FIG. 10 is a flowchart illustrating an example of recognition result verification processing. FIG. 11 is a diagram illustrating the degree of polymerization of the recognition area and the estimation area. FIG. 12 is a diagram showing a table in which criteria for determining the validity of recognition results and a method for correcting the recognition area are determined for each task of the hydraulic excavator 1. FIG. 13 is a diagram illustrating a method of correcting a recognition area when acquiring a loading task. FIG. 14 is a diagram illustrating a method of correcting a recognition area when acquiring a reaching task. FIG. 15 is a diagram showing a display example of the display device 55 when the accuracy of the recognition result is less than or equal to the threshold value.
　ステップＳ１２１において、情報処理装置５４の検証部８６は、タスク取得部８０により取得されたタスクを取得する。 In step S121, the verification unit 86 of the information processing device 54 acquires the task acquired by the task acquisition unit 80.
　ステップＳ１２２において、情報処理装置５４の検証部８６は、認識部８４により演算された認識領域を取得する。 In step S122, the verification unit 86 of the information processing device 54 acquires the recognition area calculated by the recognition unit 84.
　ステップＳ１２３において、情報処理装置５４の検証部８６は、領域推定部８５により演算された推定領域を取得する。 In step S123, the verification unit 86 of the information processing device 54 obtains the estimated area calculated by the area estimation unit 85.
　ステップＳ１２４において、情報処理装置５４の検証部８６は、ダンプトラック２００の認識結果の確度を演算する。図１１に示すように、本実施形態では、検証部８６は、ダンプトラック２００の認識結果の確度として、認識部８４により演算された認識領域と、領域推定部８５により演算された推定領域との重なり具合を示す重合度Ａｃｏｖｅｒを演算する。重合度Ａｃｏｖｅｒは、下記の式（７）によって与えられる。 In step S124, the verification unit 86 of the information processing device 54 calculates the accuracy of the recognition result of the dump truck 200. As shown in FIG. 11, in the present embodiment, the verification unit 86 determines the accuracy of the recognition result of the dump truck 200 between the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85. A degree of polymerization Acover indicating the degree of overlap is calculated. The degree of polymerization Acover is given by the following formula (7).
Figure JPOXMLDOC01-appb-M000007
Figure JPOXMLDOC01-appb-M000007
　上記の式（７）において、Ｓｄｅは、領域推定部８５により演算された推定領域の面積を示す。Ｓｃｏｖｅｒは、認識部８４により演算された認識領域（Ｓｄｒ）と、領域推定部８５により演算された推定領域（Ｓｄｅ）とが重なる領域の面積を示す。Ｓｃｏｖｅｒは、例えば、２つの凸多角形同士の交差領域の面積を求める問題に帰着することができるので、数値計算問題として解くことができる。 In the above equation (7), Sde indicates the area of the estimated region calculated by the region estimation unit 85. Scover indicates the area of a region where the recognition region (Sdr) calculated by the recognition section 84 and the estimation region (Sde) calculated by the region estimation section 85 overlap. For example, Scover can be reduced to a problem of finding the area of an intersection area between two convex polygons, and therefore can be solved as a numerical calculation problem.
　ステップＳ１２５において、情報処理装置５４の検証部８６は、演算された重合度Ａｃｏｖｅｒが、閾値Ａｔｈより高いか否かを判定する。これにより、検証部８６は、ダンプトラック２００の認識結果の妥当性を検証する。演算された重合度Ａｃｏｖｅｒが閾値Ａｔｈより高い場合、検証部８６は、ダンプトラック２００の認識結果は妥当である（補正可能である）と判定し、ステップＳ１２６に移行する。演算された重合度Ａｃｏｖｅｒが閾値Ａｔｈ以下である場合、検証部８６は、ダンプトラック２００の認識結果は妥当でない（補正不能である）と判定し、ステップＳ１２７に移行する。 In step S125, the verification unit 86 of the information processing device 54 determines whether the calculated degree of polymerization Acover is higher than the threshold value Ath. Thereby, the verification unit 86 verifies the validity of the recognition result of the dump truck 200. If the calculated degree of polymerization Acover is higher than the threshold Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (correctable), and proceeds to step S126. If the calculated degree of polymerization Acover is less than or equal to the threshold value Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is not valid (cannot be corrected), and proceeds to step S127.
　ここで、検証部８６は、図１２に示すように、タスク取得部８０により取得されたタスクに応じて、認識結果の妥当性の判定基準である重合度Ａｃｏｖｅｒの閾値Ａｔｈを変更することができる。図１２に示すテーブルは、予め記憶装置５７に保存されている。図１２に示すテーブルの欄１２０１に示すように、タスク取得部８０により取得されたタスクが積込タスクである場合、検証部８６は、閾値ＡｔｈとしてＡｔｈ１を設定する。タスク取得部８０により取得されたタスクがリーチングタスクである場合、検証部８６は、閾値ＡｔｈとしてＡｔｈ２を設定する。Ａｔｈ１及びＡｔｈ２は、予め行われた機能検証時における領域推定部８５及び認識部８４の各領域の演算精度、タスク取得時のバケット１０内の対象物の有無、フロント作業装置２及びダンプトラック２００の距離等を加味して決定されることが望ましい。 Here, as shown in FIG. 12, the verification unit 86 can change the threshold Ath of the degree of polymerization Acover, which is a criterion for determining the validity of the recognition result, according to the task acquired by the task acquisition unit 80. . The table shown in FIG. 12 is stored in the storage device 57 in advance. As shown in column 1201 of the table shown in FIG. 12, when the task acquired by the task acquisition unit 80 is a loading task, the verification unit 86 sets Ath1 as the threshold value Ath. When the task acquired by the task acquisition unit 80 is a reaching task, the verification unit 86 sets Ath2 as the threshold value Ath. Ath1 and Ath2 are based on the calculation accuracy of each area of the area estimation unit 85 and recognition unit 84 at the time of functional verification performed in advance, the presence or absence of an object in the bucket 10 at the time of task acquisition, and the accuracy of the front work device 2 and dump truck 200. It is desirable that the distance be determined in consideration of distance, etc.
　積込タスクが指定する積込動作では、油圧ショベル１は、バケット１０内に対象物を有し、フロント作業装置２及びダンプトラック２００の距離が大きい状態で始動する。これにより、フロント作業装置２がダンプトラック２００に衝突した時の損害は、リーチング動作時よりも積込動作時の方が大きいと考えられる。したがって、積込タスクが取得された場合に設定される閾値Ａｔｈ１は、リーチングタスクが取得された場合に設定される閾値Ａｔｈ２と同じ値かそれ以上であることが望ましい。 In the loading operation specified by the loading task, the hydraulic excavator 1 starts with the object in the bucket 10 and the distance between the front working device 2 and the dump truck 200 is large. As a result, it is considered that the damage caused when the front working device 2 collides with the dump truck 200 is greater during the loading operation than during the reaching operation. Therefore, it is desirable that the threshold value Ath1 that is set when a loading task is acquired is the same value as or greater than the threshold value Ath2 that is set when a leeching task is acquired.
　ステップＳ１２６において、情報処理装置５４の補正部８７は、ダンプトラック２００の認識結果を補正する。具体的には、補正部８７は、認識部８４により演算された認識領域を補正する。そして、補正部８７は、ダンプトラック２００の認識結果は妥当である（補正可能である）との判定結果、認識結果の確度（重合度）、及び、補正後の認識領域を、制御装置４０に出力する。更に、補正部８７は、制御装置４０に出力されたこれらの情報を、表示装置５５に表示してオペレータに通知したり、作業現場を管理する管理装置に送信して管理者に通知したりしてもよい。その後、情報処理装置５４は、図１０に示す認識結果検証処理を終了する。 In step S126, the correction unit 87 of the information processing device 54 corrects the recognition result of the dump truck 200. Specifically, the correction unit 87 corrects the recognition area calculated by the recognition unit 84. Then, the correction unit 87 sends the determination result that the recognition result of the dump truck 200 is appropriate (correctable), the accuracy of the recognition result (degree of superposition), and the corrected recognition area to the control device 40. Output. Further, the correction unit 87 displays the information outputted to the control device 40 on the display device 55 to notify the operator, or transmits the information to a management device that manages the work site to notify the manager. It's okay. After that, the information processing device 54 ends the recognition result verification process shown in FIG.
　ここで、補正部８７は、図１２に示すように、タスク取得部８０により取得されたタスクに応じて、認識部８４により演算された認識領域の補正方法を変更することができる。図１２に示すテーブルの欄１２０２に示すように、タスク取得部８０により取得されたタスクが積込タスクである場合、補正部８７は、認識部８４により演算された認識領域を、図１３に示すような認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とを包含する領域（Ｓｄｃ）となるように補正する。認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とを包含する領域は、認識領域（Ｓｄｒ）又は推定領域（Ｓｄｅ）が占めるＯＲ領域を包含する領域（Ｓｄｃ）である。図１３の例では、補正部８７は、認識部８４により演算された認識領域（Ｓｄｒ）を、認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とを包含する矩形の領域であって、その面積が最小となる領域（Ｓｄｃ）に補正している。 Here, as shown in FIG. 12, the correction unit 87 can change the method of correcting the recognition area calculated by the recognition unit 84 according to the task acquired by the task acquisition unit 80. As shown in column 1202 of the table shown in FIG. 12, when the task acquired by the task acquisition unit 80 is a loading task, the correction unit 87 converts the recognition area calculated by the recognition unit 84 into the recognition area shown in FIG. The correction is made so that the area (Sdc) includes the recognition area (Sdr) and the estimated area (Sde). The area that includes the recognition area (Sdr) and the estimation area (Sde) is an area (Sdc) that includes the OR area occupied by the recognition area (Sdr) or the estimation area (Sde). In the example of FIG. 13, the correction unit 87 defines the recognition area (Sdr) calculated by the recognition unit 84 as a rectangular area that includes the recognition area (Sdr) and the estimated area (Sde), and whose area is Correction is made to the minimum area (Sdc).
　積込タスクが指定する積込動作では、油圧ショベル１は、バケット１０をダンプトラック２００の外側の地面からダンプトラック２００の内側のベッセル上方に向かって積込位置まで移動させる。したがって、認識部８４により演算された認識領域が、認識領域と推定領域とを包含する領域に補正されると、積込動作時にダンプトラック２００の外側から内側に向かって移動するフロント作業装置２がダンプトラック２００に衝突するリスクが低減する。よって、タスク取得部８０により取得されたタスクが積込タスクである場合、補正部８７は、認識部８４により演算された認識領域を、認識領域と推定領域とを包含する領域となるように補正する。 In the loading operation specified by the loading task, the hydraulic excavator 1 moves the bucket 10 from the ground outside the dump truck 200 toward the upper side of the vessel inside the dump truck 200 to the loading position. Therefore, when the recognition area calculated by the recognition unit 84 is corrected to include the recognition area and the estimated area, the front working device 2, which moves from the outside to the inside of the dump truck 200 during the loading operation, The risk of colliding with the dump truck 200 is reduced. Therefore, when the task acquired by the task acquisition unit 80 is a loading task, the correction unit 87 corrects the recognition area calculated by the recognition unit 84 so that it becomes an area that includes the recognition area and the estimated area. do.
　補正部８７は、補正後の認識領域の高さ方向（Ｚ軸方向）の位置を、次のように演算することができる。すなわち、補正部８７は、フロント作業装置２とダンプトラック２００の衝突を回避するべく、認識領域及び推定領域の各頂点座標のＺ軸成分の最大値を、補正後の認識領域の高さ方向の位置として用いることができる。 The correction unit 87 can calculate the position of the corrected recognition area in the height direction (Z-axis direction) as follows. That is, in order to avoid a collision between the front working device 2 and the dump truck 200, the correction unit 87 adjusts the maximum value of the Z-axis component of each vertex coordinate of the recognition area and the estimation area to the height direction of the recognition area after correction. Can be used as a position.
　また、補正部８７は、補正後の認識領域の方位ｖｄｃを、認識領域の方位ベクトルｖｄｒと、推定領域の方位ベクトルｖｄｅとを用いて、例えば、下記の式（８）～式（９）から演算することができる。 Further, the correction unit 87 calculates the corrected orientation vdc of the recognition area using the orientation vector vdr of the recognition area and the orientation vector vde of the estimation area, for example, from the following equations (8) to (9). Can be calculated.
Figure JPOXMLDOC01-appb-M000008
Figure JPOXMLDOC01-appb-M000008
Figure JPOXMLDOC01-appb-M000009
Figure JPOXMLDOC01-appb-M000009
　上記の式（９）のゲインＧｋの演算に用いられるΔｄｒ（Δｘｄｒ，Δｙｄｒ，Δｚｄｒ）は、認識部８４における認識領域の演算精度に関する情報である。Δｄｒとしては、例えば、予め外界計測装置７０の計測精度及び認識部８４の演算精度を計測しておいたものを用いることができる。また、認識部８４の認識アルゴリズムに機械学習を用いた識別器を用いている場合、Δｄｒとしては、その識別器が出力する認識確度の情報を用いてもよい。 Δdr (Δxdr, Δydr, Δzdr) used to calculate the gain Gk in the above equation (9) is information regarding the calculation accuracy of the recognition area in the recognition unit 84. As Δdr, for example, the measurement accuracy of the external world measurement device 70 and the calculation accuracy of the recognition unit 84 can be measured in advance. Furthermore, when a discriminator using machine learning is used in the recognition algorithm of the recognition unit 84, information on recognition accuracy output by the discriminator may be used as Δdr.
　上記の式（９）のゲインＧｋの演算に用いられるΔｄｅ（Δｘｄｅ，Δｙｄｅ，Δｚｄｅ）は、領域推定部８５における推定領域の演算精度に関する情報である。Δｄｅとしては、トラック情報取得装置５６がトラック情報と同時に取得可能なダンプトラック２００の位置調整量を用いることができる。すなわち、補正部８７は、トラック情報取得装置５６により取得されたダンプトラック２００の位置調整量を用いて、認識部８４により演算された認識領域を補正することができる。トラック情報取得装置５６が当該位置調整量を取得できない場合、Δｄｅとしては、油圧ショベル１の現場において測位装置６０により取得される測位情報の誤差を予め調査した結果を用いることができる。 Δde (Δxde, Δyde, Δzde) used to calculate the gain Gk in the above equation (9) is information regarding the calculation accuracy of the estimated region in the region estimation unit 85. As Δde, the position adjustment amount of the dump truck 200 that can be acquired by the truck information acquisition device 56 at the same time as the truck information can be used. That is, the correction unit 87 can correct the recognition area calculated by the recognition unit 84 using the position adjustment amount of the dump truck 200 acquired by the truck information acquisition device 56. When the track information acquisition device 56 cannot acquire the position adjustment amount, the result of a preliminary investigation of the error in the positioning information acquired by the positioning device 60 at the site of the hydraulic excavator 1 can be used as Δde.
　また、補正部８７は、バケット１０の積込位置Ｐｄｃ（Ｘｄｃ，Ｙｄｃ，Ｚｄｃ）を演算する。具体的には、補正部８７は、例えば、認識部８４により演算された認識領域におけるベッセルの中央位置Ｐｄｒ０と、領域推定部８５により演算された推定領域におけるベッセルの中央位置Ｐｄｅ０とを演算する。そして、補正部８７は、認識領域におけるベッセルの中央位置Ｐｄｒ０と、推定領域におけるベッセルの中央位置Ｐｄｅ０とを用いて、例えば、下記の式（１０）から演算することができる。 Further, the correction unit 87 calculates the loading position Pdc (Xdc, Ydc, Zdc) of the bucket 10. Specifically, the correction unit 87 calculates, for example, the center position Pdr0 of the vessel in the recognition area calculated by the recognition unit 84 and the center position Pde0 of the vessel in the estimation area calculated by the area estimation unit 85. Then, the correction unit 87 can calculate, for example, from the following equation (10) using the center position Pdr0 of the vessel in the recognition area and the center position Pde0 of the vessel in the estimation area.
Figure JPOXMLDOC01-appb-M000010
Figure JPOXMLDOC01-appb-M000010
　上記の式（１０）において、ゲインＧｋは、Δｄｅを含む上記の式（９）によって与えられる。すなわち、補正部８７は、トラック情報取得装置５６により取得されたダンプトラック２００の位置調整量と、認識領域及び推定領域とに基づいて、バケット１０の積込位置を演算することができる。補正部８７は、演算されたバケット１０の積込位置を制御装置４０に出力する。 In the above equation (10), the gain Gk is given by the above equation (9) including Δde. That is, the correction unit 87 can calculate the loading position of the bucket 10 based on the position adjustment amount of the dump truck 200 acquired by the truck information acquisition device 56, the recognition area, and the estimated area. The correction unit 87 outputs the calculated loading position of the bucket 10 to the control device 40.
　一方、タスク取得部８０により取得されたタスクがリーチングタスクである場合、補正部８７は、認識部８４により演算された認識領域（Ｓｄｒ）を、図１４に示すような認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とが重なる領域に包含される領域（Ｓｄｃ）となるように補正する。認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とが重なる領域に包含される領域（Ｓｄｃ）は、認識領域（Ｓｄｒ）及び推定領域（Ｓｄｅ）の両方が占めるＡＮＤ領域に包含される領域（Ｓｄｃ）である。図１４の例では、補正部８７は、認識部８４により演算された認識領域（Ｓｄｒ）を、認識領域（Ｓｄｒ）と推定領域（Ｓｄｅ）とを包含する矩形の領域であって、その面積が最大となる領域（Ｓｄｃ）に補正している。 On the other hand, when the task acquired by the task acquisition unit 80 is a reaching task, the correction unit 87 estimates the recognition area (Sdr) calculated by the recognition unit 84 to be a recognition area (Sdr) as shown in FIG. Correction is made so that the area (Sdc) is included in the area overlapping with the area (Sde). The area (Sdc) included in the area where the recognition area (Sdr) and the estimation area (Sde) overlap is the area (Sdc) included in the AND area occupied by both the recognition area (Sdr) and the estimation area (Sde). It is. In the example of FIG. 14, the correction unit 87 defines the recognition area (Sdr) calculated by the recognition unit 84 as a rectangular area that includes the recognition area (Sdr) and the estimated area (Sde), and whose area is Correction is made to the maximum area (Sdc).
　リーチングタスクが指定するリーチング動作では、油圧ショベル１は、バケット１０をダンプトラック２００の内側のベッセル上方にある積込位置から、ダンプトラック２００の外側の地面まで移動させる。したがって、認識部８４により演算された認識領域が、認識領域と推定領域とが重なる領域に包含される領域に補正されると、リーチング動作時にダンプトラック２００の内側から外側に向かって移動するフロント作業装置２がダンプトラック２００に衝突するリスクが低減する。よって、タスク取得部８０により取得されたタスクがリーチングタスクである場合、補正部８７は、認識部８４により演算された認識領域を、認識領域と推定領域とが重なる領域に包含される領域となるように補正する。 In the reaching operation specified by the reaching task, the hydraulic excavator 1 moves the bucket 10 from the loading position above the vessel inside the dump truck 200 to the ground outside the dump truck 200. Therefore, when the recognition area calculated by the recognition unit 84 is corrected to an area included in the area where the recognition area and the estimated area overlap, the front work that moves from the inside to the outside of the dump truck 200 during the reaching operation The risk of the device 2 colliding with the dump truck 200 is reduced. Therefore, when the task acquired by the task acquisition unit 80 is a reaching task, the correction unit 87 sets the recognition area calculated by the recognition unit 84 to be an area included in the area where the recognition area and the estimated area overlap. Correct it as follows.
　なお、補正部８７は、積込タスクが取得された場合と同様に、補正後の認識領域の高さ方向（Ｚ軸方向）の位置として、認識領域及び推定領域の各頂点座標のＺ軸成分の最大値を用いることができる。また、補正部８７は、積込タスクが取得された場合と同様に、補正後の認識領域の方位を、上記の式（８）～式（９）から演算することができる。 Note that, similarly to the case where the loading task is acquired, the correction unit 87 uses the Z-axis component of each vertex coordinate of the recognition area and the estimated area as the position in the height direction (Z-axis direction) of the recognition area after correction. The maximum value of can be used. Further, the correction unit 87 can calculate the orientation of the corrected recognition area from the above equations (8) to (9), as in the case where the loading task is acquired.
　ステップＳ１２７において、情報処理装置５４の検証部８６は、ダンプトラック２００の認識結果は妥当でない（補正不能である）との判定結果、認識結果の確度（重合度）、認識領域及び推定領域を、制御装置４０に出力する。更に、検証部８６は、制御装置４０に出力されたこれらの情報を、表示装置５５に表示してオペレータに通知したり、作業現場を管理する管理装置に送信して管理者に通知したりする。この際、検証部８６は、油圧ショベル１の動作を継続してよいか否かを、オペレータ又は管理者等のユーザに確認する。 In step S127, the verification unit 86 of the information processing device 54 determines that the recognition result of the dump truck 200 is invalid (impossible to correct), the accuracy of the recognition result (degree of superimposition), the recognition area, and the estimated area. Output to the control device 40. Furthermore, the verification unit 86 displays the information output to the control device 40 on the display device 55 to notify the operator, or sends it to the management device that manages the work site to notify the manager. . At this time, the verification unit 86 confirms with the user, such as the operator or administrator, whether or not the operation of the hydraulic excavator 1 may be continued.
　例えば、検証部８６は、図１５に示すように、ダンプトラック２００の認識結果が妥当でない（補正不能である）との判定結果をユーザに通知すると共に油圧ショベル１の動作の継続可否をユーザに確認する画面５５１を、表示装置５５に表示させる。画面５５１は、ダンプトラック２００の認識結果が妥当でない（補正不能である）との判定結果をユーザに通知すると共に油圧ショベル１の動作の継続可否をユーザに確認するためのメッセージ５５２を含む。画面５５１は、油圧ショベル１の動作を継続するとの確認結果をユーザが入力するためのボタン５５３を含む。画面５５１は、油圧ショベル１の動作を継続しないとの確認結果をユーザが入力するためのボタン５５４を含む。画面５５１は、油圧ショベル１を自動制御からオペレータの手動操作に切り替えることをユーザが入力するためのボタン５５５を含む。また、画面５５１は、ダンプトラック２００の認識領域（Ｓｄｒ）及び推定領域（Ｓｄｅ）の演算結果を示して、ダンプトラック２００と油圧ショベル１との位置関係をユーザに通知する。 For example, as shown in FIG. 15, the verification unit 86 notifies the user of the determination result that the recognition result of the dump truck 200 is not valid (cannot be corrected), and also informs the user whether or not the operation of the hydraulic excavator 1 can be continued. A screen 551 to be confirmed is displayed on the display device 55. The screen 551 includes a message 552 for notifying the user of the determination result that the recognition result of the dump truck 200 is invalid (impossible to correct) and for confirming with the user whether or not the operation of the hydraulic excavator 1 can be continued. The screen 551 includes a button 553 for the user to input a confirmation result that the operation of the hydraulic excavator 1 will be continued. Screen 551 includes a button 554 for the user to input a confirmation result that the operation of hydraulic excavator 1 is not to be continued. Screen 551 includes a button 555 for the user to input switching the hydraulic excavator 1 from automatic control to manual operation by the operator. Further, the screen 551 shows the calculation results of the recognition area (Sdr) and the estimation area (Sde) of the dump truck 200, and notifies the user of the positional relationship between the dump truck 200 and the hydraulic excavator 1.
　検証部８６は、継続可否の確認結果をユーザから受け付けると、当該確認結果を制御装置４０に出力する。その後、情報処理装置５４は、図１０に示す認識結果検証処理を終了する。 Upon receiving the confirmation result from the user as to whether or not continuation is possible, the verification unit 86 outputs the confirmation result to the control device 40. After that, the information processing device 54 ends the recognition result verification process shown in FIG.
＜制御装置の動作制御＞
　制御装置４０は、情報処理装置５４によるダンプトラック２００の認識結果の妥当性を検証した結果に基づいて、油圧ショベル１の車体の動作（例えば、フロント作業装置２の回動動作、下部走行体５の走行動作、及び、上部旋回体７の旋回動作）を制御する。 <Operation control of control device>
The control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 (for example, the rotational movement of the front working device 2, the lower traveling body 5 (traveling operation of the upper revolving structure 7 and turning operation of the upper revolving structure 7).
　制御装置４０は、認識結果が妥当であった（補正可能であった）場合、タスク取得装置５８により取得されたタスクと、情報処理装置５４から出力された補正後の認識領域とに基づいて、油圧ショベル１の動作を制御する。具体的には、制御装置４０は、積込タスクが取得された場合には、バケット１０が地面から積込位置に到達するまでにフロント作業装置２がダンプトラック２００に衝突しないような軌跡で移動するよう、油圧ショベル１の積込動作を制御する。制御装置４０は、リーチングタスクが取得された場合には、バケット１０が積込位置から次の掘削対象の地面に到達するまでにフロント作業装置２がダンプトラック２００に衝突しないような軌跡で移動するよう、油圧ショベル１のリーチング動作を制御する。 If the recognition result is valid (correctable), the control device 40 performs the following based on the task acquired by the task acquisition device 58 and the corrected recognition area output from the information processing device 54: Controls the operation of the hydraulic excavator 1. Specifically, when the loading task is acquired, the control device 40 causes the front working device 2 to move on a trajectory such that the front working device 2 does not collide with the dump truck 200 before the bucket 10 reaches the loading position from the ground. The loading operation of the hydraulic excavator 1 is controlled so as to do so. When the reaching task is acquired, the control device 40 moves on a trajectory such that the front working device 2 does not collide with the dump truck 200 before the bucket 10 reaches the next excavation target ground from the loading position. Thus, the reaching operation of the hydraulic excavator 1 is controlled.
　また、制御装置４０は、認識結果が妥当でなかった（補正不能であった）場合、油圧ショベル１の動作の継続可否の確認結果が情報処理装置５４から出力されると、当該確認結果に応じて油圧ショベル１の動作を制御する。 Further, when the recognition result is not valid (correction is not possible) and the information processing device 54 outputs the confirmation result as to whether or not the operation of the hydraulic excavator 1 can be continued, the control device 40 responds to the confirmation result. to control the operation of the hydraulic excavator 1.
＜作用効果＞
　以上のように、実施形態１の油圧ショベル１は、対象物をダンプトラック２００に積み込む作業機械である。油圧ショベル１は、油圧ショベル１の車体の周辺環境を計測する外界計測装置７０を備える。油圧ショベル１は、外界計測装置７０の計測結果に基づいて油圧ショベル１の車体の周辺に存在するダンプトラック２００を認識する情報処理装置５４を備える。油圧ショベル１は、情報処理装置５４の認識結果に基づいて油圧ショベル１の車体の動作を制御する制御装置４０を備える。油圧ショベル１は、外部からダンプトラック２００の位置及び車格情報を取得するトラック情報取得装置５６を備える。情報処理装置５４は、トラック情報取得装置５６により取得されたダンプトラック２００の位置及び車格情報に基づいてダンプトラック２００の認識結果を補正する。制御装置４０は、補正されたダンプトラック２００の認識結果に基づいて油圧ショベル１の車体の動作を制御する。 <Effect>
As described above, the hydraulic excavator 1 of the first embodiment is a working machine that loads objects onto the dump truck 200. The hydraulic excavator 1 includes an external world measuring device 70 that measures the surrounding environment of the vehicle body of the hydraulic excavator 1 . The hydraulic excavator 1 includes an information processing device 54 that recognizes dump trucks 200 existing around the vehicle body of the hydraulic excavator 1 based on the measurement results of the external world measuring device 70. The hydraulic excavator 1 includes a control device 40 that controls the operation of the vehicle body of the hydraulic excavator 1 based on the recognition result of the information processing device 54. The hydraulic excavator 1 includes a truck information acquisition device 56 that acquires the position and vehicle class information of the dump truck 200 from the outside. The information processing device 54 corrects the recognition result of the dump truck 200 based on the position and vehicle class information of the dump truck 200 acquired by the truck information acquisition device 56. The control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 based on the corrected recognition result of the dump truck 200.
　これにより、実施形態１の油圧ショベル１は、種々の状況においてもダンプトラック２００の正確な認識結果を用いてダンプトラック２００に対する積込動作等を制御することができる。よって、実施形態１によれば、種々の状況においても運搬機械の位置及び姿勢を正確に認識することで、運搬機械への積込作業を適切に制御することが可能な作業機械を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can control the loading operation, etc. for the dump truck 200 using accurate recognition results of the dump truck 200 even in various situations. Therefore, according to the first embodiment, it is possible to provide a working machine that can appropriately control the loading work on the transport machine by accurately recognizing the position and orientation of the transport machine even in various situations. I can do it.
　従来、油圧ショベル１がダンプトラック２００に対する積込動作を行う場合、外界計測装置７０の計測結果からダンプトラック２００の認識結果を検証することが困難であった。特に、機械学習を用いた認識システムでは、学習データを収集していない現場又はダンプトラック２００に対して、認識結果の不確定性の評価を適切に行うことが困難である。更に、油圧ショベル１の開発会社と認識システムの開発会社とが異なる場合、油圧ショベル１の開発会社には、認識システムのアルゴリズムの細部まで公開されない可能性がある。油圧ショベル１の開発会社は、認識システムの認識結果を用いてダンプトラック２００に衝突することなく適切に積込作業を制御することが可能であるか、事前に十分なリスク評価を行うことが困難である。 Conventionally, when the hydraulic excavator 1 performs a loading operation on the dump truck 200, it has been difficult to verify the recognition results of the dump truck 200 from the measurement results of the external world measuring device 70. In particular, in a recognition system using machine learning, it is difficult to appropriately evaluate the uncertainty of recognition results for sites or dump trucks 200 for which no learning data has been collected. Furthermore, if the development company of the hydraulic excavator 1 and the development company of the recognition system are different, the details of the recognition system's algorithm may not be disclosed to the development company of the hydraulic excavator 1. It is difficult for the developer of Hydraulic Excavator 1 to conduct a sufficient risk assessment in advance to determine whether it is possible to appropriately control loading work without colliding with dump truck 200 using the recognition results of the recognition system. It is.
　これに対し、実施形態１の油圧ショベル１は、トラック情報取得装置５６により取得された第三者からの情報に基づいてダンプトラック２００の認識結果の妥当性を検証し、当該認識結果を補正することができる。そして、実施形態１の油圧ショベル１は、補正された認識結果に基づいて、積込動作等を適切に制御することができる。すなわち、実施形態１の油圧ショベル１は、ダンプトラック２００の位置及び姿勢等の正確な認識結果に基づいて油圧ショベル１の車体の動作を制御することができる。したがって、実施形態１の油圧ショベル１は、ダンプトラック２００との衝突リスクを低減させて、積込動作やリーチング動作を制御できるので、積込作業の安全性及び生産性を向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を適切に制御することが可能な油圧ショベル１を提供することができる。 In contrast, the hydraulic excavator 1 of the first embodiment verifies the validity of the recognition result of the dump truck 200 based on information from a third party acquired by the truck information acquisition device 56, and corrects the recognition result. be able to. The hydraulic excavator 1 of the first embodiment can appropriately control the loading operation and the like based on the corrected recognition result. That is, the hydraulic excavator 1 according to the first embodiment can control the operation of the vehicle body of the hydraulic excavator 1 based on accurate recognition results such as the position and posture of the dump truck 200. Therefore, the hydraulic excavator 1 of the first embodiment can control the loading operation and the reaching operation while reducing the risk of collision with the dump truck 200, thereby improving the safety and productivity of the loading operation. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
　更に、情報処理装置５４は、外界計測装置７０の計測結果からダンプトラック２００を認識し、認識されたダンプトラック２００が存在する領域である認識領域を演算する認識部８４を備える。情報処理装置５４は、トラック情報取得装置５６により取得されたダンプトラック２００の位置及び車格情報からダンプトラック２００が存在すると推定される領域である推定領域を演算する領域推定部８５を備える。情報処理装置５４は、推定領域に基づいて認識領域を補正する補正部８７を備える。 Further, the information processing device 54 includes a recognition unit 84 that recognizes the dump truck 200 from the measurement results of the external world measurement device 70 and calculates a recognition area where the recognized dump truck 200 exists. The information processing device 54 includes an area estimation unit 85 that calculates an estimated area where the dump truck 200 is estimated to exist based on the position and vehicle class information of the dump truck 200 acquired by the truck information acquisition device 56. The information processing device 54 includes a correction unit 87 that corrects the recognition area based on the estimated area.
　これにより、実施形態１の油圧ショベル１は、ダンプトラック２００の存在領域という比較的簡易な指標に基づいて認識結果を補正することができる。したがって、実施形態１の油圧ショベル１は、ダンプトラック２００の位置及び姿勢を正確且つ容易に認識することができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を適切に制御することが可能な油圧ショベル１を容易に提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can correct the recognition result based on a relatively simple index of the area where the dump truck 200 exists. Therefore, the hydraulic excavator 1 of the first embodiment can accurately and easily recognize the position and orientation of the dump truck 200. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be easily provided.
　更に、情報処理装置５４（検証部８６）は、認識領域を推定領域と比較することによって、ダンプトラック２００の認識結果の確度を演算する。補正部８７は、演算された確度に応じて認識領域を補正する。 Further, the information processing device 54 (verification unit 86) calculates the accuracy of the recognition result of the dump truck 200 by comparing the recognition area with the estimated area. The correction unit 87 corrects the recognition area according to the calculated accuracy.
　これにより、実施形態１の油圧ショベル１は、ダンプトラック２００の認識結果の妥当性を定量的に検証することができ、認識領域を正確且つ精細に補正することができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確且つ精細に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can quantitatively verify the validity of the recognition result of the dump truck 200, and can accurately and finely correct the recognition area. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, by accurately and precisely recognizing the position and posture of the dump truck 200 even in various situations, the hydraulic pressure that can more appropriately control the loading operation on the dump truck 200 is realized. A shovel 1 can be provided.
　更に、情報処理装置５４（検証部８６）は、認識領域と推定領域との重なり具合を示す重合度を、認識結果の確度として演算する。補正部８７は、演算された重合度に応じて認識領域を補正する。 Furthermore, the information processing device 54 (verification unit 86) calculates the degree of overlap, which indicates the degree of overlap between the recognition area and the estimated area, as the accuracy of the recognition result. The correction unit 87 corrects the recognition area according to the calculated degree of superposition.
　これにより、実施形態１の油圧ショベル１は、認識結果の確度を明確な基準で演算することができ、認識領域を更に正確且つ精細に補正することができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を更に正確且つ精細に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can calculate the accuracy of the recognition result based on a clear standard, and can correct the recognition area more accurately and finely. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, by recognizing the position and posture of the dump truck 200 more accurately and precisely in various situations, it is possible to more appropriately control the loading operation on the dump truck 200. A hydraulic excavator 1 can be provided.
　更に、トラック情報取得装置５６は、ダンプトラック２００の位置調整量に関する情報を更に取得する。領域推定部８５は、トラック情報取得装置５６により取得された位置調整量を用いて推定領域を演算する。 Furthermore, the truck information acquisition device 56 further acquires information regarding the amount of position adjustment of the dump truck 200. The area estimation unit 85 calculates an estimated area using the position adjustment amount acquired by the track information acquisition device 56.
　これにより、実施形態１の油圧ショベル１は、事前検証等により得られたダンプトラック２００の位置情報の誤差よりも現実に即した値を用いて、推定領域を演算することができる。したがって、実施形態１の油圧ショベル１は、認識領域を更に正確に補正することができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を更に正確に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can calculate the estimated area using a value that is more in line with reality than the error in the position information of the dump truck 200 obtained through preliminary verification or the like. Therefore, the hydraulic excavator 1 of the first embodiment can correct the recognition area more accurately. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator is capable of more accurately controlling the loading operation on the dump truck 200 by more accurately recognizing the position and posture of the dump truck 200 in various situations. 1 can be provided.
　更に、情報処理装置５４は、油圧ショベル１が次に行うタスクを取得するタスク取得部８０を更に備える。補正部８７は、取得されたタスクに応じて、認識領域の補正方法を変更する。 Furthermore, the information processing device 54 further includes a task acquisition unit 80 that acquires the next task to be performed by the hydraulic excavator 1. The correction unit 87 changes the recognition area correction method according to the acquired task.
　これにより、実施形態１の油圧ショベル１は、取得されたタスク毎に最適な認識領域に補正することができるので、油圧ショベル１にタスク毎に最適な動作を行わせることができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を更に正確に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of Embodiment 1 can correct the recognition area to the optimum recognition area for each acquired task, so the hydraulic excavator 1 can perform the optimum operation for each task. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator is capable of more accurately controlling the loading operation on the dump truck 200 by more accurately recognizing the position and posture of the dump truck 200 in various situations. 1 can be provided.
　更に、補正部８７は、対象物をダンプトラック２００に積み込む積込タスクが取得された場合、認識領域と推定領域とを包含する領域となるように認識領域を補正する。 Further, when a loading task for loading an object onto the dump truck 200 is acquired, the correction unit 87 corrects the recognition area so that it becomes an area that includes the recognition area and the estimation area.
　これにより、実施形態１の油圧ショベル１では、積込動作時にダンプトラック２００の外側から内側に向かって移動するフロント作業装置２がダンプトラック２００に衝突するリスクを更に低減することができる。実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, in the hydraulic excavator 1 of the first embodiment, the risk of the front working device 2 moving from the outside to the inside of the dump truck 200 colliding with the dump truck 200 during the loading operation can be further reduced. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
　更に、油圧ショベル１は、対象物を保持するバケット１０を更に備える。トラック情報取得装置５６は、ダンプトラック２００の位置調整量を更に取得する。補正部８７は、対象物をダンプトラック２００に積み込む積込タスクが取得された場合、トラック情報取得装置５６により取得された位置調整量と、認識領域及び推定領域とに基づいて、バケット１０からダンプトラック２００に対象物を積み込む時のバケット１０の位置を演算する。制御装置４０は、補正部８７により演算されたバケット１０の位置に基づいて、油圧ショベル１の積込動作を制御する。 Further, the hydraulic excavator 1 further includes a bucket 10 that holds an object. The truck information acquisition device 56 further acquires the position adjustment amount of the dump truck 200. When the loading task of loading the target object onto the dump truck 200 is acquired, the correction unit 87 performs a dump operation from the bucket 10 based on the position adjustment amount acquired by the truck information acquisition device 56, the recognition area, and the estimated area. The position of the bucket 10 when loading objects onto the truck 200 is calculated. The control device 40 controls the loading operation of the hydraulic excavator 1 based on the position of the bucket 10 calculated by the correction section 87.
　これにより、実施形態１の油圧ショベル１は、バケット１０の積込位置という制御装置４０の制御目標値をより妥当性の高い値にすることができる。したがって、実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can set the control target value of the control device 40, which is the loading position of the bucket 10, to a value with higher validity. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
　更に、補正部８７は、対象物をダンプトラック２００に積み込む積込タスクの終了後に行われるリーチングタスクが取得された場合、認識領域と推定領域とが重なる領域に包含される領域となるように認識領域を補正する。 Further, when a reaching task performed after the loading task of loading the object onto the dump truck 200 is acquired, the correction unit 87 recognizes the area so that it is included in the area where the recognition area and the estimated area overlap. Correct the area.
　これにより、実施形態１の油圧ショベル１は、リーチング動作時にダンプトラック２００の内側から外側に向かって移動するフロント作業装置２がダンプトラック２００に衝突するリスクを更に低減することができる。したがって、実施形態１の油圧ショベル１は、積込作業の安全性及び生産性を更に向上させることができる。よって、実施形態１によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を更に適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the first embodiment can further reduce the risk of the front working device 2 moving from the inside to the outside of the dump truck 200 colliding with the dump truck 200 during the reaching operation. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of loading work. Therefore, according to the first embodiment, the hydraulic excavator 1 is capable of more appropriately controlling the loading operation on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
　更に、情報処理装置５４（検証部８６）は、ダンプトラック２００の認識結果が補正不能と判定した場合、当該認識結果が補正不能との判定結果をユーザに通知すると共に油圧ショベル１の車体の動作の継続可否をユーザに確認する画面５５１を表示装置５５に表示させる。情報処理装置５４（検証部８６）は、継続可否の確認結果をユーザから受け付けると、当該確認結果を制御装置４０に出力する。制御装置４０は、当該確認結果に応じて油圧ショベル１の車体の動作を制御する。 Further, when the information processing device 54 (verification unit 86) determines that the recognition result of the dump truck 200 cannot be corrected, the information processing device 54 (verification unit 86) notifies the user of the determination result that the recognition result cannot be corrected, and also adjusts the operation of the vehicle body of the hydraulic excavator 1. A screen 551 is displayed on the display device 55 for asking the user whether or not to continue. When the information processing device 54 (verification unit 86) receives a confirmation result from the user as to whether or not continuation is possible, the information processing device 54 (verification unit 86) outputs the confirmation result to the control device 40. The control device 40 controls the operation of the vehicle body of the hydraulic excavator 1 according to the confirmation result.
　油圧ショベル１には、周辺に存在する砂埃のような現場環境の影響やセンサの異常等によって、ダンプトラック２００の認識結果が補正不能である判定される可能性がある。このような場合であっても、実施形態１の油圧ショベル１は、オペレータ又は管理者の指示を仰ぐことにより、適切な判断の下で油圧ショベル１の運用を継続することができる。よって、実施形態１によれば、ダンプトラック２００の認識結果を補正不能な場合であっても、ダンプトラック２００への積込作業を適切に制御することが可能な油圧ショベル１を提供することができる。 In the hydraulic excavator 1, there is a possibility that the recognition result of the dump truck 200 will be determined to be uncorrectable due to the influence of the site environment such as surrounding dust or abnormality of the sensor. Even in such a case, the hydraulic excavator 1 of the first embodiment can continue to operate the hydraulic excavator 1 under appropriate judgment by asking the operator or manager for instructions. Therefore, according to the first embodiment, it is possible to provide the hydraulic excavator 1 that can appropriately control the loading work on the dump truck 200 even when the recognition result of the dump truck 200 cannot be corrected. can.
［実施形態２］
　図１６及び図１７を用いて、実施形態２の作業機械について説明する。実施形態２の作業機械において、実施形態１と同様の構成及び動作については、説明を省略する。 [Embodiment 2]
The working machine of Embodiment 2 will be described using FIGS. 16 and 17. In the work machine of Embodiment 2, descriptions of the same configurations and operations as in Embodiment 1 will be omitted.
　図１６は、実施形態２の情報処理装置５４の機能的構成を説明するブロック図である。図１７は、図１６に示すトラック姿勢推定部８８の処理を説明する図である。 FIG. 16 is a block diagram illustrating the functional configuration of the information processing device 54 of the second embodiment. FIG. 17 is a diagram illustrating the processing of the truck attitude estimation unit 88 shown in FIG. 16.
　実施形態１のトラック情報取得装置５６は、ダンプトラック２００の位置、姿勢及び車格情報を含むトラック情報を取得する。但し、ダンプトラック２００の位置・配車管理システムによっては、ダンプトラック２００の地図上の２次元的な位置と方位とを管理しているものの、ダンプトラック２００の姿勢を管理していない場合がある。 The truck information acquisition device 56 of the first embodiment acquires truck information including the position, attitude, and vehicle class information of the dump truck 200. However, depending on the dump truck 200 position/vehicle allocation management system, although the two-dimensional position and orientation of the dump truck 200 on the map are managed, the attitude of the dump truck 200 may not be managed.
　そこで、実施形態２の油圧ショベル１は、油圧ショベル１の周辺の地形情報からダンプトラック２００の姿勢を推定する。具体的には、実施形態２の油圧ショベル１は、図１６に示すように、油圧ショベル１の周辺の地形情報を取得する地形情報取得装置５９を備える。実施形態２の情報処理装置５４は、地形情報取得装置５９により取得された地形情報に基づいてダンプトラック２００の姿勢を推定するトラック姿勢推定部８８を備える。実施形態２の領域推定部８５は、トラック姿勢推定部８８により推定されたダンプトラック２００の姿勢を用いて、推定領域を演算する。 Therefore, the hydraulic excavator 1 of the second embodiment estimates the attitude of the dump truck 200 from topographical information around the hydraulic excavator 1. Specifically, the hydraulic excavator 1 of the second embodiment includes a topographic information acquisition device 59 that acquires topographic information around the hydraulic excavator 1, as shown in FIG. The information processing device 54 of the second embodiment includes a truck attitude estimation unit 88 that estimates the attitude of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59. The area estimating unit 85 of the second embodiment calculates an estimated area using the attitude of the dump truck 200 estimated by the truck attitude estimating unit 88.
　地形情報取得装置５９は、油圧ショベル１の上部旋回体７に取り付けられていてもよい。地形情報取得装置５９は、例えば、外界計測装置７０とセンサを共有して油圧ショベル１の周辺の地形を計測し、その計測結果から情報処理装置５４において地形情報を取得してもよい。或いは、地形情報取得装置５９は、無線通信機能を有し、現場の地形を計測する外部システムから送信された地形情報を受信することによって、地形情報を取得してもよい。本実施形態では、地形情報の取得方法については特に限定されない。 The terrain information acquisition device 59 may be attached to the upper revolving body 7 of the hydraulic excavator 1. For example, the terrain information acquisition device 59 may share a sensor with the external world measurement device 70 to measure the terrain around the hydraulic excavator 1, and the information processing device 54 may acquire the terrain information from the measurement results. Alternatively, the topographic information acquisition device 59 may have a wireless communication function and may obtain topographic information by receiving topographic information transmitted from an external system that measures the topography of the site. In this embodiment, there are no particular limitations on the method of acquiring terrain information.
　本実施形態の地形情報は、点群データとして与えられものとする。しかしながら、地形情報は、点群データに限定されず、例えば、油圧ショベル１の周辺を格子状に分割し、格子毎に高さ情報を保持したグリッド形式のデータであってもよい。 It is assumed that the topographical information in this embodiment is given as point cloud data. However, the terrain information is not limited to point cloud data, and may be data in a grid format, for example, in which the area around the hydraulic excavator 1 is divided into grids and height information is held for each grid.
　トラック姿勢推定部８８は、トラック情報取得装置５６により取得されたトラック情報に含まれるダンプトラック２００の位置、方位及び車格情報と、地形情報取得装置５９により取得された地形情報とを取得する。トラック姿勢推定部８８は、取得されたこれらの情報に基づいて、ダンプトラック２００の存在領域２１０内の地形情報を抽出する。トラック姿勢推定部８８は、抽出された地形情報から、存在領域２１０内の地形の平面推定を行う。平面推定としては、例えば、最小二乗近似を用いることができる。そして、トラック姿勢推定部８８は、平面推定により得られた平面の傾斜角度を演算することによって、ダンプトラック２００の姿勢を推定する。具体的には、トラック姿勢推定部８８は、平面推定により得られた平面の法線ベクトルｎｔと、現場座標系５００のＹ軸及びＺ軸のそれぞれが成す角度から、ダンプトラック２００の姿勢を示す角度θｒｏｌｌ及びθｐｉｔｃｈを演算する。 The truck attitude estimation unit 88 acquires the position, orientation, and vehicle size information of the dump truck 200 included in the truck information acquired by the truck information acquisition device 56 and the terrain information acquired by the terrain information acquisition device 59. The truck attitude estimating unit 88 extracts topographical information within the presence area 210 of the dump truck 200 based on the acquired information. The truck attitude estimation unit 88 estimates the plane of the terrain within the existence area 210 from the extracted terrain information. For example, least squares approximation can be used for plane estimation. Then, the truck attitude estimating unit 88 estimates the attitude of the dump truck 200 by calculating the inclination angle of the plane obtained by plane estimation. Specifically, the truck attitude estimation unit 88 indicates the attitude of the dump truck 200 from the angle formed by the normal vector nt of the plane obtained by the plane estimation and the Y axis and the Z axis of the site coordinate system 500. Calculate angles θroll and θpitch.
　実施形態２の領域推定部８５は、トラック情報取得装置５６により取得されたダンプトラック２００の位置、方位及び車格情報と、トラック姿勢推定部８８により推定されたダンプトラック２００の姿勢とに基づいて、ダンプトラック２００の推定領域を演算する。推定領域の演算方法は、実施形態１と同様である。 The area estimation unit 85 of the second embodiment is based on the position, orientation, and vehicle size information of the dump truck 200 acquired by the truck information acquisition device 56 and the attitude of the dump truck 200 estimated by the truck attitude estimation unit 88. , calculates the estimated area of the dump truck 200. The method of calculating the estimated area is the same as in the first embodiment.
　以上のように、実施形態２の油圧ショベル１は、地形情報取得装置５９を更に備える。実施形態２の情報処理装置５４は、地形情報取得装置５９により取得された地形情報に基づいてダンプトラック２００の姿勢を推定するトラック姿勢推定部８８を備える。実施形態２の領域推定部８５は、トラック姿勢推定部８８により推定されたダンプトラック２００の姿勢を用いて、推定領域を演算する。 As described above, the hydraulic excavator 1 of the second embodiment further includes the terrain information acquisition device 59. The information processing device 54 of the second embodiment includes a truck attitude estimation unit 88 that estimates the attitude of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59. The area estimating unit 85 of the second embodiment calculates an estimated area using the attitude of the dump truck 200 estimated by the truck attitude estimating unit 88.
　これにより、実施形態２の油圧ショベル１は、ダンプトラック２００の位置・配車管理システムがダンプトラック２００の姿勢を管理していない場合であっても、認識領域を正確に補正することができる。よって、実施形態２によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確に認識することで、ダンプトラック２００への積込作業を適切に制御することが可能な油圧ショベル１を提供することができる。 Thereby, the hydraulic excavator 1 of the second embodiment can accurately correct the recognition area even if the dump truck 200 position/dispatch management system does not manage the attitude of the dump truck 200. Therefore, according to the second embodiment, the hydraulic excavator 1 is capable of appropriately controlling the loading work on the dump truck 200 by accurately recognizing the position and posture of the dump truck 200 even in various situations. can be provided.
［実施形態３］
　図１８を用いて、実施形態３の油圧ショベル１について説明する。実施形態３の油圧ショベル１において、実施形態１と同様の構成及び動作については、説明を省略する。 [Embodiment 3]
The hydraulic excavator 1 of Embodiment 3 will be described using FIG. 18. In the hydraulic excavator 1 of Embodiment 3, descriptions of the same configuration and operation as in Embodiment 1 will be omitted.
　図１８は、実施形態３の情報処理装置５４の機能的構成を説明するブロック図である。 FIG. 18 is a block diagram illustrating the functional configuration of the information processing device 54 of the third embodiment.
　実施形態３の情報処理装置５４は、外界計測装置７０により取得された点群データのうち、ダンプトラック２００及びその周辺の点群データだけを用いて、ダンプトラック２００の認識を行う。 The information processing device 54 of the third embodiment recognizes the dump truck 200 using only the point group data of the dump truck 200 and its surroundings among the point group data acquired by the external world measurement device 70.
　具体的には、実施形態３の情報処理装置５４は、外界計測装置７０により取得された点群データから、領域推定部８５により演算された推定領域を含む所定領域内の点群データを抽出するフィルタ部８９を更に備える。実施形態３の認識部８４は、フィルタ部８９により抽出された点群データからダンプトラック２００を認識し、ダンプトラック２００の認識領域を演算する。認識領域の演算方法は、実施形態１と同様である。 Specifically, the information processing device 54 of the third embodiment extracts point cloud data within a predetermined region including the estimated region calculated by the region estimation unit 85 from the point cloud data acquired by the external world measuring device 70. It further includes a filter section 89. The recognition unit 84 of the third embodiment recognizes the dump truck 200 from the point cloud data extracted by the filter unit 89 and calculates the recognition area of the dump truck 200. The method of calculating the recognition area is the same as in the first embodiment.
　外界計測装置７０により取得された点群データがダンプトラック２００以外（他の作業機械や地面）のデータを含む場合、一般的には、認識処理の精度低下や処理時間の長時間化を引き起こす可能性がある。実施形態３の油圧ショベル１は、領域推定部８５により演算された推定領域を用いて外乱となる点群データを取り除くことができる。これにより、実施形態３の油圧ショベル１は、認識処理の精度向上及び処理時間の短縮化を図ることができる。よって、実施形態３によれば、種々の状況においてもダンプトラック２００の位置及び姿勢を正確且つ迅速に認識することで、ダンプトラック２００への積込作業を適切且つ迅速に制御することが可能な油圧ショベル１を提供することができる。 If the point cloud data acquired by the external measurement device 70 includes data from sources other than the dump truck 200 (other work machines or the ground), it may generally cause a decrease in the accuracy of the recognition process or an increase in the processing time. There is sex. The hydraulic excavator 1 according to the third embodiment can use the estimated area calculated by the area estimating section 85 to remove point cloud data that causes disturbance. Thereby, the hydraulic excavator 1 of the third embodiment can improve the accuracy of recognition processing and shorten processing time. Therefore, according to the third embodiment, by accurately and quickly recognizing the position and orientation of the dump truck 200 even in various situations, it is possible to appropriately and quickly control the loading work on the dump truck 200. A hydraulic excavator 1 can be provided.
［その他］
　なお、本発明は上記の実施形態に限定されるものではなく、様々な変形例が含まれる。例えば、上記の実施形態は本発明を分かりやすく説明するために詳細に説明したものであり、必ずしも説明した全ての構成を備えるものに限定されるものではない。また、或る実施形態の構成の一部を他の実施形態の構成に置き換えることが可能であり、また、或る実施形態の構成に他の実施形態の構成を加えることも可能である。また、各実施形態の構成の一部について、他の構成の追加・削除・置換をすることが可能である。 [others]
Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above embodiments have been described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace some of the configurations of each embodiment with other configurations.
　また、上記の各構成、機能、処理部、処理手段等は、それらの一部又は全部を、例えば集積回路にて設計する等によりハードウェアによって実現してもよい。また、上記の各構成、機能等は、プロセッサがそれぞれの機能を実現するプログラムを解釈し、実行することによりソフトウェアによって実現してもよい。各機能を実現するプログラム、テープ、ファイル等の情報は、メモリや、ハードディスク、ＳＳＤ（solid state drive）等の記録装置、又は、ＩＣカード、ＳＤカード、ＤＶＤ等の記録媒体に置くことができる。 Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Further, each of the above-mentioned configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tapes, and files that implement each function can be stored in a memory, a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, SD card, or DVD.
　また、制御線や情報線は説明上必要と考えられるものを示しており、製品上必ずしも全ての制御線や情報線を示しているとは限らない。実際には殆ど全ての構成が相互に接続されていると考えてもよい。 In addition, control lines and information lines are shown that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.
　１…油圧ショベル（作業機械）、５４…情報処理装置、５５…表示装置、５６…トラック情報取得装置（運搬機械情報取得装置）、５９…地形情報取得装置、７０…外界計測装置、８０…タスク取得部、８４…認識部、８５…領域推定部、８６…検証部、８７…補正部、８８…トラック姿勢推定部、８９…フィルタ部、２００…ダンプトラック（運搬機械）、Ｓｄｅ…推定領域、Ｓｄｒ…認識領域 DESCRIPTION OF SYMBOLS 1... Hydraulic excavator (work machine), 54... Information processing device, 55... Display device, 56... Truck information acquisition device (transportation machine information acquisition device), 59... Terrain information acquisition device, 70... External world measurement device, 80... Task Acquisition unit, 84... Recognition unit, 85... Area estimation unit, 86... Verification unit, 87... Correction unit, 88... Truck posture estimation unit, 89... Filter unit, 200... Dump truck (transportation machine), Sde... Estimation area, Sdr…recognition area

Claims (12)

  1. 　対象物を運搬機械に積み込む作業機械であって、
    　前記作業機械の車体の周辺環境を計測する外界計測装置と、
    　前記外界計測装置の計測結果に基づいて前記車体の周辺に存在する前記運搬機械を認識する情報処理装置と、
    　前記情報処理装置の認識結果に基づいて前記車体の動作を制御する制御装置と、
    　外部から前記運搬機械の位置及び車格情報を取得する運搬機械情報取得装置と、を備え、
    　前記情報処理装置は、前記運搬機械情報取得装置により取得された前記運搬機械の位置及び車格情報に基づいて前記運搬機械の認識結果を補正し、
    　前記制御装置は、補正された前記運搬機械の認識結果に基づいて前記車体の動作を制御する
    　ことを特徴とする作業機械。     A working machine that loads objects onto a transport machine,
    an external world measurement device that measures the surrounding environment of the vehicle body of the working machine;
    an information processing device that recognizes the transport machine existing around the vehicle body based on measurement results of the external world measurement device;
    a control device that controls the operation of the vehicle body based on the recognition result of the information processing device;
    A transport machine information acquisition device that acquires the position and vehicle class information of the transport machine from the outside,
    The information processing device corrects the recognition result of the transport machine based on the position and vehicle class information of the transport machine acquired by the transport machine information acquisition device,
    The working machine is characterized in that the control device controls the operation of the vehicle body based on the corrected recognition result of the transport machine.
  2. 　前記情報処理装置は、
    　　前記外界計測装置の計測結果から前記運搬機械を認識し、認識された前記運搬機械が存在する領域である認識領域を演算する認識部と、
    　　前記運搬機械情報取得装置により取得された前記運搬機械の位置及び車格情報から前記運搬機械が存在すると推定される領域である推定領域を演算する領域推定部と、
    　　前記推定領域に基づいて前記認識領域を補正する補正部と、を備える
    　ことを特徴とする請求項１に記載の作業機械。     The information processing device includes:
    a recognition unit that recognizes the transport machine from the measurement results of the external world measurement device and calculates a recognition area that is an area where the recognized transport machine exists;
    an area estimation unit that calculates an estimated area where the transport machine is estimated to exist based on the position and vehicle class information of the transport machine acquired by the transport machine information acquisition device;
    The work machine according to claim 1, further comprising: a correction section that corrects the recognition area based on the estimated area.
  3. 　前記情報処理装置は、前記認識領域を前記推定領域と比較することによって前記認識結果の確度を演算し、
    　前記補正部は、演算された前記確度に応じて前記認識領域を補正する
    　ことを特徴とする請求項２に記載の作業機械。     The information processing device calculates the accuracy of the recognition result by comparing the recognition area with the estimation area,
    The working machine according to claim 2, wherein the correction unit corrects the recognition area according to the calculated accuracy.
  4. 　前記情報処理装置は、前記認識領域と前記推定領域との重なり具合を示す重合度を前記確度として演算し、
    　前記補正部は、演算された前記重合度に応じて前記認識領域を補正する
    　ことを特徴とする請求項３に記載の作業機械。     The information processing device calculates a degree of overlap indicating the degree of overlap between the recognition area and the estimation area as the accuracy;
    The working machine according to claim 3, wherein the correction unit corrects the recognition area according to the calculated degree of polymerization.
  5. 　前記運搬機械情報取得装置は、前記運搬機械の位置調整量に関する情報を更に取得し、　前記領域推定部は、前記運搬機械情報取得装置により取得された前記位置調整量を用いて前記推定領域を演算する
    　ことを特徴とする請求項２に記載の作業機械。     The material handling machine information acquisition device further obtains information regarding the position adjustment amount of the material handling machine, and the area estimating unit calculates the estimated region using the position adjustment amount acquired by the material handling machine information acquisition device. The working machine according to claim 2, characterized in that:
  6. 　前記情報処理装置は、前記作業機械が次に行うタスクを取得するタスク取得部を更に備え、
    　前記補正部は、取得された前記タスクに応じて、前記認識領域の補正方法を変更する
    　ことを特徴とする請求項２に記載の作業機械。     The information processing device further includes a task acquisition unit that acquires a task to be performed next by the work machine,
    The working machine according to claim 2, wherein the correction unit changes the method of correcting the recognition area depending on the acquired task.
  7. 　前記補正部は、前記対象物を前記運搬機械に積み込む積込タスクが取得された場合、前記認識領域と前記推定領域とを包含する領域となるように前記認識領域を補正する
    　ことを特徴とする請求項６に記載の作業機械。     The correction unit is characterized in that, when a loading task for loading the object onto the transporting machine is acquired, the correction unit corrects the recognition area so that it becomes an area that includes the recognition area and the estimation area. The working machine according to claim 6.
  8. 　前記対象物を保持するバケットを更に備え、
    　前記運搬機械情報取得装置は、前記運搬機械の位置調整量に関する情報を更に取得し、　前記補正部は、前記対象物を前記運搬機械に積み込む積込タスクが取得された場合、前記運搬機械情報取得装置により取得された前記位置調整量と、前記認識領域及び前記推定領域と、に基づいて、前記バケットから前記運搬機械に前記対象物を積み込む時の前記バケットの位置を演算し、
    　前記制御装置は、前記補正部により演算された前記バケットの前記位置に基づいて、前記対象物の前記運搬機械への積込動作を制御する
    　ことを特徴とする請求項６に記載の作業機械。     further comprising a bucket that holds the object,
    The transportation machine information acquisition device further acquires information regarding the amount of position adjustment of the transportation machine, and the correction unit is configured to acquire information about the transportation machine information when a loading task for loading the object onto the transportation machine is acquired. Calculating the position of the bucket when loading the object from the bucket to the transporting machine based on the position adjustment amount acquired by the device, the recognition area and the estimation area,
    The working machine according to claim 6, wherein the control device controls the loading operation of the object into the transporting machine based on the position of the bucket calculated by the correction unit.
  9. 　前記補正部は、前記対象物を前記運搬機械に積み込む積込タスクの終了後に行われるリーチングタスクが取得された場合、前記認識領域と前記推定領域とが重なる領域に包含される領域となるように前記認識領域を補正する
    　ことを特徴とする請求項６に記載の作業機械。     The correction unit is configured such that when a reaching task performed after the end of a loading task for loading the object onto the transporting machine is acquired, the area is included in an area where the recognition area and the estimation area overlap. The working machine according to claim 6, wherein the recognition area is corrected.
  10. 　前記情報処理装置は、
    　　前記運搬機械の前記認識結果が補正不能と判定した場合、前記認識結果が補正不能との判定結果をユーザに通知すると共に前記車体の動作の継続可否をユーザに確認する画面を表示装置に表示させ、
    　　前記継続可否の確認結果をユーザから受け付けると、前記確認結果を前記制御装置に出力し、
    　前記制御装置は、前記確認結果に応じて前記車体の動作を制御する
    　ことを特徴とする請求項２に記載の作業機械。     The information processing device includes:
    If the recognition result of the transport machine is determined to be uncorrectable, the display device notifies the user of the determination result that the recognition result is uncorrectable, and displays a screen on the display device for confirming with the user whether or not the operation of the vehicle body can be continued. ,
    Upon receiving the confirmation result from the user as to whether or not the continuation is possible, outputting the confirmation result to the control device;
    The working machine according to claim 2, wherein the control device controls the operation of the vehicle body according to the confirmation result.
  11. 　前記作業機械の周辺の地形情報を取得する地形情報取得装置を更に備え、
    　前記情報処理装置は、前記地形情報取得装置により取得された前記地形情報に基づいて前記運搬機械の姿勢を推定する姿勢推定部を更に備え、
    　前記領域推定部は、前記姿勢推定部により推定された前記運搬機械の姿勢を用いて前記推定領域を演算する
    　ことを特徴とする請求項２に記載の作業機械。     further comprising a terrain information acquisition device that acquires terrain information around the working machine,
    The information processing device further includes a posture estimation unit that estimates a posture of the transport machine based on the topographic information acquired by the topographic information acquisition device,
    The working machine according to claim 2, wherein the area estimating unit calculates the estimated area using the attitude of the transporting machine estimated by the attitude estimating unit.
  12. 　前記外界計測装置は、前記計測結果として前記周辺環境の点群データを取得し、
    　前記情報処理装置は、前記外界計測装置により取得された前記点群データから、前記推定領域を含む所定領域内の前記点群データを抽出するフィルタ部を更に備え、
    　前記認識部は、前記フィルタ部により抽出された前記点群データから前記運搬機械を認識し、前記認識領域を演算する
    　ことを特徴とする請求項２に記載の作業機械。     The external world measurement device acquires point cloud data of the surrounding environment as the measurement result,
    The information processing device further includes a filter unit that extracts the point cloud data within a predetermined area including the estimated area from the point cloud data acquired by the external world measurement device,
    The working machine according to claim 2, wherein the recognition unit recognizes the transport machine from the point cloud data extracted by the filter unit and calculates the recognition area.
PCT/JP2023/013757 2022-04-04 2023-04-03 Work machine WO2023195436A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-062487 2022-04-04
JP2022062487A JP2023152458A (en) 2022-04-04 2022-04-04 Work machine

Publications (1)

Publication Number Publication Date
WO2023195436A1 true WO2023195436A1 (en) 2023-10-12

Family

ID=88242968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/013757 WO2023195436A1 (en) 2022-04-04 2023-04-03 Work machine

Country Status (2)

Country Link
JP (1) JP2023152458A (en)
WO (1) WO2023195436A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000192514A (en) * 1998-12-28 2000-07-11 Hitachi Constr Mach Co Ltd Automatically operating construction machine and operating method thereof
JP2020035380A (en) * 2018-08-31 2020-03-05 株式会社小松製作所 Image processing system, display device, image processing method, generation method of learned model, and learning data set
JP2020126363A (en) * 2019-02-01 2020-08-20 株式会社小松製作所 Image processing system, image processing method, generation method of learnt model, and data set for leaning
JP2021056543A (en) * 2019-09-26 2021-04-08 コベルコ建機株式会社 Container measurement system
WO2022190484A1 (en) * 2021-03-08 2022-09-15 コベルコ建機株式会社 Container measurement system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000192514A (en) * 1998-12-28 2000-07-11 Hitachi Constr Mach Co Ltd Automatically operating construction machine and operating method thereof
JP2020035380A (en) * 2018-08-31 2020-03-05 株式会社小松製作所 Image processing system, display device, image processing method, generation method of learned model, and learning data set
JP2020126363A (en) * 2019-02-01 2020-08-20 株式会社小松製作所 Image processing system, image processing method, generation method of learnt model, and data set for leaning
JP2021056543A (en) * 2019-09-26 2021-04-08 コベルコ建機株式会社 Container measurement system
WO2022190484A1 (en) * 2021-03-08 2022-09-15 コベルコ建機株式会社 Container measurement system

Also Published As

Publication number Publication date
JP2023152458A (en) 2023-10-17

Similar Documents

Publication Publication Date Title
US11530920B2 (en) Controlling movement of a machine using sensor fusion
EP3597831A1 (en) Construction machinery
KR102606049B1 (en) construction machinery
JP2018035645A (en) Work machine image display system
JP7023813B2 (en) Work machine
JP7203616B2 (en) working machine
JP7071203B2 (en) Work machine
US20210246625A1 (en) Control device, loading machine, and control method
US20220254054A1 (en) Construction machine work information generation system and work information generation method
CN113944198B (en) Control method, device and system and working machine
CN114521228A (en) Container measuring system
JP2021001436A (en) Work machine
WO2021256353A1 (en) Construction machine
WO2023195436A1 (en) Work machine
US20230358014A1 (en) Work machine
KR20210000593A (en) Apparatus for generating environment data neighboring construction equipment and construction equipment including the same
US20230144985A1 (en) Positioning system for work machine, work machine, and positioning method for work machine
US20220316188A1 (en) Display system, remote operation system, and display method
JP7390991B2 (en) Work machines and construction support systems
JP7421788B2 (en) Satellite positioning method, satellite positioning device, satellite positioning system and construction machinery
WO2023136326A1 (en) Work machine
JP7489344B2 (en) Terrain information management system and work machine
WO2024004984A1 (en) Work machine
WO2021210427A1 (en) Construction method and construction system
WO2021256528A1 (en) Calibration device and calibration method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23784722

Country of ref document: EP

Kind code of ref document: A1