WO2024005182A1 - Work vehicle path plan generation system and work vehicle path plan generation method - Google Patents

Work vehicle path plan generation system and work vehicle path plan generation method Download PDF

Info

Publication number
WO2024005182A1
WO2024005182A1 PCT/JP2023/024347 JP2023024347W WO2024005182A1 WO 2024005182 A1 WO2024005182 A1 WO 2024005182A1 JP 2023024347 W JP2023024347 W JP 2023024347W WO 2024005182 A1 WO2024005182 A1 WO 2024005182A1
Authority
WO
WIPO (PCT)
Prior art keywords
work vehicle
work
route
route plan
plan generation
Prior art date
Application number
PCT/JP2023/024347
Other languages
French (fr)
Japanese (ja)
Inventor
然波 前田
正憲 逢澤
和久 高濱
Original Assignee
株式会社小松製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社小松製作所 filed Critical 株式会社小松製作所
Publication of WO2024005182A1 publication Critical patent/WO2024005182A1/en

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices

Definitions

  • the present disclosure relates to a route plan generation system for a work vehicle and a route plan generation method for a work vehicle.
  • the excavation plan creation device described in Patent Document 1 uses a machine learning model that inputs topographical information and outputs planned values for an excavation trajectory and turning direction.
  • This excavation plan creation device generates a plurality of machine-learned plan models using excavation efficiency as an evaluation value and using different parameters related to soil quality. Further, this excavation planning device estimates the soil quality, selects a planning model based on the estimated soil quality, inputs topographical information to the selected planning model, and calculates a planning value as an output of the planning model. In addition, this excavation plan creation device estimates topographical information based on time-series data of the position of the blade edge of the bucket, the load of the working machine that supports the bucket, and the estimated soil quality.
  • the excavation plan creation device described in Patent Document 1 can appropriately consider the influence of soil quality when creating an excavation plan, and can create a highly accurate excavation plan.
  • the computational processing load may become large.
  • the present disclosure has been made in view of the above circumstances, and aims to provide a work vehicle route plan generation system and a work vehicle route plan generation method that can efficiently generate a route plan for a work vehicle. shall be.
  • one aspect of the present disclosure provides a route plan generation system for a work vehicle that generates a route plan for performing excavation work on the ground of a construction target area using a work vehicle having a work machine.
  • a position detection unit that detects the position of the work vehicle; terrain shape information that indicates the shape of the terrain within the construction target area; a design surface that indicates the position of the work vehicle; and a target shape within the construction target area.
  • an information storage unit that stores information, and a work equipment route plan that indicates a travel route of the work machine and a travel route of the work vehicle based on the topographical shape information, the position of the work vehicle, and the design surface information.
  • 1 is a route plan generation system for a work vehicle, comprising: a route plan generation unit that generates a travel route plan shown in FIG.
  • a route plan for a work vehicle can be efficiently generated.
  • FIG. 2 is a plan view showing a construction target area in which excavation work is performed by the work vehicle according to the embodiment.
  • FIG. 2 is a side sectional view of a construction target area in which excavation work is performed with a work vehicle according to an embodiment.
  • 1 is a schematic block diagram showing a configuration example of a route plan generation device according to an embodiment.
  • FIG. 2 is a diagram illustrating an example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment.
  • FIG. 6 is a diagram illustrating an example of a simulation model used for simulation of travel route candidates by the route plan generation unit according to the embodiment.
  • FIG. 7 is a diagram illustrating another example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment.
  • FIG. 7 is a diagram illustrating still another example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment.
  • FIG. 2 is a side view showing the earth and sand held in front of the blade and the earth and sand in a windrow formed by overflowing on both sides of the blade when excavating with the working machine according to the embodiment.
  • FIG. 2 is a plan view illustrating the earth and sand held in front of the blade and the earth and sand in a windrow formed by overflowing on both sides of the blade when excavating with the working machine according to the embodiment.
  • FIG. 2 is a side view showing the earth and sand held in front of the blade and the earth and sand in a windrow formed by overflowing on both sides of the blade when excavating with the working machine according to the embodiment.
  • FIG. 3 is a diagram simulating a polygonal column-shaped three-dimensional model of the earth and sand that is held in front of the blade and the earth and sand that overflows and is formed in the windrow on both sides of the blade when excavating with the working machine according to the embodiment.
  • 3 is a flowchart showing the operation of the route plan generation device according to the embodiment.
  • FIG. 2 is a diagram schematically showing a flow of generating a route plan by performing learning using reinforcement learning according to an embodiment.
  • 1 is a schematic block diagram showing a configuration example of a route plan generation system for a work vehicle according to an embodiment.
  • FIG. 1 is a plan view showing a construction target area A in which excavation work is performed with a work vehicle 100 (work machine) according to the embodiment.
  • FIG. 2 is a side sectional view of a construction target area A in which excavation work is performed using the work vehicle 100 according to the embodiment.
  • the work vehicle 100 performs excavation work on the ground G in a predetermined construction target area A.
  • the work vehicle 100 forms an excavated ground surface K along a design surface S designed in advance.
  • the construction target area A shown in FIGS. 1 and 2 is only an example, and its planar shape etc. can be changed as appropriate.
  • the topographic shape of the ground surface of the ground G and the shape of the design surface S are also only examples, and can be changed as appropriate.
  • the work vehicle 100 is automatically operated by remote control at the construction site including the construction target area A, and excavates the ground G.
  • the work vehicle 100 according to the embodiment is, for example, a bulldozer.
  • Work vehicle 100 includes a lower traveling body 110, an upper vehicle body 120, and a working machine 130.
  • the lower traveling body 110 supports the work vehicle 100 so that it can travel.
  • the lower traveling body 110 includes, for example, a pair of left and right crawler belts 110a (also referred to as left crawler belt 110a) and a crawler belt 110b (also referred to as right crawler belt 110b).
  • the left and right crawler tracks 110a and 110b can independently drive the drive wheels to move forward and backward. If the left crawler belt 110a and the right crawler belt 110b are moved forward at the same time, the lower traveling body 110 moves forward, and if the left crawler belt 110a and the right crawler belt 110b are simultaneously moved backward, the lower traveling body 110 moves backward.
  • the lower traveling body 110 is rotated around the turning center. can be rotated to
  • the upper vehicle body 120 is supported on the lower traveling body 110.
  • the upper vehicle body 120 includes a driver's cab 121.
  • the operator's cab 121 is a space in which an operator (driver) rides and operates the work vehicle 100.
  • the work machine 130 includes at least a lift frame 131 and a blade 133.
  • Lift frame 131 is operably attached to undercarriage 110.
  • the blade 133 excavates earth and sand.
  • Blade 133 is operably attached to lift frame 131.
  • the work vehicle 100 excavates the ground G with the blade 133 while moving within the construction target area A along a plurality of travel routes R.
  • each of the plurality of traveling routes R is straight in a plan view, but the traveling route R is not limited to being straight, but may be curved or curved as appropriate depending on the terrain, obstacles, etc. Good too.
  • the plurality of travel routes R are set parallel to each other in a plan view, but they may extend radially, for example.
  • FIG. 3 is a schematic block diagram showing a configuration example of the route plan generation device 20 according to the embodiment.
  • the route plan generation device 20 can be configured using a computer such as a microcomputer or a CPU (Central Processing Unit), and hardware such as peripheral circuits and peripheral devices of the computer.
  • the route plan generation device 20 has an information input section 21, an information storage section 22, a route plan generation section 23, and has a functional configuration composed of a combination of hardware and software such as a program executed by a computer.
  • An information output section 24 is provided.
  • the information input unit 21 receives from the outside terrain shape information indicating the shape of the terrain such as the surface of the ground G in the construction target area A, the position of the work vehicle 100, and design information indicating the target shape in the construction target area A. Accepts input of surface information and.
  • the design information is obtained, for example, from an external CAD (Computer Aided Design) system that designs a construction site including the construction target area A.
  • the terrain shape information is acquired by a detection device such as a radar included in the work vehicle 100.
  • the detection device may be provided in another work vehicle or may be attached to a structure within the construction area A. Further, the detection device may be mounted on a flying vehicle that flies above the construction site.
  • An example of the flying object is an unmanned aerial vehicle (UAV) such as a drone.
  • UAV unmanned aerial vehicle
  • the information storage unit 22 stores the topographic shape information and design surface information input by the information input unit 21. Further, the information storage unit 22 stores various vehicle information regarding the work vehicle 100, such as the size of the blade 133, the driving force of the crawlers 110a and 110b, and the maximum traveling speed.
  • the route plan generation unit 23 generates a route plan for carrying out excavation work in the construction target area A with the work vehicle 100, as will be described in detail later.
  • the information output unit 24 outputs the route plan information generated by the route plan generation unit 23 to the outside.
  • FIG. 4 is a diagram illustrating an example of a moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
  • the route plan generation unit 23 generates a work equipment route plan and a travel route plan for the work vehicle 100 as a route plan for excavating the work vehicle 100 in the construction target area A.
  • the movement route L of the work implement 130 indicates the vertical position and angle of the blade 133 at a plurality of positions on the travel route R when the work vehicle 100 travels along the travel route R.
  • the route plan generation unit 23 generates a work implement route plan that is the optimal travel route L of the work machine 130 and a travel route plan that is the optimal travel route R of the work vehicle 100.
  • the route plan generation unit 23 generates a work machine route plan and a travel route plan that prevent slippage of the tracks 110a and 110b of the work vehicle 100 and increase the excavation efficiency of the blade 133.
  • the route plan generation unit 23 generates a work equipment route plan that is the optimal travel route L of the work machine 130 and a travel route R that is the optimal travel route R of the work vehicle 100 for the plurality of travel routes L and travel route R candidates. Reinforcement learning is performed to generate a route plan. In the present embodiment, simulations are performed multiple times for a plurality of moving routes L with various parameters related to the operation of the working machine 130 and candidates for the driving route R, while learning by reinforcement learning is performed.
  • FIG. 5 is a diagram illustrating an example of a simulation model used by the route plan generation unit 23 according to the embodiment to simulate candidates for the travel route L and the travel route R.
  • the route plan generation unit 23 decomposes the phenomena that occur during excavation by the work vehicle 100 into models for each element. For example, as shown in FIG. 5, the route plan generation unit 23 decomposes a phenomenon that occurs during excavation by the work vehicle 100 into a vehicle body model M10, a control model M20, and an earth and sand model M30, and performs reinforcement learning through simulation. Ru.
  • the simulation is executed by a simulator running on a computer.
  • the vehicle body model M10 relates to parameters related to the operation of the work vehicle 100 on the travel route R and parameters related to the operation of the work implement 130 on the travel route R. More specifically, the vehicle body model M10 includes, for example, a hydraulic model M11 related to hydraulic equipment such as a lift cylinder and a tilt cylinder of the working machine 130, a mechanism model M12 related to the mechanism of movable parts such as a lift frame 131 and a blade 133, and a lower traveling model M11. It includes an undercarriage model M13 related to the body 110.
  • the hydraulic model M11 simulates, for example, relief pressure related to hydraulic equipment such as a lift cylinder and a tilt cylinder of the working machine 130.
  • hydraulic equipment such as a lift cylinder and a tilt cylinder of the working machine 130.
  • the maximum value of the reaction force due to the operation of the lift frame 131 and the blade 133 is set as the maximum reaction force limit.
  • the relief pressure of the hydraulic equipment and the like associated with the operation of the working machine 130 is simulated within a range that does not exceed the maximum reaction force limit.
  • the excavation range that can be excavated by the blade 133 is simulated when movable parts such as the lift frame 131 and the blade 133 are operated based on the movement path L.
  • a position limit of the tip blade is set.
  • the excavation operation by the blade 133 is simulated within a range that does not exceed the position limit of the tip blade.
  • simulations are performed regarding the vehicle speed caused by the drive of the lower traveling body 110, the degree of slippage of the tracks 110a and 110b, and the like.
  • the vehicle speed when the work vehicle 100 travels along the traveling route R by driving the lower traveling body 110 is received by the traveling driving force (traction force) by the tracks 110a and 110b and by pushing the earth and sand on the ground G by the blade 133. It is calculated based on the reaction force.
  • the degree of slippage of the tracks 110a and 110b can be simulated based on the reaction force received from earth and sand when the lift frame 131 and the blade 133 perform an excavation operation, for example.
  • This reaction force is the sum of the excavation resistance (shearing resistance) when the blade 133 excavates the ground G and the soil carrying resistance due to friction when pushing forward the earth and sand held in front of the blade 133.
  • the reaction force reaches the maximum reaction force limit, it can be determined that the shoe slip limit of the tracks 110a and 110b is exceeded and slip occurs.
  • the control model M20 relates to control conditions when performing a simulation based on candidates for the travel route L and the travel route R.
  • the control model M20 includes, for example, a route following model M21 and a route planning model M22.
  • the route following model M21 when performing a simulation, it is assumed that the trajectory followability of the work vehicle 100 for the travel route R and the trajectory followability for the movement route L of the work implement 130 are, for example, 100%. Assume.
  • the route planning model M22 is set for a plurality of travel routes L and driving routes R, which are to be learned by performing simulations a plurality of times.
  • the route planning model M22 generates new travel routes L and travel routes R based on the results of simulations performed on candidates for travel routes L and travel routes R so as to further improve excavation efficiency.
  • a starting point P1 of the moving route L is temporarily set along the traveling route R, and an excavation start position P2 where the blade 133 is lowered to start excavating the ground G.
  • the excavation depth P3 at which the ground G is excavated in one excavation, the penetration angle P4 of the blade 133 into the ground G, the end point P5 of the moving route L, the vehicle speed of the work vehicle 100, etc. are varied, and the simulation target is A plurality of moving route L candidates are sequentially generated.
  • the route planning model M22 after performing a simulation on one candidate travel route L, another travel route L with different conditions is generated, and the simulations are sequentially executed.
  • the earth and sand model M30 simulates parameters related to earth and sand on the ground G to be excavated by the work equipment 130 when the work vehicle 100 is driven and the work equipment 130 is moved based on each candidate of the travel route L and the travel route R. It is for the purpose of The earth and sand model M30 is classified into a terrain model M31 related to topographic changes caused by excavating the ground G with the blade 133, and a reaction force model M32 related to the reaction force that the blade 133 receives when excavating the ground G.
  • the terrain model M31 by excavating the ground G with the blade 133, a simulation is performed on the amount of excavated soil to be excavated.
  • the amount of excavated soil is calculated based on the difference between the topographic shape information of the ground G before excavation and the design surface information about the design surface S in the area along the travel route R.
  • the amount of excavated soil in each excavation is based on the terrain shape before excavation and the one-time excavation along the movement route L. It can be calculated based on the difference from the excavated surface formed by excavation.
  • FIG. 6 is a diagram showing another example of the moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
  • the amount of dirt D1 held is A simulation is performed for a certain amount of soil to be held.
  • FIG. 7 is a diagram showing still another example of the moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
  • the excavated soil may be simulated based on the difference between the amount of excavated soil and the amount of backfilled soil that is the amount of backfilled earth and sand D2.
  • FIG. 8 is a side view showing the dirt D1 held in front of the blade 133 and the windrow dirt D5 formed by overflowing to both sides of the blade 133 when excavating with the working machine 130 according to the embodiment. be.
  • FIG. 9 is a plan view showing the dirt D1 held in front of the blade 133 and the windrow dirt D5 formed by overflowing to both sides of the blade 133 when excavating with the working machine 130 according to the embodiment. be.
  • the terrain model M31 as shown in FIGS. 8 and 9, when the ground G is excavated with the blade 133 based on the travel route L and the travel route R, so-called windrows overflow to both sides in the width direction of the blade 133.
  • a simulation is performed for the amount of earth and sand D5.
  • the amount of earth and sand D1 to be held and the amount of earth and sand D5 in the windrow may be calculated based on, for example, a preliminary experiment using a model.
  • the terrain model M31 for example, when backfilling a part of the earth and sand excavated on the travel route R at another position on the travel route R, the backfilled earth and sand is compacted by the crawler tracks 110a and 110b.
  • a simulation is performed as follows: The terrain model M31 also simulates landslides caused by earth and sand excavated on the travel route R.
  • the reaction force model M32 simulates the excavation resistance that the blade 133 receives from the earth and sand on the ground G when the blade 133 is operated along the movement path L.
  • the excavation resistance increases depending on the depth of excavation into the ground G by the blade 1330.
  • the reaction force model M32 also simulates the excavation resistance that the blade 133 receives from the backfill earth and sand D2.
  • the simulator calculates that when the work vehicle 100 is moved along the travel route R and the work implement 130 is operated along the travel route L, the reaction force that the blade 133 receives from the earth and sand is: If the shoe slip limit of the undercarriage 110 is exceeded, a penalty value (for example, ⁇ 0.05) in reinforcement learning is given.
  • a penalty value for example, ⁇ 0.05
  • the simulator uses reinforcement learning to search for the optimal travel route L and travel route R based on the calculated reward and penalty value, and generates a route plan to generate a work equipment route plan and a travel route plan.
  • Learn section 23 The route plan generation unit 23 generates a work equipment route plan for the entire construction target area A based on the work equipment route plan that is the optimal movement route L and the travel route plan that is the optimal travel route R. It is trained to generate route plans.
  • FIG. 10 shows that when excavating with the working machine 130 according to the embodiment, the earth and sand D1 held in front of the blade 133 and the earth and sand D5 of the windrow formed by overflowing on both sides of the blade 133 are arranged in a polygonal shape. It is a diagram imitating three-dimensional models Dm1 and Dm5.
  • the earth and sand D1 held in front of the blade 133 is converted into a polygonal prism-shaped (for example, triangular prism-shaped) three-dimensional model Dm1.
  • a three-dimensional model of a polygonal prism shape (for example, a quadrangular prism shape (rectangular parallelepiped shape)) is Dm5 is simplified and calculated. This makes it possible to efficiently calculate the amount of windrow earth and sand D5, which changes from moment to moment as the ground G is excavated based on the moving route L. can.
  • FIG. 11 is a flowchart showing the operation of the route plan generation device 20 according to the embodiment.
  • the route plan generation device 20 that has been trained by reinforcement learning is installed in, for example, the work vehicle 100.
  • the route plan generation device 20 generates a route plan for performing excavation work on the ground G of the construction target area A with the work vehicle 100.
  • the information input unit 21 first inputs topographical shape information indicating the shape of the ground G in the construction target area A, the position of the work vehicle 100, and the ground surface in the construction target area A.
  • Design surface information indicating the shape in which G should be excavated is acquired from the outside (S1).
  • the route plan generation unit 23 indicates a travel route R of the work vehicle 100, indicating a travel route L of the work machine 130, based on the terrain shape information, the position of the work vehicle 100, and the design surface information.
  • a travel route plan is generated (S3).
  • the information output unit 24 outputs the generated working machine route plan and traveling route plan to the outside (S4).
  • FIG. 12 is a diagram schematically showing a flow in which the route plan generation unit 23 is learned by reinforcement learning according to the embodiment.
  • Reinforcement learning can be performed using, for example, a simulator that runs on a computer outside the vehicle body 100 and executes a simulation. To do this, first, the simulator sets candidates for the travel route L of the work machine 130 and the candidate travel route R for the work vehicle 100 (S31).
  • the simulator performs a simulation based on the set travel route L candidate and the travel route R candidate for the work vehicle 100.
  • the operation of the work vehicle 100 and the work implement 130 is simulated using the vehicle body model M10 and the control model M20 (S32).
  • the work vehicle 100 travels along the travel route R when operating the work implement 130. Simulate the trajectory.
  • a movement locus of the working machine 130 when the working machine 130 is operated based on the moving route L is simulated.
  • the simulator also calculates the working time when excavating based on the travel route L and the travel route R, based on the data of the travel trajectory of the work vehicle 100 and the travel trajectory of the work equipment 130 that have been calculated. .
  • the simulator uses the earth and sand model M30 to perform a simulation of earth and sand when excavating earth and sand on the ground G based on the data of the travel trajectory of the work vehicle 100 and the movement trajectory of the work implement 130 ( S33).
  • the earth and sand model M30 for example, excavation resistance when excavating earth and sand on the ground G is calculated.
  • the calculated excavation resistance data is fed back to the vehicle body model M10 and reflected in the calculation of the vehicle speed of the work vehicle 100, etc.
  • the simulator uses the earth and sand model M30 to calculate a post-excavation topography that indicates the shape of the ground G after excavation by the working machine 130. Further, the route plan generation unit 23 uses the earth and sand model M30 to calculate the amount of earth excavated by the working machine 130, the amount of earth held, the amount of earth in the windrow, etc.
  • the simulator learns the route plan generation unit 23 by reinforcement learning based on the work time and the amount of excavated soil calculated as described above (S34). In reinforcement learning, rewards and penalties are calculated, and candidates for the travel route L and the travel route R are evaluated. At this time, the simulator shows that when the work vehicle 100 is moved along the travel route R and the work machine 130 excavates based on the travel route L, the excavation resistance exceeds the shoe slip limit of the tracks 110a and 110b. Determine whether or not the limit has been exceeded. As a result, if the excavation resistance exceeds the shoe slip limit, it is determined that the work vehicle 100 is slipping.
  • the simulator performs a simulation on one travel route L and a candidate travel route R, then performs learning based on the evaluation results for the travel route L and candidate travel route R, and then performs a simulation.
  • a candidate travel route L or travel route R is set.
  • the candidate travel route L or travel route R to be simulated next is set so that the work vehicle 100 does not slip and the reward expressed by the above formula (1) is as large as possible.
  • the route plan generation unit 23 is trained to generate a work machine route plan and a travel route plan that prevent the work vehicle 100 from slipping and have high excavation efficiency.
  • FIG. 13 is a schematic block diagram showing a configuration example of a route plan generation system 50 for a work vehicle according to the embodiment. As shown in FIG. 13, the work vehicle route plan generation system 50 includes a remote control device 60 and a work vehicle 100.
  • the remote control device 60 includes a communication section 61, an information output section 62, and a control section 63.
  • the communication unit 61 can communicate with the work vehicle 100 using a public wireless communication network or wireless communication means.
  • the information output unit 62 outputs information necessary for automatically driving the work vehicle 100.
  • the information output unit 62 acquires, for example, design surface information of the construction target area A from an external CAD system, etc., and transmits it to the work vehicle 100 via the communication unit 61.
  • the control unit 63 monitors the operating state of each part of the work vehicle 100 based on information detected by various sensors included in the work vehicle 100.
  • the work vehicle 100 automatically operates based on the route plan generated by route plan generation device 20. For this reason, the work vehicle 100 includes a communication section 71, the route plan generation device 20, a position detection section 72, a route plan storage section 73, and a vehicle control section 74.
  • the communication unit 71 can communicate with the communication unit 61 of the remote control device 60 via a public wireless communication network or wireless communication means.
  • a part or all of the configuration of work vehicle 100 may be included in remote control device 60.
  • the route plan generation device 20 shown above is provided on the work vehicle 100 side in this embodiment.
  • the route plan generation device 20 may be provided on the remote control device 60 side.
  • part or all of the configuration of the remote control device 60 may be provided in the work vehicle 100.
  • the position detection unit 72 is provided in the work vehicle 100.
  • the position detection unit 72 uses, for example, GPS, and is capable of detecting the position of the work vehicle 100.
  • the route plan storage unit 73 stores the work equipment route plan and travel route plan of the work vehicle 100, which are generated by the route plan generation device 20 and output to the outside.
  • the vehicle control unit 74 controls the operation of each part of the work machine 130 and the work vehicle 100 based on the work machine route plan and the travel route plan stored in the route plan storage unit 73.
  • the work vehicle 100 performs work while moving the work vehicle 100 based on a work implement route plan that is the optimal travel route L for the work machine 130 and a travel route plan that is the optimal travel route R for the work vehicle 100.
  • the machine 130 is operated. Thereby, the ground G can be excavated efficiently.
  • the work vehicle 100 is a bulldozer, but is not limited to this.
  • the work vehicle 100 may be a work machine including a work machine such as a hydraulic excavator, a wheel loader, a motor grader, and a running body.
  • part or all of the program executed by the computer in the above embodiments can be distributed via a computer-readable recording medium or a communication line.
  • a route plan for a work vehicle can be efficiently generated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mining & Mineral Resources (AREA)
  • Civil Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Structural Engineering (AREA)
  • Operation Control Of Excavators (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

An embodiment of the present disclosure is a work vehicle path plan generation system for generating a path plan in order for a work vehicle having a work machine to perform excavation work of the ground in a work area, said system comprising: a position detection unit for detecting a position of the work vehicle; an information storage unit for storing topographic information indicating topography in the work area, the position of the work vehicle, and design surface information indicating a shape into which the ground in the work area should be excavated; and a path plan generation unit for generating a travel path plan indicating a travel path of the work vehicle and including multiple work machine path plans on the basis of the topographic information, the position of the work vehicle, and the design surface information.

Description

作業車両の経路計画生成システム、作業車両の経路計画生成方法Work vehicle route plan generation system, work vehicle route plan generation method
　本開示は、作業車両の経路計画生成システム、作業車両の経路計画生成方法に関する。
　本願は、２０２２年６月３０日に、日本に出願された特願２０２２－１０６３７２号に基づき優先権を主張し、その内容をここに援用する。 The present disclosure relates to a route plan generation system for a work vehicle and a route plan generation method for a work vehicle.
This application claims priority based on Japanese Patent Application No. 2022-106372 filed in Japan on June 30, 2022, the contents of which are incorporated herein.
　特許文献１に記載されている掘削計画作成装置では、地形情報を入力し、掘削軌跡と旋回方向の計画値を出力する機械学習モデルを用いる。この掘削計画作成装置では、掘削効率を評価値として土質に係るパラメータをそれぞれ異ならせて機械学習された複数の計画モデルを生成する。さらに、この掘削計画作成装置では、土質を推定し、推定した土質に基づいて計画モデルを選択し、選択した計画モデルに、地形情報を入力し、計画モデルの出力として計画値を算出する。また、この掘削計画作成装置では、バケットの刃先位置とバケットを支持する作業機の負荷の時系列データと、推定した土質とに基づき、地形情報を推定する。 The excavation plan creation device described in Patent Document 1 uses a machine learning model that inputs topographical information and outputs planned values for an excavation trajectory and turning direction. This excavation plan creation device generates a plurality of machine-learned plan models using excavation efficiency as an evaluation value and using different parameters related to soil quality. Further, this excavation planning device estimates the soil quality, selects a planning model based on the estimated soil quality, inputs topographical information to the selected planning model, and calculates a planning value as an output of the planning model. In addition, this excavation plan creation device estimates topographical information based on time-series data of the position of the blade edge of the bucket, the load of the working machine that supports the bucket, and the estimated soil quality.
日本国特開２０２１－１８８３６２号公報Japanese Patent Application Publication No. 2021-188362
　特許文献１に記載されている掘削計画作成装置では、掘削計画を作成する際に土質の影響を適切に考慮し、精度の高い掘削計画を作成することができる。その一方で、例えば、バケットの刃先位置とバケットを支持する作業機の負荷の時系列データと、土質とに基づいた推定演算等を行う必要がある。特に、土質について、土砂を粒子モデルとして取り扱った場合、演算処理の負荷が大きくなる可能性があった。 The excavation plan creation device described in Patent Document 1 can appropriately consider the influence of soil quality when creating an excavation plan, and can create a highly accurate excavation plan. On the other hand, it is necessary to perform estimation calculations based on, for example, time-series data of the position of the blade edge of the bucket, the load of the working machine that supports the bucket, and the soil quality. In particular, with regard to soil quality, when soil is treated as a particle model, the computational processing load may become large.
　本開示は、上記事情に鑑みてなされたものであり、作業車両の経路計画を、効率良く生成することができる作業車両の経路計画生成システム、作業車両の経路計画生成方法を提供することを目的とする。 The present disclosure has been made in view of the above circumstances, and aims to provide a work vehicle route plan generation system and a work vehicle route plan generation method that can efficiently generate a route plan for a work vehicle. shall be.
　上記課題を解決するため、本開示の一態様は、作業機を有した作業車両で、施工対象エリアの地面に対して掘削作業を行うための経路計画を生成する作業車両の経路計画生成システムであって、前記作業車両の位置を検出する位置検出部と、前記施工対象エリア内の地形の形状を示す地形形状情報、前記作業車両の位置、及び前記施工対象エリア内の目標形状を示す設計面情報を記憶する情報記憶部と、前記地形形状情報、前記作業車両の位置、及び前記設計面情報に基づいて、前記作業機の移動経路を示す作業機経路計画、及び前記作業車両の走行経路を示す走行経路計画を生成する経路計画生成部と、を備える作業車両の経路計画生成システムである。 In order to solve the above problems, one aspect of the present disclosure provides a route plan generation system for a work vehicle that generates a route plan for performing excavation work on the ground of a construction target area using a work vehicle having a work machine. a position detection unit that detects the position of the work vehicle; terrain shape information that indicates the shape of the terrain within the construction target area; a design surface that indicates the position of the work vehicle; and a target shape within the construction target area. an information storage unit that stores information, and a work equipment route plan that indicates a travel route of the work machine and a travel route of the work vehicle based on the topographical shape information, the position of the work vehicle, and the design surface information. 1 is a route plan generation system for a work vehicle, comprising: a route plan generation unit that generates a travel route plan shown in FIG.
　本開示の各態様によれば、作業車両の経路計画を、効率良く生成することができる。 According to each aspect of the present disclosure, a route plan for a work vehicle can be efficiently generated.
実施形態に係る作業車両で掘削作業を行う施工対象エリアを示す平面図である。FIG. 2 is a plan view showing a construction target area in which excavation work is performed by the work vehicle according to the embodiment. 実施形態に係る作業車両で掘削作業を行う施工対象エリアの側断面図である。FIG. 2 is a side sectional view of a construction target area in which excavation work is performed with a work vehicle according to an embodiment. 実施形態に係る経路計画生成装置の構成例を示す概略ブロック図である。1 is a schematic block diagram showing a configuration example of a route plan generation device according to an embodiment. 実施形態に係る作業車両の走行経路における、作業機の移動経路の一例を示す図である。FIG. 2 is a diagram illustrating an example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment. 実施形態に係る経路計画生成部で、移動経路の候補についてのシミュレーションに用いるシミュレーションモデルの例を示す図である。FIG. 6 is a diagram illustrating an example of a simulation model used for simulation of travel route candidates by the route plan generation unit according to the embodiment. 実施形態に係る作業車両の走行経路における、作業機の移動経路の他の一例を示す図である。FIG. 7 is a diagram illustrating another example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment. 実施形態に係る作業車両の走行経路における、作業機の移動経路のさらに他の一例を示す図である。FIG. 7 is a diagram illustrating still another example of a moving route of a working machine in a traveling route of a working vehicle according to an embodiment. 実施形態に係る作業機で掘削を行った際に、ブレードの前方で抱え込む土砂、及びブレードの両側にあふれ出して形成されるウィンドローの土砂を示す側面図である。FIG. 2 is a side view showing the earth and sand held in front of the blade and the earth and sand in a windrow formed by overflowing on both sides of the blade when excavating with the working machine according to the embodiment. 実施形態に係る作業機で掘削を行った際に、ブレードの前方で抱え込む土砂、及びブレードの両側にあふれ出して形成されるウィンドローの土砂を示す平面図である。FIG. 2 is a plan view illustrating the earth and sand held in front of the blade and the earth and sand in a windrow formed by overflowing on both sides of the blade when excavating with the working machine according to the embodiment. 実施形態に係る作業機で掘削を行った際に、ブレードの前方で抱え込む土砂、及びブレードの両側にあふれ出して形成されるウィンドローの土砂を、多角柱状の立体モデルに模した図である。FIG. 3 is a diagram simulating a polygonal column-shaped three-dimensional model of the earth and sand that is held in front of the blade and the earth and sand that overflows and is formed in the windrow on both sides of the blade when excavating with the working machine according to the embodiment. 実施形態に係る経路計画生成装置の動作を示すフローチャートである。3 is a flowchart showing the operation of the route plan generation device according to the embodiment. 実施形態に係る強化学習による学習を行うことで、経路計画を生成する流れの概略を示す図である。FIG. 2 is a diagram schematically showing a flow of generating a route plan by performing learning using reinforcement learning according to an embodiment. 実施形態に係る作業車両の経路計画生成システムの構成例を示す概略ブロック図である。1 is a schematic block diagram showing a configuration example of a route plan generation system for a work vehicle according to an embodiment.
　以下、図面を参照して本開示の実施形態について説明する。なお、各図において同一または対応する構成には同一の符号を用いて説明を適宜省略する。 Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In addition, in each figure, the same reference numerals are used for the same or corresponding components, and the description thereof will be omitted as appropriate.
（作業車両１００の構成例）
　図１は、実施形態に係る作業車両１００（作業機械）で掘削作業を行う施工対象エリアＡを示す平面図である。図２は、実施形態に係る作業車両１００で掘削作業を行う施工対象エリアＡの側断面図である。
　図１、図２に示すように、本実施形態において、作業車両１００は、予め定められた施工対象エリアＡの地面Ｇに対して掘削作業を行う。作業車両１００は、地面Ｇを掘削することによって、予め設計された設計面Ｓに沿った掘削地表面Ｋを形成する。なお、図１、図２に示した施工対象エリアＡは一例に過ぎず、その平面形状等は適宜変更可能である。また、地面Ｇの地表面の地形形状、設計面Ｓ（掘削地表面Ｋ）の形状も一例に過ぎず、適宜変更可能である。 (Example of configuration of work vehicle 100)
FIG. 1 is a plan view showing a construction target area A in which excavation work is performed with a work vehicle 100 (work machine) according to the embodiment. FIG. 2 is a side sectional view of a construction target area A in which excavation work is performed using the work vehicle 100 according to the embodiment.
As shown in FIGS. 1 and 2, in this embodiment, the work vehicle 100 performs excavation work on the ground G in a predetermined construction target area A. By excavating the ground G, the work vehicle 100 forms an excavated ground surface K along a design surface S designed in advance. Note that the construction target area A shown in FIGS. 1 and 2 is only an example, and its planar shape etc. can be changed as appropriate. Further, the topographic shape of the ground surface of the ground G and the shape of the design surface S (excavated ground surface K) are also only examples, and can be changed as appropriate.
　作業車両１００は、施工対象エリアＡを含む施工現場にて、遠隔操作により自動的に稼働し、地面Ｇを掘削する。実施形態に係る作業車両１００は、一例としてブルドーザである。作業車両１００は、下部走行体１１０、上部車体１２０、および作業機１３０を備える。 The work vehicle 100 is automatically operated by remote control at the construction site including the construction target area A, and excavates the ground G. The work vehicle 100 according to the embodiment is, for example, a bulldozer. Work vehicle 100 includes a lower traveling body 110, an upper vehicle body 120, and a working machine 130.
　下部走行体１１０は、作業車両１００を走行可能に支持する。下部走行体１１０は、例えば左右一対の履帯１１０ａ（左履帯１１０ａとも称する）および履帯１１０ｂ（右履帯１１０ｂとも称する）を備える。左右それぞれの履帯１１０ａおよび１１０ｂは、駆動輪を独立して駆動して前進および後進させることができる。左履帯１１０ａと右履帯１１０ｂを同時に前進させれば下部走行体１１０は前進し、左履帯１１０ａと右履帯１１０ｂを同時に後進させれば下部走行体１１０は後進する。また、一方の履帯の駆動輪と、他方の履帯の駆動輪を互いに逆向きに駆動、例えば右履帯１１０ｂを前進させると同時に左履帯１１０ａを後進させると、下部走行体１１０は、旋回中心を中心に回転することができる。 The lower traveling body 110 supports the work vehicle 100 so that it can travel. The lower traveling body 110 includes, for example, a pair of left and right crawler belts 110a (also referred to as left crawler belt 110a) and a crawler belt 110b (also referred to as right crawler belt 110b). The left and right crawler tracks 110a and 110b can independently drive the drive wheels to move forward and backward. If the left crawler belt 110a and the right crawler belt 110b are moved forward at the same time, the lower traveling body 110 moves forward, and if the left crawler belt 110a and the right crawler belt 110b are simultaneously moved backward, the lower traveling body 110 moves backward. Furthermore, when the drive wheels of one crawler track and the drive wheels of the other track are driven in opposite directions, for example, when the right track 110b is moved forward and the left track 110a is moved backward at the same time, the lower traveling body 110 is rotated around the turning center. can be rotated to
　上部車体１２０は、下部走行体１１０上に支持される。上部車体１２０は、運転室１２１を備えている。運転室１２１は、オペレータ（運転者）が搭乗し、作業車両１００の操作を行うためのスペースである。 The upper vehicle body 120 is supported on the lower traveling body 110. The upper vehicle body 120 includes a driver's cab 121. The operator's cab 121 is a space in which an operator (driver) rides and operates the work vehicle 100.
　作業機１３０は、リフトフレーム１３１、及びブレード１３３を少なくとも備えている。リフトフレーム１３１は、下部走行体１１０に、動作可能に取り付けられる。ブレード１３３は、土砂などを掘削する。ブレード１３３は、リフトフレーム１３１に動作可能に取り付けられる。 The work machine 130 includes at least a lift frame 131 and a blade 133. Lift frame 131 is operably attached to undercarriage 110. The blade 133 excavates earth and sand. Blade 133 is operably attached to lift frame 131.
　図１に示すように、作業車両１００は、施工対象エリアＡ内を、複数の走行経路Ｒに沿って移動しながら、ブレード１３３で地面Ｇを掘削する。図１において、複数の走行経路Ｒの各々は、平面視で直線状としたが、走行経路Ｒは、直線状に限らず、地形や障害物等に応じて、適宜湾曲、あるいは屈曲していてもよい。また、図１において、複数の走行経路Ｒは、平面視で互いに平行に設定されているが、例えば放射状に延びる等していてもよい。 As shown in FIG. 1, the work vehicle 100 excavates the ground G with the blade 133 while moving within the construction target area A along a plurality of travel routes R. In FIG. 1, each of the plurality of traveling routes R is straight in a plan view, but the traveling route R is not limited to being straight, but may be curved or curved as appropriate depending on the terrain, obstacles, etc. Good too. Further, in FIG. 1, the plurality of travel routes R are set parallel to each other in a plan view, but they may extend radially, for example.
（経路計画生成装置２０の構成例）
　図３は、実施形態に係る経路計画生成装置２０の構成例を示す概略ブロック図である。
　図３に示すように、経路計画生成装置２０は、マイクロコンピュータ、ＣＰＵ（Central Processing Unit）等のコンピュータと、コンピュータの周辺回路や周辺装置等のハードウェアを用いて構成することができる。経路計画生成装置２０は、ハードウェアと、コンピュータが実行するプログラム等のソフトウェアとの組み合わせから構成される機能的構成として、情報入力部２１と、情報記憶部２２と、経路計画生成部２３と、情報出力部２４と、を備える。 (Example of configuration of route plan generation device 20)
FIG. 3 is a schematic block diagram showing a configuration example of the route plan generation device 20 according to the embodiment.
As shown in FIG. 3, the route plan generation device 20 can be configured using a computer such as a microcomputer or a CPU (Central Processing Unit), and hardware such as peripheral circuits and peripheral devices of the computer. The route plan generation device 20 has an information input section 21, an information storage section 22, a route plan generation section 23, and has a functional configuration composed of a combination of hardware and software such as a program executed by a computer. An information output section 24 is provided.
　情報入力部２１は、外部から、施工対象エリアＡ内の地面Ｇの地表面等の地形の形状を示す地形形状情報と、作業車両１００の位置と、施工対象エリアＡ内の目標形状を示す設計面情報と、の入力を受け付ける。設計面情報は、例えば、施工対象エリアＡを含む施工現場の設計を行う、外部のＣＡＤ(Computer Aided Design)システムから取得される。地形形状情報は、作業車両１００に備えられたレーダ等の検出装置により取得される。検出装置は、他の作業車両に備えられても良いし、施工エリアＡ内の構造物に取り付けられても良い。また、検出装置は、施工現場の上方を飛行する飛行体に搭載されてもよい。飛行体として、ドローンのような無人航空機（ＵＡＶ：Unmanned Aerial Vehicle）が例示される。 The information input unit 21 receives from the outside terrain shape information indicating the shape of the terrain such as the surface of the ground G in the construction target area A, the position of the work vehicle 100, and design information indicating the target shape in the construction target area A. Accepts input of surface information and. The design information is obtained, for example, from an external CAD (Computer Aided Design) system that designs a construction site including the construction target area A. The terrain shape information is acquired by a detection device such as a radar included in the work vehicle 100. The detection device may be provided in another work vehicle or may be attached to a structure within the construction area A. Further, the detection device may be mounted on a flying vehicle that flies above the construction site. An example of the flying object is an unmanned aerial vehicle (UAV) such as a drone.
　情報記憶部２２は、情報入力部２１で入力を受け付けた、地形形状情報、及び設計面情報を記憶する。また、情報記憶部２２には、作業車両１００に関する各種の車両情報、例えば、ブレード１３３の大きさ、履帯１１０ａ、１１０ｂによる走行駆動力、走行最大速度等が記憶されている。 The information storage unit 22 stores the topographic shape information and design surface information input by the information input unit 21. Further, the information storage unit 22 stores various vehicle information regarding the work vehicle 100, such as the size of the blade 133, the driving force of the crawlers 110a and 110b, and the maximum traveling speed.
　経路計画生成部２３は、後に詳述するように、施工対象エリアＡ内を作業車両１００で掘削施工するための、経路計画を生成する。
　情報出力部２４は、経路計画生成部２３によって生成された経路計画の情報を、外部に出力する。 The route plan generation unit 23 generates a route plan for carrying out excavation work in the construction target area A with the work vehicle 100, as will be described in detail later.
The information output unit 24 outputs the route plan information generated by the route plan generation unit 23 to the outside.
（経路計画生成部２３の構成例）
　図４は、実施形態に係る作業車両１００の走行経路Ｒにおける、作業機１３０の移動経路Ｌの一例を示す図である。
　経路計画生成部２３は、施工対象エリアＡ内を作業車両１００で掘削施工するための経路計画として、作業車両１００の作業機経路計画、及び走行経路計画を生成する。作業機１３０の移動経路Ｌは、作業車両１００が走行経路Ｒに沿って走行する際に、走行経路Ｒ上の複数の位置における、ブレード１３３の上下方向の位置、角度を示すものである。 (Example of configuration of route plan generation unit 23)
FIG. 4 is a diagram illustrating an example of a moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
The route plan generation unit 23 generates a work equipment route plan and a travel route plan for the work vehicle 100 as a route plan for excavating the work vehicle 100 in the construction target area A. The movement route L of the work implement 130 indicates the vertical position and angle of the blade 133 at a plurality of positions on the travel route R when the work vehicle 100 travels along the travel route R.
　経路計画生成部２３は、最適な作業機１３０の移動経路Ｌである作業機経路計画、及び最適な作業車両１００の走行経路Ｒである走行経路計画を生成する。経路計画生成部２３は、作業車両１００の履帯１１０ａ、１１０ｂのスリップが生じず、かつブレード１３３による掘削効率が高くなる作業機経路計画、及び走行経路計画を生成する。経路計画生成部２３は、複数の移動経路Ｌ、及び走行経路Ｒの候補について、最適な作業機１３０の移動経路Ｌである作業機経路計画、及び最適な作業車両１００の走行経路Ｒである走行経路計画を生成するように強化学習される。本実施形態では、作業機１３０の動作に関するパラメータを様々に異ならせた複数の移動経路Ｌ、及び走行経路Ｒの候補についてのシミュレーションを、強化学習による学習を行いながら、複数回行う。 The route plan generation unit 23 generates a work implement route plan that is the optimal travel route L of the work machine 130 and a travel route plan that is the optimal travel route R of the work vehicle 100. The route plan generation unit 23 generates a work machine route plan and a travel route plan that prevent slippage of the tracks 110a and 110b of the work vehicle 100 and increase the excavation efficiency of the blade 133. The route plan generation unit 23 generates a work equipment route plan that is the optimal travel route L of the work machine 130 and a travel route R that is the optimal travel route R of the work vehicle 100 for the plurality of travel routes L and travel route R candidates. Reinforcement learning is performed to generate a route plan. In the present embodiment, simulations are performed multiple times for a plurality of moving routes L with various parameters related to the operation of the working machine 130 and candidates for the driving route R, while learning by reinforcement learning is performed.
　図５は、実施形態に係る経路計画生成部２３で、移動経路Ｌ、及び走行経路Ｒの候補についてのシミュレーションに用いるシミュレーションモデルの例を示す図である。
　経路計画生成部２３は、移動経路Ｌ、及び走行経路Ｒの候補についてのシミュレーションを行うため、作業車両１００による掘削時に生じる現象を、要素毎のモデルに分解する。例えば、図５に示すように、経路計画生成部２３は、作業車両１００による掘削時に生じる現象を、車体モデルＭ１０と、制御モデルＭ２０と、土砂モデルＭ３０と、に分解してシミュレーションにより強化学習される。コンピュータ上で動作する、シミュレーションを実行するシミュレータにより、シミュレーションは実行される。 FIG. 5 is a diagram illustrating an example of a simulation model used by the route plan generation unit 23 according to the embodiment to simulate candidates for the travel route L and the travel route R.
In order to simulate the travel route L and the travel route R candidates, the route plan generation unit 23 decomposes the phenomena that occur during excavation by the work vehicle 100 into models for each element. For example, as shown in FIG. 5, the route plan generation unit 23 decomposes a phenomenon that occurs during excavation by the work vehicle 100 into a vehicle body model M10, a control model M20, and an earth and sand model M30, and performs reinforcement learning through simulation. Ru. The simulation is executed by a simulator running on a computer.
　車体モデルＭ１０では、走行経路Ｒ上における作業車両１００の動作に関するパラメータ、走行経路Ｒ上における作業機１３０の動作に関するパラメータに関する。車体モデルＭ１０では、より具体的には、例えば、作業機１３０のリフトシリンダ、チルトシリンダ等の油圧機器に関する油圧モデルＭ１１、リフトフレーム１３１、ブレード１３３等の可動部の機構に関する機構モデルＭ１２、下部走行体１１０に関する足回りモデルＭ１３、を含む。 The vehicle body model M10 relates to parameters related to the operation of the work vehicle 100 on the travel route R and parameters related to the operation of the work implement 130 on the travel route R. More specifically, the vehicle body model M10 includes, for example, a hydraulic model M11 related to hydraulic equipment such as a lift cylinder and a tilt cylinder of the working machine 130, a mechanism model M12 related to the mechanism of movable parts such as a lift frame 131 and a blade 133, and a lower traveling model M11. It includes an undercarriage model M13 related to the body 110.
　油圧モデルＭ１１では、例えば、作業機１３０のリフトシリンダ、チルトシリンダ等の油圧機器に関するリリーフ圧等をシミュレーションする。このような油圧機器には、リフトフレーム１３１、ブレード１３３の動作による反力の最大値が、最大反力制限として設定されている。油圧モデルＭ１１では、最大反力制限を超えない範囲で、作業機１３０の動作にともなう、油圧機器のリリーフ圧等についてシミュレーションする。 The hydraulic model M11 simulates, for example, relief pressure related to hydraulic equipment such as a lift cylinder and a tilt cylinder of the working machine 130. In such hydraulic equipment, the maximum value of the reaction force due to the operation of the lift frame 131 and the blade 133 is set as the maximum reaction force limit. In the hydraulic model M11, the relief pressure of the hydraulic equipment and the like associated with the operation of the working machine 130 is simulated within a range that does not exceed the maximum reaction force limit.
　機構モデルＭ１２では、移動経路Ｌに基づいてリフトフレーム１３１、ブレード１３３等の可動部を動作させた場合の、ブレード１３３により掘削可能な掘削範囲についてシミュレーションする。ブレード１３３には、先端ブレードの位置制限が設定されている。機構モデルＭ１２では、先端ブレードの位置制限を超えない範囲で、ブレード１３３による掘削動作のシミュレーションを行う。 In the mechanism model M12, the excavation range that can be excavated by the blade 133 is simulated when movable parts such as the lift frame 131 and the blade 133 are operated based on the movement path L. For the blade 133, a position limit of the tip blade is set. In the mechanism model M12, the excavation operation by the blade 133 is simulated within a range that does not exceed the position limit of the tip blade.
　足回りモデルＭ１３では、下部走行体１１０の駆動による車速、履帯１１０ａ、１１０ｂのスリップの発生度合い等について、シミュレーションを行う。下部走行体１１０の駆動によって走行経路Ｒに沿って作業車両１００が走行移動するときの車速は、履帯１１０ａ、１１０ｂによる走行駆動力（牽引力）と、ブレード１３３で地面Ｇの土砂を押すことによって受ける反力とに基づけて算出される。履帯１１０ａ、１１０ｂのスリップの発生度合いは、例えば、リフトフレーム１３１、ブレード１３３で掘削動作を行う場合に土砂から受ける反力に基づいてシミュレーションを行うことができる。この反力は、ブレード１３３で地面Ｇを掘削する際の掘削抵抗（せん断抵抗）と、ブレード１３３の前方に抱え込んだ土砂を前方に押す際の摩擦による運土抵抗との和である。足回りモデルＭ１３では、反力が最大反力制限に到達した場合に、履帯１１０ａ、１１０ｂのシュースリップ限界を超え、スリップが発生する、と判断することができる。 In the suspension model M13, simulations are performed regarding the vehicle speed caused by the drive of the lower traveling body 110, the degree of slippage of the tracks 110a and 110b, and the like. The vehicle speed when the work vehicle 100 travels along the traveling route R by driving the lower traveling body 110 is received by the traveling driving force (traction force) by the tracks 110a and 110b and by pushing the earth and sand on the ground G by the blade 133. It is calculated based on the reaction force. The degree of slippage of the tracks 110a and 110b can be simulated based on the reaction force received from earth and sand when the lift frame 131 and the blade 133 perform an excavation operation, for example. This reaction force is the sum of the excavation resistance (shearing resistance) when the blade 133 excavates the ground G and the soil carrying resistance due to friction when pushing forward the earth and sand held in front of the blade 133. In the suspension model M13, when the reaction force reaches the maximum reaction force limit, it can be determined that the shoe slip limit of the tracks 110a and 110b is exceeded and slip occurs.
　制御モデルＭ２０では、移動経路Ｌ、及び走行経路Ｒの候補に基づいてシミュレーションを行う際の、制御条件に関するものである。制御モデルＭ２０は、例えば、経路追従モデルＭ２１と、経路計画モデルＭ２２と、を有している。
　経路追従モデルＭ２１では、シミュレーションを行う際に、作業車両１００の走行経路Ｒに対する軌跡追従性、作業機１３０の移動経路Ｌに対する軌跡追従性について、例えば、それぞれ、１００％の追従性を有する、と仮定する。 The control model M20 relates to control conditions when performing a simulation based on candidates for the travel route L and the travel route R. The control model M20 includes, for example, a route following model M21 and a route planning model M22.
In the route following model M21, when performing a simulation, it is assumed that the trajectory followability of the work vehicle 100 for the travel route R and the trajectory followability for the movement route L of the work implement 130 are, for example, 100%. Assume.
　また、経路計画モデルＭ２２では、複数回のシミュレーションを行うことでの学習対象となる、複数の移動経路Ｌ、及び走行経路Ｒについての経路計画モデルＭ２２を設定する。経路計画モデルＭ２２では、移動経路Ｌ、及び走行経路Ｒの候補について行ったシミュレーションの結果に基づいて、掘削効率が、より向上するように、新たな移動経路Ｌ、及び走行経路Ｒを生成する。図４に示すように、経路計画モデルＭ２２では、例えば、走行経路Ｒに沿って仮に設定される、移動経路Ｌの始点Ｐ１、ブレード１３３を下降させて地面Ｇの掘削を開始する掘削開始位置Ｐ２、１回の掘削で地面Ｇを掘削する掘削深さＰ３、地面Ｇに対するブレード１３３の侵入角度Ｐ４、移動経路Ｌの終点Ｐ５、作業車両１００の車速等の条件を様々に異ならせて、シミュレーション対象となる複数の移動経路Ｌの候補を順次生成する。経路計画モデルＭ２２では、一つの移動経路Ｌの候補について、シミュレーションを行った後、条件が異なる他の移動経路Ｌを生成し、シミュレーションを順次実行させる。 In addition, the route planning model M22 is set for a plurality of travel routes L and driving routes R, which are to be learned by performing simulations a plurality of times. The route planning model M22 generates new travel routes L and travel routes R based on the results of simulations performed on candidates for travel routes L and travel routes R so as to further improve excavation efficiency. As shown in FIG. 4, in the route planning model M22, for example, a starting point P1 of the moving route L is temporarily set along the traveling route R, and an excavation start position P2 where the blade 133 is lowered to start excavating the ground G. , the excavation depth P3 at which the ground G is excavated in one excavation, the penetration angle P4 of the blade 133 into the ground G, the end point P5 of the moving route L, the vehicle speed of the work vehicle 100, etc. are varied, and the simulation target is A plurality of moving route L candidates are sequentially generated. In the route planning model M22, after performing a simulation on one candidate travel route L, another travel route L with different conditions is generated, and the simulations are sequentially executed.
　土砂モデルＭ３０は、移動経路Ｌ、及び走行経路Ｒの各候補に基づいて作業車両１００を走行させて作業機１３０を移動させた場合に、作業機１３０で掘削する地面Ｇの土砂に関するパラメータをシミュレーションするためのものである。土砂モデルＭ３０では、ブレード１３３で地面Ｇを掘削することで生じる地形変化に関する地形モデルＭ３１と、地面Ｇを掘削する際にブレード１３３が受ける反力に関する反力モデルＭ３２と、に分類される。 The earth and sand model M30 simulates parameters related to earth and sand on the ground G to be excavated by the work equipment 130 when the work vehicle 100 is driven and the work equipment 130 is moved based on each candidate of the travel route L and the travel route R. It is for the purpose of The earth and sand model M30 is classified into a terrain model M31 related to topographic changes caused by excavating the ground G with the blade 133, and a reaction force model M32 related to the reaction force that the blade 133 receives when excavating the ground G.
　地形モデルＭ３１では、ブレード１３３で地面Ｇを掘削することで、掘削される掘削土量についてシミュレーションを行う。掘削土量は、走行経路Ｒに沿った領域における、掘削前の地面Ｇの地形形状情報と、設計面Ｓについての設計面情報との差に基づいて算出される。地形モデルＭ３１では、地面Ｇから設計面Ｓまでの掘削を、複数回に分けて行う場合、各回の掘削時における掘削土量は、掘削前の地形形状と、移動経路Ｌに沿った１回の掘削によって形成される掘削面との差に基づいて算出することができる。 In the terrain model M31, by excavating the ground G with the blade 133, a simulation is performed on the amount of excavated soil to be excavated. The amount of excavated soil is calculated based on the difference between the topographic shape information of the ground G before excavation and the design surface information about the design surface S in the area along the travel route R. In the terrain model M31, when excavating from the ground G to the design surface S is carried out in multiple stages, the amount of excavated soil in each excavation is based on the terrain shape before excavation and the one-time excavation along the movement route L. It can be calculated based on the difference from the excavated surface formed by excavation.
　図６は、実施形態に係る作業車両１００の走行経路Ｒにおける、作業機１３０の移動経路Ｌの他の一例を示す図である。
　また、図６に示すように、移動経路Ｌに基づいてブレード１３３で地面Ｇを掘削し、ブレード１３３の前方に土砂Ｄ１を抱え込んでいる場合、地形モデルＭ３１では、抱え込んでいる土砂Ｄ１の量である抱え込み土量について、シミュレーションを行う。 FIG. 6 is a diagram showing another example of the moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
In addition, as shown in FIG. 6, when the ground G is excavated by the blade 133 based on the movement route L and dirt D1 is held in front of the blade 133, in the terrain model M31, the amount of dirt D1 held is A simulation is performed for a certain amount of soil to be held.
　図７は、実施形態に係る作業車両１００の走行経路Ｒにおける、作業機１３０の移動経路Ｌのさらに他の一例を示す図である。
　図７に示すように、ブレード１３３で走行経路Ｒ上の地面Ｇの一部を掘削した土砂Ｄ２により、走行経路Ｒ上の地面Ｇの他の部分を埋め戻す場合、地形モデルＭ３１では、掘削土量と、埋め戻した土砂Ｄ２の量である埋め戻し土量と、の差に基づき、実質的な掘削土量について、シミュレーションを行うようにしてもよい。 FIG. 7 is a diagram showing still another example of the moving route L of the working machine 130 in the traveling route R of the working vehicle 100 according to the embodiment.
As shown in FIG. 7, when backfilling other parts of the ground G on the travel route R with earth and sand D2 excavated from a part of the ground G on the travel route R with the blade 133, in the terrain model M31, the excavated soil The actual amount of excavated soil may be simulated based on the difference between the amount of excavated soil and the amount of backfilled soil that is the amount of backfilled earth and sand D2.
　図８は、実施形態に係る作業機１３０で掘削を行った際に、ブレード１３３の前方で抱え込む土砂Ｄ１、及びブレード１３３の両側にあふれ出して形成されるウィンドローの土砂Ｄ５を示す側面図である。図９は、実施形態に係る作業機１３０で掘削を行った際に、ブレード１３３の前方で抱え込む土砂Ｄ１、及びブレード１３３の両側にあふれ出して形成されるウィンドローの土砂Ｄ５を示す平面図である。
　地形モデルＭ３１では、図８、図９に示すように、移動経路Ｌ、及び走行経路Ｒに基づいてブレード１３３で地面Ｇを掘削した際に、ブレード１３３の幅方向両側にあふれ出す、いわゆるウィンドローの土砂Ｄ５の量について、シミュレーションを行う。地形モデルＭ３１では、一つの走行経路Ｒに沿って、移動経路Ｌに基づいてブレード１３３で地面Ｇを掘削することによって、地面Ｇ上にウィンドローが形成された場合、形成されたウィンドローの土砂Ｄ５の量を、他の走行経路Ｒにおいて、移動経路Ｌの候補についてシミュレーションを行う際に、地形情報に含めるものとする。 FIG. 8 is a side view showing the dirt D1 held in front of the blade 133 and the windrow dirt D5 formed by overflowing to both sides of the blade 133 when excavating with the working machine 130 according to the embodiment. be. FIG. 9 is a plan view showing the dirt D1 held in front of the blade 133 and the windrow dirt D5 formed by overflowing to both sides of the blade 133 when excavating with the working machine 130 according to the embodiment. be.
In the terrain model M31, as shown in FIGS. 8 and 9, when the ground G is excavated with the blade 133 based on the travel route L and the travel route R, so-called windrows overflow to both sides in the width direction of the blade 133. A simulation is performed for the amount of earth and sand D5. In the terrain model M31, when a windrow is formed on the ground G by excavating the ground G with the blade 133 along one traveling route R based on the moving route L, the earth and sand of the formed windrow It is assumed that the amount D5 is included in the terrain information when simulating a candidate travel route L on another travel route R.
　上記抱え込む土砂Ｄ１の土量、ウィンドローの土砂Ｄ５の土量は、例えば、模型を用いた事前実験等に基づいて算出するようにしてもよい。
　これ以外に、地形モデルＭ３１では、例えば、走行経路Ｒ上で掘削した土砂の一部を、走行経路Ｒ上の他の位置で埋め戻す場合、埋め戻した土砂を、履帯１１０ａ、１１０ｂで転圧して締め固める、として、シミュレーションを行う。
　地形モデルＭ３１では、走行経路Ｒ上で掘削した土砂についての土崩れについてもシミュレーションを行う。 The amount of earth and sand D1 to be held and the amount of earth and sand D5 in the windrow may be calculated based on, for example, a preliminary experiment using a model.
In addition to this, in the terrain model M31, for example, when backfilling a part of the earth and sand excavated on the travel route R at another position on the travel route R, the backfilled earth and sand is compacted by the crawler tracks 110a and 110b. A simulation is performed as follows:
The terrain model M31 also simulates landslides caused by earth and sand excavated on the travel route R.
　反力モデルＭ３２では、移動経路Ｌに基づいて、作業車両１００を走行経路Ｒに沿って移動させながらブレード１３３を動作させて地面Ｇを掘削した際に、ブレード１３３の前方に抱え込んだ土砂Ｄ１を、前方に押していくことによって地面Ｇとの間に生じる運土抵抗についてシミュレートする。作業車両１００が、走行経路Ｒに沿って前進していくと、ブレード１３３の前方で抱え込む土砂Ｄ１の量は、漸次増大する。つまり、運土抵抗は、時々刻々と変化する。 In the reaction force model M32, when the work vehicle 100 is moved along the travel route R based on the travel route L and the blade 133 is operated to excavate the ground G, the dirt D1 held in front of the blade 133 is removed. , the soil carrying resistance that occurs between the soil and the ground G by pushing it forward is simulated. As the work vehicle 100 moves forward along the travel route R, the amount of dirt D1 held in front of the blade 133 gradually increases. In other words, soil transport resistance changes from moment to moment.
　反力モデルＭ３２では、移動経路Ｌに沿ってブレード１３３を動作させたときに、地面Ｇの土砂からブレード１３３が受ける掘削抵抗についてシミュレートする。掘削抵抗は、地面Ｇに対するブレード１３３０による掘削深さに応じて大きくなる。図７に示すように、移動経路Ｌの終点Ｐ５よりも前方で、土砂Ｄ２を埋め戻す場合、反力モデルＭ３２は、埋め戻す土砂Ｄ２からブレード１３３が受ける掘削抵抗についてもシミュレートする。 The reaction force model M32 simulates the excavation resistance that the blade 133 receives from the earth and sand on the ground G when the blade 133 is operated along the movement path L. The excavation resistance increases depending on the depth of excavation into the ground G by the blade 1330. As shown in FIG. 7, when backfilling with earth and sand D2 ahead of the end point P5 of the moving route L, the reaction force model M32 also simulates the excavation resistance that the blade 133 receives from the backfill earth and sand D2.
　シミュレータは、複数の移動経路Ｌ、及び走行経路Ｒの候補について行った、複数回のシミュレーションの結果に基づいて、経路計画生成部２３を強化学習する。シミュレータは、各回のシミュレーションについて、作業車両１００を走行経路Ｒに沿って移動させながら、移動経路Ｌに沿って作業機１３０を動作させた場合の掘削土量と、移動経路Ｌの始点から終点まで作業機１３０で掘削を行うのに要する作業時間とに基づいて、強化学習による報酬を、例えば下式（１）に基づいて算出する。
　　報酬＝掘削土量／作業時間　・・・（１） The simulator performs reinforcement learning on the route plan generation unit 23 based on the results of multiple simulations performed on multiple travel routes L and driving route R candidates. For each simulation, the simulator calculates the amount of soil excavated when the work vehicle 100 is moved along the travel route R and the work implement 130 is operated along the travel route L, and the distance from the start point to the end point of the travel route L. Based on the work time required for excavating with the work machine 130, the reward by reinforcement learning is calculated based on the following formula (1), for example.
Remuneration = excavated soil volume / working time ... (1)
　また、シミュレータは、各回のシミュレーションについて、作業車両１００を走行経路Ｒに沿って移動させながら、移動経路Ｌに沿って作業機１３０を動作させた場合に、ブレード１３３が土砂から受ける反力が、下部走行体１１０のシュースリップ限界を超えた場合、強化学習におけるペナルティ値（例えば、－０．０５）を付与する。 In addition, for each simulation, the simulator calculates that when the work vehicle 100 is moved along the travel route R and the work implement 130 is operated along the travel route L, the reaction force that the blade 133 receives from the earth and sand is: If the shoe slip limit of the undercarriage 110 is exceeded, a penalty value (for example, −0.05) in reinforcement learning is given.
　シミュレータは、強化学習により、算出される報酬とペナルティ値とに基づいて、最適な移動経路Ｌ、及び走行経路Ｒを探索し、作業機経路計画、及び走行経路計画を生成するように経路計画生成部２３を学習する。
　経路計画生成部２３は、最適な移動経路Ｌである作業機経路計画、及び最適な走行経路Ｒである走行経路計画に基づき、施工対象エリアＡの全体を対象とした作業機経路計画、及び走行経路計画を生成するように学習される。 The simulator uses reinforcement learning to search for the optimal travel route L and travel route R based on the calculated reward and penalty value, and generates a route plan to generate a work equipment route plan and a travel route plan. Learn section 23.
The route plan generation unit 23 generates a work equipment route plan for the entire construction target area A based on the work equipment route plan that is the optimal movement route L and the travel route plan that is the optimal travel route R. It is trained to generate route plans.
　図１０は、実施形態に係る作業機１３０で掘削を行った際に、ブレード１３３の前方で抱え込む土砂Ｄ１、及びブレード１３３の両側にあふれ出して形成されるウィンドローの土砂Ｄ５を、多角形状の立体モデルＤｍ１、Ｄｍ５に模した図である。
　ところで、図８～図１０に示すように、上記のような経路計画生成部２３の地形モデルＭ３１では、ブレード１３３の前方に抱え込んでいる土砂Ｄ１を、多角柱状（例えば三角柱状）の立体モデルＤｍ１に模して単純化し、抱え込み土量を算出する。これにより、移動経路Ｌに基づいて地面Ｇを掘削するにともなって時々刻々と変化する抱え込み土量を、効率的に算出することができる。 FIG. 10 shows that when excavating with the working machine 130 according to the embodiment, the earth and sand D1 held in front of the blade 133 and the earth and sand D5 of the windrow formed by overflowing on both sides of the blade 133 are arranged in a polygonal shape. It is a diagram imitating three-dimensional models Dm1 and Dm5.
By the way, as shown in FIGS. 8 to 10, in the terrain model M31 of the route plan generation unit 23 as described above, the earth and sand D1 held in front of the blade 133 is converted into a polygonal prism-shaped (for example, triangular prism-shaped) three-dimensional model Dm1. Calculate the amount of soil to be carried by simplifying it by imitating it. Thereby, it is possible to efficiently calculate the amount of soil to be held, which changes from moment to moment as the ground G is excavated based on the moving route L.
　また、地形モデルＭ３１では、ブレード１３３の両側にはみ出すウィンドローの土砂Ｄ５の土量については、前方に抱え込んでいる土砂Ｄ１の比率に応じて、多角柱状（例えば四角柱状（直方体状）の立体モデルＤｍ５に模して単純化して算出する。これにより、移動経路Ｌに基づいて地面Ｇを掘削するにともなって時々刻々と変化するウィンドローの土砂Ｄ５の土量を、効率的に算出することができる。 In addition, in the terrain model M31, regarding the amount of soil D5 of the windrow protruding on both sides of the blade 133, a three-dimensional model of a polygonal prism shape (for example, a quadrangular prism shape (rectangular parallelepiped shape)) is Dm5 is simplified and calculated. This makes it possible to efficiently calculate the amount of windrow earth and sand D5, which changes from moment to moment as the ground G is excavated based on the moving route L. can.
（経路計画生成装置２０の動作例）
　図１１は、実施形態に係る経路計画生成装置２０の動作を示すフローチャートである。
　強化学習により学習された経路計画生成装置２０は、例えば作業車両１００に備えられる。経路計画生成装置２０は、施工対象エリアＡの地面Ｇに対して作業車両１００で掘削作業を行うための経路計画を生成する。
　これには、図１１に示すように、まず、情報入力部２１が、施工対象エリアＡ内の地面Ｇの形状を示す地形形状情報、作業車両１００の位置、及び、施工対象エリアＡ内の地面Ｇを掘削すべき形状を示す設計面情報を、外部から取得する（Ｓ１）。次に、取得した地形形状情報、作業車両１００の位置、及び設計面情報を、情報記憶部２２に記憶する（Ｓ２）。次に、経路計画生成部２３で、地形形状情報、作業車両１００の位置、及び設計面情報に基づいて、作業機１３０の移動経路Ｌを示す作業機経路計画作業車両１００の走行経路Ｒを示す走行経路計画を生成する（Ｓ３）。その後、情報出力部２４が、生成された作業機経路計画、及び走行経路計画を、外部に出力する（Ｓ４）。 (Example of operation of route plan generation device 20)
FIG. 11 is a flowchart showing the operation of the route plan generation device 20 according to the embodiment.
The route plan generation device 20 that has been trained by reinforcement learning is installed in, for example, the work vehicle 100. The route plan generation device 20 generates a route plan for performing excavation work on the ground G of the construction target area A with the work vehicle 100.
To do this, as shown in FIG. 11, the information input unit 21 first inputs topographical shape information indicating the shape of the ground G in the construction target area A, the position of the work vehicle 100, and the ground surface in the construction target area A. Design surface information indicating the shape in which G should be excavated is acquired from the outside (S1). Next, the acquired terrain shape information, the position of the work vehicle 100, and the design surface information are stored in the information storage unit 22 (S2). Next, the route plan generation unit 23 indicates a travel route R of the work vehicle 100, indicating a travel route L of the work machine 130, based on the terrain shape information, the position of the work vehicle 100, and the design surface information. A travel route plan is generated (S3). Thereafter, the information output unit 24 outputs the generated working machine route plan and traveling route plan to the outside (S4).
　図１２は、実施形態に係る強化学習により、経路計画生成部２３が学習される流れの概略を示す図である。強化学習は、例えば、車体１００の外部にあるコンピュータ上で動作する、シミュレーションを実行するシミュレータを用いて学習することができる。
　これには、まず、シミュレータが、作業機１３０の移動経路Ｌの候補と作業車両１００の走行経路Ｒの候補を設定する（Ｓ３１）。 FIG. 12 is a diagram schematically showing a flow in which the route plan generation unit 23 is learned by reinforcement learning according to the embodiment. Reinforcement learning can be performed using, for example, a simulator that runs on a computer outside the vehicle body 100 and executes a simulation.
To do this, first, the simulator sets candidates for the travel route L of the work machine 130 and the candidate travel route R for the work vehicle 100 (S31).
　シミュレータは、設定した移動経路Ｌの候補と作業車両１００の走行経路Ｒの候補に基づいて、シミュレーションを行う。具体的には、車体モデルＭ１０、及び制御モデルＭ２０を用い、作業車両１００、及び作業機１３０の動作についてのシミュレーションを行う（Ｓ３２）。作業車両１００の動作のシミュレーションでは、設定された移動経路Ｌの候補と作業車両１００の走行経路Ｒの候補に基づいて、作業機１３０を動作させるときの走行経路Ｒに沿った作業車両１００の走行軌跡をシミュレーションする。また、作業機１３０の動作のシミュレーションでは、移動経路Ｌに基づいて作業機１３０を動作させた場合における、作業機１３０の移動軌跡を、シミュレーションする。
　また、シミュレータは、算出された作業車両１００の走行軌跡、及び作業機１３０の移動軌跡のデータに基づいて、移動経路Ｌ、及び走行経路Ｒに基づいて掘削を行った際の作業時間を算出する。 The simulator performs a simulation based on the set travel route L candidate and the travel route R candidate for the work vehicle 100. Specifically, the operation of the work vehicle 100 and the work implement 130 is simulated using the vehicle body model M10 and the control model M20 (S32). In the simulation of the operation of the work vehicle 100, based on the set travel route L candidate and the travel route R candidate for the work vehicle 100, the work vehicle 100 travels along the travel route R when operating the work implement 130. Simulate the trajectory. Furthermore, in the simulation of the operation of the working machine 130, a movement locus of the working machine 130 when the working machine 130 is operated based on the moving route L is simulated.
The simulator also calculates the working time when excavating based on the travel route L and the travel route R, based on the data of the travel trajectory of the work vehicle 100 and the travel trajectory of the work equipment 130 that have been calculated. .
　さらに、シミュレータは、作業車両１００の走行軌跡、及び作業機１３０の移動軌跡のデータに基づいて、地面Ｇの土砂を掘削する際の、土砂についてのシミュレーションを、土砂モデルＭ３０を用いて実行する（Ｓ３３）。土砂モデルＭ３０では、例えば、地面Ｇの土砂を掘削する際の掘削抵抗を算出する。算出された掘削抵抗のデータは、車体モデルＭ１０にフィードバックされ、作業車両１００の車速等の算出に反映される。 Further, the simulator uses the earth and sand model M30 to perform a simulation of earth and sand when excavating earth and sand on the ground G based on the data of the travel trajectory of the work vehicle 100 and the movement trajectory of the work implement 130 ( S33). In the earth and sand model M30, for example, excavation resistance when excavating earth and sand on the ground G is calculated. The calculated excavation resistance data is fed back to the vehicle body model M10 and reflected in the calculation of the vehicle speed of the work vehicle 100, etc.
　また、シミュレータは、土砂モデルＭ３０を用い、作業機１３０による掘削後の地面Ｇの形状を示す掘削後地形を算出する。さらに、経路計画生成部２３では、土砂モデルＭ３０を用い、作業機１３０による掘削土量、抱え込み土量、ウィンドローの土量等を算出する。 Furthermore, the simulator uses the earth and sand model M30 to calculate a post-excavation topography that indicates the shape of the ground G after excavation by the working machine 130. Further, the route plan generation unit 23 uses the earth and sand model M30 to calculate the amount of earth excavated by the working machine 130, the amount of earth held, the amount of earth in the windrow, etc.
　シミュレータは、上記のようにして算出された作業時間と、掘削土量とに基づいて、強化学習により経路計画生成部２３を学習する（Ｓ３４）。強化学習では、報酬、及びペナルティを算出し、移動経路Ｌ、及び走行経路Ｒの候補についての評価を行う。このとき、シミュレータは、作業車両１００を走行経路Ｒに沿って移動させながら、移動経路Ｌに基づいて作業機１３０による掘削を行った際に、掘削抵抗が、履帯１１０ａ、１１０ｂのシュースリップ限界を超えたか否かを判定する。その結果、掘削抵抗が、シュースリップ限界を超えていた場合、作業車両１００のスリップが生じていた、と判定する。 The simulator learns the route plan generation unit 23 by reinforcement learning based on the work time and the amount of excavated soil calculated as described above (S34). In reinforcement learning, rewards and penalties are calculated, and candidates for the travel route L and the travel route R are evaluated. At this time, the simulator shows that when the work vehicle 100 is moved along the travel route R and the work machine 130 excavates based on the travel route L, the excavation resistance exceeds the shoe slip limit of the tracks 110a and 110b. Determine whether or not the limit has been exceeded. As a result, if the excavation resistance exceeds the shoe slip limit, it is determined that the work vehicle 100 is slipping.
　シミュレータは、一つの移動経路Ｌ、及び走行経路Ｒの候補についてシミュレーションを行った後、その移動経路Ｌ、及び走行経路Ｒの候補についての評価結果に基づいた学習が行われ、次にシミュレーションを行うべき移動経路Ｌ、又は走行経路Ｒの候補を設定する。次にシミュレーションを行うべき移動経路Ｌ、又は走行経路Ｒの候補は、作業車両１００のスリップが生じず、かつ上式（１）で表される報酬がなるべく大きくなるように、設定される。 The simulator performs a simulation on one travel route L and a candidate travel route R, then performs learning based on the evaluation results for the travel route L and candidate travel route R, and then performs a simulation. A candidate travel route L or travel route R is set. The candidate travel route L or travel route R to be simulated next is set so that the work vehicle 100 does not slip and the reward expressed by the above formula (1) is as large as possible.
　このような移動経路Ｌ、及び走行経路Ｒの候補についてのシミュレーションを、強化学習による学習を行いながら複数回繰り返す。これにより、作業車両１００のスリップが生じず、かつ掘削効率の高い作業機経路計画、及び走行経路計画が生成できるように経路計画生成部２３が学習される。 Simulations of such candidates for the travel route L and the travel route R are repeated multiple times while learning by reinforcement learning. As a result, the route plan generation unit 23 is trained to generate a work machine route plan and a travel route plan that prevent the work vehicle 100 from slipping and have high excavation efficiency.
（作業車両の経路計画生成システムの構成）
　図１３は、実施形態に係る作業車両の経路計画生成システム５０の構成例を示す概略ブロック図である。
　図１３に示すように、作業車両の経路計画生成システム５０は、遠隔制御装置６０と、作業車両１００と、を備えている。 (Configuration of work vehicle route plan generation system)
FIG. 13 is a schematic block diagram showing a configuration example of a route plan generation system 50 for a work vehicle according to the embodiment.
As shown in FIG. 13, the work vehicle route plan generation system 50 includes a remote control device 60 and a work vehicle 100.
　遠隔制御装置６０は、通信部６１と、情報出力部６２と、管制部６３と、を備えている。
　通信部６１は、公衆無線通信網、無線通信手段により、作業車両１００と通信可能である。 The remote control device 60 includes a communication section 61, an information output section 62, and a control section 63.
The communication unit 61 can communicate with the work vehicle 100 using a public wireless communication network or wireless communication means.
　情報出力部６２は、作業車両１００を自動運転するために必要な情報を出力する。情報出力部６２は、例えば、施工対象エリアＡの設計面情報等を、外部のＣＡＤシステムなどから取得し、通信部６１を介して作業車両１００に送信する。
　管制部６３は、作業車両１００に備えられた各種のセンサにより検出された情報に基づき、作業車両１００の各部の動作状態を監視する。 The information output unit 62 outputs information necessary for automatically driving the work vehicle 100. The information output unit 62 acquires, for example, design surface information of the construction target area A from an external CAD system, etc., and transmits it to the work vehicle 100 via the communication unit 61.
The control unit 63 monitors the operating state of each part of the work vehicle 100 based on information detected by various sensors included in the work vehicle 100.
　作業車両１００は、経路計画生成装置２０によって生成された経路計画に基づいて、自動的に動作する。
　このため、作業車両１００は、通信部７１と、上記経路計画生成装置２０と、位置検出部７２と、経路計画記憶部７３と、車両制御部７４と、を備えている。 Work vehicle 100 automatically operates based on the route plan generated by route plan generation device 20.
For this reason, the work vehicle 100 includes a communication section 71, the route plan generation device 20, a position detection section 72, a route plan storage section 73, and a vehicle control section 74.
　通信部７１は、公衆無線通信網、無線通信手段により、遠隔制御装置６０の通信部６１と通信可能である。
　なお、作業車両１００の一部または全部の構成は、遠隔制御装置６０に備えられても良い。例えば、上記で示した経路計画生成装置２０は、本実施形態において、作業車両１００側に備えられている。経路計画生成装置２０は、遠隔制御装置６０側に備えられていてもよい。また、遠隔制御装置６０の一部または全部の構成は、作業車両１００に備えられても良い。 The communication unit 71 can communicate with the communication unit 61 of the remote control device 60 via a public wireless communication network or wireless communication means.
Note that a part or all of the configuration of work vehicle 100 may be included in remote control device 60. For example, the route plan generation device 20 shown above is provided on the work vehicle 100 side in this embodiment. The route plan generation device 20 may be provided on the remote control device 60 side. Further, part or all of the configuration of the remote control device 60 may be provided in the work vehicle 100.
　位置検出部７２は、作業車両１００に備えられている。位置検出部７２は、例えばＧＰＳ等が用いられ、作業車両１００の位置が検出可能とされている。 The position detection unit 72 is provided in the work vehicle 100. The position detection unit 72 uses, for example, GPS, and is capable of detecting the position of the work vehicle 100.
　経路計画記憶部７３は、経路計画生成装置２０によって生成され、外部に出力された、作業車両１００の作業機経路計画、及び走行経路計画を記憶する。
　車両制御部７４は、経路計画記憶部７３に記憶された作業機経路計画、及び走行経路計画に基づいて、作業機１３０、及び作業車両１００の各部動作を制御する。作業車両１００は、作業機１３０の最適な移動経路Ｌである作業機経路計画と、作業車両１００の最適な走行経路Ｒである走行経路計画とに基づいて、作業車両１００を移動させながら、作業機１３０を動作させる。これにより、地面Ｇの掘削が効率良く行われる。 The route plan storage unit 73 stores the work equipment route plan and travel route plan of the work vehicle 100, which are generated by the route plan generation device 20 and output to the outside.
The vehicle control unit 74 controls the operation of each part of the work machine 130 and the work vehicle 100 based on the work machine route plan and the travel route plan stored in the route plan storage unit 73. The work vehicle 100 performs work while moving the work vehicle 100 based on a work implement route plan that is the optimal travel route L for the work machine 130 and a travel route plan that is the optimal travel route R for the work vehicle 100. The machine 130 is operated. Thereby, the ground G can be excavated efficiently.
（作用・効果）
　本実施形態によれば、作業車両１０の経路計画を、効率良く生成することができる。 (action/effect)
According to this embodiment, a route plan for the work vehicle 10 can be efficiently generated.
　以上、本開示の実施形態について図面を参照して説明してきたが、具体的な構成は上記実施形態に限られるものではなく、本開示の要旨を逸脱しない範囲の設計変更等も含まれる。 Although the embodiments of the present disclosure have been described above with reference to the drawings, the specific configuration is not limited to the above embodiments, and may include design changes without departing from the gist of the present disclosure.
　例えば、上述した実施形態に係る作業車両１００は、ブルドーザであるが、これに限られない。例えば、作業車両１００は、油圧ショベル、ホイールローダ、モータグレーダ等の作業機及び走行体を有する作業機械であってもよい。 For example, the work vehicle 100 according to the embodiment described above is a bulldozer, but is not limited to this. For example, the work vehicle 100 may be a work machine including a work machine such as a hydraulic excavator, a wheel loader, a motor grader, and a running body.
　また、上記実施形態でコンピュータが実行するプログラムの一部または全部は、コンピュータが読取可能な記録媒体や通信回線を介して頒布することができる。 Further, part or all of the program executed by the computer in the above embodiments can be distributed via a computer-readable recording medium or a communication line.
　上述した一態様によれば、作業車両の経路計画を、効率良く生成することができる。 According to one aspect described above, a route plan for a work vehicle can be efficiently generated.
１０…作業車両　２０…経路計画生成装置　２２…情報記憶部　２３…経路計画生成部　５０…作業車両の経路計画生成システム　７２…位置検出部　７３…経路計画記憶部　７４…車両制御部　１００…作業車両　１３０…作業機　Ａ…施工対象エリア　Ｄ１…土砂　Ｄ５…土砂　Ｄｍ１、Ｄｍ５…立体モデル　Ｇ…地面　Ｌ…移動経路　Ｒ…走行経路　Ｓ…設計面 10...Work vehicle 20...Route plan generation device 22...Information storage section 23...Route plan generation section 50...Route plan generation system for work vehicle 72...Position detection section 73...Route plan storage section 74...Vehicle control section 100...Work vehicle 130...Work equipment A...Construction target area D1...Earth and sand D5...Earth and sand Dm1, Dm5...3D model G...Ground L...Movement route R...Traveling route S...Design surface

Claims (9)

  1. 　作業機を有した作業車両で、施工対象エリアの地面に対して掘削作業を行うための経路計画を生成する作業車両の経路計画生成システムであって、
    　前記作業車両の位置を検出する位置検出部と、
    　前記施工対象エリア内の地形の形状を示す地形形状情報、前記作業車両の位置、及び前記施工対象エリア内の目標形状を示す設計面情報を記憶する情報記憶部と、
    　前記地形形状情報、前記作業車両の位置、及び前記設計面情報に基づいて、前記作業機の移動経路を示す作業機経路計画、及び前記作業車両の走行経路を示す走行経路計画を生成する経路計画生成部と、
    　を備える作業車両の経路計画生成システム。     A route plan generation system for a work vehicle that generates a route plan for performing excavation work on the ground in a construction target area using a work vehicle having a work machine,
    a position detection unit that detects the position of the work vehicle;
    an information storage unit that stores topographical shape information indicating the shape of the terrain within the construction target area, the position of the work vehicle, and design surface information indicating the target shape within the construction target area;
    A route plan that generates a work equipment route plan showing a travel route of the work equipment and a travel route plan showing a travel route of the work vehicle based on the terrain shape information, the position of the work vehicle, and the design surface information. A generation section,
    A work vehicle route plan generation system comprising:
  2. 　前記経路計画生成部は、最適な前記走行経路、及び最適な前記移動経路を生成するように強化学習される、
    　請求項１に記載の作業車両の経路計画生成システム。     The route plan generation unit performs reinforcement learning to generate the optimal travel route and the optimal travel route.
    The route plan generation system for a work vehicle according to claim 1.
  3. 　前記経路計画生成部は、少なくとも、前記作業機により掘削される土砂、及び、前記走行経路上における前記作業車両に関するパラメータを用いたシミュレーションを複数回行うことで、前記強化学習される、
    　請求項２に記載の作業車両の経路計画生成システム。     The route plan generation unit performs the reinforcement learning by performing a simulation multiple times using at least parameters related to earth and sand excavated by the work machine and the work vehicle on the travel route.
    The route plan generation system for a work vehicle according to claim 2.
  4. 　前記経路計画生成部は、前記走行経路に沿って前記作業車両を移動させたときの、掘削土量、及び作業時間の少なくとも一方を用いた報酬に基づいて、前記強化学習される、
    　請求項２又は３に記載の作業車両の経路計画生成システム。     The route plan generation unit performs the reinforcement learning based on a reward using at least one of excavated soil volume and work time when moving the work vehicle along the travel route.
    A route plan generation system for a work vehicle according to claim 2 or 3.
  5. 　前記経路計画生成部は、前記走行経路に沿って前記作業車両を移動させ、前記作業機で前記地面を掘削する前と後における地形の差に基づく、前記作業機での抱え込み土量に基づいて、算出された前記掘削土量を用いて前記強化学習される、
    　請求項４に記載の作業車両の経路計画生成システム。     The route plan generation unit moves the work vehicle along the travel route, and calculates the amount of soil to be carried by the work machine based on the difference in topography before and after excavating the ground with the work machine. , the reinforcement learning is performed using the calculated excavated soil volume,
    A route plan generation system for a work vehicle according to claim 4.
  6. 　前記経路計画生成部は、前記走行経路に沿って前記作業車両を移動させた後、前記作業機の前方に抱え込まれた土砂を、多角柱状の立体モデルに模すことで、算出された前記抱え込み土量を用いて前記強化学習される、
    　請求項５に記載の作業車両の経路計画生成システム。     The route plan generation unit moves the work vehicle along the travel route and then models the dirt held in front of the working machine into a polygonal column-shaped three-dimensional model, thereby generating the calculated held amount. The reinforcement learning is performed using the volume of soil,
    The route plan generation system for a work vehicle according to claim 5.
  7. 　前記経路計画生成部は、前記走行経路に沿って前記作業車両を移動させた場合に、前記作業機の両側にはみ出すうね状のウィンドローの土量に基づいて、算出された前記掘削土量を用いて前記強化学習される、
    　請求項５に記載の作業車両の経路計画生成システム。     The route plan generation unit calculates the amount of excavated soil calculated based on the amount of soil of a ridge-shaped windrow that protrudes on both sides of the work machine when the work vehicle is moved along the travel route. The reinforcement learning is performed using the
    The route plan generation system for a work vehicle according to claim 5.
  8. 　前記経路計画生成部は、前記ウィンドローを、多角柱状の立体モデルに模すことで、算出された前記ウィンドローの土量を用いて前記強化学習される、
    　請求項７に記載の作業車両の経路計画生成システム。     The route plan generation unit performs the reinforcement learning using the calculated earth volume of the windrow by modeling the windrow into a polygonal columnar three-dimensional model.
    The route plan generation system for a work vehicle according to claim 7.
  9. 　作業機を有した作業車両で、施工対象エリアの地面に対して掘削作業を行うための経路計画を生成する作業車両の経路計画生成方法であって、
    　前記作業車両の位置を検出するステップと、
    　前記施工対象エリア内の地形の形状を示す地形形状情報、前記作業車両の位置、及び前記施工対象エリア内の前記地面を掘削すべき形状を示す設計面情報を記憶するステップと、
    　前記地形形状情報、前記作業車両の位置、及び前記設計面情報に基づいて、前記作業機の移動経路を示す作業機経路計画、及び前記作業車両の走行経路を示す走行経路計画を生成するステップと、
    　を含む、作業車両の経路計画生成方法。     A route plan generation method for a work vehicle that generates a route plan for performing excavation work on the ground of a construction target area with a work vehicle having a work machine, the method comprising:
    detecting the position of the work vehicle;
    storing topographical shape information indicating the shape of the terrain in the construction target area, the position of the work vehicle, and design surface information indicating the shape in which the ground in the construction target area should be excavated;
    generating a work equipment route plan indicating a travel route of the work equipment and a travel route plan representing a travel route of the work vehicle based on the terrain shape information, the position of the work vehicle, and the design surface information; ,
    A method for generating a route plan for a work vehicle, including:
PCT/JP2023/024347 2022-06-30 2023-06-30 Work vehicle path plan generation system and work vehicle path plan generation method WO2024005182A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-106372 2022-06-30
JP2022106372A JP2024005910A (en) 2022-06-30 2022-06-30 Route plan generation system of work vehicle, and route plan generation method of work vehicle

Publications (1)

Publication Number Publication Date
WO2024005182A1 true WO2024005182A1 (en) 2024-01-04

Family

ID=89380784

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/024347 WO2024005182A1 (en) 2022-06-30 2023-06-30 Work vehicle path plan generation system and work vehicle path plan generation method

Country Status (2)

Country Link
JP (1) JP2024005910A (en)
WO (1) WO2024005182A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120136508A1 (en) * 2010-11-30 2012-05-31 Taylor Michael A System for automated excavation planning and control
US20190055715A1 (en) * 2017-08-15 2019-02-21 Caterpillar Inc. System and method for controlling earthmoving machines
WO2019189888A1 (en) * 2018-03-30 2019-10-03 住友重機械工業株式会社 Construction machine operation assistance system, and construction machine
US20200105072A1 (en) * 2016-12-23 2020-04-02 Caterpillar Sarl Monitoring The Operation Of A Work Machine
WO2020226848A1 (en) * 2019-05-03 2020-11-12 Caterpillar Inc. System for controlling the position of a work implement
JP2021113487A (en) * 2020-01-17 2021-08-05 バイドゥ ユーエスエイ エルエルシーBaidu USA LLC Neural task planning part for autonomous travelling vehicle

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120136508A1 (en) * 2010-11-30 2012-05-31 Taylor Michael A System for automated excavation planning and control
US20200105072A1 (en) * 2016-12-23 2020-04-02 Caterpillar Sarl Monitoring The Operation Of A Work Machine
US20190055715A1 (en) * 2017-08-15 2019-02-21 Caterpillar Inc. System and method for controlling earthmoving machines
WO2019189888A1 (en) * 2018-03-30 2019-10-03 住友重機械工業株式会社 Construction machine operation assistance system, and construction machine
WO2020226848A1 (en) * 2019-05-03 2020-11-12 Caterpillar Inc. System for controlling the position of a work implement
JP2021113487A (en) * 2020-01-17 2021-08-05 バイドゥ ユーエスエイ エルエルシーBaidu USA LLC Neural task planning part for autonomous travelling vehicle

Also Published As

Publication number Publication date
JP2024005910A (en) 2024-01-17

Similar Documents

Publication Publication Date Title
AU2013260731B2 (en) System and method for optimizing a cut location
US8948981B2 (en) System and method for optimizing a cut location
US9228315B2 (en) System and method for modifying a path for a machine
US9487929B2 (en) Systems and methods for adjusting pass depth in view of excess materials
US10101723B2 (en) System and method for optimizing a work implement path
US20160077514A1 (en) System and Method for Optimizing a Work Implement Path
US9260837B1 (en) Intelligent pass jump control
US10883248B2 (en) Road maintenance using stored maintenance passes
US20140180547A1 (en) System and Method for Estimating Material Characteristics
AU2015224458A1 (en) System and method for optimizing a work implement path
US10056009B2 (en) Training apparatus
Chacko et al. State of the art in excavators
CN108222094B (en) Double-shovel board appliance system
WO2021241258A1 (en) Excavation plan creation device, working machine, and excavation plan creation method
US11634887B2 (en) Method and system for controlling construction machinery
WO2024005182A1 (en) Work vehicle path plan generation system and work vehicle path plan generation method
US20240068202A1 (en) Autonomous Control Of Operations Of Powered Earth-Moving Vehicles Using Data From On-Vehicle Perception Systems
CN112211237A (en) System and method for operating an underground machine
El Samid Infrastructure robotics: A trade-off study examining both autonomously and manually controlled approaches to lunar excavation and construction
CN114737553B (en) Method for compacting soil and soil compactor
US20210222406A1 (en) Methods and systems for guiding mission planning in a machine system
CN114737553A (en) Method for compacting soil and soil compactor
AU2020217430A1 (en) System and method for controlling operation of a machine
Mimura Development of a Bulldozer Operation Simulator

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23831621

Country of ref document: EP

Kind code of ref document: A1