EP3352037B1 - Autonomously traveling work vehicle and method for autonomous travel control - Google Patents
Autonomously traveling work vehicle and method for autonomous travel control Download PDFInfo
- Publication number
- EP3352037B1 EP3352037B1 EP17207001.3A EP17207001A EP3352037B1 EP 3352037 B1 EP3352037 B1 EP 3352037B1 EP 17207001 A EP17207001 A EP 17207001A EP 3352037 B1 EP3352037 B1 EP 3352037B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- obstruction
- sensor
- sensors
- autonomous traveling
- work vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 107
- 238000012545 processing Methods 0.000 claims description 42
- 238000001514 detection method Methods 0.000 claims description 31
- 238000012790 confirmation Methods 0.000 claims description 4
- 206010061876 Obstruction Diseases 0.000 description 176
- 230000006870 function Effects 0.000 description 12
- 230000002950 deficient Effects 0.000 description 8
- 230000002159 abnormal effect Effects 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 239000000126 substance Substances 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000007257 malfunction Effects 0.000 description 3
- 239000003595 mist Substances 0.000 description 3
- -1 rain Substances 0.000 description 3
- 239000000779 smoke Substances 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 240000007594 Oryza sativa Species 0.000 description 1
- 235000007164 Oryza sativa Nutrition 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000009566 rice Nutrition 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0055—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots with safety arrangements
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0234—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using optical markers or beacons
- G05D1/0236—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using optical markers or beacons in combination with a laser
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01B—SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL
- A01B69/00—Steering of agricultural machines or implements; Guiding agricultural machines or implements on a desired track
- A01B69/007—Steering or guiding of agricultural vehicles, e.g. steering of the tractor to keep the plough in the furrow
- A01B69/008—Steering or guiding of agricultural vehicles, e.g. steering of the tractor to keep the plough in the furrow automatic
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
- G05D1/024—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0255—Control of position or course in two dimensions specially adapted to land vehicles using acoustic signals, e.g. ultra-sonic singals
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
- G05D1/0278—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
- G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0808—Diagnosing performance data
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01B—SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL
- A01B33/00—Tilling implements with rotary driven tools, e.g. in combination with fertiliser distributors or seeders, with grubbing chains, with sloping axles, with driven discs
- A01B33/02—Tilling implements with rotary driven tools, e.g. in combination with fertiliser distributors or seeders, with grubbing chains, with sloping axles, with driven discs with tools on horizontal shaft transverse to direction of travel
- A01B33/021—Tilling implements with rotary driven tools, e.g. in combination with fertiliser distributors or seeders, with grubbing chains, with sloping axles, with driven discs with tools on horizontal shaft transverse to direction of travel with rigid tools
Definitions
- the disclosure relates to an autonomous traveling work vehicle that performs work while autonomously traveling along a travel route and a method for autonomous travel.
- An autonomous traveling work vehicle may have an obstruction sensor that detects, in a noncontact manner, an obstruction with which the autonomous traveling work vehicle is likely to collide during traveling. Upon detection of an obstruction, avoidance steering or emergency stop may be executed in order to avoid collision with the obstruction. Therefore, autonomous traveling may be prohibited when the obstruction sensor is not normally working.
- An autonomous traveling work vehicle disclosed in JP 2015-222503 A has an obstruction sensor or a camera as an obstruction detection unit and checks whether or not an obstruction is present ahead of, behind, and beside a machine body.
- a warning is issued, and the autonomous traveling work vehicle is controlled to slow down or stop.
- an engine is started by operating a key switch or when a check switch provided on a remote control device or provided close to a driver's seat is operated, whether the obstruction sensor is normal or abnormal is automatically checked before start of autonomous traveling.
- this abnormality checking whether the obstruction sensor is outputting a predetermined output voltage (or outputting an abnormal voltage), short circuit, and disconnection are checked.
- the obstruction sensor has an abnormality, the name of the sensor and contents of the abnormality are displayed on a display, and autonomous traveling is not permitted.
- a defective operation of the obstruction sensor is considered to be caused by malfunction of a signal system such as disconnection or short circuit of a signal line, and a state and the like of a sensor signal are checked to determine presence or absence of a defective operation.
- a signal system such as disconnection or short circuit of a signal line, and a state and the like of a sensor signal are checked to determine presence or absence of a defective operation.
- an obstruction sensor such as an ultrasonic sonar, a laser scanner, or a camera
- an obstruction sensor such as an ultrasonic sonar, a laser scanner, or a camera
- an external environment in which the obstruction sensor properly operates i.e., a proper operation environment in order to avoid a defective operation of the obstruction sensor caused by an external environment factor.
- a defective operation of an obstruction sensor caused by an external environment factor may be solved, for example, just by removing an attached object.
- autonomous traveling work vehicle disclosed in JP 2015-222503 A autonomous traveling is promptly prohibited even in a case where such a defective operation occurs.
- a signal system of an obstruction sensor has no problem, there is a risk of failure to properly detect an obstruction because of an inappropriate evaluation system such as a detection threshold value.
- JP H09-269829 A discloses an autonomous vehicle having means to automatically confirm the operation of an obstruction sensor.
- the vehicle checks an obstruction sensor and a light projection part of the sensor by means of a reflection unit. Based on the reflection, a sensitivity of the sensor can be adapted.
- JP H06-195121 A discloses an autonomous vehicle with obstruction sensors.
- the quality of each sensor can be checked by means of diagnostic master pieces which are arranged at predetermined positions of the travel. At the predetermined positions detecting signals of the obstruction diagnostic master piece and a mark pattern diagnostic master piece are collated with a normal signal pattern stored in advance.
- An autonomous driving vehicle disclosed in US 2016/0282874 A1 has sensors to detect an obstacle and sensor state evaluation means configured to evaluate a state of performance degradation of the sensors.
- Speed and steering angle control value setting means are provided configured to provide limit values to the driving speed and the steering angle based on a state of performance degradation of the sensor.
- US 2016/0084737 A1 discloses a method and system for auto safety verification of automatic guided vehicle (AGV) sensors.
- Obstruction sensors can be tested by means of a test object that is brought into the sensor field of the obstruction sensors.
- An auto safety verification test can be performed autonomously by the AGV to ensure that sensors on the AGV are configured correctly and working properly.
- an autonomous traveling work vehicle to autonomously travel along a travel route as well as a methods for autonomous travel of an autonomous traveling work vehicle along a travel route according to the independent claims are provided. Further embodiments are disclosed in dependent claims.
- an autonomous traveling work vehicle to autonomously travel along a travel route comprising: a travel control unit to control autonomous traveling; one or more obstruction sensors to detect an obstruction to traveling; an obstruction sensor determining unit to perform a sensor determining process for determining whether or not the obstruction sensors are normally operating; and an obstruction sensor processing unit to output a prohibition command to prohibit autonomous traveling controlled by the travel control unit in case the obstruction sensor determining unit determines that the obstruction sensors are not normally operating.
- the sensor determining process includes an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors_by removing an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors and a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected.
- the sensor trial process is performed only under the proper operation environment.
- a method for autonomous travel of an autonomous traveling work vehicle along a travel route comprising: controlling autonomous traveling by a travel control unit of the autonomous traveling work vehicle; performing, by an obstruction sensor determining unit of the autonomous traveling work vehicle, a sensor determining process for determining whether or not one or more obstruction sensors of the autonomous traveling work vehicle, configured for detecting an obstruction to traveling, are normally operating, the sensor determining process including an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensor by removing an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors and a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected; and outputting, by an obstruction sensor processing unit of the autonomous traveling work vehicle, a prohibition command to prohibit autonomous traveling controlled by the travel control unit when the obstruction sensor determining unit determines that the obstruction sensors are not normally operating.
- the sensor trial process is performed only under the proper operation
- a sensor determining process for determining whether or not the obstruction sensors are normally operating starts from an operation environment securing process.
- the operation environment securing process may be a process for checking whether or not a condition of an obstruction sensor operation environment needed for a normal operation of the obstruction sensors is met, for example, whether or not sensor beam exit opening and incident opening are blocked, whether or not a direction of emission of a sensor beam is deviated, or whether or not a hindering member is present on a path of a sensor beam in a case where the obstruction sensors are ones using a sensor beam such as ultrasonic sonars or laser scanners.
- the obstruction sensors are cameras, for example, whether or not a lens is fogged up or whether or not a foreign substance is attached to a lens may be checked. Unless such a condition of an obstruction sensor operation environment is met, the obstruction sensors may not operate normally even if a signal system of the obstruction sensors is normal. Therefore, first, a defective operation of the obstruction sensors caused by an operation environment that is an external factor may be avoided by securing an operation environment through this operation environment securing process.
- a main role of the operation environment securing process may be not to find out malfunction of the obstruction sensors themselves but to remove an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors.
- User's maintenance and inspection may be important to remove such inconvenience in an operation environment (surrounding environment), for example, attachment of a foreign substance.
- the operation environment securing process may include a notification process for notifying a user of an instruction to secure the proper operation environment and a confirmation process for receiving, from the user, information indicating that the proper operation environment has been secured.
- a user when the operation environment securing process is started, a user is given an instruction to secure a proper operation environment for the obstruction sensors (e.g., remove a foreign substance attached to the obstruction sensors), and the user may perform an operation for securing the proper operation environment based on this instruction.
- Contents of this instruction may be stored in advance and selected automatically in accordance with a season or a characteristic of a work place or selected based on a request from a control center or a supervisor, the user may be conveniently notified of a proper instruction according to a situation.
- the user who has performed the operation for securing a proper operation environment based on this instruction may input completion of the operation through an input operation device such as a touch panel. Thereby, a control system may recognize that the operation environment has been secured.
- the sensor determining process may include a sensor trial process for determining whether or not the obstruction sensors are normally operating under the proper operation environment by trial.
- a dummy piece may be provided at a corner of a detection region of an obstruction sensor in order to determine whether or not the obstruction sensor is normally operating by actual trial. In this case, however, the dummy piece itself may reduce the detection region. Furthermore, this may not be effective for solution of a problem that detection of an obstruction becomes unstable, for example, because of mist, rain, or smoke. In view of this, in the sensor trial process, it may be determined that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected. This may ensure that the obstruction sensors detect an obstruction, thereby enabling safe autonomous traveling.
- the user may conveniently be notified of a proper instruction according to a situation. Furthermore, in a case where the user is notified, by a display such as a touch panel, a buzzer, or a lamp, of recognition by a control system of a work vehicle that it has been determined in the sensor trial process that the obstruction sensors are normally operating, the user may feel safe to proceed to a next operation.
- a display such as a touch panel, a buzzer, or a lamp
- the obstruction sensors are configured by cameras and where an obstruction is detected based on photographed images acquired from the cameras, it may be confirmed that the obstruction sensors (cameras) are normally operating when it is confirmed that an obstruction that has entered field of views of the cameras can be detected.
- the obstruction sensors may be cameras having a face recognition function; and in the sensor trial process, it may be determined that the cameras are normally operating in case a face of a person that enters the field of views at least one of the cameras is recognized by the cameras. According to this configuration, it may be determined that the obstruction sensors (cameras) are normally operating in a case where the cameras recognize a face when a user stands in front of the cameras.
- the sensor determining process may include a signal line determining process for detecting disconnection and short circuit of a signal line of the obstruction sensors.
- the sensor determining process performed by the obstruction sensor determining unit may be performed when the main key is turned on. However, in a case where the main key is turned off and on repeatedly in a short period, it may be annoying that the sensor determining process for determining an operation of the obstruction sensors is activated every time the main key is turned on. In order to avoid this problem the sensor determining process performed by the obstruction sensor determining unit may be performed when a main key is turned on; and within a predetermined period from an immediately preceding time at which the main key is turned off to a subsequent time at which the main key is turned on, a last result of determination indicating a normal operation in the sensor determining process is may be maintained while the sensor determining process may be omitted.
- An abnormality of an obstruction sensor that perform the function of eyes of a driver during autonomous traveling poses a significant problem in autonomous traveling.
- a driver can visually confirm an obstruction, and therefore manual traveling may be permitted during occurrence of an abnormality of an obstruction sensor.
- this output of the prohibition command from the obstruction sensor processing unit may be suspended during manual traveling.
- a condition for autonomous traveling may be that the obstruction sensors are normally operating.
- a defective operation caused by an external environment factor such as attachment of mud or rainwater to the obstruction sensors may sometimes be less likely to occur, for example, depending on characteristics of a work place, characteristics of traveling working, or characteristics of a work device. In such cases, the sensor trial process may be more important than the operation environment securing process.
- an embodiment in which the sensor determining process includes a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected may make it possible to determine a detection state of the obstruction sensors with certainty even in a situation where detection of an obstruction is unstable, for example, because of mist, rain, or smoke.
- the work vehicle is a tractor that performs work and travels in a field (work field) separated by ridges as boundaries.
- working traveling used herein includes working while traveling, traveling itself, working itself, a state where at least one of working and traveling, traveling itself, and working itself is being performed, and a state where such traveling working is temporarily stopped.
- a tractor is provided with an operation unit 20 at a center of a vehicle body 1 supported by front wheels 11 and rear wheels 12.
- a work device 30 that is a rotary tilling machine is mounted via a hydraulic lifting mechanism 31.
- the front wheels 11 function as steering control wheels through which the tractor changes a travel direction when a steering angle of the steering control wheels is changed.
- the steering angle of the front wheels 11 is changed by an operation of a steering mechanism 13.
- the steering mechanism 13 includes a steering motor 14 for automatic steering.
- the front wheels 11 can be steered by operating a steering wheel 22 disposed on the operation unit 20.
- a satellite positioning module 80 configured as a global navigation satellite system (GNSS) module is provided.
- GNSS global navigation satellite system
- a satellite antenna for receiving GNSS signals is attached at a ceiling area of the cabin 21.
- An inertial navigation module incorporated with a gyro acceleration sensor and a magnetic direction sensor may be combined with the satellite positioning module 80 for complementing satellite navigation.
- the inertial navigation module may also be provided in a different location from that of the satellite positioning module 80.
- the vehicle body 1 of the tractor is provided with obstruction sensors 7 that detect an obstruction to traveling.
- the obstruction sensors 7 include, as obstruction sensors, ultrasonic sonars 71, laser scanners (lidars) 72, and cameras 73.
- Two ultrasonic sonars 71 are provided on each of front, rear, left, and right sides of the vehicle body 1. That is, eight ultrasonic sonars 71 in total are provided.
- the ultrasonic sonars 71 detect an obstruction in almost all of the surrounding region around the vehicle body 1.
- a single laser scanner 72 is provided on each of the front and rear sides of the vehicle body 1. That is, two laser scanners 72 in total are provided.
- the laser scanners 72 detect an obstruction in a backward direction and a forward direction of the vehicle body 1.
- a single camera 73 is provided on each of the front, rear, left, and right sides of the vehicle body 1. That is, four cameras 73 in total are provided.
- the cameras 73 photograph all of the surrounding region around the vehicle body 1. In this exemplary embodiment, the cameras 73 are not used just to detect an obstruction around the vehicle body 1. Photographed images output from the four cameras 73 are used as monitoring images as they are, or are used as an overhead image obtained by a view point conversion process.
- attribute information indicating that a face has been recognized may be added to data of photographed images output from the camera 73 in a case where a face is recognized in a photographed image.
- Fig. 3 illustrates a control system constructed in a tractor.
- a function of determining whether or not the obstruction sensors 7 are normally operating is constructed mainly in a first control unit 4, and a function of controlling working traveling while using a detection result of the obstruction sensors 7 is constructed mainly in a second control unit 5.
- the first control unit 4, the second control unit 5, and an input output signal processing unit 6 are connected to one another by an on-board LAN.
- the input output signal processing unit 6 processes input output signals handled in working traveling of a tractor.
- the satellite positioning module 80 is also connected to the same on-board LAN.
- the obstruction sensors 7 are connected to the input output signal processing unit 6.
- the cameras 73 are connected to the input output signal processing unit 6 through an image processing unit 70 that performs processes such as a process for generating a monitoring image from photographed images acquired from the cameras 73.
- the image processing unit 70 may be constructed in the input output signal processing unit 6 or may be constructed in the cameras 73.
- the face recognition process may be performed in the image processing unit 70.
- vehicle travel instruments 91, work device instruments 92, a notification device 93, an autonomous/manual switch 94, traveling state detection sensors 81, and work state detection sensors 82 are connected to the input output signal processing unit 6.
- the vehicle travel instruments 91 include the steering motor 14 and control instruments attached to a transmission mechanism and an engine unit controlled for vehicle traveling.
- the work device instruments 92 include control instruments for driving the work device 30 and the lifting mechanism 31.
- the notification device 93 may include, a display, a lamp, a speaker, and the like for giving an alert on work and travel to a driver and a supervisor.
- the autonomous/manual switch 94 is a switch for selecting either an autonomous travel mode for traveling with autonomous steering or a manual steering mode for traveling with manual steering.
- operating the autonomous/manual switch 94 while the work vehicle is traveling in the autonomous steering mode may cause the work vehicle to switch to traveling with manual steering
- operating the autonomous/manual switch 94 while the work vehicle is traveling with manual steering may cause the work vehicle to switch to traveling with autonomous steering.
- the traveling state detection sensors 81 include sensors for detecting travel states such as a steering angle, an engine speed, and a transmission state.
- the work state detection sensors 82 include use state detection sensors that detect use states such as orientation, a height above ground, and a use width of the work device 30.
- the input output signal processing unit 6 includes a communication unit (not illustrated).
- the communication unit exchanges data with an external computer in conformity to a wireless communication standard or a wired communication standard.
- Examples of the external computer include a control computer constructed in a control center or another facility in a remote location, and a tablet computer or a smartphone (mobile phone) held by a driver or a supervisor.
- the first control unit 4 includes a touch panel 43 that functions as a graphical user interface and an obstruction sensor determining unit 40 that performs a sensor determining process for determining whether the obstruction sensors 7 are normal or abnormal.
- the ultrasonic sonars 71, the laser scanners 72, and the cameras 73 are provided as the obstruction sensors 7 in this exemplary embodiment, other sensors may be provided alternatively or additionally and a common term "obstruction sensors" is used in a case where they need not be distinguished.
- An operation environment securing processing unit 41 and a sensor trial processing unit 42 are constructed in the obstruction sensor determining unit 40 in order to perform the sensor determining process.
- the operation environment securing processing unit 41 performs an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors.
- a user a driver or a worker
- the operation environment securing processing unit 41 confirms that the operation environment has been secured through a user input operation indicating completion of the securing operation.
- the sensor trial processing unit 42 performs a sensor trial process for checking whether the obstruction sensors are normal or abnormal under the proper operation environment by trial.
- a user (a driver or a worker) is instructed to enter a detection region of the obstruction sensors, for example, through the input output device for a touch panel, and the sensor trial processing unit 42 confirms that the obstruction sensors are normally operating when the obstruction sensors detect the user as an obstruction.
- the obstruction sensor determining unit 40 may also have a function, known as such, of performing a signal line determining process for checking short circuit and disconnection of a signal line of the obstruction sensors.
- the second control unit 5 includes a work travel control unit 50, a travel route setting unit 53, an own vehicle position calculating unit 54, a work travel command generating unit 55, and an obstruction sensor processing unit 56.
- the travel route setting unit 53 loads a travel route that is a target route for autonomous traveling into a memory in a readable manner. The travel route may be generated on a tractor side or may be generated on another computer and then downloaded into the second control unit 5.
- the own vehicle position calculating unit 54 calculates a coordinate position of the vehicle body 1 on a map based on positioning data obtained from the satellite positioning module 80 such as a GPS.
- the work travel command generating unit 55 finds a displacement amount between the own vehicle position calculated by the own vehicle position calculating unit 54 and the travel route set by the travel route setting unit 53 and generates a traveling command (including a steering command and a vehicle speed command) to reduce this displacement amount.
- the work travel control unit 50 includes a travel control unit 51 and a work control unit 52.
- the travel control unit 51 has an autonomous travel control function (autonomous travel mode) and a manual travel control function (manual travel mode).
- autonomous travel mode autonomous travel mode
- manual travel mode manual travel mode
- the travel control unit 51 gives a control signal to the vehicle travel instruments 91 based on the traveling command received from the work travel command generating unit 55.
- the work control unit 52 gives a control signal to the work device instruments 92 based on a work command received from the work travel command generating unit 55.
- the manual travel mode is selected, manual traveling based on a driver's operation of the steering wheel 22 is performed.
- the work control unit 52 has an autonomous work control function for causing the work device instruments 92 to autonomously operate and a manual work control function of manually controlling the work device instruments 92 by using various operating tools.
- the obstruction sensor processing unit 56 may perform different processes based on a detection result of the obstruction sensors 7 in three situations, i.e., before the start of autonomous traveling, during autonomous traveling, and during manual traveling. Before the start of autonomous traveling, the obstruction sensor processing unit 56 may give a permission command to permit autonomous traveling to the work travel control unit 50 in case a normal operation of the obstruction sensors is confirmed (the obstruction sensors are determined to be normal) based on a determination result output from the obstruction sensor determining unit 40. The obstruction sensor processing unit 56 gives a prohibition command to prohibit autonomous traveling to the work travel control unit 50 in a case where a normal operation of the obstruction sensors is not confirmed (the obstruction sensors are determined to be abnormal).
- a stopping command to stop the vehicle body 1 or a slowdown command to decrease a vehicle speed may be given to the work travel control unit 50.
- a slowdown command to decrease a vehicle speed may be given to the work travel control unit 50 even in a case where an obstruction is detected in a surrounding region other than the traveling direction of the vehicle body 1.
- a notification about contents of the command may be given through the notification device 93.
- the sensor determining process is activated when a main key is turned on.
- the operation environment securing process is performed by the operation environment securing processing unit 41, and a message requesting an operation environment securing operation such as cleaning laser emission openings of the laser scanners 72 or lenses of the cameras 73 is displayed on the touch panel 43 as an example of the notification process.
- a user e.g., a driver or a supervisor
- the operation environment securing processing unit 41 confirms that a proper operation environment has been secured by this user input operation.
- the sensor trial process is performed by the sensor trial processing unit 42, and a message requesting that a user enters paths of laser beams emitted from the laser scanners 72 (a detection region of the obstruction sensors) (artificial entry of an obstruction) or the field of view of at least one of the cameras 73 is displayed on the touch panel 43.
- the obstruction sensors 7 are operating, and therefore when the user who entered the paths of laser beams (a detection region of the obstruction sensors) or an object carried by the user is detected as an obstruction, a result of the detection is returned to the sensor trial processing unit 42 as a normal operation confirmation signal.
- the sensor trial processing unit 42 may confirm a normal operation through an actual trial of the obstruction sensors. Since the obstruction sensors 7 are disposed so as to cover the whole region around the tractor, the user may complete the sensor trial process by going around the tractor.
- the obstruction sensor determining unit 40 may perform a signal line determining process for checking short circuit and disconnection of a signal line of the obstruction sensors at any time.
- the obstruction sensor processing unit 56 gives a permission command to the travel control unit 51 of the work travel control unit 50. In this way, autonomous traveling is permitted.
- the obstruction sensor processing unit 56 gives a prohibition command to the travel control unit 51 of the work travel control unit 50. In this way, autonomous traveling is prohibited.
- a control routine for start of traveling is activated, whether or not the main key of the tractor is on is checked (#01). In a case where the main key is off (No in #01), this routine is finished. In a case where the main key is on (Yes in #01), it is checked which of the autonomous traveling mode and the manual traveling mode is set (#02). In case the autonomous traveling mode is set (Yes in #02), a period T elapsed from an immediately preceding time at which the main key was turned off is calculated (#03). The elapsed period T is compared with a threshold value H (#04). This threshold value H is a period set in order to avoid repetition of an unnecessary sensor determining process that occurs when the main key is turned off and on repeatedly within a short period.
- the operation environment securing process starts, and a user is instructed to perform an operation for securing an operation environment through the touch panel 43 (#05).
- the user performs the operation for securing an operation environment based on this instruction.
- the operation environment securing processing unit 41 determines that it has been confirmed that the operation environment has been secured (Yes in #06), and then the sensor trial process for determining whether or not the obstruction sensors are normally operating by trial starts (#07).
- the sensor trial process is performed as follows:
- the result of the determination is checked (#09), and in case the result of the determination indicates a normal operation (Yes in #09), the obstruction sensor processing unit 56 gives a permission command to the work travel control unit 50 (#10). In this way, autonomous traveling starts (#11). In case the result of the determination does not indicate a normal operation (No in #09), the obstruction sensor processing unit 56 gives a prohibition command to the work travel control unit 50 (#12). As a result, autonomous traveling is not started, and the routine returns to step #01.
- step #14 whether or not a last result of sensor determination indicates a normal operation is checked (#14), and in a case where the last result of the sensor determination indicates a normal operation (Yes in #14), step #10 is performed, and a permission command is given to the work travel control unit 50 without performing a new sensor determining process. In a case the last result of the sensor determination does not indicate a normal operation (No in #14), step #01 is performed again.
- step #05 may be performed in which the sensor determination process is started instead of returning to step #01.
- the technologies described herein are applicable to an autonomous traveling work vehicle for working and traveling along a travel route.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Electromagnetism (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Optics & Photonics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Environmental Sciences (AREA)
- Soil Sciences (AREA)
- Mechanical Engineering (AREA)
- Game Theory and Decision Science (AREA)
- Business, Economics & Management (AREA)
- Acoustics & Sound (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Guiding Agricultural Machines (AREA)
- Steering Control In Accordance With Driving Conditions (AREA)
- Traffic Control Systems (AREA)
Description
- The disclosure relates to an autonomous traveling work vehicle that performs work while autonomously traveling along a travel route and a method for autonomous travel.
- An autonomous traveling work vehicle may have an obstruction sensor that detects, in a noncontact manner, an obstruction with which the autonomous traveling work vehicle is likely to collide during traveling. Upon detection of an obstruction, avoidance steering or emergency stop may be executed in order to avoid collision with the obstruction. Therefore, autonomous traveling may be prohibited when the obstruction sensor is not normally working.
- An autonomous traveling work vehicle disclosed in
JP 2015-222503 A - Upon detection of an obstruction, a warning is issued, and the autonomous traveling work vehicle is controlled to slow down or stop. When an engine is started by operating a key switch or when a check switch provided on a remote control device or provided close to a driver's seat is operated, whether the obstruction sensor is normal or abnormal is automatically checked before start of autonomous traveling. In this abnormality checking, whether the obstruction sensor is outputting a predetermined output voltage (or outputting an abnormal voltage), short circuit, and disconnection are checked. In a case where the obstruction sensor has an abnormality, the name of the sensor and contents of the abnormality are displayed on a display, and autonomous traveling is not permitted.
- In the autonomous traveling work vehicle disclosed in
JP 2015-222503 A JP 2015-222503 A -
JP H09-269829 A -
JP H06-195121 A - An autonomous driving vehicle disclosed in
US 2016/0282874 A1 has sensors to detect an obstacle and sensor state evaluation means configured to evaluate a state of performance degradation of the sensors. Speed and steering angle control value setting means are provided configured to provide limit values to the driving speed and the steering angle based on a state of performance degradation of the sensor. When the sensor suffers from performance degradation, the vehicle stops after it drives to a point at which the vehicle does not obstruct movement of other vehicles. -
US 2016/0084737 A1 discloses a method and system for auto safety verification of automatic guided vehicle (AGV) sensors. Obstruction sensors can be tested by means of a test object that is brought into the sensor field of the obstruction sensors. An auto safety verification test can be performed autonomously by the AGV to ensure that sensors on the AGV are configured correctly and working properly. - It is an object to provide an autonomous traveling work vehicle and a method for autonomous travel of an autonomous traveling work vehicle that can properly determine whether or not an obstruction sensor is normally operating by also taking into consideration factors other than malfunction of a signal system.
- For solving the problem, an autonomous traveling work vehicle to autonomously travel along a travel route as well as a methods for autonomous travel of an autonomous traveling work vehicle along a travel route according to the independent claims are provided. Further embodiments are disclosed in dependent claims.
- According to an aspect, an autonomous traveling work vehicle to autonomously travel along a travel route is provided, comprising: a travel control unit to control autonomous traveling; one or more obstruction sensors to detect an obstruction to traveling; an obstruction sensor determining unit to perform a sensor determining process for determining whether or not the obstruction sensors are normally operating; and an obstruction sensor processing unit to output a prohibition command to prohibit autonomous traveling controlled by the travel control unit in case the obstruction sensor determining unit determines that the obstruction sensors are not normally operating. The sensor determining process includes an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors_by removing an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors and a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected. The sensor trial process is performed only under the proper operation environment.
- According to a further aspect, a method for autonomous travel of an autonomous traveling work vehicle along a travel route is provided, the method comprising: controlling autonomous traveling by a travel control unit of the autonomous traveling work vehicle; performing, by an obstruction sensor determining unit of the autonomous traveling work vehicle, a sensor determining process for determining whether or not one or more obstruction sensors of the autonomous traveling work vehicle, configured for detecting an obstruction to traveling, are normally operating, the sensor determining process including an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensor by removing an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors and a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected; and outputting, by an obstruction sensor processing unit of the autonomous traveling work vehicle, a prohibition command to prohibit autonomous traveling controlled by the travel control unit when the obstruction sensor determining unit determines that the obstruction sensors are not normally operating. The sensor trial process is performed only under the proper operation environment.
- According to this configuration, a sensor determining process for determining whether or not the obstruction sensors are normally operating starts from an operation environment securing process. The operation environment securing process may be a process for checking whether or not a condition of an obstruction sensor operation environment needed for a normal operation of the obstruction sensors is met, for example, whether or not sensor beam exit opening and incident opening are blocked, whether or not a direction of emission of a sensor beam is deviated, or whether or not a hindering member is present on a path of a sensor beam in a case where the obstruction sensors are ones using a sensor beam such as ultrasonic sonars or laser scanners. Alternatively, in a case where the obstruction sensors are cameras, for example, whether or not a lens is fogged up or whether or not a foreign substance is attached to a lens may be checked. Unless such a condition of an obstruction sensor operation environment is met, the obstruction sensors may not operate normally even if a signal system of the obstruction sensors is normal. Therefore, first, a defective operation of the obstruction sensors caused by an operation environment that is an external factor may be avoided by securing an operation environment through this operation environment securing process.
- A main role of the operation environment securing process may be not to find out malfunction of the obstruction sensors themselves but to remove an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors. User's maintenance and inspection may be important to remove such inconvenience in an operation environment (surrounding environment), for example, attachment of a foreign substance. However, a user may tend to forget such maintenance and inspection. In view of this, the operation environment securing process may include a notification process for notifying a user of an instruction to secure the proper operation environment and a confirmation process for receiving, from the user, information indicating that the proper operation environment has been secured. According to this configuration, when the operation environment securing process is started, a user is given an instruction to secure a proper operation environment for the obstruction sensors (e.g., remove a foreign substance attached to the obstruction sensors), and the user may perform an operation for securing the proper operation environment based on this instruction. Contents of this instruction may be stored in advance and selected automatically in accordance with a season or a characteristic of a work place or selected based on a request from a control center or a supervisor, the user may be conveniently notified of a proper instruction according to a situation. The user who has performed the operation for securing a proper operation environment based on this instruction may input completion of the operation through an input operation device such as a touch panel. Thereby, a control system may recognize that the operation environment has been secured.
- Even in a case where sensor beam exit opening and incident opening are brought into a good condition by removal of a foreign substance attached to the obstruction sensors, detection of an obstruction may become unstable, for example, because of mist, rain, or smoke. In order to solve such a problem, it may be effective to examine whether or not the obstruction sensors detect an actual obstruction by trial. In view of this, the sensor determining process may include a sensor trial process for determining whether or not the obstruction sensors are normally operating under the proper operation environment by trial.
- Conventionally, a dummy piece may be provided at a corner of a detection region of an obstruction sensor in order to determine whether or not the obstruction sensor is normally operating by actual trial. In this case, however, the dummy piece itself may reduce the detection region. Furthermore, this may not be effective for solution of a problem that detection of an obstruction becomes unstable, for example, because of mist, rain, or smoke. In view of this, in the sensor trial process, it may be determined that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected. This may ensure that the obstruction sensors detect an obstruction, thereby enabling safe autonomous traveling. Also in this case, in case contents of an instruction given to a user in this sensor trial process are selected automatically or selected based on a request from a control center or a supervisor, the user may conveniently be notified of a proper instruction according to a situation. Furthermore, in a case where the user is notified, by a display such as a touch panel, a buzzer, or a lamp, of recognition by a control system of a work vehicle that it has been determined in the sensor trial process that the obstruction sensors are normally operating, the user may feel safe to proceed to a next operation.
- In an embodiment in which the obstruction sensors are configured by cameras and where an obstruction is detected based on photographed images acquired from the cameras, it may be confirmed that the obstruction sensors (cameras) are normally operating when it is confirmed that an obstruction that has entered field of views of the cameras can be detected. In view of this, the obstruction sensors may be cameras having a face recognition function; and in the sensor trial process, it may be determined that the cameras are normally operating in case a face of a person that enters the field of views at least one of the cameras is recognized by the cameras. According to this configuration, it may be determined that the obstruction sensors (cameras) are normally operating in a case where the cameras recognize a face when a user stands in front of the cameras.
- Conventional checking of short circuit and disconnection of a signal line may also be important to determine whether or not the obstruction sensors themselves are normally operating. In view of this, the sensor determining process may include a signal line determining process for detecting disconnection and short circuit of a signal line of the obstruction sensors.
- The sensor determining process performed by the obstruction sensor determining unit may be performed when the main key is turned on. However, in a case where the main key is turned off and on repeatedly in a short period, it may be annoying that the sensor determining process for determining an operation of the obstruction sensors is activated every time the main key is turned on. In order to avoid this problem the sensor determining process performed by the obstruction sensor determining unit may be performed when a main key is turned on; and within a predetermined period from an immediately preceding time at which the main key is turned off to a subsequent time at which the main key is turned on, a last result of determination indicating a normal operation in the sensor determining process is may be maintained while the sensor determining process may be omitted.
- An abnormality of an obstruction sensor that perform the function of eyes of a driver during autonomous traveling poses a significant problem in autonomous traveling. However, during manual traveling, a driver can visually confirm an obstruction, and therefore manual traveling may be permitted during occurrence of an abnormality of an obstruction sensor. In view of this output of the prohibition command from the obstruction sensor processing unit may be suspended during manual traveling. In other words, a condition for autonomous traveling may be that the obstruction sensors are normally operating.
- A defective operation caused by an external environment factor such as attachment of mud or rainwater to the obstruction sensors may sometimes be less likely to occur, for example, depending on characteristics of a work place, characteristics of traveling working, or characteristics of a work device. In such cases, the sensor trial process may be more important than the operation environment securing process.
- In view of this, an embodiment in which the sensor determining process includes a sensor trial process for determining that the obstruction sensors are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors is detected, may make it possible to determine a detection state of the obstruction sensors with certainty even in a situation where detection of an obstruction is unstable, for example, because of mist, rain, or smoke.
- The embodiments disclosed above with regard to the autonomous traveling work vehicle may apply to the method for autonomous travel of an autonomous traveling work vehicle mutatis mutandis.
-
- Fig. 1
- is a side view of a tractor that is an example of an autonomous working vehicle that performs work while autonomously traveling along a travel route.
- Fig. 2
- is a schematic plan view of a tractor illustrating obstruction detection sensors disposed on the tractor.
- Fig. 3
- is a functional block diagram illustrating a control system of a tractor.
- Fig. 4
- is an explanatory view illustrating a flow of control in a sensor determining process.
- Fig. 5
- is a flowchart illustrating an example of a sensor determining process.
- Next, an exemplary embodiment of an autonomous traveling work vehicle will be described below with reference to the drawings. In this embodiment, the work vehicle is a tractor that performs work and travels in a field (work field) separated by ridges as boundaries. The term "working traveling" used herein includes working while traveling, traveling itself, working itself, a state where at least one of working and traveling, traveling itself, and working itself is being performed, and a state where such traveling working is temporarily stopped.
- As illustrated in
Fig. 1 , a tractor is provided with anoperation unit 20 at a center of avehicle body 1 supported byfront wheels 11 andrear wheels 12. At a rear of thevehicle body 1, awork device 30 that is a rotary tilling machine is mounted via ahydraulic lifting mechanism 31. Thefront wheels 11 function as steering control wheels through which the tractor changes a travel direction when a steering angle of the steering control wheels is changed. The steering angle of thefront wheels 11 is changed by an operation of asteering mechanism 13. Thesteering mechanism 13 includes asteering motor 14 for automatic steering. For manual traveling, thefront wheels 11 can be steered by operating asteering wheel 22 disposed on theoperation unit 20. In acabin 21 of the tractor, asatellite positioning module 80 configured as a global navigation satellite system (GNSS) module is provided. As a component of thesatellite positioning module 80, a satellite antenna for receiving GNSS signals (including GPS signals) is attached at a ceiling area of thecabin 21. An inertial navigation module incorporated with a gyro acceleration sensor and a magnetic direction sensor may be combined with thesatellite positioning module 80 for complementing satellite navigation. The inertial navigation module may also be provided in a different location from that of thesatellite positioning module 80. - In an embodiment, as schematically illustrated in
Fig. 2 , thevehicle body 1 of the tractor is provided withobstruction sensors 7 that detect an obstruction to traveling. Theobstruction sensors 7 include, as obstruction sensors,ultrasonic sonars 71, laser scanners (lidars) 72, andcameras 73. Twoultrasonic sonars 71 are provided on each of front, rear, left, and right sides of thevehicle body 1. That is, eightultrasonic sonars 71 in total are provided. Theultrasonic sonars 71 detect an obstruction in almost all of the surrounding region around thevehicle body 1. Asingle laser scanner 72 is provided on each of the front and rear sides of thevehicle body 1. That is, twolaser scanners 72 in total are provided. Thelaser scanners 72 detect an obstruction in a backward direction and a forward direction of thevehicle body 1. Asingle camera 73 is provided on each of the front, rear, left, and right sides of thevehicle body 1. That is, fourcameras 73 in total are provided. Thecameras 73 photograph all of the surrounding region around thevehicle body 1. In this exemplary embodiment, thecameras 73 are not used just to detect an obstruction around thevehicle body 1. Photographed images output from the fourcameras 73 are used as monitoring images as they are, or are used as an overhead image obtained by a view point conversion process. Furthermore, in embodiments in which thecameras 73 have a face recognition function, attribute information indicating that a face has been recognized may be added to data of photographed images output from thecamera 73 in a case where a face is recognized in a photographed image. -
Fig. 3 illustrates a control system constructed in a tractor. In the control system according to this exemplary embodiment, a function of determining whether or not theobstruction sensors 7 are normally operating is constructed mainly in afirst control unit 4, and a function of controlling working traveling while using a detection result of theobstruction sensors 7 is constructed mainly in asecond control unit 5. Thefirst control unit 4, thesecond control unit 5, and an input outputsignal processing unit 6 are connected to one another by an on-board LAN. The input outputsignal processing unit 6 processes input output signals handled in working traveling of a tractor. Furthermore, thesatellite positioning module 80 is also connected to the same on-board LAN. - The
obstruction sensors 7 are connected to the input outputsignal processing unit 6. InFig. 3 , thecameras 73 are connected to the input outputsignal processing unit 6 through animage processing unit 70 that performs processes such as a process for generating a monitoring image from photographed images acquired from thecameras 73. Theimage processing unit 70 may be constructed in the input outputsignal processing unit 6 or may be constructed in thecameras 73. The face recognition process may be performed in theimage processing unit 70. - In addition,
vehicle travel instruments 91,work device instruments 92, anotification device 93, an autonomous/manual switch 94, travelingstate detection sensors 81, and workstate detection sensors 82 are connected to the input outputsignal processing unit 6. Thevehicle travel instruments 91 include thesteering motor 14 and control instruments attached to a transmission mechanism and an engine unit controlled for vehicle traveling. Thework device instruments 92 include control instruments for driving thework device 30 and thelifting mechanism 31. Thenotification device 93 may include, a display, a lamp, a speaker, and the like for giving an alert on work and travel to a driver and a supervisor. The autonomous/manual switch 94 is a switch for selecting either an autonomous travel mode for traveling with autonomous steering or a manual steering mode for traveling with manual steering. For example, operating the autonomous/manual switch 94 while the work vehicle is traveling in the autonomous steering mode may cause the work vehicle to switch to traveling with manual steering, and operating the autonomous/manual switch 94 while the work vehicle is traveling with manual steering may cause the work vehicle to switch to traveling with autonomous steering. The travelingstate detection sensors 81 include sensors for detecting travel states such as a steering angle, an engine speed, and a transmission state. The workstate detection sensors 82 include use state detection sensors that detect use states such as orientation, a height above ground, and a use width of thework device 30. - The input output
signal processing unit 6 includes a communication unit (not illustrated). The communication unit exchanges data with an external computer in conformity to a wireless communication standard or a wired communication standard. Examples of the external computer include a control computer constructed in a control center or another facility in a remote location, and a tablet computer or a smartphone (mobile phone) held by a driver or a supervisor. - The
first control unit 4 includes atouch panel 43 that functions as a graphical user interface and an obstructionsensor determining unit 40 that performs a sensor determining process for determining whether theobstruction sensors 7 are normal or abnormal. Although theultrasonic sonars 71, thelaser scanners 72, and thecameras 73 are provided as theobstruction sensors 7 in this exemplary embodiment, other sensors may be provided alternatively or additionally and a common term "obstruction sensors" is used in a case where they need not be distinguished. An operation environment securingprocessing unit 41 and a sensortrial processing unit 42 are constructed in the obstructionsensor determining unit 40 in order to perform the sensor determining process. The operation environment securingprocessing unit 41 performs an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors. In an example of the operation environment securing process, a user (a driver or a worker) is instructed to secure an operation environment, for example, through an input output device for a touch panel, and the operation environment securingprocessing unit 41 confirms that the operation environment has been secured through a user input operation indicating completion of the securing operation. The sensortrial processing unit 42 performs a sensor trial process for checking whether the obstruction sensors are normal or abnormal under the proper operation environment by trial. In an example of the sensor trial process, a user (a driver or a worker) is instructed to enter a detection region of the obstruction sensors, for example, through the input output device for a touch panel, and the sensortrial processing unit 42 confirms that the obstruction sensors are normally operating when the obstruction sensors detect the user as an obstruction. The obstructionsensor determining unit 40 may also have a function, known as such, of performing a signal line determining process for checking short circuit and disconnection of a signal line of the obstruction sensors. - The
second control unit 5 includes a worktravel control unit 50, a travelroute setting unit 53, an own vehicleposition calculating unit 54, a work travelcommand generating unit 55, and an obstructionsensor processing unit 56. The travelroute setting unit 53 loads a travel route that is a target route for autonomous traveling into a memory in a readable manner. The travel route may be generated on a tractor side or may be generated on another computer and then downloaded into thesecond control unit 5. The own vehicleposition calculating unit 54 calculates a coordinate position of thevehicle body 1 on a map based on positioning data obtained from thesatellite positioning module 80 such as a GPS. The work travelcommand generating unit 55 finds a displacement amount between the own vehicle position calculated by the own vehicleposition calculating unit 54 and the travel route set by the travelroute setting unit 53 and generates a traveling command (including a steering command and a vehicle speed command) to reduce this displacement amount. - The work
travel control unit 50 includes atravel control unit 51 and awork control unit 52. Thetravel control unit 51 has an autonomous travel control function (autonomous travel mode) and a manual travel control function (manual travel mode). When the autonomous travel mode is selected, thetravel control unit 51 gives a control signal to thevehicle travel instruments 91 based on the traveling command received from the work travelcommand generating unit 55. Thework control unit 52 gives a control signal to thework device instruments 92 based on a work command received from the work travelcommand generating unit 55. When the manual travel mode is selected, manual traveling based on a driver's operation of thesteering wheel 22 is performed. Similarly, thework control unit 52 has an autonomous work control function for causing thework device instruments 92 to autonomously operate and a manual work control function of manually controlling thework device instruments 92 by using various operating tools. - The obstruction
sensor processing unit 56 may perform different processes based on a detection result of theobstruction sensors 7 in three situations, i.e., before the start of autonomous traveling, during autonomous traveling, and during manual traveling. Before the start of autonomous traveling, the obstructionsensor processing unit 56 may give a permission command to permit autonomous traveling to the worktravel control unit 50 in case a normal operation of the obstruction sensors is confirmed (the obstruction sensors are determined to be normal) based on a determination result output from the obstructionsensor determining unit 40. The obstructionsensor processing unit 56 gives a prohibition command to prohibit autonomous traveling to the worktravel control unit 50 in a case where a normal operation of the obstruction sensors is not confirmed (the obstruction sensors are determined to be abnormal). During autonomous traveling and manual working, when an obstruction sensor detects an obstruction in a traveling direction of thevehicle body 1, a stopping command to stop thevehicle body 1 or a slowdown command to decrease a vehicle speed may be given to the worktravel control unit 50. A slowdown command to decrease a vehicle speed may be given to the worktravel control unit 50 even in a case where an obstruction is detected in a surrounding region other than the traveling direction of thevehicle body 1. When some sort of command is given from the obstructionsensor processing unit 56 to the worktravel control unit 50, a notification about contents of the command may be given through thenotification device 93. During manual traveling, in principle, when an obstruction is detected by theobstruction sensors 7, a notification about the detection may be given, but traveling itself may be permitted. However, in a case where an obstruction is detected ahead of thevehicle body 1 in the traveling direction, emergency stop may be activated. - Next, a flow of information in the sensor determining process performed in the obstruction
sensor determining unit 40 to determine whether or not theobstruction sensors 7 are normally operating is described by using thelaser scanners 72 and thecameras 73 as an example with reference toFig. 4 . In this sensor determining process, theultrasonic sonars 71 are handled in a similar manner to thelaser scanners 72. - In this exemplary embodiment, the sensor determining process is activated when a main key is turned on. When the sensor determining process is activated, the operation environment securing process is performed by the operation environment securing
processing unit 41, and a message requesting an operation environment securing operation such as cleaning laser emission openings of thelaser scanners 72 or lenses of thecameras 73 is displayed on thetouch panel 43 as an example of the notification process. A user (e.g., a driver or a supervisor) performs the operation environment securing operation based on the displayed contents and then performs a user input operation indicating completion of the operation environment securing operation through thetouch panel 43. The operation environment securingprocessing unit 41 confirms that a proper operation environment has been secured by this user input operation. Next, the sensor trial process is performed by the sensortrial processing unit 42, and a message requesting that a user enters paths of laser beams emitted from the laser scanners 72 (a detection region of the obstruction sensors) (artificial entry of an obstruction) or the field of view of at least one of thecameras 73 is displayed on thetouch panel 43. In the sensor trial process, theobstruction sensors 7 are operating, and therefore when the user who entered the paths of laser beams (a detection region of the obstruction sensors) or an object carried by the user is detected as an obstruction, a result of the detection is returned to the sensortrial processing unit 42 as a normal operation confirmation signal. In case data of photographed images obtained from thecameras 73 includes attribute information indicating that a face of a user who has entered the field of view has been recognized, this attribute information becomes normal operation confirmation information of thecameras 73. In this way, the sensortrial processing unit 42 may confirm a normal operation through an actual trial of the obstruction sensors. Since theobstruction sensors 7 are disposed so as to cover the whole region around the tractor, the user may complete the sensor trial process by going around the tractor. The obstructionsensor determining unit 40 may perform a signal line determining process for checking short circuit and disconnection of a signal line of the obstruction sensors at any time. - When the obstruction
sensor determining unit 40 outputs a determination result indicating that theobstruction sensors 7 are normally operating through the operation environment securing process and the sensor trial process, the obstructionsensor processing unit 56 gives a permission command to thetravel control unit 51 of the worktravel control unit 50. In this way, autonomous traveling is permitted. When the obstructionsensor determining unit 40 outputs a determination result indicating that a normal operation of theobstruction sensors 7 is not confirmed, the obstructionsensor processing unit 56 gives a prohibition command to thetravel control unit 51 of the worktravel control unit 50. In this way, autonomous traveling is prohibited. - Next, an example of a flow of control from the sensor determining process to the start of traveling of a tractor is described with reference to the flowchart of
Fig. 5 . - When a control routine for start of traveling is activated, whether or not the main key of the tractor is on is checked (#01). In a case where the main key is off (No in #01), this routine is finished. In a case where the main key is on (Yes in #01), it is checked which of the autonomous traveling mode and the manual traveling mode is set (#02). In case the autonomous traveling mode is set (Yes in #02), a period T elapsed from an immediately preceding time at which the main key was turned off is calculated (#03). The elapsed period T is compared with a threshold value H (#04). This threshold value H is a period set in order to avoid repetition of an unnecessary sensor determining process that occurs when the main key is turned off and on repeatedly within a short period.
- In case the elapsed period T is longer than the threshold value H, the operation environment securing process starts, and a user is instructed to perform an operation for securing an operation environment through the touch panel 43 (#05). The user performs the operation for securing an operation environment based on this instruction. When the operation for securing an operation environment is completed and the user inputs completion of the operation through the
touch panel 43, the operation environment securingprocessing unit 41 determines that it has been confirmed that the operation environment has been secured (Yes in #06), and then the sensor trial process for determining whether or not the obstruction sensors are normally operating by trial starts (#07). - In an embodiment, the sensor trial process is performed as follows:
- (1) A user is requested to go around the tractor by displaying a message "press a trial operation completion button after going around the tractor" on the
touch panel 43. - (2) All obstruction sensors are caused to operate in a trial mode.
- (3) Whether all of the obstruction sensors have detected an obstruction (user) is checked when the user presses the trial operation completion button.
- (4) In case all of the obstruction sensors have detected an obstruction (user), it is determined that a sensor operation state is normal. In case one or more of the obstruction sensors has not detected an obstruction (user), it is determined that a sensor operation state is not normal, and the obstruction sensor(s) that has(have) not detected an obstruction (user) is (are) displayed on the
touch panel 43. - (5) A result of the determination is output (#08).
- The result of the determination is checked (#09), and in case the result of the determination indicates a normal operation (Yes in #09), the obstruction
sensor processing unit 56 gives a permission command to the work travel control unit 50 (#10). In this way, autonomous traveling starts (#11). In case the result of the determination does not indicate a normal operation (No in #09), the obstructionsensor processing unit 56 gives a prohibition command to the work travel control unit 50 (#12). As a result, autonomous traveling is not started, and the routine returns to step #01. - When a prohibition command is issued, autonomous traveling is prohibited, and therefore the user may switch the traveling mode to manual traveling. Alternatively, it is also possible to employ an arrangement in which the traveling mode is automatically switched to manual traveling. When the traveling mode is switched to the manual traveling mode (NO in step #02), manual traveling is started (#13), and therefore the tractor can be moved to a place where maintenance and inspection of the
obstruction sensors 7 are possible. That is, output of a prohibition command is suspended during manual traveling. - In case the elapsed period T is equal to or less than the threshold value H in step #04 (No in #04), whether or not a last result of sensor determination indicates a normal operation is checked (#14), and in a case where the last result of the sensor determination indicates a normal operation (Yes in #14),
step # 10 is performed, and a permission command is given to the worktravel control unit 50 without performing a new sensor determining process. In a case the last result of the sensor determination does not indicate a normal operation (No in #14),step # 01 is performed again. In case a period elapsed from an immediately preceding timing at which the main key is turned off to a subsequent time at which the main key is turned on is equal to or less than a threshold value (a predetermined period set so that mud or the like is unlikely to be attached), a last result of determination indicating a normal operation in the sensor determination process is maintained. In a case where a last result of the sensor determination does not indicate a normal operation,step # 05 may be performed in which the sensor determination process is started instead of returning to step #01. - (1) In the example illustrated in the flowchart of
Fig. 5 , the sensor trial process is performed after the operation environment securing process. However, in a case where the operation environment securing process is unnecessary, only the sensor trial process may be performed while the operation environment securing process is omitted. Alternatively, only the operation environment securing process may be performed while omitting the sensor trial process. - (2) In the flowchart illustrated in
Fig. 5 , the sensor determining process is performed at a timing at which the main key is turned on. Alternatively, it is possible to employ an arrangement in which the sensor determining process is performed at any timing or upon receipt of an outside instruction. - (3) Although a tractor equipped with a tilling machine is taken as an example of a work vehicle in the above description, the present invention is also applicable to a tractor equipped with a
work device 30 other than a tilling machine, farm machines, such as a combine harvester or a rice planting machine, and construction machines. - (4) In the above exemplary embodiment, the
first control unit 4, thesecond control unit 5, and the input outputsignal processing unit 6 are connected through an on-board LAN and are provided in the tractor. However, it is also possible to employ an arrangement in which thefirst control unit 4 is a tablet computer, a notebook computer, a smartphone, or the like that is carried by a supervisor and wirelessly exchanges data with the control system of the tractor. Furthermore, thefirst control unit 4 may be configured as a computer that is placed in a control center in a remote location or at a user's home or the like, and is connected to the control system of the tractor through an Internet line. - (5) The way in which the functional units in the functional block diagram illustrated in
Fig. 3 are divided is an example for easy understanding of the description, and various functional units can be freely unified and a single functional unit can be freely divided into a plurality of units. - The technologies described herein are applicable to an autonomous traveling work vehicle for working and traveling along a travel route.
-
- 40:
- obstruction sensor determining unit
- 41:
- operation environment securing processing unit
- 42:
- sensor trial processing unit
- 43:
- touch panel
- 50:
- work travel control unit
- 51:
- travel control unit
- 56:
- obstruction sensor processing unit
- 7:
- obstruction sensors
- 70:
- image processing unit
- 71:
- ultrasonic sonar
- 72:
- laser scanner
- 73:
- camera
- 81:
- traveling state detection sensors
- 82:
- work state detection sensors
- 91:
- vehicle travel instruments
- 92:
- work device instruments
- 93:
- notification device
Claims (7)
- An autonomous traveling work vehicle to autonomously travel along a travel route, comprising:- a travel control unit (51) adapted to control autonomous traveling;- one or more obstruction sensors (7) configured to detect an obstruction to traveling;- an obstruction sensor determining unit (40) adapted to perform a sensor determining process for determining whether or not the obstruction sensors (7) are normally operating; and- an obstruction sensor processing unit (56) adapted to output a prohibition command to prohibit autonomous traveling controlled by the travel control unit (51) in case the obstruction sensor determining unit (40) determines that the obstruction sensors (7) are not normally operating,
wherein the sensor determining process includes- an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors (7) so as to remove, by a user, an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors (7), and- a sensor trial process, performed only under the proper operation environment, for determining that the obstruction sensors (7) are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors (7) is detected. - The autonomous traveling work vehicle according to claim 1, wherein the operation environment securing process includes- a notification process for notifying a user of an instruction to secure the proper operation environment and- a confirmation process for receiving, from the user, information indicating that the proper operation environment has been secured.
- The autonomous traveling work vehicle according to claim 1 or 2, wherein- the obstruction sensors (7) are cameras (73) having a face recognition function; and- in the sensor trial process, it is determined that the cameras (73) are normally operating in case the cameras (73) recognize a face of a person who enters the field of view of at least one of the cameras (73).
- The autonomous traveling work vehicle according to any one of claims 1 to 3, wherein the sensor determining process includes a signal line determining process for detecting disconnection and short circuit of a signal line of the obstruction sensors.
- The autonomous traveling work vehicle according to any one of claims 1 to 4, wherein- the sensor determining process is performed by the obstruction sensor determining unit (40) when a main key is turned on; and- within a predetermined period from an immediately preceding time at which the main key is turned off to a subsequent time at which the main key is turned on, a last result of determination indicating a normal operation in the sensor determining process is maintained while the sensor determining process is omitted.
- The autonomous traveling work vehicle according to any one of claims 1 to 5, wherein output of the prohibition command from the obstruction sensor processing unit (56) is suspended during manual traveling.
- A method for autonomous travel of an autonomous traveling work vehicle along a travel route, comprising:- controlling autonomous traveling by a travel control unit (51) of the autonomous traveling work vehicle;- performing, by an obstruction sensor determining unit (40) of the autonomous traveling work vehicle, a sensor determining process for determining whether or not one or more obstruction sensors (7) of the autonomous traveling work vehicle, configured for detecting an obstruction to traveling, are normally operating; and- outputting, by an obstruction sensor processing unit (56) of the autonomous traveling work vehicle, a prohibition command to prohibit autonomous traveling controlled by the travel control unit (51) when the obstruction sensor determining unit (40) determines that the obstruction sensors (7) are not normally operating.
wherein the sensor determining process includes- an operation environment securing process for securing a proper operation environment for a normal operation of the obstruction sensors (7) so as to remove, by a user, an external factor that is inconvenient for a normal operation of the obstruction sensors in an environment surrounding the obstruction sensors (7), and- a sensor trial process, performed only under the proper operation environment, for determining that the obstruction sensors (7) are normally operating in case an obstruction artificially brought into a detection region of the obstruction sensors (7) is detected.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017008352A JP6800028B2 (en) | 2017-01-20 | 2017-01-20 | Self-driving work vehicle |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3352037A1 EP3352037A1 (en) | 2018-07-25 |
EP3352037B1 true EP3352037B1 (en) | 2021-08-25 |
Family
ID=60673505
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17207001.3A Active EP3352037B1 (en) | 2017-01-20 | 2017-12-13 | Autonomously traveling work vehicle and method for autonomous travel control |
Country Status (4)
Country | Link |
---|---|
US (1) | US10635100B2 (en) |
EP (1) | EP3352037B1 (en) |
JP (1) | JP6800028B2 (en) |
CN (1) | CN108334069A (en) |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6593241B2 (en) * | 2016-04-05 | 2019-10-23 | 株式会社デンソー | Electronic control unit |
JP6800028B2 (en) * | 2017-01-20 | 2020-12-16 | 株式会社クボタ | Self-driving work vehicle |
US11479213B1 (en) * | 2017-12-11 | 2022-10-25 | Zoox, Inc. | Sensor obstruction detection and mitigation |
JP6959179B2 (en) * | 2018-04-26 | 2021-11-02 | 株式会社クボタ | Work platform |
US11243531B2 (en) * | 2018-08-09 | 2022-02-08 | Caterpillar Paving Products Inc. | Navigation system for a machine |
JP6863358B2 (en) * | 2018-10-26 | 2021-04-21 | 井関農機株式会社 | Work vehicle |
JP7423542B2 (en) | 2018-10-30 | 2024-01-29 | 東京計器株式会社 | Handle angle control device |
CN111513629B (en) * | 2019-02-01 | 2023-12-05 | 北京奇虎科技有限公司 | Barrier sensor shielding detection method and device and sweeper |
CN110199590A (en) * | 2019-04-09 | 2019-09-06 | 丰疆智能科技股份有限公司 | Intelligent mini-tiller and its hard object bypassing method |
CN109964556B (en) * | 2019-04-11 | 2023-06-27 | 湖南龙舟农机股份有限公司 | Rotary cultivator control system and method |
CN113766824B (en) * | 2019-04-25 | 2023-07-14 | 株式会社久保田 | Harvester, obstacle determination program, recording medium, and obstacle determination method |
US11301694B2 (en) | 2019-06-06 | 2022-04-12 | Cnh Industrial America Llc | Detecting plugging of ground-engaging tools of an agricultural implement from imagery of a field using a machine-learned classification model |
JP7162571B2 (en) * | 2019-06-24 | 2022-10-28 | 株式会社クボタ | work vehicle |
US20210243941A1 (en) * | 2020-02-07 | 2021-08-12 | Ag Leader Technology | Planter Obstruction Monitoring And Associated Devices And Methods |
DE102020207777A1 (en) | 2020-06-23 | 2021-12-23 | Deere & Company | Agricultural tractor with a system for detecting following road users |
WO2023095342A1 (en) * | 2021-11-29 | 2023-06-01 | 日産自動車株式会社 | Power supply system and method for controlling power supply system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015178446A1 (en) * | 2014-05-22 | 2015-11-26 | ヤンマー株式会社 | Running system of work vehicle |
US20160084737A1 (en) * | 2014-09-19 | 2016-03-24 | Swisslog Logistics Inc. | Method and System for Auto Safety Verification of AGV Sensors |
Family Cites Families (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06195121A (en) * | 1992-12-22 | 1994-07-15 | Toyota Motor Corp | Sensor check system for unmanned carrying car |
JPH0716148A (en) * | 1993-06-30 | 1995-01-20 | Sharp Corp | Warmer type rice cooker |
JPH09269829A (en) * | 1996-03-29 | 1997-10-14 | Mazda Motor Corp | Operation confirmation device for safety sensor of vehicle |
US6642839B1 (en) * | 2000-02-16 | 2003-11-04 | Altra Technologies Incorporated | System and method of providing scalable sensor systems based on stand alone sensor modules |
JP4501719B2 (en) * | 2005-02-22 | 2010-07-14 | トヨタ自動車株式会社 | Vehicle remote control device |
JP5284186B2 (en) * | 2009-06-09 | 2013-09-11 | 京セラドキュメントソリューションズ株式会社 | Image forming apparatus |
JP5189565B2 (en) * | 2009-07-21 | 2013-04-24 | 株式会社日立製作所 | Elevator door control device |
US8509982B2 (en) * | 2010-10-05 | 2013-08-13 | Google Inc. | Zone driving |
JP2014006243A (en) * | 2012-05-28 | 2014-01-16 | Ricoh Co Ltd | Abnormality diagnostic device, abnormality diagnostic method, imaging apparatus, moving body control system and moving body |
WO2015068249A1 (en) * | 2013-11-08 | 2015-05-14 | 株式会社日立製作所 | Autonomous driving vehicle and autonomous driving system |
US10319039B1 (en) * | 2014-05-20 | 2019-06-11 | State Farm Mutual Automobile Insurance Company | Accident fault determination for autonomous vehicles |
US10223479B1 (en) * | 2014-05-20 | 2019-03-05 | State Farm Mutual Automobile Insurance Company | Autonomous vehicle operation feature evaluation |
JP2015222503A (en) * | 2014-05-22 | 2015-12-10 | ヤンマー株式会社 | Autonomous travel working vehicle |
JP6386794B2 (en) * | 2014-06-04 | 2018-09-05 | 日立建機株式会社 | Transport vehicle |
US10124805B2 (en) * | 2014-07-31 | 2018-11-13 | Komatsu Ltd. | Work vehicle and control method for work vehicle |
US10571911B2 (en) * | 2014-12-07 | 2020-02-25 | Toyota Motor Engineering & Manufacturing North America, Inc. | Mixed autonomous and manual control of a vehicle |
US20170341236A1 (en) * | 2016-05-27 | 2017-11-30 | General Electric Company | Integrated robotic system and method for autonomous vehicle maintenance |
CN105353872A (en) * | 2015-10-28 | 2016-02-24 | 广东欧珀移动通信有限公司 | Control signal sending method and control signal sending apparatus |
US10397019B2 (en) * | 2015-11-16 | 2019-08-27 | Polysync Technologies, Inc. | Autonomous vehicle platform and safety architecture |
US10149468B2 (en) * | 2016-05-10 | 2018-12-11 | Crinklaw Farm Services, Inc. | Robotic agricultural system and method |
CN106227214B (en) * | 2016-08-30 | 2019-08-13 | 北京小米移动软件有限公司 | Autonomous robot, apparatus and method for detecting failure |
JP6800028B2 (en) * | 2017-01-20 | 2020-12-16 | 株式会社クボタ | Self-driving work vehicle |
US10474916B2 (en) * | 2017-11-20 | 2019-11-12 | Ashok Krishnan | Training of vehicles to improve autonomous capabilities |
US10514700B2 (en) * | 2017-12-15 | 2019-12-24 | Walmart Apollo, Llc | System and method for managing a vehicle storage area |
US11417111B2 (en) * | 2017-12-22 | 2022-08-16 | Terra Scientia, Llc | Method, system and material for detecting objects of high interest with laser scanning systems |
JP6900915B2 (en) * | 2018-01-24 | 2021-07-07 | トヨタ自動車株式会社 | Autonomous driving system |
-
2017
- 2017-01-20 JP JP2017008352A patent/JP6800028B2/en active Active
- 2017-11-30 CN CN201711238194.9A patent/CN108334069A/en active Pending
- 2017-12-13 EP EP17207001.3A patent/EP3352037B1/en active Active
- 2017-12-14 US US15/842,810 patent/US10635100B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015178446A1 (en) * | 2014-05-22 | 2015-11-26 | ヤンマー株式会社 | Running system of work vehicle |
US20160084737A1 (en) * | 2014-09-19 | 2016-03-24 | Swisslog Logistics Inc. | Method and System for Auto Safety Verification of AGV Sensors |
Also Published As
Publication number | Publication date |
---|---|
EP3352037A1 (en) | 2018-07-25 |
JP2018116611A (en) | 2018-07-26 |
US20180210443A1 (en) | 2018-07-26 |
JP6800028B2 (en) | 2020-12-16 |
US10635100B2 (en) | 2020-04-28 |
CN108334069A (en) | 2018-07-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3352037B1 (en) | Autonomously traveling work vehicle and method for autonomous travel control | |
US10732628B2 (en) | Sensor management device and sensor management method | |
JP6739364B2 (en) | Self-driving work vehicle | |
JP6579903B2 (en) | Driving support device, vehicle, and garage cooperative control system | |
JP7154362B2 (en) | work vehicle | |
JP6870270B2 (en) | Remote control method and remote control device for unmanned driving system | |
WO2018092265A1 (en) | Driving assistance device and driving assistance method | |
EP3146823B1 (en) | Running system of work vehicle | |
US20180024563A1 (en) | Work vehicle, slope travel control system for work vehicle, and slope travel control method for work vehicle | |
US11377103B2 (en) | Vehicle control device and recording medium | |
WO2014148025A1 (en) | Travel control device | |
KR20210102540A (en) | Method for alerting danger situations of moving object and apparatus for the same | |
JP2016181032A (en) | Automatic travel control device and automatic travel control system | |
US20230264699A1 (en) | Drive planning device, storage medium storing computer program for drive planning and drive planning method | |
JP2021049892A (en) | Automatic operation system | |
JP6575451B2 (en) | Driving support device and driving support program | |
KR20210097233A (en) | Remote control support system for a self driving car and method of the same | |
US20210300340A1 (en) | Predetermined area management device | |
US20210304606A1 (en) | Accommodation area management device | |
CN114572243A (en) | Target object detection device and vehicle equipped with the same | |
JP2022067561A (en) | Automatic operation system | |
US20220319201A1 (en) | Act-of-looking-aside determination device | |
JP2019145952A (en) | Vehicle and camera system | |
JP7429111B2 (en) | Traffic control equipment and vehicle control equipment | |
US20230391370A1 (en) | Vehicle control device, storage medium for storing computer program for vehicle control, and method for controlling vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190125 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190522 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20210526 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017044634 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Ref country code: AT Ref legal event code: REF Ref document number: 1424449 Country of ref document: AT Kind code of ref document: T Effective date: 20210915 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20210825 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1424449 Country of ref document: AT Kind code of ref document: T Effective date: 20210825 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211227 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211125 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211125 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211126 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602017044634 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
26N | No opposition filed |
Effective date: 20220527 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20211213 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20211231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211213 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211213 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211213 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211231 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20171213 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231108 Year of fee payment: 7 Ref country code: DE Payment date: 20231031 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210825 |