CA3148445A1 - Weed picking and disposal module and method - Google Patents

Abstract

There is disclosed a method for efficiently picking and disposing of weeds in an agricultural field using an implement provided on an agricultural vehicle, the method comprising capturing images of a field travelled by the agricultural vehicle, the field comprising weeds and crops; generating a map from the captured images, comprising coordinates of the weeds and crops; inputting disposal lanes into the generated map, the disposal lanes being regions where the weeds are to be disposed, and determining, based on the coordinates of the weeds, the coordinates of the crops and the disposal lanes, a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of weeds, while avoiding crops when moving to the disposal lanes; and removing weeds with the implement and disposing of said weeds in the disposal lanes by following the picking and disposing trajectory determined.

Description

WEED PICKING AND DISPOSAL MODULE AND METHOD
TECHNICAL FIELD
[001] The technical field generally relates to weed picking and disposal implements for use on agricultural vehicles, and more specifically to modules and methods of optimizing weed or immature crop disposal with automated implements.
BACKGROUND

[002] Weeds compete with crops for resources, including water, nutrients, and sunlight.
Picking of weeds from agricultural fields is a continuous process which improves crop yield by removing competition for resources. Once weeds are removed from an agricultural field, the weeds must be properly disposed of. If, for example, a weed is simply removed and thrown arbitrarily in the field, there is a chance that the weed will take root once more.
There is therefore a desire for methods that allow for proper weed disposal without disproportionately adding to the time required to carry out weed removal and disposal.
SUMMARY

[003] In one aspect, there is provided a method for picking and disposing of weeds using an implement provided on an agricultural vehicle. The method may comprise capturing images of a field travelled by the agricultural vehicle, the field comprising weeds and crops.
The method may also comprise generating a map from the captured images, comprising coordinates of the weeds and crops; and inputting disposal lanes into the generated map, the disposal lanes being regions where the weeds are to be disposed. The method may comprise determining, based on the coordinates of the weeds, the coordinates of the crops and the disposal lanes, a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of weeds, while avoiding crops when moving to the disposal lanes. The method may also comprise removing weeds with the implement and disposing of the weeds in the disposal lanes by following the picking and disposing trajectory determined. In possible embodiment, immature crops or other plants can picked and disposed of according to this method.

[004] In one embodiment, removing weeds comprises picking one or more weeds at a time with the implement.
Date Recue/Date Received 2022-02-14

[005] In one embodiment, the picking and disposing trajectory is iteratively built and modified as the agricultural vehicle travels the field.

[006] In another embodiment, the implement avoids crop avoidance zones as it follows the picking and disposing trajectory to the disposal lanes.

[007] In another embodiment, the implement follows the picking and disposing trajectory by a combination of rotational and translational movement.

[008] In another embodiment, avoiding contact with the crops comprises maintaining a minimum lateral distance between the implement and the crops.

[009] In another embodiment, avoiding contact with the crops comprises maintaining a minimum vertical distance between the implement and the crops.

[0010] In another embodiment, determining the picking and disposing trajectory comprises applying a travelling salesman algorithm, which reduces travel distance to the disposal lanes, modified with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

[0011] In another embodiment, determining the picking and disposing trajectory comprises a step of concatenating a plurality of path segments, each path segment corresponding to a linear distance between one of the weed coordinates and disposal coordinates located within the disposal lanes.

[0012] In another embodiment, determining the picking and disposing trajectory comprises a step of smoothing an initial path comprising the concatenated path segments.

[0013] In another embodiment, determining the picking and disposing trajectory comprises determining a speed of the implement based on the shape of the trajectory and based on constraints inherent to a controller controlling movement of the implement.

[0014] In another embodiment, accelerations and decelerations are determined based on torque limits of the controller.

[0015] In another embodiment, the implement is a gripper and removing weeds comprises controlling jaws of the gripper to pinch and extract weeds from the ground.
Date Recue/Date Received 2022-02-14

[0016] In another embodiment, the images are captured using at least one camera on the agricultural vehicle.

[0017] In another aspect, there is provided a weed picking and disposal module provided on or attached to an agricultural vehicle. The module, or assembly, comprises an agricultural implement translatable along three degrees of freedom; a controller configured to control the movement of the agricultural implement; and a processing module. The processing module comprises input ports for receiving coordinates of weeds and crops determined from images and for receiving positions of disposal lanes; and a non-transitory memory and a processor, the non-transitory memory storing the coordinates and instructions for causing the processor to generate a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of weeds, while avoiding crops when moving to the disposal lanes.

[0018] In another embodiment, the implement is also rotatable along at least one degree of freedom.

[0019] In another embodiment, the agricultural implement is a gripper comprising jaws configured and adapted to pinch and retrieve the weeds.

[0020] In another embodiment, one of a delta robot or a gantry system operatively connected to the agricultural implement, for translating the implement along the three degrees of freedom.

[0021] In another embodiment, the controller comprises at least one first actuator for controlling movement of the gripper and a second actuator for actuation of the gripper.

[0022] In another embodiment, the non-transitory memory has stored thereon instructions to apply a travelling salesman algorithm to an initial path, which reduces travel distance to the disposal lanes, and instructions to modify the initial path with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

[0023] In another embodiment, the non-transitory memory has stored thereon instructions to determine a speed of the implement based on the shape of the modified path and based on constraints inherent to a controller controlling movement of the implement.
Date Recue/Date Received 2022-02-14

[0024] In another embodiment, the non-transitory memory has stored thereon instructions to determine accelerations and decelerations based on torque limits of the controller.

[0025] In another aspect, there is provided a method for picking and disposing of unwanted plants using an implement provided on an agricultural vehicle. The method comprises capturing images of a field travelled by the agricultural vehicle, the field comprising unwanted plants and crop; generating a map from the captured images, comprising coordinates of the unwanted plants and crops; inputting disposal lanes into the generated map, the disposal lanes being regions where the unwanted plants are to be disposed, and determining, based on the coordinates of the unwanted plants, the coordinates of the crops and the disposal lanes, a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of unwanted plants, while avoiding crops when moving to the disposal lanes; and removing unwanted plants with the implement and disposing of said unwanted plants in the disposal lanes by following the picking and disposing trajectory determined.
BRIEF DESCRIPTION OF DRAWINGS

[0026] Fig. 1 is a perspective view of an example implementation of an agricultural vehicle having a weed or immature crop disposal module (not shown) mounted underneath, according to a possible embodiment;

[0027] Fig. 2 is a side view of a disassembled portion of the body of the vehicle of Fig.
1, along with the weed picking and disposal module visible underneath;

[0028] Fig. 3 is a top-down view of the body of the vehicle and the weed picking and disposal module Fig. 2 placed side-by-side;

[0029] Fig. 4 is a view of an agricultural implement assembly, specifically a gripper assembly, that may be used in accordance with the methods described herein;

[0030] Fig. 5 is a top-down view of an example field shown in map form having crop lanes and disposal lanes, with weeds and crops shown in the crop lanes;
Date Recue/Date Received 2022-02-14

[0031] Fig. 6 is a top-down view of the example field of Fig. 5 shown in map form having crop lanes, intermediate lanes and disposal lanes, with weeds and crops shown in the crop and intermediate lanes;

[0032] Fig. 7 is a top-down view of the example field of Fig. 5 shown in map form having crop lanes and disposal lanes, showing a weed disposal location in the weed disposal lane;

[0033] Fig. 8 is a top-down view of example lanes of a field, with an implement picking and disposing trajectory going from a first weed to a first weed drop disposal location, then to a second weed while respecting crop avoidance;

[0034] Fig. 9 is a top-down view of the example lanes of the field of Fig. 8, with an improved (shorter) implement picking and disposing trajectory going from a first weed to a first weed drop disposal location, then to a second weed while respecting crop avoidance;

[0035] Fig. 10 is a top-down view of the example lanes of the field of Fig. 8, with an implement picking and disposing trajectory going from a first weed to a first weed drop disposal location, then to a second weed all in straight lines without respecting crop avoidance;

[0036] Fig. 11 is a top-down view of the example lanes of the field of Fig.
10, with an implement picking and disposing trajectory going from a first weed to a first weed drop disposal location, then to a second weed all in straight lines while respecting a crop avoidance zone;

[0037] Fig. 12 is a top-down view of the example lanes of the field of Fig.
11, with an implement picking and disposing trajectory going from a first weed to a first weed drop disposal location, then to a second weed all in one smoothed curve while respecting the crop avoidance zone;

[0038] Fig. 13 is a top-down view of the example lanes of the field of Fig.
12, with an improved (shorter) implement picking and disposing trajectory going from a first weed to an improved first weed drop disposal location, then to a second weed all in one smoothed curve while respecting the crop avoidance zone;
Date Recue/Date Received 2022-02-14

[0039] Fig. 14 is a top down view of the example lanes of the field of Fig. 8, displaying an example of a full implement picking and disposing trajectory going from a first weed to a last weed while stopping at weed disposal locations in between each weed;

[0040] Fig. 15 is a top down view of the example lanes of the field of Fig. 8, displaying another example of a full implement picking and disposing trajectory going from a first weed to a last weed while stopping at weed disposal locations in between each weed; and

[0041] Fig. 16 is an example flow chart of possible steps carried out to pick and dispose of weeds along a picking and disposing trajectory that minimizes the time needed to pick and dispose of weeds, in accordance with one implementation.
DETAILED DESCRIPTION

[0042] Broadly described, the method of weed disposal described herein can be used on or with an agricultural vehicle, such as an autonomous weeding machine, capable of navigating farm fields while identifying weeds and pulling them using a controllable implement, such as a robot gripper. The method focuses on disposal of weeds or other undesired plants growing close to the crop in an appropriate location after removal. The weeds are picked by physically gripping them and pulling them from the earth, and then discarding them at a separate location to avoid rerooting. Although reference is made to weeds throughout this application, it is understood that crops or other objects of interest may similarly be picked or disposed of in accordance with the present disclosure.

[0043] A key performance indicator of any weeding method is the rate at which weeds may be removed. Accordingly, a weed disposal method should have as little impact on the weed removal rate as possible. For example, once a weed is removed from the ground, the simplest disposal method would be to release the weed in the same spot. As mentioned previously, however, such a disposal method may result in the weed rerooting at the same spot. Accordingly, the present method aims to minimize the impact on the weed removal rate while disposing of weeds in an appropriate manner.
Date Recue/Date Received 2022-02-14

[0044] With reference to Fig. 1 and in accordance with one implementation, there is shown an agricultural vehicle 10 for picking weeds or for thinning crops from an agricultural field.
The vehicle 10 shown therein is configured to travel along lanes in an agricultural field, over longitudinal sections of soil, to pick, remove and dispose of weeds. In possible implementations, the vehicle is an autonomous vehicle, comprising cameras, sensors and processors to control electric motors and wheels connected to the motor(s). In other possible implementations, the vehicle can be a tractor or other agricultural vehicle, driven by a driver. The crops can be of different types, including vegetable crops, such as lettuces, carrots, and onions.

[0045] With reference to Figs. 2 and 3, some components of the autonomous vehicle 10 are shown disassembled. The components shown include a frame, or chassis, 12 and a picking and disposal module 100 mounted thereon, including an agricultural implement 200 for picking weeds. The picking module can be affixed underneath the vehicle or attached to one of its sides and tracked. The agricultural implement may also be configured and adapted for thinning the field, where in this case, selected immature crops (or otherwise unwanted plants) are removed to provide enough room to allow stronger crops to grow. As such, the picking and disposal module can also be referred to a weeding and/or a thinning and disposal module. In possible implementations, such as the one illustrated, the agricultural implement 200 can be a gripper 210. In the illustrated implementation, there is provided three grippers 210a, 210b, 210c, each capable of weed picking and removal in a predefined zone underneath the vehicle.
Alternatively, there may be provided a single gripper for serving the entire zone, or multiple grippers serving overlapping zones.

[0046] Still referring to Figs. 2 and 3, and also to Fig. 4, in one implementation, it is preferable that the grippers are configured to not only grab weeds from the above-ground portion of the weed (e.g., the leaves), but to pinch the roots of the weeds and fully retrieve them from the ground. Picking at the root of the weed or plant may reduce the likelihood of the weed regrowing whether in place or displaced elsewhere. In the illustrated implementation, the grippers 210 are installed at an end of a delta robot 150, with the delta robot 150 providing translational and rotational movement, as will be explained in more detail. Accordingly, the grippers 210 may grasp weeds at first location and dispose of them at a second location.
Date Recue/Date Received 2022-02-14

[0047] In a possible implementation, and with reference to the example illustrated in Fig.
4, the gripper may comprise two jaws or plates 212, which can be controlled such that they can be spaced apart or brought together to pinch the weeds or plants, such as the stems, preferably near or within the soil, close to the roots. Other implement or gripper configurations are possible.

[0048] The picking and disposal module 100 may comprise frame 102 onto which the grippers 210 and other parts can be mounted, as well as a processing module comprising one or more processor(s) 414, including a CPU, and non-transitory memory 412, including volatile and non-volatile memory, for storing and carrying out computer instructions (see Fig. 16). The processing module 400 can be mounted within of the picking and disposal module 100 or on the vehicle 10, such that several picking modules can share the processing and/or storage capacity. In possible embodiments, a portion of the processing capacity can be remotely accessed. Different configurations are possible.
The frame 102 can include structural members, such as tubes, and side plates to protect the components of the modules from dust, mud and/or water. The frame 102 may also include connecting members (not shown) adapted to connect to the frame 12 of the agricultural vehicle 10. In possible implementations, the picking and disposal module 100 can be a distinct, independent module, mountable onto different types of agricultural vehicles, which may or not be autonomous vehicles. In other implementations, the picking and disposal module 100 can be provided as part of an integral to the vehicle, such as in the illustrated implementation. The picking and disposal module may also be attached and tracked behind a tractor or other similar vehicle.

[0049] The picking and disposal module 100 additionally may include cameras mounted thereon for identifying and determining the positioning of weeds, extra or immature plants and crops in the agricultural field. The cameras 220 may be coupled to lights mounted in proximity thereof for illuminating the field of vision of the cameras 220, providing improved images. In the illustrated implementation as shown in Figs.
2 and 3, there is shown a picking and disposal module 100 with cameras 220 mounted onto the frame 102 of the picking and disposal module 100. Alternatively, the cameras 220 may be mounted proximate to the gripper 210, such as on the end of the delta robot 150. In one implementation, the cameras 220 may instead, or additionally, be mounted onto the vehicle 10. The lights coupled to the cameras 220 may also be mounted either onto the picking and disposal module 100, or the vehicle 10.
Date Recue/Date Received 2022-02-14

[0050] With the implementation shown, when the autonomous agricultural vehicle passes in an agricultural field, the picking and disposal module 100 is activated. Once the picking and disposal module 100 is activated, the cameras 220 begin capturing images of the field. In one implementation, the cameras 220 are coupled to a transceiver module for transmitting images to a remote site where an artificial intelligence (Al) system identifies weeds and crops. In one implementation, the images transmitted by the cameras 220 are interpreted by an operator at the remote site who marks crops and weeds, and also optionally immature crops that are to be removed for thinning. The cameras 220 may transmit the images through any wireless communication system to the remote site. In yet other implementations, the Al module and image analysis can be conducted locally, in a processing device provided on the vehicle 10 and in communication with one or more controller(s) controlling the weed picking implement 200.

[0051] According to the images generated and marked with respect to weeds and crops, and possibly other plants such as immature crops, a map 500 may be generated identifying the coordinates of weeds 520 and crops 510 in the area captured by the images (see Fig. 7). Weeds 520 may represent a singular weed or a group of weeds which may be handled by the agricultural implement 200. The weeds and crops appearing in the images captured by the image sensors or cameras can be first identified by pixel analysis.
Their coordinates can then be determined. For example, a Euclidean map can be generated, using Simultaneous Localization and Mapping (SLAM) algorithms, which allows building a map and localizing the vehicle and/or picking modules at the same time.
In a possible implementation, pixels of the map 500 can be classified as weed 520, crop 510 or other, such as other types of plants or immature (or weak) crops, and this classification can be used to determine center coordinates of the plants, such as weeds and crops. In possible implementations, different types of weeds and plants could be detected, and the weed picking and disposal method could be adapted accordingly. In a possible implementation, machine learning models, such as deep neural network models, can be used to differentiate between weeds or immature crops and desired crops. For example, Al classifiers can be used to distinguish between crops and weeds, based on the outer contour of the plant.

[0052] In one implementation, the map 500 may instead, or in addition to the cameras of the picking and disposal module 100, be provided by another image capture system such as a satellite image. As the vehicle travels through the agricultural field, the map 500 may Date Recue/Date Received 2022-02-14 be continuously updated with images captured by the cameras. Additionally, a speed of the vehicle may be aligned with the camera capture rate and the processor. The speed of the vehicle may be calibrated so that the vehicle moves through the field at a rate that allows for the cameras and the processor to capture and process images, and to then effect weed picking and disposal by the gripper 210.

[0053] The processing module 400 is configured to receive inputs, such as the map 500 or the weed and crop coordinates, process the inputs, and output data for controlling the gripper 210. For example, the map 500 or map data may include coordinates of weeds 520 and crops 510. The coordinates can be provided or accessed through input ports 410 to the memory 412 and the processor 414, the processor 414 generating a position and/or other parameters for feeding motor drives (such as speed, accelerations, decelerations) along a trajectory for the gripper 210 to move and rotate to, based on the received coordinates, and sending the position information (or other drive parameters) via output ports 416 to one or more controller (such as motor drives) for controlling the gripper 210.
The controller controls the movement of the grippers 210 based on the coordinates of the weeds 520 and crops 510, by adjusting actuators associated with X, Y and Z
motion of the implement 200 or gripper 210. In one implementation, the controller comprises one or more actuators for controlling movement of the gripper 210 and another actuator for actuation of the jaws (opening and closing) of the gripper 210. In one implementation, the set of actuators comprises four actuators, each one corresponding to translation along the x, y and z axes, as well as rotation around the z axis. The controllers can comprise motor drives. The controller controls the movement of the grippers 210 based on the coordinates of the weeds 520 and crops 510, and more specifically based on a picking and disposing trajectory determined by the processing module 400. For thinning purposes, the position of the implement can be optimized to maximize the number of immature crops to remove near the desired, stronger crops.

[0054] In one implementation, the processor 414 determines crop lanes 550 and weed disposal lanes 560 based on the coordinates of the weeds and crops. The disposal lanes correspond to regions where the weeds are to be disposed. Many crops are planted in lanes to simplify picking and planting and to ensure adequate space is provided for each crop. Accordingly, in one implementation a crop lane 550 may be defined as having a length spanning a first crop to a last crop generally in a longitudinal (or y-axis) direction in an agricultural field, and a width in a lateral (or x-axis) direction which respects a minimum Date Recue/Date Received 2022-02-14 distance between each crop and an acceptable weed disposal location. For example, a minimum distance of 0.5 m may be desired between the crop and a potential weed disposal location. If the crops are planted in a straight line, the crop lane will span 1 m in width, and the disposal lane 560 will comprise any point after that while respecting the crop lanes 550 of adjacent crops. In one implementation, a collection vehicle may then go through the disposal lanes 560 to collect any weeds disposed thereon. The above values have been provided as examples. Any other value as deemed appropriate, for example based on the type of crop or local soil conditions, may be selected.

[0055] The disposal of the weeds along an agricultural field is carried out while respecting constraints. The constraints may include any one or a combination of the following. In one implementation, the constraints include a minimum distance to the crop. A
minimum distance to the crop prevents the agricultural implement from getting closer than a specified distance to the crop, whether the distance is lateral and/or vertical. This constraint may avoid dirt being dropped on the crop or ensure collision avoidance with the crop (see crop avoidance described further below). In one implementation, the constraints include setting of the crop lane width. The crop lane width determines a minimum distance that the weed must be dropped away from the crops. In one implementation, the constraints include a maximum distance between a weed and a crop, so that any weeds exceeding this distance are ignored. This constraint may improve weed removal/disposal rate by avoiding removal of all possible weeds, and instead only focusing on weeds that may be most likely to affect crop growth.

[0056] Accordingly, the agricultural field can be mapped by either a binary crop lane 550 and disposal lane 560 configuration (as shown in Fig. 5), or one where the crop lane 550 corresponds to a minimum distance from the crop 510. The disposal lane 560 may then be yet further away from the bounds of the crop lane 550 (as shown in Fig. 6).
In the configuration illustrated in Fig.6, the weeds 575 outside the crop lanes 550 are ignored by the agricultural implement 200, while the weeds 520 removed from the crop lanes 550 must be disposed of in the disposal lanes 560, and not in the intermediate lanes 570 between the crop lanes 550 and disposal lanes 560. This may ensure, for example, that all weeds 520 are disposed of in a narrow corridor where they may later be collected by a tractor or any other suitable method/device.
Date Recue/Date Received 2022-02-14

[0057] The above constraints relate to weed disposal but may also be coupled with constraints related to weed removal to provide clear parameters for the agricultural implement to follow during weed removal and disposal as the vehicle 10 moves in an agricultural field.

[0058] With reference to Fig. 7, there is provided an example map 500 of an agricultural field. If weed removal consisted of removing a single weed, such as weed 520a, the simplest way to remove said weed from the crop lane 550 would be to evaluate the distance between the weed 520a and the two disposal lanes 560 adjacent to the crop lane 550. The processor would then simply choose the closest disposal lane 560 and guide the agricultural implement along a straight horizontal line from the weed to the disposal lane 560 and drop the weed in the closest point within the disposal lane 560. In a possible implementation, the processing module may be configured and adapted to control the implement for picking more than one weed at a time, such as when the weeds form clusters of weeds. In this case, the disposal process remains the same, that is, the picked weeds must be dropped in the disposal lanes, while preferably avoiding going over and/or near the crops.

[0059] With reference to Figs. 8 and 9, in a crop lane 550 where there are multiple weeds the processor should consider not only a single weed, but all the weeds that need to be removed across the crop lane 550. The algorithm may then determine an optimal path that .. would result in the shortest distance while visiting all the weeds and weed disposal locations present in the crop lane 550. As illustrated in Fig. 8, simply drawing a horizontal line from the first weed 520a to the nearest disposal lane 560 (while also avoiding the crop) and then attempting to reach the second weed 520 may result in a path that is longer than if the processor considered the location of the second weed 520b before determining which disposal lane 560 to dispose of the first weed 520a in, as illustrated in Fig. 9. The path to the disposal lane may additionally not simply be a horizontal line, but diagonal.

[0060] An additional factor for the processor to consider is that the vehicle 10 is moving through the crop lane 550, and so there is a finite amount of time whereby the agricultural implement 200 can remove a particular weed before it is out of range of the agricultural implement 200. Accordingly, the solution to removing all the weeds is not purely abstract, as in the travelling salesman problem but also constrained by practical considerations such as limited time to both remove and dispose of a weed, as well as the computational Date Recue/Date Received 2022-02-14 time required to calculate the optimal path. In one implementation, the vehicle 10 may accelerate, slow down, or stop along a crop lane 550 depending on the number of weeds or the time required to remove and dispose of weeds. In one implementation, the speed of the vehicle 10 may be constant. Accordingly, as the map 500 is updated with movement of the vehicle 10, the optimal path may be iteratively built and modified as the agricultural vehicle 10 travels through the agricultural field as new solutions become possible.

[0061] In the travelling salesman problem, there is provided a number of points that must be visited by the hypothetical salesman, the solution algorithm to which allows for all points to be visited in the shortest distance or time. The problem is complicated to solve analytically due to the rapidly increasing complexity of the problem, which is on the scale of n! (with n representing the number of points). For example, the number of possible paths connecting a problem having 11 points is approximately 40 million, which is approximately 35 million more possible paths than a problem with 10 points. There are a number of potential approaches or algorithms to the problem which obviate the need to solve for each of these possible paths then comparing all the results. One potential travelling salesman algorithm that does not require an analytical solution consists of simply selecting the next nearest point at every step, or the "nearest neighbour" algorithm. Such an algorithm reduces the complexity of the problem from n! to n2, a reduction in complexity of several orders of magnitude using the same examples of problems with 10 and 11 points.
Given that zones or regions must be avoided between the picking and dropping locations, to avoid dropping soil onto crops for example, simply applying a travelling salesman algorithm may be insufficient. As such, a possible implementation of the method comprises applying a travelling salesman algorithm, which reduces travel distance to the disposal lanes, modified with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

[0062] In one implementation and in consideration of the above, the processor may use a "nearest neighbour" algorithm to determine the optimal path. In this algorithm, the weed nearest to the agricultural implement is selected, such as first weed 520a in the embodiment illustrated in Figs. 8 and 9, to start at for first picking and disposal. The agricultural implement then selects the second weed 520b for picking and disposal by determining the next nearest weed. The second weed 520b may then be picked after the first weed 520a has been disposed of in a disposal lane 560. The following weed would then be selected based on the nearest weed to the previous weed disposed of in the Date Recue/Date Received 2022-02-14 disposal lane, and the implement would visit every nearest weed step by step until every weed in the crop lane 550 has been removed. Alternatively, the processor may consider a plurality of potential paths depending on different starting weeds, then choosing the optimal path that may result in the least time or distance travelled for the agricultural implement to dispose of all the weeds. The "nearest neighbour" algorithm has been explained due to simplicity above, though any other algorithm for determining the optimal path may alternatively be used. In one implementation, the processor iterates up to 3 potential solutions using a different starting weed and selects the optimal solution based on shortest total distance. In another implementation, the processor iterates up to 6 potential solutions based on a different starting weed and selects the optimal solution based on shortest total distance. The number of iterations may be limited to set an upper limit on the use of computational resources and based on the time available between two picks. Other iteration values may also be selected. This possible "nearest neighbor"
implementation can also be modified with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

[0063] With reference to Fig. 10, the path of the agricultural implement 200 should be as short as possible. Each line between two points is a path segment representing the path of the agricultural implement. Each path segment corresponds to a linear distance (straight line) between the coordinates of a weed and the coordinates of a weed disposal location.
If a straight line is drawn from the weed disposal location 530a to the second weed 520b, it would be the shortest distance between said locations, as shown in Fig. 10.
If a straight line is drawn concatenating every weed to its respective weed disposal position to generate several concatenated path segments, an initial path corresponding to the picking and disposing trajectory is generated. The process of determining the picking and disposing trajectory may thus comprise a step of concatenating the different path segments, each path segment corresponding to a linear distance between one of the weed coordinates and disposal coordinates located within the disposal lanes.

[0064] With reference to Fig. 11, in one implementation each crop 510 comprises a crop avoidance (or collision avoidance) zone 512. The crop avoidance zone 515 extends outwardly from the crop 510 and may be defined by a field which pushes out any point of the path segment passing over the crop outwards. The field may, for example, be generated using a virtual potential field function in the algorithm. The pushed-out points may be connected by a series of discrete lines 514 representing the path of the agricultural Date Recue/Date Received 2022-02-14 implement 200. The discrete lines may then be smoothed to form one continuous curve 516 from the first weed 520a to the second weed 520b, including the weed disposal location 530a, as illustrated in Fig. 12. The path comprising the concatenated path segments, modified to avoid the crop avoidance zones, can thus be smoothed.
The agricultural implement may additionally be able to travel along a smooth curve, such as the continuous curve 516, faster than a number of straight sections, such as the discrete lines 514.

[0065] In a next step, it may also be found that the optimal path no longer includes the weed disposal location 530a simply being placed at the shortest horizontal distance from the first weed 520a. The weed disposal location 530a may be moved to correspond to a location along the continuous curve 516 which overlaps with the weed disposal lane 560.
The algorithm may therefore adjust the positioning of the weed disposal location 530a as a result of the virtual potential field pathing. The resulting curve as illustrated in Fig. 13 may be shorter, for example in comparison to the curve illustrated in Fig. 12.
In one implementation, the coordinates of the weed disposal location 530a may be optimized using a gradient descent algorithm to produce the shortest path between the two weeds 520a, 520b that passes through the disposal lane 560.

[0066] In one implementation, the processor may generate at least three potential picking and disposing trajectories comprising the smoothed concatenated path segments with optimized weed disposal locations, selecting the one having the shortest distance or representing the shortest amount of time for the implement to follow, as described below.

[0067] The real constraints of the robot configuration, such as torque limits of the controller, may also be considered by the algorithm when evaluating potential paths to provide a speed of the agricultural implement along each path. For example, although the .. steps described above relate generally to a picking and disposing trajectory based on physical distance only, the processor can additionally, or in lieu of, provide acceleration and speed information to the controller so that acceleration and speed may be determined along each point in the picking and disposing trajectory. The processor may then be able to select a path that, in view of the dynamics of the robot or constraints of the controller, is the optimized path representing the shortest amount of time for the implement to follow.
The determination of the picking and disposing trajectory can thus comprise a step of calculating a maximum speed of the implement based on the shape of the trajectory and Date Recue/Date Received 2022-02-14 based on constraints inherent to the controller controlling movement of the implement, such as the different motor drives constraints. Similarly, determination the picking and disposing trajectory can thus comprise a step of calculating accelerations and decelerations based on limits of the controller. Constraints inherent to the controller(s) can include, for example, minimum and maximum acceleration and deceleration, minimum and maximum speeds, which can be translated into voltage ranges.

[0068] With reference to Figs. 14 and 15 there is shown two example solutions comprising two separate paths for the agricultural implement to follow, including different weed disposal locations, for removing all weeds in the crop lane 550. The solutions may be evaluated by the algorithm and optimized with respect to time, or distance travelled, to find the solution which is most efficient. The algorithm may be solved to N
iterations comprising multiple solutions using varied orders of picking and disposing, as well as varied disposal locations, to find the most efficient solution. In one implementation, the algorithm is solved out to 3 iterations. In one implementation, the algorithm is solved to 6 iterations. The number of iterations may be limited to set an upper limit on the use of computational resources. Other iteration values may also be selected. The method is thus conceived such that determining and/or updating the weed picking and disposing trajectory can be realized by the processor in less time than the time needed between two picks.

[0069] With reference to Fig. 16, there is provided an example flow chart based on the embodiments disclosed herein. In a first step, images are captured from a camera (or cameras), mounted onto the autonomous vehicle. In a second step, the processing module 400 of the vehicle then processes the images to generate a map 500 of the captured area which comprises coordinates of the weeds and crops. In a third step, disposal lanes 560 (i.e., positional information) are inputted to the map 500.
In a fourth step, a disposal and picking trajectory that minimizes the time needed to pick and dispose of weeds is determined. In a fifth step, the agricultural implement follows the trajectory to pick and dispose of weeds.

[0070] Although reference and examples have been made to weed disposal in accordance with one embodiment of the present disclosure, it is equally envisaged that other uses may be possible. For example, the agricultural implement may, instead of disposal weeds in a field, dispose of extra or immature crops. Crop thinning is the process of removing young crops from a field to make space for other, stronger crops.
For example, Date Recue/Date Received 2022-02-14 carrots are often planted in a row in a field. It is sometimes necessary to remove some carrots from the row to allow other carrots to grow to full size. It may also be necessary to remove carrots in areas where growth has been less than desired, due to poor sunlight or other factors. The removed carrots may then be replanted elsewhere or disposed of.
Accordingly, instead of identifying, removing and disposing of weeds from a field, the present disclosure may instead be directed to identifying, removing and disposing of immature or selected crops (i.e., unwanted plants), such as young carrots.
Date Recue/Date Received 2022-02-14

Claims (23)

18

1. A method for picking and disposing of weeds using an implement provided on an agricultural vehicle, the method comprising:
capturing images of a field travelled by the agricultural vehicle, the field comprising weeds and crops;
generating a map from the captured images, comprising coordinates of the weeds and crops;
inputting disposal lanes into the generated map, the disposal lanes being regions where the weeds are to be disposed, and determining, based on the coordinates of the weeds, the coordinates of the crops and the disposal lanes, a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of weeds, while avoiding crops when moving to the disposal lanes; and removing weeds with the implement and disposing of said weeds in the disposal lanes by following the picking and disposing trajectory determined.

2. The method of claim 1, wherein removing weeds comprises picking one or more weeds at a time with the implement.

3. The method of claim 1 or 2, wherein the picking and disposing trajectory is iteratively built and modified as the agricultural vehicle travels the field.

4. The method of any one of claims 1 to 3, wherein the implement avoids crop avoidance zones as it follows the picking and disposing trajectory to the disposal lanes.

5. The method of any one of claims 1 to 4, wherein the implement follows the picking and disposing trajectory by a combination of rotational and translational movement.

6. The method of any one of claims 1 to 5, wherein avoiding contact with the crops comprises maintaining a minimum lateral distance between the implement and the crops.

7. The method of any one of claims 1 to 6, wherein avoiding contact with the crops comprises maintaining a minimum vertical distance between the implement and the crops.

8. The method of any one of claims 1 to 7, wherein determining the picking and disposing trajectory comprises applying a travelling salesman algorithm, which reduces travel distance to the disposal lanes, modified with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

9. The method of any one of claims 1 to 8, wherein determining the picking and disposing trajectory comprises a step of concatenating a plurality of path segments, each path segment corresponding to a linear distance between one of the weed coordinates and disposal coordinates located within the disposal lanes.

10. The method of claim 9, wherein determining the picking and disposing trajectory comprises a step of smoothing an initial path comprising the concatenated path segments.

11. The method of any one of claims 1 to 10, wherein determining the picking and disposing trajectory comprises determining a speed of the implement based on the shape of the trajectory and based on constraints inherent to a controller controlling movement of the implement.

12. The method of claim 11, further comprising determining accelerations and decelerations based on torque limits of the controller.

13. The method of any one of claims 1 to 12, wherein the implement is a gripper and removing weeds comprises controlling jaws of the gripper to pinch and extract weeds from the ground.

14. The method of any one of claims 1 to 13, wherein the images are captured using at least one camera on the agricultural vehicle.

15. A weed picking and disposal module provided on or attached to an agricultural vehicle, the module comprising:

an agricultural implement translatable along three degrees of freedom;
a controller configured to control the movement of the agricultural implement; and a processing module comprising:
input ports for receiving coordinates of weeds and crops determined from images and for receiving positions of disposal lanes; and a non-transitory memory and a processor, the non-transitory memory storing the coordinates and instructions for causing the processor to generate a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of weeds, while avoiding crops when moving to the disposal lanes.

16. The weed picking and disposal module of claim 15, wherein the implement is also rotatable along at least one degree of freedom.

17. The weed picking and disposal module of claim 15 or 16, wherein the agricultural implement is a gripper comprising jaws configured and adapted to pinch and retrieve the weeds.

18. The weed picking and disposal module of any one of claims 15 to 17, comprising one of a delta robot or a gantry system operatively connected to the agricultural implement, for translating the implement along the three degrees of freedom.

19. The weed picking and disposal module of any one of claims 15 to 18, wherein the controller comprises at least one first actuator for controlling movement of the gripper and a second actuator for actuation of the gripper.

20. The weed picking and disposal module of any one of claims 15 to 19, wherein the non-transitory memory has stored thereon instructions to apply a travelling salesman algorithm to an initial path, which reduces travel distance to the disposal lanes, and instructions to modify the initial path with a collision avoidance algorithm, which avoids travelling over the crops while disposing of the weeds.

21. The weed picking and disposal module of claim 20, wherein the non-transitory memory has stored thereon instructions to determine a speed of the implement based on the shape of the modified path and based on constraints inherent to a controller controlling movement of the implement.

22. The weed picking and disposal module of claim 20, wherein the non-transitory memory has stored thereon instructions to determine accelerations and decelerations based on torque limits of the controller.

23. A method for picking and disposing of unwanted plants using an implement provided on an agricultural vehicle, the method comprising:
capturing images of a field travelled by the agricultural vehicle, the field comprising unwanted plants and crops;
generating a map from the captured images, comprising coordinates of the unwanted plants and crops;
inputting disposal lanes into the generated map, the disposal lanes being regions where the unwanted plants are to be disposed, and determining, based on the coordinates of the unwanted plants, the coordinates of the crops and the disposal lanes, a picking and disposing trajectory for the implement to follow that minimizes the time needed to pick and dispose of unwanted plants, while avoiding crops when moving to the disposal lanes; and removing unwanted plants with the implement and disposing of said unwanted plants in the disposal lanes by following the picking and disposing trajectory determined.