GB2517049A

GB2517049A - Artificial intelligence for detecting and filling void areas of agricultural commodity containers

Info

Publication number: GB2517049A
Application number: GB1410199.2A
Authority: GB
Inventors: Matthew J Darr; Robert Mcnaull; John Kruckeberg; Zach T Bonefas
Original assignee: University of Iowa Research Foundation UIRF; Deere and Co; Iowa State University Research Foundation ISURF
Current assignee: University of Iowa Research Foundation UIRF; Deere and Co; Iowa State University Research Foundation ISURF
Priority date: 2013-07-28
Filing date: 2014-06-09
Publication date: 2015-02-11
Anticipated expiration: 2034-06-09
Also published as: DE102014108449A1; DE102014108449B4; GB2517049B; GB201410199D0

Abstract

A system and method for transferring agricultural material from a transferring vehicle 100 to a storage portion 106 of a receiving vehicle 102, and detecting and filling void areas of the agricultural material is disclosed. A transferring vehicle having a spout 112 transfers agricultural material to a storage portion of a receiving vehicle. An imaging device 126 associated with the transferring vehicle and facing towards the storage portion of the receiving vehicle collects imaging data. An image processing module (302 figure 3) is operable to, based on the imaging data, estimate fill levels of agricultural material within the storage portion; identify an area of the storage portion as targeted to fill; and identify a first void within the targeted area. A controller (324 figure 3) in communication with the image processing module is configured to command the spout to a position to direct agricultural material to the first void. Also disclosed is a method of performing the above.

Description

ARTIFICIAL INTELLIGENCE FOR DETECTING AND FILLING VOID

AREAS OF AGRICULTURAL OMMO1MTY CONTAINERS

TECHNICAL FIELD

100011 The present disclosure pertains to agricultural material transfer and, more particularly, to detecting and filling voids during the transfer of agricultural material.

BACKGROUND

[0002j A harvesting machine, such as a combine or selfpropei1ed forage harvester, i typicaliy used to harvest large (luantlt'es of agricultural materia1 suLh a grain or silage, in a field. Typically, harvesting machines utilize a spout (e.g., an unloading auger) to transfer awicultural material to a storage portion (e.g., a grain cart or wagon), of a receiving vehicle (e.g., a tractor). The material transfer/unloading process may occur when the harvesting machine and the receiving vehicle are stationary. Alternatively, the material transfer/unloading process may occur on-the-go, while the harvesting machine and the receiving vehicle are moving and the harvesting machine is simultaneously harvesting agricultural material.

[0003j When unloading on-the-go. the operators of the harvesting machine and the receiving vehicle must work together to maintain cooperative aliment therebetween to prevent spillage of agricultural material outside of the storage portion and to achieve a desired distribution of material within the storage portion. Desired distributions of material may optimize the material fill level by avoiding unused space within the storage portion. By achieving a desired distribution and avoiding spiUagc, the operational efficiency of the transportation of agricultural material from a field to

I

a grain bin may he optimized. Controlling the spout position while maintaining cooperative alignment is a challenging process because an operator of the harvesting machine must monitor various controls relating to harvesting the crop in front of the harvesting madiine. while simultaneously monitoring and controlling the material transfer process to the side of the harvesting machine. Further difficulties arise because the operator may not have a clear line of sight into the storage portion.

SUMMARY

9OO4l An example system fhr facilitating transfer of agricultural material is disclosed A transfernug vehicle nas a piopelled poiton for propclling the tranfening vehicle and a spout for n ansfernng the agricultural material to a storage portion of the receiving vehicle. The spout has a discharge portion. A first imaging device associated with the transferring vehicle and faces towards the storage portion of the receiving vehicle and collects first image data. An image processing module is operable to estimate an observed fill level of the storage portion based on the first image data and to estimate a distribution of agricultural material in the stoxage portion I he imagc processing module is arranged to identify at least one void in the estimated distribution and an associated target position in an opening directly above the at least one void. A controller, responsive to the observed fill level meeting or exceeding a target partial fill level of agricultural material in the storage portion.

triggers shifting of the discharge portion of the spout to the target position to fill the void in the agricultural material.

[0005] An example method of transferring agricultural material includes collecting image data by an imaging device facing towards a storage portion of a receiving venicle The storage portion of the receiving vehicle is capalc of stonng agncultural material transferred to the storage portion from a transferring vehicle via a spout.

Whik the agricultural matenal is bcmg transfcntd to the storage portion the image data is processed to determine a fill level protiic of the agricultural material ssithm the storage portion. Based on the fill level profile, a first void is identified within the storage portion and the spout is commanded to a first position to direct agricultural material to the first void.

0006J An etample systcm meiidcs a transferrrng sehiele basing a spout for transferring agricultural material to a storage portion of a receiving vehicle. An imaging device associated with the transferring vehicle faces towards the storage portion of the receiving vehicle and collectb nnagmg data An image processing module is operable to, based on the imaging data, estimate fill levels of agricultural material within the storage portion; identify an area of the storage portion as targeted to fill; and identify a first void within the targeted area. A controller is in communication with the image processing module and is configured to command the spout to a position to direct agricultural material to the first void.

BRIEF DEScRIFI'ION OF THE DRAWINGS [00071 FIG. IA is a plan view showing example imaging devices mounted on a transferring vehicle and facing toward a receiving vehicle, [(10081 FIG, I B is a view in a horizontal plane as viewed along reference line I B--lB in PlCT. IA.

[0009] FIG. 2A is a plan view showing an example imaging device (e.g., a stereo vision system) mounted on a receiving vehicle and facing a storage pothon of the receiving vehicle, 0010] FIG. 213 is a view in a horizontal plane as viewed along reference line 2ff- 2B in FIG. 2A.

[0011] FIG. 3 is a block diagram of an example machine vision-augmented guidance syscm for a tiansfemng vehicle for fac3htatmg the unloadng or trarsfernng of agricultural material from the transferring vchicle (e.g., a combine or a self-propelled forage harvester).

O012] FIG 4. is a block diagiam of another example machne vision-augmented guidance for a transferring vehicle for facilitating the unloading or transferring of agricultural material from the transferring vehicle (e.g., a self-propelled forage harvester).

[00131 FIG. 5A is a block diagram of an example machine vision-augmented guidance system fbr a receiving vehicle for facilitating the unloading or transferring of agricultural material from a transferring vehicle to the receiving vehicle (e.g., grain cart and tractor).

[0014] FIG. SB is a block diagram of an example electronic guidance system for a receiving vehicle for cooperation with the machine vision augmentcd guidance system of FIG. 3 or FIG. 4.

[0015] FIG 6 is a two-dimenscmal iep'csertation of various possible ilhistrative distributions of material in the interior of a container or storage portion, consistent with a cross-sectional view along reference line 2C-2C in FIG. 2k [0916] FIG. 7 is a plan view of a transferring vehicle and a receiving vehicle, where the transferring vehicle is aligned within a matrix of possible offset positions.

10917] FIG. 8k is an example fill model of a storage portion of the receiving vehicle illustrating cells.

[0018] FIG. SB is the example fill model of FIG. SA illustrating columns.

[0019] FIG. SC is the example fill model of FIG. 8A illustrating zones.

[0020 FIG. SD is the example fill model of FIG. SA illustrating avoid area.

[0021] FIG. 9 is the example fill model of FIG. 8A with example till data.

[0022] FIG. lois the example fill model of FIG, 8A with example void volume calculations from the fill data of FIG. 9.

100231 FIG, 11 is a flowchart of an example fifl strategy.

[0024] I IC 12 1', a flowchart of an example gencral fill mode [0025] FEC]. 13 is a flowchart of an example zone fill mode.

[0026] FRi. 14 is a flowchart of an example void fill mode.

[0027] FIG. 15 is a block diagram of an example processor platform to execute nr utilize the machine visionaugmented guidance system. 2'

DETAILED DESCRIPTION

100281 Systems and methods for controlling the transfer of agricultural material from a transferring vehicle 100 to a receiving vehicle 102 are disclosed herein. As depicted in FIGS. 1A-2B, the transferring vehicle 100 is shown as a combine, whereas the receiving vehicle 102 is shown as a tractor and a grain cart. More generally, the icen mg vehic1e 1 0 may mdude the.nmhmation of a propelled portion 104 and a storage porbon 106 of a container 108 (i.e.. a towed storage unit). FJowever, other examples may utilize other types of working machines. For example, the transferring vehicle 100 may he a self-propelled forage harvester. Moreover, the systems and methods disclosed herein are applicable to the transfer of any type of agricultural material, including hut riot knifed to corn, maize, soybeans, silage wheat, oats, rye, barley, flax, oiiseed, fiber, vegetables, fruit, nuts, seeds, or other agricultural material.

t0029] In certain examples. harvested agricultural material is transferred to a grain tank 110 (e.g., a hopper), where it may be stored until a material transfer (i.e., unloading) process is initiated. During a material transfer process, the agricultural material is transferred from the grain tank 110 to the storage portion 106 of the receiving vehicle 102 via a spout 112 (e.g., an unloading auger). Other examples, such as examples utilizing self-propelled forage harvesters, do not utilize the grain tank 110, In such examples, the agricultural material is transferred directly to the storage portion 106 as it is harvested.

100301 To achieve a desired distribution of agricultural material within the storage portion 106, an auger rotation system is configured to sense a spout rotation angIe 114 of FIGS. 1A and 2A and/or 116 ofFIGS. 18 and 28 arid rotate the spout 1i2 to (3 change the relative position of a spuut end 1. 18 of the spout 112 with respect to a container pert eter 120 of the storage portion 116 of the container 108. Additionally or alternatively, an aligrunent. module may he adapted for generating command data to the transfi'wring velncle 100 and/or the propelled portion 104 of the receiving vehicle 102 to steer the storage portion lOb and'or the propelled portion 104 to change the relative position of the spout 112 with respect to the container perimeter 120.

100311 Various control methodologies maybe used to control the position of the spout I P during a material transfer process to achieve a desired material rnstnbutaon In some examples, voids within the distribution of agricultural material are identified and the position of the spout 112 is adused to fill the voids fins and other cxamples will be described in detail below.

100321 There arc many benefits to achieving a desired distribution of agricultural material within the storage portion 106. For example, desired distributions may optimize the mnatenal handling capabilities by avoiding spi'lage from and unused space within the storage portion 106m order to maxunuc the opeiational etIkienc'y of the transportation of agricultural material between two locations, e.g.. from a field to a grain bin. Furthen'nore, desired distributions may prevent uneven loading and localized stresses on portions of the receiving vehicle 102 imparted by the weight of the aicultural material within the storage portion 1 06. For example, an uneven distribution may cause localized stresses on a tongue 122 and/or a hitch 124 that eouple% the con4atncr 108 to the propelled portion 104 of the receiving vehele £02 Furthermore, an unieven distribution may increase localized soil compaction imparted by certain wheels of the container 108. which may negatively affect the emergence of next year's crop. Thus, achieving a desired distribution of agricultural material within the storage portion 106 results in significant benefits.

100331 Example systems and methods described herein may utilize imaging devices such as a first imaging device 126 and may optionally utilize a second iniaging device 128 and/or other sensors to monitor the fill eveI and distribution of agricultural material within the storage portion 106. For the sake of clarity and brevity, the first and second imaging devices 126, 128 are referred to herein as suck However, ref erei es of the first and second Jnagmg devices 126, 1 28 are intended to mci ide example systems including one of the first imaging device 126 or the second imaging device 128. In some examples, the second imaging device 128 is optional and provides redundancy to the first imaging device 126 in case of failure, malfunction or unavailability of image data from the first imaging device I 2& Similarly, the first imaging device 126 may provide redundancy to the second imaging device 128.

OO34] The first and second imaging devices 126, 128 may be stereo imaging devices that may provide digital data output as stereo video image data or as a series of stereo still frame images at regular or periodic intervals, or at other sampling intervals. Each stereo image (e.g, the first image data or the second image data) has two component images of the same scene or a portion of the same scene. In an alternate example the first and second imaging devices 126, 128 may be monocular imaging devices and may output first and second monocular image data, respectively.

In an example, the first and second imaging devices 126, 128 may include a charge-coupled device (CCI)), a complementary metal-oxide semiconductor (CMOS array, or another suitable device for detection or collection of image data.

[0035] As shown in FIGS. IA and 1 B, the first and second imaging devices 126, 128 may be mounted on the transferring vehicle 100. Uowever, in alternate examples as shown in FIGS. 2A and 2B, the first imaging device 126 and/or the optional second imaging device 128 may be mounted on the receiving vehicle 102 or the propelled portion 104 of the receiving vehicle 102.

[0036] The first imaging device 126 has a first field of view 130 and the second imaging device 128 has a second field of view 132, each of which are indicated by dashed lines in FIG. 1A. The boundaries of the fields of view 130, 132 arc merely shown for illustrative purposes and will vary in actual practice. The spout end 118 (c g a b1⁄4'ot) of the spout 112 is generally aligred over a cenLal zone j34 within the container perimeter 120 defined by edges 136 of the storage portion 106 for unloading material from the transferring vehicle 100 to the receiving vehicle 102, Similarly, the transferring vehicle 100 and the receiving vehicle 102 may be aligned in the position as shown in FIGS. 1A-2B, regardless of whether the velucles move together in a forward motion (e.g.. with coordinated or tracked vehicle headings) during harvesting, as is typical, or arc stationary. Aligmnent between the transferring vehicle 100 and the receiving vehicle 102 may be defined by a lateral offset 138 and a fbre/aft offset 140.

E00371 In FIG. I B, the first and second imaging devices 126, 128 are mounted on the transferring vehicle 100, where the first imaging device 126 is mounted on a first support 202 (e.g., a monopole platform with tilt and/or pan adjustments) to provide a first downward field of view 144, As shown in FIG. lB, a down-tilt angle 146 is measured with respect to a vertical axis, A tilt scnsor on or in the first imaging device 126, or associated with the rnastimaging device mounting hardware, may measure the downiit angle 146 (e.g., declination angle). For example, the tilt sensor may comprise one or more accelerometers, such as those that are commercially available for portable consumer electronic devices.

[0038] if the first imaging device 126 is elevated or mounted on the transferring i ekck 100 sufficiently high with respect to tht storage portion 106, the first imaging device 126 will have the visibility or downward field of view 144 into the storage portion 106 of the container 108 sufficient to observe and profile the surface (i.e., height (z) sersus respective x, y coordmatcs in the container) of the agneultural material (e.g., grain) as the agricultural material fills the storage portion 106 of the container 108 The first imaging device 126 may be mounted on the roor of the transfemng \ehiclc 100 facing ot looking directly away fron the side of the transferring vehicle 100 with the spout 112 for unloading agricultural material.

[0039] if the first imaging device 126 is Thither from the storage portion 106 or the container 108 than the second imaging device 128 during unloading of the awiculturai material, the first imaging device 126 may have suitable view of the storage portion 106 to facilitate easier tracking of the relative position of the storage portion 106 to the transferring vehicle 100, 100401 In one illustrative configuration, consistent with the downward field of view 144 in FIG. IB, the optical axes of the first and second imaging devices 126, 128 are perpendicular to a respective lens thereof and are tilted downward from a generally horizontal plane at the down-tilted angle 146 (e.g., approximately 60 to 85 degrees downward from the vertical axis or approximately 1 0 to 25 degrees downward from a horizontal axis or horizontal plane as shown in FIG. I B). There are several advantages to having a field of view or optical axis of the first or secoiid imaging device 126, 128 tilted downward from a generally horizontal plane.

100411 First, less of the sky is visible in the field of view of the first imaging device 126 such the collected image data tends to have a more unifhrm image intensity profile. The tilted configuration of the optical axis (which are perpendicular to the lenses of the imaging devices 126 is well suited for mitigating the potential dynamic range issues caused by bright sunlight or intermediate cloud cover, for nstance In one example, the first or second imaging device 126, 128 is mounted with a down4ilt angle 146 to avoid washout of; corruption ot or interference with collected image data during a traisient exposure time penod to sunlight, a reflection, or a light source that exceeds a threshold brightness level for a material portion of the pixels in the collected image data. Second, the bottom part of the storage portion 106 becomes more visible in the image data to enable the recording of the image data related to one or more wheels of the storage portion 106, The wheel is a feature on the storage portion I 06 that may be robustly tracked by image processng teclmique Third, tilting the stereo camera down may mitigate the accumulation of dust and other debris on the lens or external window of the first or second imaging device 126, 128.

[0042 FIG. 2A shows the first imaging device 126 on the rear of the propelled portion 104 (e.g., a tractor) of the receiving vehicle 102. The first imaging device 126 has a first field of view 200 indicated by the dashed lincs. The spout end 118 of the spout 112 is generally aligned over the central zone 134 (e.g., central region or target area) of the storage pornoti iOó of the eontaincr 1 08 for unloading material from the transferring vehicle 100 to the receiving vehicle 102. Similarly, the transferring vehicle 1 00 and the receiving vehicle 102 arc aligned in position as shown, even as

II

the vehicles 102, 100 move with coordinated headings or gencTally parallel headings with no or minimal relative velocity with respect to each other.

[0043J In an alternative example of FIG. 2A, the optional second imaging device 128 may be mounted on the receiving vehicle 102 with a second field of view, which may he slightly offset from, overlapped with, or aligned with the first field of view to provide redundancy should the first imaging device 126 fail, malfunction, be unavailable, he unreliable, or provide poor quality image data, For example, the first imaging device 126 may not operate reliably if it is obscured by dust, fog, salt, or air-born contaminants, or if it is exposed to inadequate ambient light conditions or excessive glare from sunlight or reflected light in FIG. 2i, an image processing module may estimate the distance or range from the first imaging device 126, the second imaging device 128, or both to an ohiect in the image such as thc spout 112, the spout end 118, the container perimeter 120, or the level or profile of aqçieultural material in storage portion 106 of the container 108 (e.g., at various positions or coordinates within the storage portion 1 06).

[00441 FiG, 2B illustrates a. view in a horizontal plane as viewed along reference line 2B-2l3 in FIG. 2A. In one example, the first imaging device 126 is mounted on the receiving vehicle 102 on a first support 142 (e.g., monopole with tilt or pan adjustment) to provide a first downward field of view 204 or a first down-tilted field of view.

100451 If the first imaging device 126 is elevated or mounted on the receiving vehicle 102 sufficiently high with respect to the storage portion 106, the first imaging device 126 will have visibility or the first downward field of view 204 into the storage portion 106 sufficient to observe and profile the surface (i.e., height z venws respectne x, coordinates in the container) of the agricultural material (e g, g'ain) as the agricultural material fills the storage portion 106 of the container 108. The first imaging device 126 may be mounted on the roof or cab of the propelled portion 104 of the receiving vehicle 102 theing or looking directly away from the side of the transferring vehicle 100 with the spout 112 for unloading agriculmral material.

100461 The block diagram of FIG. 3 illustrates an example system 300 fbi facilitating the transfer of agricultural material in connection with the equipment of FIGS. lA-2B. The first and second imaging devices 1.26, 128 transmit collected image data to an image processing module 302. The image processing module 302 may he coupled to a vehicle data bus 304 and may include a container module 306, a spout module 308, and alignment module 310, an arbiter 312, and a material profile module 314 1 he image processing module 30' ana1yie' toe collected image data and generates command data to various components of the sssteni 300 to facihtate material transfer. For example, a steering controller 316, a braking controller 318, a propulsion controller 320, a user interface 322, a vehicle controller 324, and a user interface processing module 326, among other optional components, may he coupled to the vehicic data bus 304 o receive command data to facintate rnatenal tiarsier 100471 The first and second image devices 126, 128 are coupled to the image processing module 302 and transmit collected image data thereto. Tile collected image data may mclude steico image data or monoculai irnagt data If the collected image data is monocular image data, the image pro.cssmg n'odule 302 ni create a stereo image from the first and second monocular image data (e.g., right and left image data, respectively) with reference to the relative position and orientation of the first and second imaging devices 1 26, respectively. The image processing module 302 determines: (1) at least two points on a common visual axis that bisects the lenses of both the first arid second imaging devices 126, 128. and (2) a linear spatial separation between the first and second imaging devices 126, 128, where the first field of view (in FIG. IA) of the first imaging device 126 and the second field of view 132 of the second imaging device 128 overlap, at least partially, to capture the spout 112, the spoilt end 118 and the container perimeter 120 in the collected image data.

[0048] The image processing module 302 may utilize the rotation angie 114 of the spout 112 to facilitate fusion of image data from the first imaging device 126 and the second imaging device 128, or to construct sereo image data %shere the first imaging device 126 and the second imaging device 128 individually provide monocular image data fbr the same scene or object. In any arrangement of imaging devices 126, 128 dp,closed herein where the fields of viesv 130, 132 ovcrlap data fusion of image data from a first imaging device 126 and a second imaging device 128 enables the image processing module 302 to create a virtual profile of the material distribution level inside the storage portion 106 of the container 108, even when the entire surface of the agricultural material is not visible to one of the first or second imaging devices 126, 128. Even if the second imaging device 128 is not mounted on the spout 112 in certain configurations, a rotation sensor 328 may utilize the spout end 11$ as a reference point in any col cctcd image data to facilitate, e g, fusion, virtual stitching or alignment of image data from different mtagmg devices The irtual profile of the entire surface of the auicultural material in the storage portion 106 enables the system 300 to intelligently execute a material transfer strategy.

0O49 First and second optical sensors 330, 332 may be associated with the first and second imaging devices 126, 128 respectively, and may be coupled to the image processing module 302. The first and second optical sensors 330, 332 may include, e.g.. a light meter, a. photo-sensor. a photo-resistor, a photo-sensitive device, or a cadmium-sulfide cell, among other sensors, The image processing module 302 may be coupled, directly or indirectly, to optional lights 334 on the transferring vehicle (FIGS. IA-28) for illumination ofthe storage portion 106 and/or the spout 1 U. For example, the image processing module 302 may control drivers, relays or switches, which in turn control the activation or deactivation of the optional lights 334. The lights 334 may be activated, for example, if an ambient light measurement from the first and second optical scosors 330. 332 is below a predetermined minimum threshold.

W050] The cuntairier module 306 may identify a set of two-dimensional or three dimensional points (e g in Cartesian toordinatcs or Polar Loordmates) corresponding to pixel positions in irnagcs collected by the first and second imaging devices 126 128. The set of two-dimensional or three dimensional points may define the position of the receiving vehicle 102. For example, the set of points may define a portion of the container perimeter 120 (FIGS. 1A and 2A) of the storage portion 106.

[0051J The container module 306 may use or retrieve container reference data to thither define objects within die collected images. The container reference data may comprise one or more of the following; reference dimensions (e.g., length, width, height), volume, reference shape, drawings, models, layout, and configuration of the storage poruon 100, the containei penmeter 120, and the container edges 136, reference dimensions, reference shape, drawings, models, layout, and configuration of

IS

the entire container 108 of the receiving vehicle 102; wheelbase, turning radius, and hitch configuration of the propelled portion 104 and/or the container 108; and distance between a pivot point of the bitch 124 and the wheelbase of the container 108. The container reference data may be stored and retrieved from a data storage device 336 (e.g., non-volatile electronic memory). For example, the container reference datamay he stored by, retncab1e by, or indexed by a eorresondmg vehicle identititr of the receiving vehiclt 102 in the data storage de%ice 336 of the systcm 300 of the traasfemng vihicle 100 For each ehic1e idcnificr of the receivmg vehicle 102 then may bc a corresponding unique container reference data stored in the data storage device 336.

0O52J In one example, the transferring vehicle 100 receives a data message from the receiving vehicle 102 in which a vehicle identifier of the receiving vehicle 102 is regularly (e g. periodically) tiansmitted In another example, the transferring vehicle interrogates the receiving vehicle 102 for its vehicle identifier or establishes a communication channel between the transferring vehicle 100 and the receiving vehicle 102 in preparation for unloading via a wireless communication device 338. In yet another example, the receiving vehicle 102 transmits its vehicle identifier to the transferrirg vehicle 100 hen the i cceivmg vcbicle 102 approadies the ransfemng vehicle 100 within a certain radial distance, in still another example, only one known configuration of the receiving vehicle 102 is used with a corresponding transferring vehicle 100 and the contamei reference dsta is stored or saed in thc data toiage dcs'ice 336 In the attei example, the traisfemng %ehlcle 100 is progianirned, at least tcrnponnly solely for receiving vehicles 102 with identical containers 108, which may be identical in dimensions, capacity, proportion and shape.

100531 in one configuration, the container module 306 identifies the position of the container 108 as follows. If the linear orientation of a set of pixels in the collected image data conforms to one or more edges 136 of the perimeter 120 of the container 108 as prescribed by the container reference data, the position of the container 108 is considered to be identified I or etampie, the cential zone 134 of the contamer 108 may he identified by dividing (i.e., dividing by two) the distance (e.g., the shortest distance or the surface normal distance) between opposite sides of the container, or, alternatively, by identifying corners of the container and subsequently identifying where diagonal lines that intercept the corners intersect. In one configuration, the central zone 134 may be defined as an opening (e.g, circular, elliptical, or rectangular) in the contaner with an opening surface area that is greater than or equal to the cross-sectional surtact rca of the spout end II S by a tactor of at icast two, although other surface areas fall within the scope of the claims.

[0054] The spout module 308 is configured to identify the spout olthc' transferring vehicle 100 in the collected image data. In one configuration, the spout module 308 identifies one or more of the following (I) spout pixels on at least a portion of the spout 112, (2) spout end pixels that are associated with the spout end 118, 3) spout pixels associated with the spout 112 1 he spout module 308 may use color discnmmation, intensity rnscrimiration, ot texture dhcnrninaflon to identity background pixels from one or more selected spout pixels with associated spout pixel patterns or attributes (e g, eoioi or color patterns te g Red Green Blue (ROB) pixci values), pixel intensity patterns, texture patterns, luminosity, brightness, hue, or reflectivity) used on the spout 112 or on the spout end 118 for identification purposes 100551 The alignment module 310 and/or a master/slave controller 340 estimate or determine motion commands at regular intervals to maintain alignment of the spout end 118 over the central zone 134 of the storage portion 106 for unloading agricultural material. The aligmnent module 310 and/or the master/slave controller 340 may send commands or requests to the transferring vehicle 100 with respect to its speed, velocity, or heading to maintain alignment of the position of the transferring vehicle 100 with respect to the receiving vehicle 102. For example, the alignment module 3 10 may transmit a request for a change in a spatial otThet between the vehicles 102, 100 to the master/slave controller 340. in response, the master/slave controller 340 or a coordination module 342 transmits a steering command or heading command to the steering controller 316, a braking or deceleration command to a braking system 344, and a propulsion, acceleration or torque command to the propulsion controller 320 to achieve the target spatial offset or change in spatial offset. Further, similar command data may be transmitted via the wireless communication device 338 to the receiving vehicle 102 for observational purposes or for control of the receiving vehicle 1 02.

1100561 In another configuration, the alignment module 310 or the image processnig module 302 may regularly oi penodicali) move, adjust oi rotate the spout end 118 to the central zone 134 dunug loading of the storage portion 106 of tue iecervtng vehicle 102 to promote even filling, a uniform height, a uniform distribution, or a desired non-uniform distribution of the agricultural material in the storage portion 106. lii such a configuration, the image processing module 302 identifles the fill state of the agricultural material in the image data from the material profile module 314.

[0057] The arbiter 312 evaluates image data quality. For example, the arbiter 312 may include an evaluator, a judging module, Boolean logic circuitry, an electronic module, a software module, or software instructions to determine: (1) whether or not to use the first image data (Le., from the first imaging device 126) and/or the second image nata (i e from the econd imaging dewee 128), nd'or () whether or not to use the image processing module 302 or its output data to align the spout end 118 with respect to the container perimeter 120 o' to adjust the spatial offset between the vehicles 102, 100. In one example, the arbiter 312 determines whether or not to use the first image data, the second imagi data, and the output data ofthe image processing module 302 based on the evaluation of o'ie 01 inorc of the following metrics, factors or criteria during one or more sampling periods: material variation of intensity of pixel data, material variation in ambient light conditions, image rectification quality, disparity image quality, stereo correspondence data quality, reliability, extent or degree of identification of edges of one or more image objects (e g, the spoat 112 the wont end 118, the container 7enmcter 120, the storage portion 106), and rehabthty of coordinate estimation (e g, thiee dmitnsional coordinates) of one or more objects in the image. The arbiter 312 may communicate one or more quality indicators (c.g, a third indicator message) to an optional mode controller 346, for example, via a data bus, a logical data path, a physical data path, or a virtual data path.

100581 The material profile module 314 is configured to detect a one-dimensional, two-dimensional, or three-dimensional representation of the fill level or a volumetric distribution of the agricultural material in the storage portion 106. Example fill level representations will be discuss in flurther detail in connection with FIG. 6.

[0059J The vehicle controller 324 may include an auger rotation system 348 amid an auger drive 350. The auger rotation system. 348 may include, for example, (1) the rotation sensor 328 for sensing the spout rotation angle 114 and/or 116 of the spout 2 with respect to one or more axes of rotation, and (2) a rotation actuator 352 for moving the spout 112 to change the spout rotation angle 114 and/or 116, and hence, the position of the spout end 118 of the spout 112 with respect to the storage portion 106. The rotation actuator 352 may include a motor, a linear motor, an electro-hydrauhe device, a ratcheting or cable--actuated mechanical device, or another device for moving the spout 112. The spout rotation angie 114 and/or 116 may be a simple angle, a compound angle or multi-diniemisional angles that is measured with reference to a reference axis parallel to the direction of travel of the transferring vehicle I 00.

the rotation sensol 328, the rotation actuator 352, aridor the auger dnw $50 may communicate with the vehicle controller 324 through a transmission line or a.

secondary data bus.

100601 An example rotation actuator 352 that includes an electro-hydraulic device may titihze proportional or non-proportional control valves. Proportional control valves facilitate finer adjustments of the spout rotatIon angle 114 and/or 116 than what is possihic with non-proportional (c g bang-bang) valves I ikewise, the rotation actuator 52 may also facilitate fine adjustincnts of the pout rotation angle 114 and'or 116 by utilizing an electric motom Sytemns utthnng electro-hydraulic devices with non-proportional control valves, absent sophisticated control systems as described herein may bi prone to fill the storage container with an mctficient multi-modal or humped distribution of agricultural material with local high areas and local low areas. However, the same systems utilizing control methods as described herein may achieve a distribution of agricultural material closer to a desired distribution.

Further control accuracy may he achieved by systems that utilize rotation actuators 352 comprising proporl'onal a1ves or electnc motors I hese and othei cxamp1e distributions of agricultural material will be further discussed below in cormection with FIG. 6.

[00611 The vehicle controller 324 may be coupled to the vehicle data bus 304 to provide a data message to in dicate when the auger drive 350 for transferring agricultural material from the transferring vehicle 100 through the spout 112 is active or inactive. In one example, the auger drive 350 may include a rotating helical screw on an auger shaft, an electric motor for driving the auger shaft, and a rotation sensor for sensing rotation or the rate of rotation of the auger shaft. If the vehicle controller 324 indicates that the auger drive 350 of the transferring vehicle 100 is active, the image processing module 302 may activate the container module 306 and the spout module 308. Thus, the auger rotation system 348 and/or the vehicle controller 324 may conserve data processing resources or energy consumption by placing the container module 306 and the spout module 308 in an inactive state (i.e., standby mode) while the transferring vehicle 100 is harvesting, but not transferring, the agricultural material to the receiving vehicle 1 00.

10062] The steering controller 316 or the coordination module 342 may adjust the rclaivc position (e g, the offset components illustra'ed in FIG IA) of the tianstcrrmg vehicle 100 to the receiving vehicle 102.For example, the steering controller 316 associated with the steering system 362 of the transferring vehicle 100 may steer the transferring vehicle 100 in accordance with a cooperative alignment (e.g., an alignment suitable far efficiently transferring material from the transferring vehicle to the receiving vehicie 102 as both are in generally forward motion) based on location data and motion data from a first location-determining receiver 354 of the Iratisferring vehicle I 00 and a second location-determining receiver 504 (FIG, 5A) of the receiving vehicle 102. The alignment module 310, the coordination module 342, and/or the auger rotation system 348 may control the relative position of the spout 112 or the spout end 118 to the container perimeter 120 to achieve an even fill to the desired fill level, with or without such location and motion data from the first and/or second locatiowdetermining receivers 354, 504 (FIG. SA). For example, the rotation actuator 352 or the auger rotation system 348, alone or in combination with the vehicle Lontioller 124 may adjust the spout angle 114 and/ot 116 10063] The spout end 118 may be adjusted for unloading agricultural matenal by shifting its position within the container perimeter 120 and within a tolerance clearance of the container perimeter 120 to prevent spillage. The spout end 118 may be adjusted by anous techniques that way be apphcd alternately or eumulativciy Under a first technique, the ahgnmevt module 310 adu'ts the spout end 118 by shifting the spout angle 114 and/or 116. Based on the collected image data for a sampling period, the rotation actuator 352 may adjust the spout angie 114 and/or 116 to place the spout end 118 in cooperative alignment with the storage portion 106.

10064] Under a second technique, the alignment module 310 requests or commands thc coordination module 342 to adjust the fore/aft offset adjustment 140 and/or the lateral adjustment 138. Under such a technique, the coordination module 342 manages or choreographs the relative fore/aft offset 140 and lateral offset 138 between the transferring vehicle 100 and the receiving vehicle 102 by moving the transferring vehicle 100 and/or the receiving vehicle 102. Under a third technique, the alignment module 310 primarily adjusts the spout end 118 for unloading agricultural material by shifting the spout angle 114 and/or 116, and the coordination module 342 secondarily and regularly (eg,, periodically) moves the fore/aft offset and the lateral offset by the fore/aft offset adjustment 140 and the lateral adjustment 138, respectively, to achieve a uniform fill state or a level loading of the storage portion 106 with the agricultural material. Accordingly, to achieve a target alignment or desired even distribution of filling the storage portion 106 with agricultural material, the spout end 118 may be adjusted regular'y (e g, m a matnx of one or more iO\4' or olumns orpkset otfet positiois) by shifting the spatial relationship between the transfcmng ?ehlLle 100 and the receiving vehicle 102 by a ibre and aft offset or a lateral offset. The spout angle adjustment maybe used for fine tuning of the distribution of the agricultural material ithin the storage portion 106 (e g, from each position wrth'n the matrix) 10065! The optional mode controller 346 is coupled to the data bus 304. Although the mode controller 346 is discussed in connection with the system 300 of FIG. 3, the mode controller 346 may similarly be utilized by systems 400 and 500 of FIGS. 2 and 5A, respectively. The mode controller 346 may include a perception quality evaluator, a judging moduic Boolcan logic circuitry, an eleetionic module a softwarc module or software instructions for determining whether to operate the machine-vision-augmented guidance system 300 in one of a plurality of operating modes. Example operating modes include (1) an operator-directed marual mode in which one or more human operator' tcer the receiv3ng ehicth 102 and or the tranfcnmg vehicle 100 as agricultural material is transferred from the transferring vehicle 1 00 to the receiving vehicle 102; (2) an automated mode in which the receiving vehicle 102 arid/or the transternng vel'iele 100 are steered and aligned automatical'y as agneultural inateria is transferred from the transferring vehicle 100 to the receiving vehicle 102; and (3) a senuautomated mode or partially automated mode in which one or more operators supervise and may override the automated steering and alignment of the transferring vehicle lOt) arid the receiving vehicle 102. For example, the mode controller 346 may determine whether to use an automated control mode of the spout 112 or an operator-directed manual control mode of the spout 112 based on a first operational status of the first loeation-dctermming receiver 354 associated with the transferring vehicle 100, a second operational status of the second location-determining receiver 504 (FIG. 5k) associated with the receiving vehicle 102, and a third operational status of the first imaging device 126 or the image processing module 302.

10066] In one configuration, the automated contiol mode ncludes an operational mode whcre the image processing module 302 proctsses the colkcted imnagc data to facilitate the dctcirmnation of a position afthe spout 112 (e g, the spout end 118) relative to the storage porbon 106 the container penmefer 120 ci the Lentral zone 134 of the storage portion 106. Further, in the automated control mode, the image processing module 302 may generate courmand data to place the spout end 118 relative to the storage portion 106, the container perimeter 120, or the central zone 134 of the storage portion 106, auth that the spout end 118 is aligned tbr transferring of agricultural material into the storage portion 106, The command data may comprise one or more of the following: steering command data for the receiving vehicle 102, steering command data for the transferring vehicle 100, or actuator command data for rotating, tilting, deflecting, or otherwise manipulating the spout 112.

100671 In one example, the mode controller 346 includes a perception quality caluator tiat eauates the fimctionahty diagnostics, perforniance, tests or quality of one or more 1ocationdetennining receivers 354, 504 (FIG. 5A), first and second imaging devices 126, 128, range finders, odometrical sensor 356, deathreckoning sensors.]er1al sensors 358, navigation sensors, or other perccption sensors In one illustrative example, the first operational status is acceptable if the first location-determining receiver 354 provides reliable position data that meets or exceeds a dilution of precision threshold or another navigation satellite reliability measurement during a sampling period. The second operational status is acceptable if the second location-determining receiver 504 (FIG. 5A) provides reliable position data that meets or exceeds a dilution of precision threshold or another navigation satellite reliability measurement (e g,total equnalent user iangt error) dunng a samphngpcriod Further, the third operational status is acceptable if the first imaging device 126 provides reliable image data in which the container module 306, the spout module 308, or the respective edge detection modules therein are capable of any of the thllowing: (I) reliably identifying or resolving one or more edges of the spout 112, the spout end 118, the container perimeter 120 or a portion thereof, or the storage portion 106 in the collected image data during a sampling time period, (2) reliably identifying on a time percentage basis (e.g., at least 99,99 % of the time) one or more reference obiects (e g a refereLce pattern or reference image on the spout 1! 2 or the receiving vehicle 102) or objeets in the image data. (3) reliably determining (e.g., via stereo eorespondencc or disparity processing) coordinates (e g, in U ret' din ensions) of a material portion (e.g., a group or constellation of reference points) on one or more of the spout 11?, the spout end 118, the storage portion 106, eontair'ei perimeter 120 or a portion thereof during a sampling period, or (4) reliably identifying one or more edges of the spout end 118 and one or more edges 136 of the container perimeter 120 of the storage portion 106.

[0068] The position and/or the coordinates (e.g., three dimensional coordinates) of the spout 112 and/or the spout end 118 may he calculated using the detected rotation angie 114 and/or 116 of the spout 112 along with a known length of the spout 112.

However, where the rotation angle 114 and/or 116 or the rotation sensor 328 is not piesent, operating, or in communication with the image processing module 302 the spout module 308 may use image ptoccssmg for any of the following (1) to estimate a spout angle 114 and/or 116, (2) to estimate a spout position (e.g., three dimensional coordinates) of the spout 112, and 3) to estimate a spout position (e.g., three dimensional coordinates) of the spout end 118.

[0069] Dilution of precision provides a figure of merit of the performance of the localion-determinmg receivers 354, 504 (FIG SA) that uses a satellite navigation system, such as the G1obal Positioning SysLern (GPS) or the Global Navigation Satellite System (GLONASS. Dilution of precision captures the time-varying impact of spatial geometry and separation between the location-determining receivers 354, 504 (FIG. 5A and satellite signals that are received by the location-determining receivers 354, 504 (FIG. 5A), without clock errors, ionospheric errors, multipath errors and othc errors The precision in pseudo-range estimate to ca Ii satellite may affect the accuracy of the determination of a three dimensional position estimate and may affcct thc time estiniate of the Ioeation-deternnnmg receiveis 354, 504 (FIG 5A). If receivable navigation satellites are spatially too close together in orbit for location-determining receivers 354. 504 (FIG. SA) at a particular time, accuracy of the position estimate may be compromised and the dilution of precision value may be higher than what is normal or acceptable.

[0070] In one example, the first location-determining receiver 354 provides a first indicator message tnat indicates that the first Ic cation-detcnnrnmg recenel 354 meets or exceeds a precision dilution threshold. The second location-determining receiver 504 (FiG. 5A) provides a second indicator message that indicates that the second location-determining receiving 504 (FiG. SA) exceeds a precision dilution threshold.

The image processing module 302 provides a third indicator message indicating that the image processing module 302 is capable of reliably identifying one or morc edges of the spout rid ii S and one or more edges of the container perimeter 120 of the storage portion 106 during a sampling period.

[0071J Ef the mode controller 3-46 is separate from the image processing module 302, a measure of redundancy and reliability is added to the system 300 because any failute or disruption of the functioning of the unage processing module 302 or the hardware or the software of the image processing module 302 is generally separate and distinct from any failure or disruption of the thnetioning of Ihe mode controllei 346 Although the mode controller 346 is shown separate from and outside of the image processing module 302 in an alternate example the mode controllet 346 may be incorporated into the image processing module 302 to potentially reduce the cost of the system 300.

100721 The master/slave controller 340 is coupled to the data bus()4. In one example, the master/slave controller 340 includes an auto-guidance module 360 and the coordinatior module 342 The auto-guidane module 160 or 1e mastci/slave controller 340 may control the transferring vehicle 100 in accordance with location data from the first location-detennining receiver 354 and a path plan or desired vehicle path (e.g, a desired vehicle path stored in a data storage device 336), The auto-guidaiwe module 360 or the master/slave controller 340 sends command data to the steering controller 316, the braking controller 3 13, and the propulsion controller 320 to contiol the path of the 1ranstemng vehiee 100 to track automatically a path plan or to track manually a steered course of an operator via the user interface $22 or the steering system 362.

[O073 The coordination module 342 may facilitate alignment of movement (eg., choreography) between the transfen'ing vehicle 100 and the receiving vehicle 102 during unloading or transferring of agricultural material between the vehicles, For example, the coordination module 342 may facilitate maintenance of the unifbrrn lateral offret 138 and the uniform fore/aft offset 140 between the vehicles 102, 100 during unloading of the agricultural material, subject to any adjustments for attainment of a unifbrm distñbtion of material in the storage portion 106.

Collectively, the uniform lateral offSet 138 and the uniform fore/aft offset 140 may be referred to as a uniform spatia! offset. in certain examples, maintenance of the lateral offset 138 and the fore/aft offset 140 or coordination of any shift in the lateral offset 1 38 and thc fore/aft offset 140 (e g pursuant to a two-dimensional matrix of prc-estabhshcd pthitlons (x, y potits) fot unifrn i loadmg of a rcspectne particu±ar container 108 or the storagc portion 106 is a necessary or desired precondition to implementing spout angle adjustment of tiie spout 112 or the spout end i 18 by the alignment module 310, [00741 In one example, the transferring vehicle 1 00, in a leader mode, is steered by the auto--guidance module 360 or die steering cotitroller 3 16 in accordance with path plan or by a human operator. The master/slave controller 340 or the coordination module 342 controls the receiving vehicle 102 in a follower mode via a slave/master controller 508 (FIG. 5A) in which the transferring vehicle 100 operates in the leader mode. if the transferring vehicle 100 operates in an automated mode or an auto-steering node, the master/slave controller 340 provides command data locally to the steering controller 316, braking controller 318, and the propulsmn contioller 321) of the transferring vehicle. Such command data may be normalized (i.e., scaled), tune stamped, and communicated to the receiving vehicle 102 via the wireless communication devices 338, 506 (FIG. SA) for processing by the si-ave/master controller 503 (FIG 5A) Altcrnativdy, the velocity the acceleration, and the heading data of the transferring vehicle 100 is communicated to the receiving vehicle 102 via the wireless communication devices 338, 506 (FIG. 5t) to enable the receiving vehicle 102 to follow the path of the transferring vehicle 100 with a minimal time delay. In an automated mode and in a leader-follower mode, the receiving vehicle 102 and/or the transferring vehicle 100 are steered and aligned automatically during transfer of agric altural material front the transferring vehicle 100 to the rcceiving vehicle 102.

100751 Thc in1age processing nodu.e 302 provides image data to the Lser interface processing module 326 that pio ides, directly or rndrectiy, status message data and performance message data to the user interface 322. As illustrated in FIG. 3, the image processing module 302 communwatcc with the vehicle data bus 301 (c g, Controller Area Network (CAN) data bus).

[0076] in one example, the locatton-dctcrmmirig receiver 354, the first wireless communication device 338, the vehicle controller 324, the steering controller 316, the braking controller 318, and the propulsion controller 320 are capable of communicating over the vehicle data bus 304. in turn, the steering controller 3 1 6 is coupled to a steering system 362 of the transferring vehicle 100, the braking controller 3 18 is coupled to the braking system 344 of the transferring vehicle 100, and the propulsion controller 320 is coupled to a propulsion system 364 of the transferring vehicle 100.

[0077] The steering system 362 may have an electrically-driven steering system, an electro-hydrauhic steering system, a gear driven steering system, a rack and pinion gear steering system, or another steering system that changes the heading of the vehicle or one or more wheels of the vehicle. The braking system 344 may comprise a regenerative braking system, an electra--hydraulic braking system, a mechanical breaking system, or another braking system capable of stopping the vehicle by hydraulic, mechanical, friction or electrical forces. The propulsion system 364 may have one or more of the following: (1) a combination of an electric motor and an electric controller, (2) an internal combustion engine that is controlled by an electronic fuel median system or another fuel me4cnng deice that may he controlled by electncal signals, or (3) a hybrid vehicle in which an internal combustion engine drives a electrical generator that is coupled to one or more electric drive motors.

[0078] An optionJ mast controller 366 may he cot pled to the vehicle data bus 304, the implement data bus, or the image processing module 302 to control an optional first support 142. 202 lix mounting and ad ustably positiomng the first and/or second imaging device 126, 128. The mast controller 366 is adapted to change the orientation (e.g., the compound angular orientation) or height above ground of the first and/or second irnagrng device 126 128 motmted to the first support 142, 202 the orientation may be cxpressed as for example, a lit angle a pan angle, a down-tilt angle, a depression angle, or a rotation angle.

19079] In one configuratioii, the user interfiice 322 is arranged Ibr entering container reference data or dimensional parameters related to the receiving vehicle 102 tot example, the container reference data or dimensonal parameters tay include a distance between a trailer hitch or pivot point, which interconnects the propelled portion 104 and the storage portion 106, and front wheel rotational axis of the storage portion 106 of the receiving vehicle 102) 0080J In an alternate example, as illustrated by the dashed lines in FIG. 3, the system 300 further may include the optional odornetrical sensor 356 (e.g., odometer) and the optional inertial sensor 358. To estimate a distance traveled by the transferring vehicle during a measurement time period or a ground speed of the transferring vehicle, the odometrical sensor 356 may comprise a rnaicnc rotation censor, a gear driven sensor, or a contactlcss sensor for measuring the rotation of one or more wheels of the transferring vehicle. The odometrical sensor 356 may be coupled to the vehicle data bus 304 or an implement data bus['he inertial sensor 358 may have one or more accelerometers, gyroscopes, or other inertial devices coupled to the vehicle data bus 304 or an implement data bus. The optional odomctricai sensor 356 and the optional inertial sensor 358 may augment or supplement position data or motion data provided by the first location-determining receiver 354, 10081] The vision-augmented guidance system 400 of FiG. 4 is similar to the system 300 of FIG. 3; however, the system 400 of FIG. 4 further includes an implement data bus 402, a gateway 404, aad a chicle controller 406 coupled to the vehicle data bus 304 for optional lights 408 and a spout 410. The spout 410 is well suited for use with a self-propelled thrage harvester, whereas the spout 112 is well suited for use with a combine. However, either spouts 410 or 112 may be used with combines, harvesters, or other heavy equipment. The vehicle controller 406 controls the lights 408 a spout cortioller 412 controls the spout 410 -in one or more f the kflowing: a rotation actuator 414, a tilt actuator 416, or a deflector actuator 418. For example, the rotation actuator 414 may rotate the spout 410 about a first axis that is generally perpendicular to the ground, The tilt actuator 416 may tilt or rotate the spout 410 at an upward or domward angle along a second axis that may he generally parallel to the ground or substantially perpendicular to the first axis, The deflector actuator 418 may activate a deflector at or near an end. of the spout 410 to avoid overshooting or undershooting the storage portion 106 of the receiving vehicle 102 with harvested material. In one example, each of the rotation actuator 414, the tilt actuator 416, and the deflector actuator 418 may ha%-e a servo-motor, an electric motor, or an electro-hydraulic mechanism for moving or adjusting the orientation or spout angle of the spout 410 or the end thereof.

110082] In one configuration, the spout controller -412 controls the spout 410 based on operator input In anoti er configuration, the spout controller 412 controls the spout 410 via the image processing module 302 based on sensor data from one or more of the following sensors: a rotation sensor 420, a tilt sensor 422, and a deflector sensor 424. The rotation sensor 420 measures a first angle of rotation of the spout 410 about a first axis that is generally perpendicular to the -ound; the tilt sensor 422 measures a second angle of rotation (i.e., a tilt angle) of the spout 410 at an upward or downward angle along a second axis that may he generally parallel to the ground or substantially perpendicular to the first axis. The deflector sensor 424 may measure a deflector angle, a deflector active status, or a deflector inactive status at or near an end of the spout 410 to avoid overshooting the storage portion 106 of the recetving vehicle 102 with harvested material. In one example, each of the rotation sensor 420 and the tilt sensor 422 may have a magnetic field sensor, a Hall Effect sensor, a inagnetostrictive sensor, a magnetoresistive sensor, a variable resistor, or another suitable sensor for measuring an angle between the spaut 112 and the transferring vehicle 100, with respect to rotation along the first axis, the second axis, or othenvise In one configuration, the deflector sensor 424 may include a contact sensor or contact switch to provide a status message or status signal indicative of whether the deflector is extended or retracted with respect to the spout end 11 8.

0083] In one configuration, the implement data bus 402 may comprise a Controlicr Area Network (CAN) implement data bus, Similarly, the vehicle data bus 304 may comprise a controller area network (CAN) data bus. In an alternate example, the implement data bus 402 and/or the vehicle data bus 304 may comprise an ISO (International Organization for Standardization) data bus or ISOBUS, Ethernet, or another data protocol or communicitions standard [0084] The gateway 404 supports secure or controlled communications between the implement data bus 402 and the vehicle data bus 304. The gateway 404 may include a fliewall (C g hardwate o software), a conmmmcations router, or another secu-ity device that may restrict or prevent a network element or device on the implement data bus 402 from communicating (e.g, unauthorized communication) with the vehicle data bus 304 or a. network element or device on the vehicle data bus 304. The security device o2the g:teway 404 may allow such communication if the network clement or device on the implement data bus 402 follows a certain security protocol, handshake, nasswoxd nd key, or another -eeunty measure Further in one example, the gaLcwa, 404 may encrypt conmrunications to the vehicle data bus 304 and decrypt communications fiom the selucle data bus 304 if a piope encryption key is entered or other ecunt) measures ale satisfied The gateway 404 may allow network deqces on the implement data bus 402 that communicate via an open standard or third party hardware and software suppliers. In contrast the network devices on the vehicle data bus 304 are solely provided by the manufacturer of the transferring vehicle (e.g., self-propelled forage harvcstcr) or those authorized by the manufacturer.

100851 In FIG. 4, a first location-determining receiver 354, a user interface 322, a usa interface processing module 326, and the gateway 404 are coupled to the implement data bus 402. In oilier examples, such elements or network devices may be connected to the vehicle data bus 304. The vehicle controller 406 and the spout controller 412 arc eouplcd to the vehicle data bus 304. In turn, the vehicle controller 406 and the spout controller 412 are coupled, directly or indirectly, to optional lights 408 on the iransferring vehicle 100 and the spout 112 of the transferring vehicle 100.

Although the system. 400 of FIG. 4 is well suited for use or installation on a self-propelled thrage harvester, the systcrn 400 of FIG. 4 may also be applied to combines, harvesters, or other heavy equipment.

[0086] The system 300 of FIG. 3 and the system 400 of FIG. 4 apply to the transferring vehicle 100. whereas systems 500 and 502 of FIGS. SA and SB, respectively, apply to the receiving vehicle 102. In particular, the system 500 of FIG. 5\ is presented as an image processing module 302 that is positioned on the receiving vehicle 102, and the system 502 of FIG. 5B is nfig.sred to cooperate with the system 300 or 400 of FIG. 3 or FIG. 4, 10087] The system 500 of FIG. 5A includes the first imaging device 126, the image processing module 302, the user interface processing module 326, the gateway 404, a second incation-determining receiver 504, a second wireless communication device 506, and a slave/master controller 508, among other devices illustrated in FIG, 3. En one example, the first imaging device 126 is mounted on the propelled portion 104 (c g, a tractor) of the receiving vehide 102 facing to'..ards the storage portion 106 The second wireless communication device 506 of the recewn'g vehicle 102 maybe adapted for communicating data with the first wireless communication device 338 of the transferring vehicle 100 of FIG. 3 or FIG. 4. The second location-determining recener 504 may orovide position data 1ocation data, altitude, velocity, ot acceleration data.

oosj As illustrated in FIC. 5A, the image processing module 302 may estimate the relative posmoii of the transftmng vehcle 100 amt the receiving vehicle 102 Thc image processing module 102 rosy £o estimate the ielative orteitatton of the spoil end 11% x th respect to the stoiage portloll 106 to direct or eortrol the steering systaii 362, the braking system 344, and the propulsion systen 364 of the receiving vchiclc 102 via one or more controllers 316, 318, 320. The image processing module 302 may make such estimates to place the transferring vehicle 100 and receiving vehicle 102 in a targct tranJening position for transferring of matenal from the spoat end II S to the storage portion 106. For example, the target transferring position or cooperative aliwiment may refer to registration or alignment of the spout position and the container position (e.g., for one or more sampling time periods). Meanwhile, the transferring vehicle 100 may be controlled (with respect to, e.g., steering, velocity, and acceleration) by its own operator or the first location-determining receiver 354.

For exanpk, the 3.ctem 500 or the m'age processing module 302 may identi' the spout eid 1 18 compute (via stereo correspondence, disparity or other image processing) the relative position of the spout end 118 to the storage portion 106, he container penmeter 120 of Vie storage portion 106, and'or tFc een'ral Zone 134 of the storage portion 106.

[0089] In the example of FLU. 5A, the steering controller 316 associated with the steering system 362 of the receiving vehicle 102 may steer the propelled portion 104 of the receiving vehicle 102 in accordance with a cooperative alignment (eg., ahrnient suitable for effluent trari'ferimg of material horn time transfernug matenal to the receiving vehicle 102) based on location data and motion data am the first location-determining receiver 354 and the second location-determining receiver 504, With or without such location and motion data from one or more of the location-determining receivers 354, 504, the image processing module 302 may steer the propelled portion 104 of the receiving vehicle 102 to maintain the cooperative alignment for unloading or txansfemng material between the vehicles by using image data to align the spout 112 or the spout end 118 with the storage portion 106 or the container perimeter 120 of the receiving vehicle.

100901 In an example configuration of FIG. SB, the transferring vehicle 100 and the receiving vehicle 102 operate in a ieaderfol1ower conflguration in which the transferring vehicle 100 is in the lead role and the receiving vehicle 102 is in the following role, such that the receiving vehicle 102 tracks or follows the heading, the velocity, and optionally the acceleration of the lead vehicle with a target spatial offset.

in FIG. 5B, the image processing module 302 may he active only on the transferring vehicle 100 and may be inactive or not present on the receiving vehicle 102. On the transferring vehicle 10, the image processing module 302 and the first location-determining receiver 354 may provide image guidance data, if available and/or reliable, and first motion data (e.g., location, heading, velocity, and acceleration), respectively, that are used by the transferring vehicle 100 and the receiving vehicle 102 for guidance.

[0091J In one example configuration, the image guidance data and/or the first motion data are tranrnitted by the first wireless comnumication device 338 on the transferring vehicle 100 to the second wireless communication device 506 on the receiving vehicle 102. On the receiving véhicle 102, the slave/master controller 508 may receive the first motion data and the first location data from the first location-determining receiver 354 on the transferring vehicle 1.00, and the second motion data and second location data from the second location-determining receiver 504 on the receiving vehicle 102. The slave/master controller 508 may generate an or control data fbr controlling die steering system 362, the braking system 344, and the propulsion system 364 of the receiving vehicle 102. The error or control data may be based on a difference between the first motion data and the second motion data and/or the difference between the first location data and the second location data.

100921 In another example configuration, the master/slave controller 340 on the transferring vehicle 100 may send control signals or control data messages to the slave/master controller 508 on the receiving vehicle 102 to control the steering system 362, the braking system 344, or the propulsion system 364 of the receiving vehicle 102. The master/slave controller 340 may send such signals or messages to the slave/master controller 508 to optimize the relative position of the spout end 118 to the storage portion 106 of the containei 1 08, to optimize the target spatial separation between the vehicles, or to avoid an imminent collision between the vehicles 102, 100. On the receiving vehicle 102, the slave/master controller 508 may operate in a slave mode or follower mode undr the control of the master/slave controller 340 An auto-guidance module 5.0 and a coordination module 511 within the slave/master controller 508 may provide guidance of the receiving vehicle 102 consistent with location data and a path plan or consistent with other guidance data or command data fiom the nasteri slave controller 340 to the transfcrnng vchicle 100 [0093] In FIG. 5A or FIG. SB, the system 500 or 502 for the receiving vehicle 102 may be used in conjunction with the system 300 or 400 of the transferring vehicle 100 of FIG, 3 or FIG. 4. The second wireless communication device 506 maybe coupled to the vehicle data bus 304. The wireless communication devices 338, 506 may exchange or communicate position data, relative position data, command data, or control data for controlling, adjusting or coordinating the position and orientation of the ehics, and more particularis', the position and the onettauon of the spout 117 or spout end 11$ o em the central ione 134 ol the container 108 The communicated data between the wireless communication de ices 338, 506 may ucluue any of the folio wing data (I) position data or location daa from either locationdeteimming receiver 354 or 504. (2) command or guidance data from an image processing module 302 on the transferring vehicle 100 or the receiving vehicle 102, (3) command or guidance data from the master/slave controller 340 or the coordination module 512, (4) command or guidance data from the slave/master controller 508 or the coordination module 512 or (5) alignment data (e.g., data concerning relative position of the imaging devices, relative position of reference points on the vehicles, and relative alignment between the spout end 118 and the container perimeter 120) from the alignment module 310. For example, to determine a relative alignment between the spout end 118 and the container perimeter 120, the image processing module 302 or alignment module 310 may use first location data of the first location-determining receiver 354 and second location data of the second location-determining receiver 504 to determine a relative position or spatial offset between the two vehicles 102, 100 or a relative position of the first miaging de%ice 126 and the second imaging devicc 128 100941 The system 500 of FIG. SA may support different configurations or combinations of electronic systems (e.g., systems 300 and 500 or 400 and 500) of the transferring and receiving vehicles 100, 102. In a first corifiraiiori, the first imaging device 126 may be disposed on the recev'ng vehicle 102 instead of, or in addifion to one or more of the first or second imaging devices 126, 128 on the transferring vehicle 100. In a second configuration, the system 500 of FIG. 5A may provide collected image processing data from the receiving vehicle 102 to the transferring vehiclc 100 via the transmission of the collected image processing data from the second wireless communication device 50$ and to the first wireless communication device 338. 1-icre, in a second configuration, the collected imaging processing data from the receiving vehicle 102 may be referred to as supplementary data, comp ementary image data, or additional image data TI he additional mage data may pnn ide additional petspeutlve or vespoin4 s that may supplement the image data collected by the transferring vehicle 100. For example, the additional image data may provide more accurate or supplement image data when the image data collected by the transferring vehicle 100 is affected by moisture, dust, poor ambient lighting, glare or reflections that do not similarly impair or impact the additional image data.

[0095] An optional odometrical sensor 356 may be coupled to the vehicle data bus 304 or the implement data bus 402. An inertial sensor 358 may have one or more accelerometers, gyroscopes or other inertial devices coupled to the vehicle data bus 3U4 or the implement data bus 402.

10096] FIG-. 6 illustrates a twodiniensioria1 representation of various possible illustrative distributions of material in the container 108, consistent with a view along reference line 6-6 in FIG, 2A. In one configuration, the y axis is coincident with the longitadinal axis or direction of travel of the containet 108, the z axis is comcideni with the height of material in the container 108, and the x axis is perpendicular to the direction of travel of the container 108. In such a configuration, the x, y and z axes are generally mutually orthogonal to each other.

[0097] In the chart of FIG. 6, the vertical axis represents a mean height () 600 of the matenal in the container 108, the honiontal axis reprcsent a longitudinal axis (y) 602 of the contaIner 108, and the dashed line on the vertical ax's indicates the maximuru tapaury or container capacity 604 A front 606 of the contamer 108 is tocateci at the origin, whereas a back 608 of the container 108 is located on the vertical axis.

[0098] FIG. 6 shows tbur illustrative distributions of material within the container 108, The first distribution is a bimodal profile 610 in which there are two main peaks in the distribution of material in the container 108 The bimodal profile 61 C) is shown as a dotted inc. The bimodal profile 61.0 may occur where the spout angle adjustment is governed by an electro4iydrauiic system with nonproportional valves. Likewise, multimodal distributions with more than two main peaks may be observed. Bimodal profiles 610 or multimodal profiJes may exist if material transfer occurs at a small number of discrete reiaive positions between the spout end 11 8 and the containa 108. In this example, the transferred material forms discrete piles, thereby defining the bimodal 610 or multirnodal distribution profiles.

[0099J I h& second distribution s the frontsKewed modal profile 612 in which there is single peak of material toward the front of the container 108. The front-skewed modal profile 612 is shown as alternating long and short dashes. The second distribution may occur where the volume or length (y) of the container 108 is greater than a minimum threshold and where the relative alignment between the spout end 118 and the container 108 is generally stationary during a substantial pothon of unloading of the material.

[00160] The third distribution is a desired profile, such as a uTnform profile 614 which may be achieved by following a suitable flU strategy as disclosed in this document. For example, during unloading, the angle of the spout 112 maybe adjusted to promote uniform distribution of the agricultural material in the storage portion 106 of the container 108. In other examples, the desired profile may be a sloped profile 616. The angle of the spout 112 may be adjusted in lieu ot or in addition to adjusting the relative offset spacing between the transferring vehicle 100 and the receiving vehicle 102.

1001011 FIG. 7 illustrates example strategies of adjusting the relative offset spacing between the transferring vehicle 100 and the receiving vehicle 102. FIG. 7 is a plan view of a transferring vehicle 100 and a receiving vehicle 102, in which the transferring vehicle 100 is aligned within a matrix 700 of possible offset positions 702, 704 between the transferring and receiving vehicle 102, 100. Each offset position 702, 704 may be defined in terms of a combination of a unique lateral offset 138 and a unique fbi a/aft offset 140 between the ehcles 102, 100 As shown, thc matrix 00 is a two-dimensional, 2 x 3 matrix (i.e., 2 columns by 3 rows) of possible otThet positions 702. 704, Although six possible matrix positions 702, 704 are shown in FIG. 7. in alternate examples the matrix 700 may have any number of possible offset positions greater than or equal to two. Here, the transferring vehicle 100 occupies a eunent offset position 704 in toe first column and tin second row of the matrix 700, whereas the other possible offset positIons 702 are not occupied by the transferring vehicle 100. Any of the systems 300, 400, 500 may direct the image processing module 302, the master/slave controller 340 of the transferring vehicle 100, or the slaveiniaster controlk'i 508 of the receiving ehiele 102 to shift to any unoccupied or other possible offset positions 702 within the matrix 700 to promote or facilitate an even distribution of agricultural material within the container 108 or storage portion of the receiving vehicle 102. The spatial offset between the transferring vehicle 100 and the receiving vehicle 102 may be adjusted in accordance with the matrix 700 or another matrix of preset positions of spatial offset to promote even distribution of agricultural material in the storage portion of the receiving vehicle 102, if such matrix is associated with a unique, relative lateral offset 138 and fore/aft offset140 between the vehicles 102, 100.

1001021 in one example of FIG. 7, both the transferring vehicle 100 and the receiving vehicle 102 may be moving forward at approximately the same velocity and heading (e.g., within a tolerance or error of the conirol systems during harvesting).

where the relative position of the receiving vehicle 1 02 is generally fixed or constant w F respec to cad' positior 702, 704 in the matrix 700 that the transfcrnng ehic1e may occupy 1001 03j In an alternate example, the receiving vehicle 102 may be shown as occupying a two dirnnsiona1 rna4nt (e g, a 3 X 3 matrix harnghrce columns and three rows) of possible offset positions, while the position of the trar1sfethng vehicle is generally fixed or constant with respect to each position of matrix that the receiving vehicle 102 may occupy. As directed by any of the systems 300, 400, 500, the image proccsslng module 302 or the slave1master cocirciler 508 of the reeivrng vehicle 102 may shtfl to any unoccupied or other possible offset pot.ons within the matrix to promote or facilitate an even distribution of aieu1tural material within the storage portion 106 of the container 108 of the receiving vehicle 102.

j00i04 FIGS, 8A-10 described an example control strategy for achieving a desired fill profile. In an example strategy, the systc 300 may include the ability to detect and fill voids in a-iculturaI material within the storage portion 106 of the receiving vehicle 102. Voids may be thought of as underutilized space within the storage portion 106. By detecting and filling voids, the distribution of agricultural material within the storage portion 106 may be opti mzed to a desired flIl profile. Voids may be related to the difference between a fill level of agricultural material and a desired fill level set point. Additionally or altematively, voids may be related to the relative volumetric differericc in agricultural material between different sections of the storage portion 106.

1001051 FIG. 8A is an example fill model 800 of the storage portion 06 ofthe container 108 of the receiving vehicle 102 that is shown relative to a plan view of the storage portIon i0& The example fill model 800 is one possible model that the image proeess1ng module 302 may create from inputs from he system 300 Such nput may include hut are not limited to inputs from the first and second imaging devices 1 26, 128, the vehicle controller 324, the vehicle data bth 304, and modules w thin the image processing module 302. The system 300 may use the fill model 800 to identify the fill level and the fill profile of agricultural material within various sections of the storage portion 106 to identify voids within the awicultural material and to command the spout 113 to fill the voids.

1001061 The fill model 800 includes the container 108, which is bounded by its perimeter 120. The fill model 800 is considered relative to a top view of the container 108, and the perimeter 120 may represent the top rim of the container 108. The perimeter 120 defines the storage portion 106 inside of the perimeter 120 and an external portion 802 outside of the perimeter 120, The till model 800 may further include a buffer zone (not shown) to offset the central zone 134 of the storage portion 106 from the perimeter 120. The buffer zone may be utilized to ensure that material is not transferred or spilled outside of the storage portion 106 and into the external portion 802 1 he storage nortion 106 Thay be dis ided into a pLurality of cells one of which is designated with reference numeral 804.

[00107] The cells 804 in the example of FIG. 8A are rectangular in shape and define a rectilinear grid within the storage portion 106; however, in other examples, the cells 804 may be shaped as squares, rectangles, triangles, hexagons, octagons or any other polygonal shapes [he cells 804 may each represent individual meaurcnients of matenel fill kvcls within the to"age po't'on 106 In other examples, the fill model 800 may include more or fewer cells 804.

[00108] Turning to FIG SB, the storagt portion 106 is showii as divided into a plurality ot columns 806 [he colunms 806 are subst&thatly perpendicuiar to the longitudinal axis 602 of the storage portion 106 of the container 108. In the example fill model 800 as depicted in FIG. 8B, the columns 806 span the width of the container 108, and each column 806 includes a plurality of adjacent cells 804. While the columns 806 are shown as an aggregation of a single column of cells 804. the colunms 806 may aggregate cells 804 in any suitable manner.

1001091 For example, as shown in FIG. 8C, the storage portion 106 may be divided into a plurality of zones 808 with each zone 808 comprising a plurality of columns 806.

[001101 FIG. SD illustrates the example fill model 800 having cells 804. columns 806, zones 808, and an example void area 810. FIG. 8D illustrates that the void area 810 may extend across multiple columns 806 aml zones 808, The parameters defining the zones 808 and the void areas 810, respectively, are independent of each other.

Furthermore, the respective sizes of the cells 804, the columns 806, the zones 808, and/or the void areas 810 may be adjustable depending on operator preferences, and/or characteristics of the transferring vehicle 100 and/or the storage portion 106 of the container 08. The cells 804, the columns 806, the zones 808, and the void area 810 all represent sections of the storage portion 1 06 according to one example of the system 300 and are not intended to limit the scope of this disclosure. in other examples, the system 300 may utilize one or more of the cells 804, the columns 806.

the zones 808, and/or the void areas 810.

100111] FIG 9 depicts an cxample lii. model 800 of FIGS 8k-SD with example fill level data determined by the image processing module 302 based on the image data collected by the first and!or second imaging devices 126, 128, as described above Each eel 804 within thc storage portion 106 is assigned a value reptesetative of the fill level of the section of the storage portion 106 corresponding to the cell 804 or a value representative of areas that are unreadable or that are external to the storage portion 106. The fill level is the height to which the storage portion 106 may be filled with aw-iculturai material, as measured or dctcrrninedby the image processing module 302 based on the image data collected by the first and/or second imaging devices 126, 128, as described above. ftc fill level represents a single dimensional value represeiting a measured length; however, material volume may be calculated based on the fill level with respect to the size and geometry of the section of the storage portion 106 corresponding to a cell 804.

[00112] In some circumstances as shown in FIG, 9, the fill level of one or more cells 804 is unreadable and is asiigned a value indicating as such, which in the example of FIG. 9 is 127. The cells outside of the storage portion 106 in the external portion 802 are assigned a value indicating as such, which in the example of FIG. 9 is 126. The remaining values correspond to the till levels of sections of the storage portion 106 corresponding to each cell 804, relative to a datum. In the example of FIG. 9, the datum is the top rim of the storage portion 106; however, in other examples, the datum may be the bottom surface of the storage portion 106, the pound, the position of the wout end 118, or other daturns I hc fill level vaLes represent the fiil lee1 with!tspeLt to the datum or to an offset from the datum Furthermore, each increment of a fill level value may represent a measurement of length. For example, each incremental fill level value may represent 5 centimeters, such that a fill level value of I represents 5 cm above the datum, a fill level value of-I represents S centimeters below the datum, and a fill level of 4 represents 20 centimeters above the datum hi other exampies it may be desirable to use a non-linear fill level scaic [001131 The fill model 800 is assigned one or more fill level set points, which represent the desired fill level for eaeh eel. In the example of FIG, 9, the fill level set point is 4, and is uniform across the entire storage portion 106. In other examples, the fill level set points may vary amongst the cells S04 of the stoiage portion 106 For example, it may be desirable to have a higher fill in the center of the storage portion 106 to achieve the sloped profile 507 as described above in connection with FIG. 6.

OOl14J FIG. 10 shows void volume levels for each cell of the fill model 800. Void volume represents the difference between the fill level set point (e.g., 4) and the actual fill level of a cell, for cells in which the fill level is lass than the fill level set point.

Thus, the void volume rtprcscnts the olurne of matenal that must he filled into a ccli fbr the fill level of the cell to equal the till level set point.

[001151 There are many different ways to identify a void. In one example in conjunction with FIG. 7. a void is defined as an area (e.g., a collection of cells 804) of the storage portion 106 that has an average fill level less than the fill level set point and a total void volume above a predetermined value. In other words, for an area to be classified as a void, the area must not be hill and must require a certain amount of material to be transterred Into it until it i,. con',idcrcd full In other examples, other mathematical relationships for void identification relating to fill level may be used.

Typically, harvesting machines have a very high material flow rate. Thus, the criterion for identifying voids should ensure that the identified voids are sufficiently sized so that an attempt to fill the void does not result in over4illing. which may result in material spillage.

[00116] In the example depicted in FIGS. 9 and 10, voids are determined with respect to a predetermined number of adjacent columns having respective fill levels below the fill level set point and having a total void voiwne above a predetermined value The manner in which vo1ds are identtheu is carefully designed with repeLt to the tolerance of control systems ofthc transferring vehicle 100 (e.g., the spout module 308 and the steering system 362) and the material flow rate of the agricultural material being transferred from the transferring vehicle 100 to the storage portion 106 of the receiving vehicle 102 via the spout 112 Other examples utiliie alternative methods of identifying voids to take into account higher or lower control tolerances of systems of various transferring vehicles 100. Moreover, it is desirable to transfer material via the spout 112 at the maximum flow rate of the spout 112 throughout the unloading process. Therefore, the manner in which voids are identified should ensure that voids are at least a pathcuar two-dimensioral size (e g cover at least a minimum area) and. at lea'a a particular tee-dnrensionai volume, such that the material being transferred to the void at the maximum flow rate of the spout 11 2 does not overfi1 the void or adaent sections of the storage portion 106 before the spout 112 is commanded to another section of the storage portion 106.

00I 17j To identify voids, the image processing module first determines the fill levels of each of the cells 804 of the storage portion 106 based on the image data provided by the first and/or second imaging devices 126, 128. Next, the average fill level is calcuAated foi each colt1mn 806 If a column ontains an unreadable value, the unreadable value may be iored, or may be assied the average fill level of the remaining cells 804 within the column 806. In this example, three adjacent columns must have an average fill level below the fill level set point in order for the area comprising the three adjacent columns to be considered a void. In the example of FIG. 9, each of the thec adjacent columns that define the void area have an average fill level of 3, which is less than the fill level set point, which is 4 in this example. Next, the total void volume of the area is calculated. If the total void volume of the area comprising the three adjacent columns is eatcr than a predetermined value, which in this example is 50, the area is classified as a void As shown in FIG. 10, each of the three adjacent columns that define the void area have a total void volume of 54 (1 8+21 ± I 5=54), which is greater than the predetermined value, which is 50 in this example. Thus, the area is classified as a void.

[00118j In other examples, fewer or more columns may be considered in identifying voids, in certain examples, the number of columns required for a void may be adjusted based on an "aggressiveness factor." Requiring a greater number of columns tbr void classification results in a less agressive fill strategy because fewer %olds will he identified Accordingly iequinrg fewer columns lbr void classification results in a more aggressive fill, strategy because more voids will be identified, which will result in a more unifbrm fill profile. FIG. SD shows an example void that is identified in conjunction with the system of FIGS. 9 and 10 as described above.

[00119] While an example manner of implementing the system 300 of FIGS. 4-58 is illustrated in FICS. BA-i 4, one or more of the elements, processes and/or devices illustrated in FIGS. 8A-14 may be combined, dividl, rcarranged, omittl, eliminated and/or implemented in any other way. Further, the image processing module 302 the container module 306, the spout module 308, the a1ignment module 310, the arbiter 3 12, the user interface processing module 326, the material profile module 314, the guidance modules 360, 510, the coordination modules 342, 512, the master/slave controller 340, the slave/master controller 508. the mode controller 346, and/or, more generally, the example system 300 of FIGS. I SB may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example the image processing module 302, the container module 306, the spout module 308, the alignment module 310, the arbiter 312, the user interface processing module 326, the material profile module 314, the guidance modules 360, 510, the coordination modules 342, 512, the master/slave controller 340, the slave/master controfler 508, the mode controller 346, and/or, more generally, the example system 300 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(sl) and/or field programmable logic deice(s) (FPLD(s)) When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example image processing module 302., the container module 306, the spout module 308, the alignment module 310, the arbiter 312, the user interface processing module 326, the material profile module 314, the guidance modules 360, 510, the coordination modules 342, 512, the master/slave controller 340, the slave/magWr controller 508. and the mode controller 346 are hereby expressly defined to include a tangible computer readable storage or storage disk such as a memory, a digital versatile d'5k (DVD), a compac disk (CD), a Bin-ray disk, etc. storing the software and/or firmware. Further still, the example system 300 of FIGS 1-5B may inciude one or more tlerntnts, processes and/or devices in addition to, or instead of, those illustrated in FIGS. 84-14, and/or may include more than one of any or all of the illustrated elements, processes and devices.

[0012(fl A flowchart representative of example processes, which may be implemented by machine readable instructions, for implementing the system 300 of FIGS. 1-SB is shown in FIGS. 844. In this example, the machine readable instructions comprise a program for execution by a processor such as the processor 1512. shown in the exampie processor platforn 1500 discussed below in connection with FIG. 15. The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a mu-ray disk or a memory associated with the processor 1512, hut the entire program and/or parts fiiereof could alternatively be executed by a device other than the processor 1512 and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowcharts illustrated in FIGS. 8-14, many other methods of implementing the example system 300 may alternatively he used, For example, the order of execution of the blocks may be changed, and/or some of the blocks described may he changed, eliminated, or combined, 1001211 As mentioned above, the example processes of FIGS. 8A14 maybe implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a tangible computer readable storage medium such as a hard disk drive, a hash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a c ache, a random-access inenioly (RAM) and'oi any oilier storage device oi stoi agc disk in which mionnanon is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or fhr caching of the information). As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device andtor storage disk and to exclude propagetng signals. As used herein, "tangible computer readable storage medium" and "tangible machine readable storage medium" are used interchangeably. Additionally or alternatively, the example processes of FIGS. SA-14 maybe implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/oi any other storage devi.t or storage disk in shicn intbrmation is stored fbr any duration (e.g. for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the inlbrrnation), As used herein, the term non-transitory computer readable medium is expressy defined to include any type of computer readable device or disk and to exclude propagating signals. As used herein, whcn the phrase "at least" is used as the transition toni in a preamble of a elaini, it is openended in the same manner as the toni "comprising" is open ended.

[90122] FIG. 11 depicts an example material transfer filling sequence 1100 that utilizes a general fill mode 1102, a zone fill mode 1106, and a void fill mode 11 lOin sequence. The procedure tbr identif'ing voids as described above may be used in conjunction with the example tilling sequence 1100. In other examples, the system 300 may utilize additional or fewem fill modec hi the example filling sequence 11 00, the operator tills the storage portion 106 in the generi fill mode 1102 until a partial fill threshold is met at block 1104, at which point the storage portion 106 is consr&rcd partially full For example a partIal fill tlueshold may he met when the storage portion 106 is two--thirds thU or threequarters full. Once the partial fill threshold is met, the transferring vehicle 100 enters the zone filling mode 1106, in which each zone is filled until the average fill level in each zone reaches the fill level set point at block 1108, such that all of the zones ate considered hill [Jpon completion, the transfen-ing vehicle 100 enters the void filling mode 1110, in which any remaining voids are identified and filled at block 1112. Once all of the voids are filled, the transferring vehicle 100 stops material transfer at block 1114.

[Q0123 Although the example filling sequence 1100 is described with the general fill mode 1102, the zone fill mode 1106, and the void fill mode 1110 performed in sencs, the respective fill modes may be performed in any order or combmation or exampk, in a filling sequence 1100, block 1108 irtay require that a snglc zone is considered thll before entering the void filling mode at block 1110. The void filling mode 1110 may then he performed on that zone until it is considered full at block 1112, at which point the system may revert to block 11(16 to fill the next zone.

[001241 FIG. 12 illustrates a general fill mode 1200, 1202. In this example the operator of the transferring vehicle 100 unloads agricultural material into the receiving vehicle 102 in the general fill mode at block 1204. At block 1206, the image processing module 302 analyzes signals from the first and/or second imaging devices 126, 128 to determine the material fill level for each cell 804 of the fill model 800, At block 1208 the image processing module 302 calculates the average Jill Icscl ot the entre storage portion 106 At block 1210, the average fill level of the storage poition 106 is compared o a parti4 fill threshold It the average fill level s less than the desired partial fill threshold, the transferring vehicle 100 continues to transfer agncuitural n"atcnal into thc storage portion 106 Onu the a'icrage fill level is gieatei than or equal to the partial fill threshold, at block 1212, the general fill mode 1200 is complete and the transfernng vàicle 100 begins the tone fill mode I he optional mode controller 346 may control fill 1Tiodes..he general fill mode may be fully automated, semi-automated, or manual (e.g., opcratordirccted manual). In a fully automated mode, the system 300 may fill the storage portion 106 in one or more controlled locations. In a manual mode, the operator controls the fill location. A semi-automated mode may include controls to prevent spillage. For example, the system 300 may include controls to sense if the spout 112 is near the perimeter 120 of the storage portion 106, to prevent an operator from moving the spout 112 beyond the perimeter 120 and/or to stop material transfer if the spout 112 moves too close to the perimeter 120.

[00i25 FIG. 13 illustrates an example zone fill mode 1300, 1302. At block 1304, the image processing module 302 determines the first desired fill zone and moves the spout I 12 to the desired. fill zone, In one example, the first desired fill zone is the zone at thc end othe storage portion 106 nearest the current position of the spout 112 Blocks 1306-1310 are then performed as explained above in conjunction with FIGS. 8ASD and 9-10 At block 1312 the image proccssing modLle 102 detrmines whcthcr Cit aerage fill level of the zone s grtater thu the desued fill lcvel Jr not, at block 13 14, the tranLsfcrring vehicle TOO fills the zone and recalculates fill levels at blocks 13061310 until the average fill level of the zone is greater than the desired fill level, at which time, the zone is classified as full at block 13 16. At block 1318, the image processing module 302 considers whether all of the zones arc fill. If not, at block 1320, it determines the next fill zone. In one example, the next fill zone may be the zone adjacent the current zone. In another example, the transferring vehicle 100 may begin at the center zone and selectively move outward. Each zone is filled until all of the zones are classified as liii!, at which point, the transferring vehicle 100 begins void filling mode at block 1322. In other examples, the transferring vehicle begins the void filling mode at block 1:322 when fewer than all of the zones are classified as fill, [001261 FIG 14 illustrates an example void filling mode 1400, 1402 At block 1404 the image proessing module $02 determines the fill lesels and void volumes as explained above in conjunction with FIGS. SA-8D and 9-10. At block 1406, the image processing system searches for voids. As explained above, the system may define a void in many different ways. In one example, an area is considered a void when the average fill level in the area is less than a desired fill level and the total void volume of the area is greater than a predetermined value. In another example., an area is considered a void when the average fill level of each of a predetermined number of adjacent columns is less than a desired fill level and the total void volume of the predetermined number of adjacent columns is greater than a predetermined value. If the result of block 1406 is "yes," the area is classified as a void in block 1408. lii block 1410, the void is filled. The fill level is determined in block 1412 arid the void continues to he filled until, in block 1414, it is determined to be full, The criteria that are used to define a void may also he used to determine whether the void is thU. In another example, other criteria are used to determine whether the void is fill. For example, the total void volume requirement for defining a void maybe lower than that for determining whether the void is full. Once the void is filled, the image processing module $02 searches for more voids and performs blocks 14081 414 on any additional voids. Once no voids are identified, the tilling mode ends at block 1416 and the transfening vehicle 100 stops material from being transfeTred to the storage portion 106 9O127] Example void filling modes 1403 may include further steps to control the order n which voids are filled In one nample the void filling mode 1400 may command the spout 112 to sweep from hack4o-front ot front4oback and selectively fill voids as they are identified. In another example, the void filling mode 1400 may begin filling the void with thc largest void volume and sequentially filing smallei voids until all voids are tilled. Certain examples utilize a control strategy that minimizes actuaLon energy of the spoat 112 by mini nizu'g movt.ment of the spout 112 OtPer examples may est1 ret the oid filling inodc to portions of a zone dee-id1ng on the material distnbutinn tthin the storage portion 106 In an exampk in which the void filling mode 1400 is entered within a zone prior to all of the zones being considered frill, the void filling mode 1400 may iore portions of a zone that are adjacent cells or columns having lower fill levels. Otherwise, voids may he falsely identified when grain is merely flowing into an empty zone. Subsequent void filling modes 1400 may consider the previously ignored portions.

1001281 In each of the fill modcs decnbed ahoe, the tmige processing modu.e 30' miy determine the fill kd of the storage portion 106 continuously as material is being transferred. This allows the system 300 to adaptively control the fill location based on the actual dynamic fill levels oimate.rial within the storage portion. One challenge with determining fill level from images taken by imaging devices is in determining the fill lev&l in the area to which material is being transferred. To address this challenge, the image processing module 302 may estimate the change in fill level in an area to which material is bemg transferred by consiaenng the flow rate of the material being transferred and the amount of transfer time. The. volumetric flow rate of the unloading auger of many combines is known Depending on th model at tin combine, material transfer may occur at a rate of 4-6 bushels per second. Typically, self-propelled forage harvcsters have a mass flow sensor on the spout that measures the mass flow rate ofmatenai that is tiansferred to a ieceiving vehicle 102 Assuming a relatively constant density of forage, the volumetric flow rate and, hence, the volume of material transferred could be calculated. In operation, the image processing module 302 would consider the fill, level or void volume of an area prior to unloading in. that area, as well as the fill level or void volume of adjacent areas. Once the transferring vehicle 100 begins transferring material to the area, the image processing module 302 may model the expected dynamic fill level of the area by considering the material flow rate, unloading time, arid the expected flow conditions of the material to adjacent areas.

OOi29j FIG. IS is a block diagram of an example processor platform 1500 capable of executing the instructions of FIGS. 8-14 to implement the system 300 of FIGS. 1 SB. The processor platform 1500 can be, for example, an electronic control module (e g, a vhcIe control module or an engme control module) a sewer, a personal computer, a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPadTM), a personal digital assistant (PDA), an Internet appliance or any other type of computing device.

1001301 The processor platform 1500 of the illustrated example includes a processor 1512. The processor 1512 of the illustrated example is hardware. For example the processor 1512 can be implemented by one or more intewated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer.

fOOl3fl The processor 1512 of the nlusiratcd example mciudcs a lo a! n emory 1513 (e.g., a cache). The processor 1512 of the illustrated example is in communication with a main memory including a volatile memory 1 5 14 and a non-volatile memory 1516 via a bus 1518. The volatile memory 1514 may be implemented by Synchronous Dynamic Random Access Memory (SORAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (UDRAM) and/or any other type of random access memory device. Tue non- -olatilc memory 1510 may be implemented by flash memory and/or any othei debired type of memory device. Access to the main memory 1514, 1516 is controlled by a memory controller.

[00132] The processor platfbrm 1500 of the illustrated example also includes an interface circuit 1520. The interface circuit 1520 may be implemented by any type of interthce standard, such as an Ethernet interface, a universal serial bus (USB), and/or 00133J In the illustrated example, one or more input devices 1522 are connected to the interface circuit 1520. The input device(s)! 522 permit(s) a user to enter data and commands into the processor 1512. The input device(s) can be implemented by, for example, a knob, a lever, an audio sensor, a microphone, a camera (still or video), a kcyboid, a burton, a mouse, a tovLhscreen, a traekpad a ftackball, isopornt and/or a voice recognition system.

[001341 One or more output devices 1524 arc also connected to the interface circuit 1520 of the illustrated example. The output devices 1524 can he implemented, thr example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touchscreen, a tactile output device, a light emitting diode (LED), a printer and/or spcakers) The interface circuit 1520 of the il1utratec1 exampic, thus typ'eaFy includes a waplücs driver card, a graphics driver chip or a graphics driver processor.

[001351 The interface circuit 1520 of the illustrated example also includes a communtcation device such as a transmitter, a receiver, a transceiver, a modem andtor network interface card to facilitate exchange of data with external machines (e.g, computing devices of any kind) via a network 1526 (e.g., an Ethernet connection, a digital subscriber line (DSJ4, a telephone line, coaxial cable, a cellular telephone system, etc.).

001361 The processor platform 1500 of the illustrated example also includes one or more mass storage devices 1528 for storing software and/or data. Exanipes of such mass storage devices 1528 include floppy disk drives, hard drive disks, compact disk drives, Biuray disk drives, BAlI) systems, and digital versatile disk (DVD) drives.

[001371 The coded instructions 1532 of FIGS 844 may he stored in the mass storage device I 28 in the ohthle mcmory 1514, ra the noa-volatile mernozy 1516 and/or on a removable tangible computer readable storage medium such as a CD or DVD, 00i38} From the thregoing, it vi11 appreciate that the above disclosed methods and apparatus of detecting and filling voids improves operational efficiency of harvesting agricultural material by optimally filling storage portions with agricultural material, improves safety by automating at least a portion of the material unloading process so that the operator of a harvesting machine can focus his attention on harvesting, reduces grain spillage, prevents damage to equipment by evenly distributing load imparted on the equipment by the agricultural material, and reduces harvesting time by facilitating seamless unloading while harvesting.

1001391 Allhough certain example methods, apparatus arid articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto.

On the contrary, this patent covers all apparatuses, methods and articles of manufacture fairly falling within the scope of the appended claims either literally or under the doctrine of equivalents.